Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ASIM Registry Event schema parser with its sample and test data for Trend Micro Vision One #8680

Merged
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
4149e07
ASIM Registry Event schema parser with its sample and test data for T…
Jul 31, 2023
95e4e4e
Fixed kql validation error.
Aug 18, 2023
4e5979a
Updated PR by implementing previously suggested some general changes …
Aug 29, 2023
94d87a5
Fixed the KqlValidation error by adding _ItemId field
Aug 29, 2023
95edc33
Updated and added raw logs.
Sep 21, 2023
1fb08e7
Resolved conflicts.
Sep 25, 2023
abdd270
Fixed the suggested review1 changes
Oct 3, 2023
406e545
Merge branch 'master' into TrendMicroVisionRegistry
Oct 16, 2023
3dabf25
Fixed suggested review2 changes.
Oct 16, 2023
0a8b652
Fixed the kql validation error by correcting the TimeGenerated field …
Oct 16, 2023
0a81e48
Resolved merge conflicts.
niralishah-crest Oct 23, 2023
b4d961d
Removed file changes of commit 95e21fb which is pushed and merge here…
niralishah-crest Oct 23, 2023
85d84bd
Merged master into TrendMicroVisionResgistry
niralishah-crest Nov 23, 2023
df41d17
Resolved merge conflicts.
Nov 24, 2023
8d8642f
Resolved Merge Conflicts
Nov 24, 2023
75b840d
Done suggested change in imRegistry.yaml file
niralishah-crest Nov 27, 2023
4ac50b4
Fixed KQLValidation error.
niralishah-crest Nov 27, 2023
b9a95bd
resolved merge conflicts
niralishah-crest Nov 28, 2023
1edcb75
Resolved merge conflicts.
niralishah-crest Dec 5, 2023
ad27128
Removed unnecessary changes from this PR.
niralishah-crest Dec 5, 2023
163c5da
Removed unnecessary changes from this PR.
niralishah-crest Dec 5, 2023
c180c4d
Added TrendMicro parser in ASimRegistry.yaml
niralishah-crest Dec 5, 2023
9584142
Merge branch 'master' into pr/8680
v-atulyadav Dec 6, 2023
b8c3887
Merge branch 'TrendMicroVisionRegistry' of https://github.com/jayeshp…
v-atulyadav Dec 6, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,889 changes: 1,886 additions & 3 deletions .script/tests/KqlvalidationsTests/CustomTables/TrendMicro_XDR_OAT_CL.json

Large diffs are not rendered by default.

7 changes: 5 additions & 2 deletions ASIM/dev/ASimTester/ASimTester.csv
Original file line number Diff line number Diff line change
Expand Up @@ -488,6 +488,7 @@ EventOriginalSeverity,string,Optional,Dns,,,
EventOriginalSeverity,string,Optional,FileEvent,,,
EventOriginalSeverity,string,Optional,NetworkSession,,,
EventOriginalSeverity,string,Optional,ProcessEvent,,,
EventOriginalSeverity,string,Optional,RegistryEvent,,,
EventOriginalSeverity,string,Optional,UserManagement,,,
EventOriginalSeverity,string,Optional,WebSession,,,
EventOriginalSubType,string,Optional,AuditEvent,,,
Expand Down Expand Up @@ -542,7 +543,7 @@ EventProduct,string,Mandatory,Dns,Enumerated,Umbrella|Azure Firewall|DNS Server|
EventProduct,string,Mandatory,FileEvent,Enumerated,Sysmon for Linux|Sysmon|M365 Defender for Endpoint|Azure File Storage|SharePoint|OneDrive|SentinelOne,
EventProduct,string,Mandatory,NetworkSession,Enumerated,Fortigate|IOS|ISE|SDP|Vectra Stream|NSGFlow|Fireware|VPC|Azure Defender for IoT|Azure Firewall|M365 Defender for Endpoint|Sysmon|Sysmon for Linux|Windows Firewall|WireData|ZIA Firewall|CDL|PanOS|VMConnection|Meraki|Zeek|Firewall|ASA|Cynerio|SentinelOne|WAF|Firepower,
EventProduct,string,Mandatory,ProcessEvent,Enumerated,M365 Defender for Endpoint|Sysmon for Linux|Sysmon|Azure Defender for IoT|Security Events|SentinelOne,
EventProduct,string,Mandatory,RegistryEvent,Enumerated,M365 Defender for Endpoint|Security Events|Sysmon|Windows Event|SentinelOne|Carbon Black Cloud,
EventProduct,string,Mandatory,RegistryEvent,Enumerated,M365 Defender for Endpoint|Security Events|Sysmon|Windows Event|SentinelOne|Carbon Black Cloud|Vision One,
EventProduct,string,Mandatory,WebSession,Enumerated,IIS|Squid Proxy|ZIA Proxy|Vectra Stream|PanOS|CDL|Fireware|Meraki|Web Security Gateway|Zeek|Dataminr Pulse|HTTP Server|Fortigate|WAF,
EventProduct,string,Mandatory,RegistryEvent,Enumerated,M365 Defender for Endpoint|Security Events|Sysmon|Windows Event|SentinelOne,
EventProduct,string,Mandatory,WebSession,Enumerated,IIS|Squid Proxy|ZIA Proxy|Vectra Stream|PanOS|CDL|Fireware|Meraki|Web Security Gateway|Zeek|Dataminr Pulse|HTTP Server|Fortigate|WAF|NetScaler,
Expand Down Expand Up @@ -601,6 +602,7 @@ EventSchema,string,Mandatory,NetworkSession,Enumerated,NetworkSession,
EventSchema,string,Mandatory,UserManagement,Enumerated,UserManagement,
EventSchema,string,Mandatory,WebSession,Enumerated,WebSession,
EventSchema,string,Recommended,ProcessEvent,,ProcessEvent,
EventSchema,string,Mandatory,RegistryEvent,,RegistryEvent,
EventSchemaVersion,string,Mandatory,AuditEvent,SchemaVersion,,
EventSchemaVersion,string,Mandatory,Authentication,SchemaVersion,,
EventSchemaVersion,string,Mandatory,Common,SchemaVersion,,
Expand All @@ -617,6 +619,7 @@ EventSeverity,string,Mandatory,UserManagement,Enumerated,Informational|Low|Mediu
EventSeverity,string,Mandatory,WebSession,Enumerated,Informational|Low|Medium|High,
EventSeverity,string,Optional,Dns,Enumerated,Informational|Low|Medium|High,
EventSeverity,string,Optional,ProcessEvent,Enumerated,Informational|Low|Medium|High,
EventSeverity,string,Recommended,RegistryEvent,Enumerated,Informational|Low|Medium|High,
EventSeverity,string,Recommended,AuditEvent,Enumerated,Informational|Low|Medium|High,
EventSeverity,string,Recommended,Authentication,Enumerated,Informational|Low|Medium|High,
EventSeverity,string,Recommended,Dhcp,Enumerated,Informational|Low|Medium|High,
Expand Down Expand Up @@ -785,7 +788,7 @@ ParentProcessFileVersion,string,Optional,ProcessEvent,,,
ParentProcessGuid,string,Optional,ProcessEvent,,,
ParentProcessGuid,string,Optional,RegistryEvent,,,
ParentProcessId,string,Recommended,ProcessEvent,,,
ParentProcessId,string,Recommended,RegistryEvent,,,
ParentProcessId,string,Mandatory,RegistryEvent,,,
ParentProcessIMPHASH,string,Optional,ProcessEvent,,,
ParentProcessInjectedAddress,string,Optional,ProcessEvent,,,
ParentProcessIntegrityLevel,string,Optional,ProcessEvent,,,
Expand Down
4 changes: 3 additions & 1 deletion Parsers/ASimRegistryEvent/Parsers/ASimRegistry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@ Parsers:
- _ASim_RegistryEvent_MicrosoftWindowsEvent
- _ASim_RegistryEvent_SentinelOne
- _ASim_RegistryEvent_VMwareCarbonBlackCloud
- _ASim_RegistryEvent_TrendMicroVisionOne
ParserQuery: |
let DisabledParsers=materialize(_GetWatchlist('ASimDisabledParsers') | where SearchKey in ('Any', 'ExcludeASimRegistry') | extend SourceSpecificParser=column_ifexists('SourceSpecificParser','') | distinct SourceSpecificParser| where isnotempty(SourceSpecificParser));
let ASimBuiltInDisabled=toscalar('ExcludeASimRegistryEventBuiltIn' in (DisabledParsers) or 'Any' in (DisabledParsers));
jayeshprajapaticrest marked this conversation as resolved.
Show resolved Hide resolved
Expand All @@ -37,6 +38,7 @@ ParserQuery: |
ASimRegistryEventMicrosoftSysmon(disabled=(ASimBuiltInDisabled or ('ExcludeASimRegistryEventMicrosoftSysmon' in (DisabledParsers) ))),
ASimRegistryEventMicrosoftWindowsEvent(disabled=(ASimBuiltInDisabled or ('ExcludeASimRegistryEventMicrosoftWindowsEvent' in (DisabledParsers) ))),
ASimRegistryEventSentinelOne(disabled=(ASimBuiltInDisabled or ('ExcludeASimRegistryEventSentinelOne' in (DisabledParsers) ))),
ASimRegistryEventVMwareCarbonBlackCloud(disabled=(ASimBuiltInDisabled or ('ExcludeASimRegistryEventVMwareCarbonBlackCloud' in (DisabledParsers) )))
ASimRegistryEventVMwareCarbonBlackCloud(disabled=(ASimBuiltInDisabled or ('ExcludeASimRegistryEventVMwareCarbonBlackCloud' in (DisabledParsers) ))),
ASimRegistryEventTrendMicroVisionOne(disabled=(ASimBuiltInDisabled or ('ExcludeASimRegistryEventTrendMicroVisionOne' in (DisabledParsers) )))
};
parser (pack=pack)
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
Parser:
Title: Registry Event ASIM Parser for Trend Micro Vision One
Version: '0.1.0'
LastUpdated: Oct 12, 2023
Product:
Name: Trend Micro Vision One
Normalization:
Schema: RegistryEvent
Version: '0.1.2'
References:
- Title: ASIM Registry Schema
Link: https://aka.ms/ASimRegistryEventDoc
- Title: ASIM
Link: https:/aka.ms/AboutASIM
- Title: Trend Micro Vision One documentation
Link:
https://docs.trendmicro.com/en-us/enterprise/trend-vision-one/xdr-part/search-app/data-mapping-intro/data-mapping-detecti.aspx
https://automation.trendmicro.com/xdr/api-v3#tag/Observed-Attack-Techniques-Pipeline/paths/~1v3.0~1oat~1dataPipelines~1%7Bid%7D~1packages~1%7BpackageId%7D/get
https://automation.trendmicro.com/xdr/api-v3#tag/Observed-Attack-Techniques/paths/~1v3.0~1oat~1detections/get
Description: |
This ASIM parser supports normalizing Trend Micro Vision One logs to the ASIM Registry Event normalized schema. Trend Micro Vision One events are captured through Trend Vision One data connector which ingests XDR logs into Microsoft Sentinel through the Trend Vision One API.
ParserName: ASimRegistryEventTrendMicroVisionOne
EquivalentBuiltInParser: _ASim_RegistryEvent_TrendMicroVisionOne
ParserParams:
- Name: disabled
Type: bool
Default: false
ParserQuery: |
let EventTypeLookup = datatable(detail_eventSubId_s: string, EventType: string)[
"TELEMETRY_REGISTRY_CREATE", "RegistryKeyCreated",
"TELEMETRY_REGISTRY_SET", "RegistryValueSet",
"TELEMETRY_REGISTRY_DELETE", "RegistryKeyDeleted",
"TELEMETRY_REGISTRY_RENAME", "RegistryKeyRenamed"
];
let RegistryKeyPrefixLookup = datatable(
RegistryKeyPrefix: string,
RegistryKeyNormalizedPrefix: string
)[
"HKLM", "HKEY_LOCAL_MACHINE",
"HKU", "HKEY_USERS",
"HKCU", "HKEY_CURRENT_USER",
"HKCR", "HKEY_CLASSES_ROOT",
"HKCC", "HKEY_CURRENT_CONFIG"
];
let RegistryValueTypeLookup = datatable (detail_objectRegType_d: real, RegistryValueType: string)[
0, "Reg_None",
1, "Reg_Sz",
2, "Reg_Expand_Sz",
3, "Reg_Binary",
4, "Reg_DWord",
5, "Reg_DWord",
7, "Reg_Multi_Sz",
11, "Reg_QWord"
];
let EventSeverityLookup = datatable(detail_filterRiskLevel_s: string, EventSeverity: string)[
"low", "Low",
"medium", "Medium",
"high", "High",
"info", "Informational",
"critical", "High"
];
let parser = (disabled: bool=false) {
TrendMicro_XDR_OAT_CL
| where not(disabled)
| where detail_eventId_s == "TELEMETRY_REGISTRY"
| parse filters_s with * "[" filters: string "]"
| parse-kv filters as (description: string, name: string) with (pair_delimiter=",", kv_delimiter=":", quote='"')
| lookup EventTypeLookup on detail_eventSubId_s
| lookup RegistryValueTypeLookup on detail_objectRegType_d
| lookup EventSeverityLookup on detail_filterRiskLevel_s
| invoke _ASIM_ResolveDvcFQDN('detail_endpointHostName_s')
| extend RegistryKeyPrefix = tostring(split(detail_objectRegistryKeyHandle_s, @'\')[0])
| lookup RegistryKeyPrefixLookup on RegistryKeyPrefix
| extend
RegistryKey = replace_string(detail_objectRegistryKeyHandle_s, RegistryKeyPrefix, RegistryKeyNormalizedPrefix),
ActingProcessId = tostring(toint(detail_processPid_d)),
ParentProcessId = tostring(toint(detail_parentPid_d)),
ActorSessionId = tostring(toint(detail_authId_d)),
AdditionalFields = bag_pack(
"name", name,
"tags", detail_tags_s,
"objectRegType", detail_objectRegType_d
)
| extend
EventCount = int(1),
EventProduct = "Vision One",
EventVendor = "Trend Micro",
EventSchema = "RegistryEvent",
EventSchemaVersion = "0.1.2",
EventResult = "Success",
DvcAction = "Allowed"
| project-rename
ActorUsername = detail_processUser_s,
EventStartTime = detail_eventTimeDT_t,
RegistryValue = detail_objectRegistryValue_s,
RegistryValueData = detail_objectRegistryData_s,
ActingProcessName = detail_processName_s,
DvcId = detail_endpointGuid_g,
DvcOs = detail_osName_s,
DvcOsVersion = detail_osVer_s,
EventUid = _ItemId,
EventOriginalSubType = detail_eventSubId_s,
EventOriginalType = detail_eventId_s,
EventOriginalUid = detail_uuid_g,
EventOriginalSeverity = detail_filterRiskLevel_s,
EventProductVersion = detail_pver_s,
EventMessage = description
| extend
User = ActorUsername,
ActorUsernameType = iff(isnotempty(ActorUsername), "Simple", ""),
ActorUserType = _ASIM_GetUserType(ActorUsername,""),
Dvc = coalesce(DvcFQDN, DvcId, DvcHostname),
DvcIdType = iff(isnotempty(DvcId), "Other", ""),
Process = ActingProcessName,
EventEndTime = EventStartTime,
RegistryPreviousKey = RegistryKey,
RegistryPreviousValue = RegistryValue,
RegistryPreviousValueData = RegistryValueData,
RegistryPreviousValueType = RegistryValueType
| project-away
*_d,
*_s,
*_g,
*_t,
*_b,
_ResourceId,
Computer,
MG,
ManagementGroupName,
RawData,
SourceSystem,
TenantId,
name,
filters,
*Prefix
};
parser(disabled = disabled)
4 changes: 3 additions & 1 deletion Parsers/ASimRegistryEvent/Parsers/imRegistry.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,7 @@ Parsers:
- _Im_RegistryEvent_MicrosoftWindowsEvent
- _Im_RegistryEvent_SentinelOne
- _Im_RegistryEvent_VMwareCarbonBlackCloud
- _Im_RegistryEvent_TrendMicroVisionOne
ParserQuery: |
let DisabledParsers=materialize(_GetWatchlist('ASimDisabledParsers') | where SearchKey in ('Any', 'ExcludevimRegistry') | extend SourceSpecificParser=column_ifexists('SourceSpecificParser','') | distinct SourceSpecificParser | where isnotempty(SourceSpecificParser));
let vimBuiltInDisabled=toscalar('ExcludevimRegistryEventBuiltIn' in (DisabledParsers) or 'Any' in (DisabledParsers));
Expand All @@ -75,6 +76,7 @@ ParserQuery: |
vimRegistryEventMicrosoftSysmon(starttime = starttime, endtime = endtime, eventtype_in = eventtype_in, actorusername_has_any = actorusername_has_any, registrykey_has_any = registrykey_has_any, registryvalue_has_any = registryvalue_has_any, registrydata_has_any = registrydata_has_any, dvchostname_has_any= dvchostname_has_any, disabled=(vimBuiltInDisabled or('ExcludevimRegistryEventMicrosoftSysmon' in (DisabledParsers) ))),
vimRegistryEventMicrosoftWindowsEvent (starttime = starttime, endtime = endtime, eventtype_in = eventtype_in, actorusername_has_any = actorusername_has_any, registrykey_has_any = registrykey_has_any, registryvalue_has_any = registryvalue_has_any, registrydata_has_any = registrydata_has_any, dvchostname_has_any= dvchostname_has_any, disabled=(vimBuiltInDisabled or('ExcludevimRegistryEventMicrosoftWindowsEvent' in (DisabledParsers) ))),
vimRegistryEventSentinelOne (starttime = starttime, endtime = endtime, eventtype_in = eventtype_in, actorusername_has_any = actorusername_has_any, registrykey_has_any = registrykey_has_any, registryvalue_has_any = registryvalue_has_any, registrydata_has_any = registrydata_has_any, dvchostname_has_any= dvchostname_has_any, disabled=(vimBuiltInDisabled or('ExcludevimRegistryEventSentinelOne' in (DisabledParsers) ))),
vimRegistryEventVMwareCarbonBlackCloud(starttime = starttime, endtime = endtime, eventtype_in = eventtype_in, actorusername_has_any = actorusername_has_any, registrykey_has_any = registrykey_has_any, registryvalue_has_any = registryvalue_has_any, registryvaluedata_has_any = registrydata_has_any, dvchostname_has_any= dvchostname_has_any, disabled=(vimBuiltInDisabled or('ExcludevimRegistryEventVMwareCarbonBlackCloud' in (DisabledParsers) )))
vimRegistryEventVMwareCarbonBlackCloud(starttime = starttime, endtime = endtime, eventtype_in = eventtype_in, actorusername_has_any = actorusername_has_any, registrykey_has_any = registrykey_has_any, registryvalue_has_any = registryvalue_has_any, registryvaluedata_has_any = registrydata_has_any, dvchostname_has_any= dvchostname_has_any, disabled=(vimBuiltInDisabled or('ExcludevimRegistryEventVMwareCarbonBlackCloud' in (DisabledParsers) ))),
vimRegistryEventTrendMicroVisionOne (starttime=starttime, endtime=endtime, eventtype_in=eventtype_in, actorusername_has_any=actorusername_has_any, registrykey_has_any=registrykey_has_any, registryvalue_has_any=registryvalue_has_any, registryvaluedata_has_any=registrydata_has_any, dvchostname_has_any=dvchostname_has_any, disabled= (vimBuiltInDisabled or('ExcludevimRegistryEventTrendMicroVisionOne' in (DisabledParsers) )))
};
parser(starttime = starttime, endtime = endtime, eventtype_in = eventtype_in, actorusername_has_any = actorusername_has_any, registrykey_has_any = registrykey_has_any, registryvalue_has_any = registryvalue_has_any, registrydata_has_any = registrydata_has_any, dvchostname_has_any= dvchostname_has_any, pack=pack)
Loading