Skip to content

Commit

Permalink
Merge pull request #9001 from Azure/v-rusraut/ExtraHopReveal(x)-MMAto…
Browse files Browse the repository at this point in the history
…AMAMigration

Repackaging - ExtraHop Reveal(x) (MMA to AMA Migration)
  • Loading branch information
v-atulyadav authored Sep 15, 2023
2 parents fc0fca4 + 2c6bede commit 03146b4
Show file tree
Hide file tree
Showing 9 changed files with 622 additions and 84 deletions.
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"id": "ExtraHopNetworks",
"title": "ExtraHop Reveal(x)",
"title": "[Deprecated] ExtraHop Reveal(x) via Legacy Agent",
"publisher": "ExtraHop Networks",
"logo": "ExtraHopLogo.svg",
"descriptionMarkdown": "The ExtraHop Reveal(x) data connector enables you to easily connect your Reveal(x) system with Microsoft Sentinel to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
{
"id": "ExtraHopNetworksAma",
"title": "[Recommended] ExtraHop Reveal(x) via AMA",
"publisher": "ExtraHop Networks",
"logo": "ExtraHopLogo.svg",
"descriptionMarkdown": "The ExtraHop Reveal(x) data connector enables you to easily connect your Reveal(x) system with Microsoft Sentinel to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.",
"graphQueries": [
{
"metricName": "Total data received",
"legend": "ExtraHop Networks",
"baseQuery": "CommonSecurityLog\n |where DeviceVendor =~ 'ExtraHop'\n |extend sent_by_ama = column_ifexists('CollectorHostName','')\n |where isnotempty(sent_by_ama)"
}
],
"sampleQueries": [
{
"description": "All logs",
"query": "\nCommonSecurityLog\n| where DeviceVendor == \"ExtraHop\"\n\n | sort by TimeGenerated"
},
{
"description": "All detections, de-duplicated",
"query": "\nCommonSecurityLog\n| where DeviceVendor == \"ExtraHop\"\n\n | extend categories = iif(DeviceCustomString2 != \"\", split(DeviceCustomString2, \",\"),dynamic(null))\n     | extend StartTime = extract(\"start=([0-9-]+T[0-9:.]+Z)\", 1, AdditionalExtensions,typeof(datetime))\n     | extend EndTime = extract(\"end=([0-9-]+T[0-9:.]+Z)\", 1, AdditionalExtensions,typeof(datetime))\n     | project      \n     DeviceEventClassID=\"ExtraHop Detection\",\n     Title=Activity,\n     Description=Message,\n     riskScore=DeviceCustomNumber2,     \n     SourceIP,\n     DestinationIP,\n     detectionID=tostring(DeviceCustomNumber1),\n     updateTime=todatetime(ReceiptTime),\n     StartTime,\n     EndTime,\n     detectionURI=DeviceCustomString1,\n     categories,\n     Computer\n     | summarize arg_max(updateTime, *) by detectionID\n     | sort by detectionID desc"
}
],
"connectivityCriterias": [
{
"type": "IsConnectedQuery",
"value": [
"CommonSecurityLog\n |where DeviceVendor =~ 'ExtraHop'\n |extend sent_by_ama = column_ifexists('CollectorHostName','')\n |where isnotempty(sent_by_ama)\n | summarize LastLogReceived = max(TimeGenerated)\n | project IsConnected = LastLogReceived > ago(30d)"
]
}
],
"dataTypes": [
{
"name": "CommonSecurityLog (‘ExtraHop’)",
"lastDataReceivedQuery": "CommonSecurityLog\n |where DeviceVendor =~ 'ExtraHop'\n |extend sent_by_ama = column_ifexists('CollectorHostName','')\n |where isnotempty(sent_by_ama)\n | summarize Time = max(TimeGenerated)\n | where isnotempty(Time)"
}
],
"availability": {
"status": 1,
"isPreview": false
},
"permissions": {
"resourceProvider": [
{
"provider": "Microsoft.OperationalInsights/workspaces",
"permissionsDisplayText": "read and write permissions.",
"providerDisplayName": "Workspace",
"scope": "Workspace",
"requiredPermissions": {
"read": true,
"write": true,
"delete": true
}
},
{
"provider": "Microsoft.OperationalInsights/workspaces/sharedKeys",
"permissionsDisplayText": "read permissions to shared keys for the workspace. [See the documentation to learn more about workspace keys](https://docs.microsoft.com/azure/azure-monitor/platform/agent-windows#obtain-workspace-id-and-key).",
"providerDisplayName": "Keys",
"scope": "Workspace",
"requiredPermissions": {
"action": true
}
}
],
"customs": [
{
"description": "To collect data from non-Azure VMs, they must have Azure Arc installed and enabled. [Learn more](https://docs.microsoft.com/azure/azure-monitor/agents/azure-monitor-agent-install?tabs=ARMAgentPowerShell,PowerShellWindows,PowerShellWindowsArc,CLIWindows,CLIWindowsArc)"
},
{
"description": "Common Event Format (CEF) via AMA and Syslog via AMA data connectors must be installed [Learn more](https://learn.microsoft.com/azure/sentinel/connect-cef-ama#open-the-connector-page-and-create-the-dcr)"
}
]
},
"instructionSteps": [
{
"title": "",
"description": "",
"instructions": [
{
"parameters": {
"title": "1. Kindly follow the steps to configure the data connector",
"instructionSteps": [
{
"title": "Step A. Configure the Common Event Format (CEF) via AMA data connector",
"description": "_Note:- CEF logs are collected only from Linux Agents_\n\n1. Navigate to Microsoft Sentinel workspace ---> configuration ---> Data connector blade .\n\n2. Search for 'Common Event Format (CEF) via AMA' data connector and open it.\n\n3. Check If there is no existing DCR configured to collect required facility of logs, Create a new DCR (Data Collection Rule)\n\n\t_Note:- It is recommended to install minimum 1.27 version of AMA agent [Learn more](https://learn.microsoft.com/azure/azure-monitor/agents/azure-monitor-agent-manage?tabs=azure-portal ) and ensure there is no duplicate DCR as it can cause log duplicacy_\n\n4. Run the command provided in the CEF via AMA data connector page to configure the CEF collector on the machine",
"instructions": [
]
},
{
"title": "Step B. Forward ExtraHop Networks logs to Syslog agent",
"description": "1. Set your security solution to send Syslog messages in CEF format to the proxy machine. Make sure to send the logs to port 514 TCP on the machine IP address.\n2. Follow the directions to install the [ExtraHop Detection SIEM Connector bundle](https://aka.ms/asi-syslog-extrahop-forwarding) on your Reveal(x) system. The SIEM Connector is required for this integration.\n3. Enable the trigger for **ExtraHop Detection SIEM Connector - CEF**\n4. Update the trigger with the ODS syslog targets you created \n5. The Reveal(x) system formats syslog messages in Common Event Format (CEF) and then sends data to Microsoft Sentinel.",
"instructions": [
]
},
{
"title": "Step C. Validate connection",
"description": "Follow the instructions to validate your connectivity:\n\nOpen Log Analytics to check if the logs are received using the CommonSecurityLog schema.\n\nIt may take about 20 minutes until the connection streams data to your workspace.\n\nIf the logs are not received, run the following connectivity validation script:\n\n 1. Make sure that you have Python on your machine using the following command: python -version\n\n2. You must have elevated permissions (sudo) on your machine",
"instructions": [
{
"parameters": {
"label": "Run the following command to validate your connectivity:",
"value": "sudo wget -O Sentinel_AMA_troubleshoot.py https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/DataConnectors/Syslog/Sentinel_AMA_troubleshoot.py&&sudo python Sentinel_AMA_troubleshoot.py --cef"
},
"type": "CopyableLabel"
}
]
}
]
},
"type": "InstructionStepsGroup"
}
]
},

{
"title": "2. Secure your machine ",
"description": "Make sure to configure the machine's security according to your organization's security policy\n\n\n[Learn more >](https://aka.ms/SecureCEF)"
}
],
"metadata": {
"id": "3c3af19d-b178-44cd-a130-c48b366758c4",
"version": "1.0.0",
"kind": "dataConnector",
"source": {
"kind": "community"
},
"author": {
"name": "ExtraHop Networks"
},
"support": {
"name": "ExtraHop Networks",
"link": "https://www.extrahop.com/support/",
"tier": "developer"
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,17 @@
"Name": "ExtraHop Reveal(x)",
"Author": "ExtraHop",
"Logo": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/ExtraHopLogo.svg\" width=\"75px\" height=\"75px\">",
"Description": "The [ExtraHop Reveal(x)](https://www.extrahop.com/products/security/) Solution for Microsoft Sentinel enables ingestion of Common Event Format (CEF) logs into Microsoft Sentinel. This solution enables you to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities./n **Underlying Microsoft Technologies used:**\n\nThis solution takes a dependency on the following technologies, and some of these dependencies either may be in [Preview](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) state or might result in additional ingestion or operational costs:/na. [Agent-based log collection (CEF over Syslog) ](https://docs.microsoft.com/azure/sentinel/connect-common-event-format)",
"Description": "The [ExtraHop Reveal(x)](https://www.extrahop.com/products/security/) Solution for Microsoft Sentinel enables ingestion of Common Event Format (CEF) logs into Microsoft Sentinel. This solution enables you to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.\n\r\n1. **ExtraHop Reveal(x) via AMA** - This data connector helps in ingesting ExtraHop Reveal(x) logs into your Log Analytics Workspace using the new Azure Monitor Agent. Learn more about ingesting using the new Azure Monitor Agent [here](https://learn.microsoft.com/azure/sentinel/connect-cef-ama). **Microsoft recommends using this Data Connector**.\n\r\n2. **ExtraHop Reveal(x) via Legacy Agent** - This data connector helps in ingesting ExtraHop Reveal(x) logs into your Log Analytics Workspace using the legacy Log Analytics agent.\n\n**NOTE:** Microsoft recommends installation of ExtraHop Reveal(x) via AMA Connector. Legacy connector uses the Log Analytics agent which is about to be deprecated by **Aug 31, 2024,** and thus should only be installed where AMA is not supported. Using MMA and AMA on same machine can cause log duplication and extra ingestion cost [more details](https://learn.microsoft.com/en-us/azure/sentinel/ama-migrate).",
"Data Connectors": [
"Data Connectors/template_ExtraHopNetworks.json"
"Data Connectors/template_ExtraHopNetworks.json",
"Data Connectors/template_ExtraHopReveal(x)AMA.json"
],
"Workbooks": [
"Workbooks/ExtraHopDetectionSummary.json"
],
"Metadata": "SolutionMetadata.json",
"BasePath": "C:\\Sentinel-Repos\\19.05.22\\Azure-Sentinel\\Solutions\\ExtraHop Reveal(x)",
"Version": "2.0.1",
"Version": "3.0.0",
"TemplateSpec": true,
"Is1PConnector": false
}
30 changes: 30 additions & 0 deletions Solutions/ExtraHop Reveal(x)/Data/system_generated_metadata.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
{
"Name": "ExtraHop Reveal(x)",
"Author": "ExtraHop",
"Logo": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/ExtraHopLogo.svg\" width=\"75px\" height=\"75px\">",
"Description": "The [ExtraHop Reveal(x)](https://www.extrahop.com/products/security/) Solution for Microsoft Sentinel enables ingestion of Common Event Format (CEF) logs into Microsoft Sentinel. This solution enables you to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.\n\r\n1. **ExtraHop Reveal(x) via AMA** - This data connector helps in ingesting ExtraHop Reveal(x) logs into your Log Analytics Workspace using the new Azure Monitor Agent. Learn more about ingesting using the new Azure Monitor Agent [here](https://learn.microsoft.com/azure/sentinel/connect-cef-ama). **Microsoft recommends using this Data Connector**.\n\r\n2. **ExtraHop Reveal(x) via Legacy Agent** - This data connector helps in ingesting ExtraHop Reveal(x) logs into your Log Analytics Workspace using the legacy Log Analytics agent.\n\n**NOTE:** Microsoft recommends installation of ExtraHop Reveal(x) via AMA Connector. Legacy connector uses the Log Analytics agent which is about to be deprecated by **Aug 31, 2024,** and thus should only be installed where AMA is not supported. Using MMA and AMA on same machine can cause log duplication and extra ingestion cost [more details](https://learn.microsoft.com/en-us/azure/sentinel/ama-migrate).",
"Metadata": "SolutionMetadata.json",
"BasePath": "C:\\Sentinel-Repos\\19.05.22\\Azure-Sentinel\\Solutions\\ExtraHop Reveal(x)",
"Version": "3.0.0",
"TemplateSpec": true,
"Is1PConnector": false,
"publisherId": "extrahop",
"offerId": "extrahop_revealx_mss",
"providers": [
"ExtraHop"
],
"categories": {
"domains": [
"Security - Threat Protection",
"Security - Cloud Security"
]
},
"firstPublishDate": "2022-05-19",
"support": {
"name": "ExtraHop",
"tier": "partner",
"link": "https://www.extrahop.com/support/"
},
"Data Connectors": "[\n \"Data Connectors/template_ExtraHopNetworks.json\",\n \"Data Connectors/template_ExtraHopReveal(x)AMA.json\"\n]",
"Workbooks": "[\n \"Workbooks/ExtraHopDetectionSummary.json\"\n]"
}
Binary file added Solutions/ExtraHop Reveal(x)/Package/3.0.0.zip
Binary file not shown.
40 changes: 15 additions & 25 deletions Solutions/ExtraHop Reveal(x)/Package/createUiDefinition.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
"config": {
"isWizard": false,
"basics": {
"description": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/ExtraHopLogo.svg\" width=\"75px\" height=\"75px\">\n\n**Note:** _There may be [known issues](https://aka.ms/sentinelsolutionsknownissues) pertaining to this Solution, please refer to them before installing._\n\nThe [ExtraHop Reveal(x)](https://www.extrahop.com/products/security/) Solution for Microsoft Sentinel enables ingestion of Common Event Format (CEF) logs into Microsoft Sentinel. This solution enables you to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.\r\n \r\n **Underlying Microsoft Technologies used:**\n\nThis solution takes a dependency on the following technologies, and some of these dependencies either may be in [Preview](https://azure.microsoft.com/support/legal/preview-supplemental-terms/) state or might result in additional ingestion or operational costs: \r\n \r\n a. [Agent-based log collection (CEF over Syslog) ](https://docs.microsoft.com/azure/sentinel/connect-common-event-format)\n\n**Data Connectors:** 1, **Workbooks:** 1\n\n[Learn more about Microsoft Sentinel](https://aka.ms/azuresentinel) | [Learn more about Solutions](https://aka.ms/azuresentinelsolutionsdoc)",
"description": "<img src=\"https://raw.githubusercontent.com/Azure/Azure-Sentinel/master/Logos/ExtraHopLogo.svg\" width=\"75px\" height=\"75px\">\n\n**Note:** _There may be [known issues](https://aka.ms/sentinelsolutionsknownissues) pertaining to this Solution, please refer to them before installing._\n\nThe [ExtraHop Reveal(x)](https://www.extrahop.com/products/security/) Solution for Microsoft Sentinel enables ingestion of Common Event Format (CEF) logs into Microsoft Sentinel. This solution enables you to view dashboards, create custom alerts, and improve investigation. This integration gives you the ability to gain insight into your organization's network and improve your security operation capabilities.\n\r\n1. **ExtraHop Reveal(x) via AMA** - This data connector helps in ingesting ExtraHop Reveal(x) logs into your Log Analytics Workspace using the new Azure Monitor Agent. Learn more about ingesting using the new Azure Monitor Agent [here](https://learn.microsoft.com/azure/sentinel/connect-cef-ama). **Microsoft recommends using this Data Connector**.\n\r\n2. **ExtraHop Reveal(x) via Legacy Agent** - This data connector helps in ingesting ExtraHop Reveal(x) logs into your Log Analytics Workspace using the legacy Log Analytics agent.\n\n**NOTE:** Microsoft recommends installation of ExtraHop Reveal(x) via AMA Connector. Legacy connector uses the Log Analytics agent which is about to be deprecated by **Aug 31, 2024,** and thus should only be installed where AMA is not supported. Using MMA and AMA on same machine can cause log duplication and extra ingestion cost [more details](https://learn.microsoft.com/en-us/azure/sentinel/ama-migrate).\n\n**Workbooks:** 1\n\n[Learn more about Microsoft Sentinel](https://aka.ms/azuresentinel) | [Learn more about Solutions](https://aka.ms/azuresentinelsolutionsdoc)",
"subscription": {
"resourceProviders": [
"Microsoft.OperationsManagement/solutions",
Expand Down Expand Up @@ -51,30 +51,6 @@
}
],
"steps": [
{
"name": "dataconnectors",
"label": "Data Connectors",
"bladeTitle": "Data Connectors",
"elements": [
{
"name": "dataconnectors1-text",
"type": "Microsoft.Common.TextBlock",
"options": {
"text": "This solution installs the data connector for ingesting ExtraHop Reveal Events in the CEF format into Microsoft Sentinel. After installing the solution, configure and enable this data connector by following guidance in Manage solution view."
}
},
{
"name": "dataconnectors-link2",
"type": "Microsoft.Common.TextBlock",
"options": {
"link": {
"label": "Learn more about connecting data sources",
"uri": "https://docs.microsoft.com/azure/sentinel/connect-data-sources"
}
}
}
]
},
{
"name": "workbooks",
"label": "Workbooks",
Expand All @@ -100,6 +76,20 @@
"uri": "https://docs.microsoft.com/azure/sentinel/tutorial-monitor-your-data"
}
}
},
{
"name": "workbook1",
"type": "Microsoft.Common.Section",
"label": "ExtraHop",
"elements": [
{
"name": "workbook1-text",
"type": "Microsoft.Common.TextBlock",
"options": {
"text": "Gain insights into ExtraHop Reveal(x) detections by analyzing traffic and activities.\nThis workbook provides an overview of security detections in your organization's network, including high-risk detections and top participants."
}
}
]
}
]
}
Expand Down
Loading

0 comments on commit 03146b4

Please sign in to comment.