Azure Sentinel Entity Population Delay

[busyb0x]

Description:
The Azure Sentinel incidents have 6-10 minutes delay in populating/displaying related entities once the incident is created in the sentinel portal. As a result the logic app that depends on the incident entities is failing since the entity list in the json array is empty.

The automation rule does not have any option to create a delay, I tried running a different playbook with delay before the actual playbook/logic app in the automation rule, with no luck as playbook/logic app will not wait for the playbook/logic app 1 before witha delay.

NOTE: This is true for incidents from MDO alerts using a Data Connector for MDO.

To Reproduce
Steps to reproduce the behavior:

1. Setup a MDO connector, and ingest alerts within sentinel.
2. Create an automation rule to run a playbook either with alert or incident as a trigger to run.
3. Logic App will fail with a empty entity list.
4. Once an incident is triggered in sentinel check the entities. Wait 6 mins for entities to update.
5. Run the logic app manually using actions in the incident information panel on the right.
6. Logic app will now run successfully.

Expected behavior
Entities should appear without a delay when MDO alerts are ingested.

Screenshots
If applicable, add screenshots to help explain your problem.

[github-actions]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[busyb0x]

Hello! Can someone look into this?

[github-actions]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[BenjiSec]

Thank you for the feedback, the product group are aware of this issue and are looking into how it can be resolved, however at this time we don’t have a workaround.
Adding @lior-tamir for visibility.

[github-actions]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[glarrick757]

I am encountering this issue in the 2023-05-01-preview version of the API when attempting to get a Security Alert Entity via the Get Entities endpoint. I experience the endpoint returning a 404 for up to 3 minutes following incident creation.

[sv3nb]

I am experiencing a similar issue where logic apps triggered by Sentinel Incidents (coming from Defender for Cloud) are missing the entities due to a delay in the population process, causing the logic app to fail in a later stage. Is there currently any work-around to delay the Logic app kick-off with a few minutes?

[mlaraibkhan]

When using logic apps, I've encountered similar issue where parsing entities from the incident trigger, while the entities are delayed when a new incident is created. This delay causes the entire logic to fail. I attempted to use a delay action in logic, but the incident trigger is set first, so any subsequent delay doesn't work when parsing the entities as the JSON output of the incident is already set without any entities at the trigger point. Any hope?

[BenWoodcockKroll]

Hi We are experiencing the same issues with the entities not showing all the details for a good 30 mins which breaks our Incident escalation. Is there any workaround yet as this looks to be an issue for sometime?!

[Kaloszer]

We're seeing a similar issue at times

[ol1v]

This has not been resolved yet. Experiencing the same issues with both entities and Custom Alert details. Please revisit this!

[cmartinez1045]

This is still an issue, but not even just for logic apps, it impacts pulling the entities via the API as well.

[Kaloszer]

Still an issue, having an enrichment work 80% of the time (due to this issue) is really not optimal. There needs to be a way to delay automation rule execution prior to entity population, or ensure that the incident is only available in sentinel with entities already bound.

@sarah-yo - please re-open as this is still an issue 2 years later.

[Dave365CH]

We're expriencing the same issue.. is there no option to delay the the automation rule?

[cmartinez1045]

This pertains to pulling incidents via the API and the availability of entities. This is the same issue as if you are running automations from a logic app stand point as well. Extremely bad security posture.

I have tried to escalate these tickets multiple times, and open it with multiple different subscriptions and keep going in circles and have been working on trying to get this resolved for well over a year at this point........

Most recent response from MS support:

"Entities are part of the SecurityAlert table. When an incident is synced from XDR or created using Analytic Rule, the corresponding alert does not sync simultaneously. Unlike Sentinel, it generates incidents without completing the alert, and the entities are gradually populated in the alert. However, the alert is not pushed into Sentinel until the processing end time is generated. This results in a delay for the entities to appear in the incidents.

To verify this, you can take a sample incident and compare the incident creation time, alert creation time, and the processing end time of the alert. You will notice a few minutes' difference.

Currently, there is no absolute resolution to this issue. However, if you are using any automation with RESTAPI, we recommend adding a few minutes of delay before triggering the RESTAPI as a workaround."

My response:

"The only way we could make this work would be to pull only alerts that are from ~12+ minutes in the past? (please note that 12 minutes was the maximum seen in testing, it could very well be higher).

So I would run the API call at 7:30am for alerts that were generated at ~7:18 at a minimum of a delay to make up for the delay in Entities being available which would in turn have alerts be pulled with the Ingestion Time + Query Time + Entity Availability Time making it 20+ minutes after potentially malicious activity has occurred to have the entities available?

From an automation perspective that would mean if we wanted to perform an action such as isolating a host, blocking an active attacker, disabling a user, etc., we would not be able to perform that action after the Ingestion Time + Query Time + Entity Availability Time making it well over 20+ minutes AFTER malicious activity has been performed to run a potentially crucial automation??"

How does MS think that is acceptable from a security perspective?"

[Dave365CH]

@cmartinez1045 totally agree... did you implement a delay or any other workaround for that?

[cmartinez1045]

I have not, currently we are just pulling the incidents as they come in and essentially forgoing any sort of automation currently. I don’t want to pull an incident or have an automation run 20+ minutes after the fact. It means a lot more manual work; however it ensures a more rapid response. Really wish MS would fix this

[Kaloszer]

@cmartinez1045 it's not, they simply don't care

[cmartinez1045]

Lol if this doesn't get resolved to reduce this time, people need to run FAR away from Sentinel.....