GoogleWorkspaceReports Data Connector - API start time error

[s-khan14]

Description
The recent update to GoogleWorkspaceReport Data Connector has caused function app to fail. The python script uses a start_time and end_time to fetch logs for GoogleWorkspaceAPI. Start_date is based on past_time, that depends on Function App's state. The function app state is returning unexpected date on first run.

To Reproduce
Steps to reproduce the behavior:

1. Deploy functionApp for GoogleWorkspaceReports Data Connector from Microsoft Sentinel.
2. After function deployment, open functionApp.
3. Click on function Name GWorkspaceReportsAPISentinelConnector
4. Click Run/Test
5. View Error in Monitor tab.
6. Unexpected: The start date can be 1-2 years in the past.

Expected behavior
If Function App state returns a date more that 24h in the past, use the default start_time parameter (which is 1 day as of now).

Error Message
Something wrong while getting the results. Exception error text: <HttpError 400 when requesting hxxps://admin.googleapis.com/admin/reports/v1/activity/users/all/applications/user_accounts?maxResults=1000&startTime=2022-11-21T20%3A00%3A00.000Z&endTime=2022-11-21T20%3A15%3A00.000000Z&alt=json returned "Invalid request: Start time and end time should be within last 180 days. Start time should be less than end time.". Details: "[{'message': 'Invalid request: Start time and end time should be within last 180 days. Start time should be less than end time.', 'domain': 'global', 'reason': 'invalid'}]">

Additional context

• Location of FunctionApp code: Solutions/GoogleWorkspaceReports/Data Connectors/GWorkspaceReportsAPISentinelConnector/init.py
• Currently the Function App uses StateManager to get the past_time: Line 121 on init.py
• Suggestion: If the date returned by StateManager is more than 24h, use default start_time and at the end of execution, set the time_stamp using state, which will be used for next run. line 124

[github-actions]

Thank you for submitting an Issue to the Azure Sentinel GitHub repo! You should expect an initial response to your Issue from the team within 5 business days. Note that this response may be delayed during holiday periods. For urgent, production-affecting issues please raise a support ticket via the Azure Portal.

[v-muuppugund]

Hi @s-khan14 , Thanks for flagging this issue, we will investigate this issue and get back to you with some updates by 19Dec2023. Thanks!

[v-muuppugund]

Hi @s-khan14 ,The start time and end time both are same ,so there is an issue,please check the followiing the statemarkerfile share in azure containers and it has start time based on last run
Please follow the steps below
1.Browse to function app application settings and go to storage accounts->Please go to the storage accounts-->Containers->FileShare-->Click on -->funcstatemarkershare->Browse-->Click on ... at the end->Select Edit,Please find below screen shot for reference

2.Please check the delays configured in function app settings as the end time is based on that as there are no default delays configured in code

Please let me know if you have any isuses,if can have a call and do further troubleshooting

[s-khan14]

This is the content of funcstatemarkershare.

{"access_transparency": "2023-03-31T10:20:00.000Z", "admin": "2023-12-07T15:34:33.317Z", "calendar": "2023-12-07T10:49:45.310Z", "chat": "2023-12-06T12:20:24.605Z", "drive": "2023-12-06T18:37:39.076Z", "gcp": "2023-03-31T10:20:00.000Z", "gplus": "2023-03-31T10:20:00.000Z", "groups": "2023-12-07T13:33:19.547Z", "groups_enterprise": "2023-12-07T13:33:18.874Z", "jamboard": "2023-03-31T10:20:00.000Z", "login": "2023-12-07T10:58:48.470Z", "meet": "2023-12-07T16:39:50.821Z", "mobile": "2023-03-31T10:20:00.000Z", "rules": "2023-03-31T10:20:00.000Z", "saml": "2023-03-31T10:20:00.000Z", "token": "2023-12-07T16:51:59.220Z", "user_accounts": "2023-12-19T17:10:00.000Z", "context_aware_access": "2023-03-31T10:20:00.000Z", "chrome": "2023-12-07T14:53:47.593Z", "data_studio": "2023-03-31T10:20:00.000Z"}

Still the function is grabbing timestamps other than this.

Suggestion
Add a check statement to validate timestamps grabbed from state manager. if timestamp do not align or are more than 1d (or normal period) in the past, replace them with default look_back value.

Why:

1. In case of current situation, where statemanager is returning dates from 2022.
2. What if function didn't run for a long time. The timestamps in state will be expired/not valid for google API.

Note: I was able to solve the problem by doing a manual deployment via VS code - Not from ARM template

[v-muuppugund]

Hi @s-khan14 , Sorry to hear this issue, got the issue, but want to have more details, so can proceed with changes,

Want to have these data points based on that will do the changes,Could you please share couple of time slots and email id,so will block the calendar.

[v-muuppugund]

@s-khan14 Gentle Reminder,got the issue, but want to have more details, so can proceed with changes,
Want to have these data points based on that will do the changes, Could you please share couple of time slots and email id, so will block the calendar.

[s-khan14]

@s-khan14 Gentle Reminder,got the issue, but want to have more details, so can proceed with changes, Want to have these data points based on that will do the changes, Could you please share couple of time slots and email id, so will block the calendar.

Hi @v-muuppugund, sorry for the delayed response. you can contact me at this email and then we can arrange a meeting from there.

[v-muuppugund]

Hi @s-khan14, Apologies for the delayed response,Will schedule a teams meeting, please let me know if this time isn't convenient

[v-muuppugund]

Hi @s-khan14 ,I waited for some time in the call today,as you are unable to join will work on changes and share package,please let me know if you have any concern

[s-khan14]

Did you add the right email address to the invite? I never got the teams invite. Checked both spam and inbox.

[v-muuppugund]

Yes @s-khan14 , I am unable to open this email and asked my collogue to open it and got the email as i.e. [maced73@yahoo.com] (mailto:maced73@yahoo.com), Could you please drop an email to (v-muuppugund@microsoft.com) .

[v-muuppugund]

Hi @s-khan14 ,Gentle Reminder,, I am unable to open this email and asked my collogue to open it and got the email as i.e. [maced73@yahoo.com] (mailto:maced73@yahoo.com), Could you please drop an email to (v-muuppugund@microsoft.com) .

[v-muuppugund]

@s-khan14 , Since we have not received a response in the last 5 days, we are closing your issue (#9617) as per our standard operating procedures. If you still need support for this issue, feel free to re-open at any time. Thank you for your co-operation!