Azure Data Connector cli doesn't work

[zoxendine]

Describe the bug

the azure data connector create does not create the desired data connection

Related command

az sentinel data-connector create -n AzureActivity -g rg -w workspace

Errors

The command failed with an unexpected error. Here is the traceback:
"Model 'AAZObjectType' has no field named 'kind'"
Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 697, in _run_job
result = cmd_copy(params)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_command.py", line 154, in call
return self._handler(*args, **kwargs)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 31, in _handler
self._execute_operations()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1105, in _execute_operations
self.DataConnectorsCreateOrUpdate(ctx=self.ctx)()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1115, in call
request = self.make_request()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_operation.py", line 318, in make_request
self.content, self.form_content, self.stream_content)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1209, in content
_builder.discriminate_by("kind", "APIPolling")
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_content_builder.py", line 159, in discriminate_by
schema.discriminate_by(prop_name, prop_value)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_field_type.py", line 243, in discriminate_by
raise AAZUnknownFieldError(self, key)
azure.cli.core.aaz.exceptions.AAZUnknownFieldError: "Model 'AAZObjectType' has no field named 'kind'"

Issue script & Debug output

msal.application: Broker enabled? False
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 697, in _run_job
result = cmd_copy(params)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_command.py", line 154, in call
return self._handler(*args, **kwargs)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 31, in _handler
self._execute_operations()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1105, in _execute_operations
self.DataConnectorsCreateOrUpdate(ctx=self.ctx)()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1115, in call
request = self.make_request()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_operation.py", line 318, in make_request
self.content, self.form_content, self.stream_content)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1209, in content
_builder.discriminate_by("kind", "APIPolling")
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_content_builder.py", line 159, in discriminate_by
schema.discriminate_by(prop_name, prop_value)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_field_type.py", line 243, in discriminate_by
raise AAZUnknownFieldError(self, key)
azure.cli.core.aaz.exceptions.AAZUnknownFieldError: "Model 'AAZObjectType' has no field named 'kind'"

cli.azure.cli.core.azclierror: The command failed with an unexpected error. Here is the traceback:
az_command_data_logger: The command failed with an unexpected error. Here is the traceback:
cli.azure.cli.core.azclierror: "Model 'AAZObjectType' has no field named 'kind'"
Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 697, in _run_job
result = cmd_copy(params)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_command.py", line 154, in call
return self._handler(*args, **kwargs)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 31, in _handler
self._execute_operations()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1105, in _execute_operations
self.DataConnectorsCreateOrUpdate(ctx=self.ctx)()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1115, in call
request = self.make_request()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_operation.py", line 318, in make_request
self.content, self.form_content, self.stream_content)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1209, in content
_builder.discriminate_by("kind", "APIPolling")
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_content_builder.py", line 159, in discriminate_by
schema.discriminate_by(prop_name, prop_value)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_field_type.py", line 243, in discriminate_by
raise AAZUnknownFieldError(self, key)
azure.cli.core.aaz.exceptions.AAZUnknownFieldError: "Model 'AAZObjectType' has no field named 'kind'"
az_command_data_logger: "Model 'AAZObjectType' has no field named 'kind'"
Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 697, in _run_job
result = cmd_copy(params)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_command.py", line 154, in call
return self._handler(*args, **kwargs)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 31, in _handler
self._execute_operations()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1105, in _execute_operations
self.DataConnectorsCreateOrUpdate(ctx=self.ctx)()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1115, in call
request = self.make_request()
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_operation.py", line 318, in make_request
self.content, self.form_content, self.stream_content)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1209, in content
_builder.discriminate_by("kind", "APIPolling")
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_content_builder.py", line 159, in discriminate_by
schema.discriminate_by(prop_name, prop_value)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_field_type.py", line 243, in discriminate_by
raise AAZUnknownFieldError(self, key)
azure.cli.core.aaz.exceptions.AAZUnknownFieldError: "Model 'AAZObjectType' has no field named 'kind'"
To check existing issues, please visit: https://github.com/Azure/azure-cli/issues
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7fb2bc4ea790>]
az_command_data_logger: exit code: 1
cli.main: Command ran in 2.296 seconds (init: 1.040, invoke: 1.257)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 7412 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/usr/bin/python3.9 /usr/lib64/az/lib/python3.9/site-packages/azure/cli/telemetry/init.py /home/vagrant/.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.

Expected behavior

data connection work

Environment Summary

azure-cli 2.51

Additional context

No response

[yonzhan]

Thank you for opening this issue, we will look into it.

[microsoft-github-policy-service]

Thank you for your feedback. This has been routed to the support team for assistance.

[jsntcy]

@necusjz, please help take a look.

[navba-MSFT]

@zoxendine Thanks for reaching out to us and reporting this issue. While running the az sentinel data-connector create CLI command please pass the --azure-active-directory parameter as shown below and check if that helps.

More info here.

[zoxendine]

@navba-MSFT Why is active-directory required for Azure Activity? What permissions are required for sentinel connections with AAD as I can't find any documentation with this information, as I am seeing a permissions error that must be due to AAD access as I was able to use the data connector for defender with no errors.

az sentinel data-connector create --data-connector-id AzureActivity \ --resource-group my-rg \ --workspace-name my-workspace \ --azure-active-directory "{data-types:{alerts:{state:Enabled}},tenant-id:my-tenant-id}" This command is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus (Unauthorized) Access denied Code: Unauthorized Message: Access denied

[necusjz]

@zoxendine What about other similar arguments? Is there any argument meet your scenario? If so, then try to fill it.

[zoxendine]

I get access denied when attempt to create an AAD connection. We need to know what perms are required for these data connections to take place

az sentinel data-connector create --data-connector-id AzureActiveDirectory --resource-group rg --workspace-nameworkspace --azure-active-directory "{data-types:{alerts:{state:Enabled}},tenant-id:id" This command is experimental and under development. Reference and support levels: https://aka.ms/CLI_refstatus (Unauthorized) Access denied Code: Unauthorized Message: Access denied

[necusjz]

Could you please provide the debug log by appending --debug?

[microsoft-github-policy-service]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @amirkeren.

[zoxendine]

Could you please provide the debug log by appending --debug?

Still seeing an access denied issue are there no documentation on what perms are required for data connections?

`cli.azure.cli.core.sdk.policies: Request URL: 'https://management.usgovcloudapi.net/subscriptions/sub-id/resourceGroups/rg/providers/Microsoft.OperationalInsights/workspaces/workspace/providers/Microsoft.SecurityInsights/dataConnectors/AzureActiveDirectory?api-version=2022-06-01-preview'
cli.azure.cli.core.sdk.policies: Request method: 'PUT'
cli.azure.cli.core.sdk.policies: Request headers:
cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json'
cli.azure.cli.core.sdk.policies: 'Accept': 'application/json'
cli.azure.cli.core.sdk.policies: 'Content-Length': '147'
cli.azure.cli.core.sdk.policies: 'CommandName': 'sentinel data-connector create'
cli.azure.cli.core.sdk.policies: 'ParameterSetName': '--data-connector-id --resource-group --workspace-name --azure-active-directory --debug'
cli.azure.cli.core.sdk.policies: 'User-Agent': 'AZURECLI/2.51.0 (RPM) (AAZ) azsdk-python-core/1.26.0 Python/3.9.16 (Linux-6.1.11-200.fc37.x86_64-x86_64-with-glibc2.36)'
cli.azure.cli.core.sdk.policies: 'Authorization': '*****'
cli.azure.cli.core.sdk.policies: Request body:
cli.azure.cli.core.sdk.policies: {"kind": "AzureActiveDirectory", "properties": {"dataTypes": {"alerts": {"state": "Enabled"}}, "tenantId": "id"}}
urllib3.connectionpool: Starting new HTTPS connection (1): management.usgovcloudapi.n/id/resourceGroups/rg/providers/Microsoft.OperationalInsights/workspaces/workspace/providers/Microsoft.SecurityInsights/dataConnectors/AzureActiveDirectory?api-version=2022-06-01-preview HTTP/1.1" 401 59
cli.azure.cli.core.sdk.policies: Response status: 401
cli.azure.cli.core.sdk.policies: Response headers:
cli.azure.cli.core.sdk.policies: 'Cache-Control': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Pragma': 'no-cache'
cli.azure.cli.core.sdk.policies: 'Content-Length': '59'
cli.azure.cli.core.sdk.policies: 'Content-Type': 'application/json; charset=utf-8'
cli.azure.cli.core.sdk.policies: 'Expires': '-1'
cli.azure.cli.core.sdk.policies: 'Server': 'Kestrel'
cli.azure.cli.core.sdk.policies: 'x-ms-ratelimit-remaining-subscription-writes': '1199'
cli.azure.cli.core.sdk.policies: 'X-Content-Type-Options': 'nosniff'
cli.azure.cli.core.sdk.policies: 'Date': 'Tue, 22 Aug 2023 12:48:27 GMT'
cli.azure.cli.core.sdk.policies: 'Connection': 'close'
cli.azure.cli.core.sdk.policies: Response content:
cli.azure.cli.core.sdk.policies: {"error":{"code":"Unauthorized","message":"Access denied"}}
cli.azure.cli.core.azclierror: Traceback (most recent call last):
File "/usr/lib64/az/lib/python3.9/site-packages/knack/cli.py", line 233, in invoke
cmd_result = self.invocation.execute(args)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 663, in execute
raise ex
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 726, in _run_jobs_serially
results.append(self._run_job(expanded_arg, cmd_copy))
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/commands/init.py", line 697, in _run_job
result = cmd_copy(params)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_command.py", line 154, in call
return self._handler(*args, **kwargs)
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 31, in _handler
self._execute_operations()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1105, in _execute_operations
self.DataConnectorsCreateOrUpdate(ctx=self.ctx)()
File "/home/vagrant/.azure/cliextensions/sentinel/azext_sentinel/aaz/latest/sentinel/data_connector/_create.py", line 1120, in call
return self.on_error(session.http_response)
File "/usr/lib64/az/lib/python3.9/site-packages/azure/cli/core/aaz/_operation.py", line 329, in on_error
raise error_type(response=response)
azure.core.exceptions.ClientAuthenticationError: (Unauthorized) Access denied
Code: Unauthorized
Message: Access denied

cli.azure.cli.core.azclierror: (Unauthorized) Access denied
Code: Unauthorized
Message: Access denied
az_command_data_logger: (Unauthorized) Access denied
Code: Unauthorized
Message: Access denied
cli.knack.cli: Event: Cli.PostExecute [<function AzCliLogging.deinit_cmd_metadata_logging at 0x7f9c39929820>]
az_command_data_logger: exit code: 1
cli.main: Command ran in 3.993 seconds (init: 1.243, invoke: 2.750)
telemetry.main: Begin splitting cli events and extra events, total events: 1
telemetry.client: Accumulated 0 events. Flush the clients.
telemetry.main: Finish splitting cli events and extra events, cli events: 1
telemetry.save: Save telemetry record of length 3725 in cache
telemetry.main: Begin creating telemetry upload process.
telemetry.process: Creating upload process: "/usr/bin/python3.9 /usr/lib64/az/lib/python3.9/site-packages/azure/cli/telemetry/init.py /home/vagrant/.azure"
telemetry.process: Return from creating process
telemetry.main: Finish creating telemetry upload process.
`

[necusjz]

Thank you for your info, I'll contact service team for the root cause.

[zoxendine]

@necusjz Any update on this matter?

[necusjz]

@necusjz Any update on this matter?

Waiting for reply from service team.

[zoxendine]

Bumping for assistance @necusjz

[necusjz]

Bumping for assistance @necusjz

I'll keep you updated, but unfortunately...

[zoxendine]

Any updates?

[necusjz]

az sentinel data-connector create

No feedback from service team. But I found some hints from client telemetry, there are parameters of successful execution in recent 30 days:

It seems --azure-security-center and --office365 may help your case.

[Kaloszer]

@necusjz

These are completely different data connectors so this does not help this case at all unfortunately. Azure Activity currently applies through policy and it had changed sometime back so my bet is that it had never been implemented in az cli.

[zoxendine]

Am I correct to assume that the az cli doesn't support/work with data connectors at this point; and this needs to be done manually through the Portal? I am also attempting to use terraform for automation but get authorization issues with that as well.

[necusjz]

Am I correct to assume that the az cli doesn't support/work with data connectors at this point; and this needs to be done manually through the Portal? I am also attempting to use terraform for automation but get authorization issues with that as well.

I think so.

[Kaloszer]

@necusjz

Any timeline or feedback on this issue. Can we expect az cli to implement these at some point? This is really an issue with automation of IaC steps. Applying the policy through code seems wonky and I couldn't get it to work under:
Azure/Azure-Sentinel#8871

I still have yet to raise a support case on that one but this is a blocker for us.

[danwilcock]

I have the same issue deploying through terraform. The service principle has data connector update perms but returns a 401. Any update? Details on perms for the user/spn creating the service connector through the apis would be helpful