feat: add precommit

[juju4]

Change(s):

• add pre-commit client config and github workflow+azure-pipelines to run server side

Reason for Change(s):

• add more early controls in "python standard" way

Version Updated:

• No

Testing Completed:

• Yes (github workflow only and private azure-pipelines)

Checked that the validations are passing and have addressed any issues that are present:

• No change to code. Add validation check only.

---

This PR adds precommit config

Example run as is without fixing anything
https://github.com/juju4/Azure-Sentinel/actions/runs/3867376910/jobs/6592111638#step:4:52

Following fixed initial config, not sure if you prefer to be in this PR or a different one
https://github.com/juju4/Azure-Sentinel/actions/runs/3867831878
master...juju4:Azure-Sentinel:devel-precommit2

Follow-up hooks here or new PR:

• codespell - https://github.com/codespell-project/codespell
  juju4@b079d36
  a bit of remediation (there is a lot here...)
  https://github.com/juju4/Azure-Sentinel/actions/runs/3867789949/jobs/6592801053
• detect-secrets - https://github.com/Yelp/detect-secrets
• PSScriptAnalyzer - https://github.com/PowerShell/PSScriptAnalyzer

[v-amolpatil]

@juju4 Could you please resolve the conflicts?
Also could you please fix the build errors.
Can you please provide more details on the pre-commit? Is it for validating the spelling mistakes in any of the file.

[v-amolpatil]

@juju4 Could you please take a look at previous comment?

[juju4]

I merged latest upstream to address conflict.
As for precommit pipelines, please confirm that I should merge remediation branches too.

On codespell remediations, only doing part of them as too many. Each content owner should address their own.
To avoid pipeline failing, I'm putting the non-remediated folders in exclude list

arm-ttk pipeline failing part is new and should not be related to PR as no functional change.

[v-amolpatil]

We will take a look. Thanks!

[juju4]

I applied remediation changes to branch to fix precommitValidation pipeline (but impacting tons of files as json prettify/standardize, end of files, UTF-8 BOM, duplicate key and so on)
Not sure about the failing check if related or not as can't see details for ADO tests.

codespell and the rest in future PR

[juju4]

Any input here?

Should I revert remediation and put test in continue-on-error like #7323 ?

[juju4]

News ?

[v-atulyadav]

Hi @mkchiliveri, please provide your response on this. Thanks

[v-atulyadav]

Hi @daxesht, could you please provide your feedback on this. Thanks

[v-atulyadav]

Hi @daxesht, please provide your feedback on this. Thanks

[v-atulyadav]

Hi @daxesht, could you please provide your feedback on this. Thanks

[v-atulyadav]

Hi @daxesht, please provide your feedback on this. Thanks

[juju4]

Any update on this or path forward?

[v-atulyadav]

Hi @juju4, I apologize for the inconvenience. We will provide you with an update by the 13th. Please update your branch from master in the meantime. Thanks

[v-atulyadav]

Hi @juju4, Having analyzed the contents, we are not able to determine the reason for this change. Could you please elaborate on this and provide screenshots of this functionality if possible? Also the changes you are requesting are for both Github workflows and Azure pipelines, can't we handle this in one place instead of both? It would also be great if we could connect over a call to discuss further. Thanks

[v-atulyadav]

Hi @juju4, waiting for your response over above comments and also please resolve branch conflicts. Thanks

[v-atulyadav]

Hi @juju4, please check comments above and act accordingly. Thanks

[v-atulyadav]

Hi @juju4, I am looking forward to your response to the above comments, as well as the resolution of branch conflicts.
Thanks

[v-atulyadav]

Hi @juju4, please respond over above comments and also please resolve branch conflicts. Thanks

[v-atulyadav]

Hi @juju4, please check comments above and act accordingly. Thanks

[v-atulyadav]

Hi @juju4, hope you are doing well. Just wanted to check if you got a chance to look at the feedback shared. Please feel free to reach out to us for any queries and/or support. Thanks

[juju4]

The content of the PR has been explained in #7056 (comment)
It applies to both Github Action and ADO because ultimately it should be available to whatever backend is used as pipeline with Sentinel Repository feature.

It adds no functionality outside of validating more content but goes further than existing validations. same than #7323

[v-atulyadav]

Noted @juju4,
We will check internally and provide you with an update as soon as possible.
Thanks

[v-atulyadav]

Hi @juju4,
We will provide you with an update on this PR by the middle of next week, and if we find any difficulty, we will reach out to you.
Thanks.

[mkchiliveri]

Hi @juju4 ,

As discussed, can you please check on if we can enforce this rule only on files committed as part of PR so that author can take an action immediately instead of the full review.

[juju4]

Per pre-commit-ci/issues#90, this is not possible. at least, not with the tool.

As said, the normal way would be cleaning existing repo progressively. Part work done in following commits that were reverted to keep this PR small. Can push them back after but for separate PR.
84ca86a
f24f041
8422184

[v-atulyadav]

Hi @juju4, as discussed we will take this as internal feature and try to develop this in future. Thanks