[ci][enh]: Github workflow to run tests during commits and pull requests

[MashyBasker]

It would be good to have a github workflow that runs the tests in the test folder against the proposed changes in a commit/PR.
This would require adding an api_base in the config.json and providing an API key as a repository secret, which would be used in running the tests with pytest.

[sauravpanda]

We are planning to offer a SaaS-based version where they can use our API key, and we manage the LLM tokens for them. The main reason is that we want to store the errors they get so that they can tackle them.

But adding a general version on OSS is a good idea. Let me think if it's better to have it here or make it a separate repo, given that actions will change a lot in the language, and we need to make language-specific configurations!