Allow arbitrary python code

[Maleware]

Description

This PR is the outcome of the decision https://github.com/stackabletech/decisions/issues/21

Contains:

• CONFIG_OVERRIDE_FILE_HEADER_KEY (operator-rs constant)
• CONFIG_OVERRIDE_FILE_FOOTER_KEY (operator-rs constant)
• Bump stackable-operator from 0.73.0 to 0.74.0
• Adding Error for writeln!

Definition of Done Checklist

• Not all of these items are applicable to all PRs, the author should update this template to only leave the boxes in that are relevant
• Please make sure all these things are done and tick the boxes

Author

Preview Give feedback

1. Changes are OpenShift compatible
2. CRD changes approved
3. CRD documentation for all fields, following the style guide.
4. Helm chart can be installed and deployed operator works
5. Integration tests passed (for non trivial changes)
6. Changes need to be "offline" compatible

Reviewer

Preview Give feedback

1. Code contains useful comments
2. Code contains useful logging statements
3. (Integration-)Test cases added
4. Documentation added or updated. Follows the style guide.
5. Changelog updated
6. Cargo.toml only contains references to git tags (not specific commits or branches)

Acceptance

Preview Give feedback

1. Feature Tracker has been updated
2. Proper release label has been added
3. Roadmap has been updated

[Maleware]

Tests green:
🟢

--- PASS: kuttl (206.16s)
    --- PASS: kuttl/harness (0.00s)
        --- PASS: kuttl/harness/smoke_airflow-2.9.2_openshift-false_executor-celery (206.12s)
PASS

[sbernauer]

LGTM otherwise

[sbernauer]

Thanks!

[lfrancke]

The decision is not public, can you summarize what this is about and if it is something that is documented please include a link to the docs. If it's not documented can you briefly summarize why note?

Does this need to be mentioned in the release notes?

[Maleware]

This is to allow the user to add arbitrary python code to the webserver_config.py.

We've got it documented in our superset docs and airflow docs

However, this can be used for any case where superset allows us to replace functions with custom ones. It was driven by superset documentation about custom OIDC manager and we found that we should add it to airflow too.

We've reached a decision on that this is how airflow wants to be configured ( no matter how ugly it might appear ) and thus we want the operator to be useful in this case.

We've had an customer for airflow who wrote their whole config for themself ( basically ignoring the operators work and thus it's not used ) to achieve having own classes and functions. They now can use the configs the operator provides and have their custom code as a config.

I can't really make a objective statement weather or not we should mention it in the release notes. For me it should as it enables some customers and they might wanna know.

Left to do: Do we want to add python libraries to the Airflow Dockerfile to have OIDC supported properly, if a customer chooses to do so?

[lfrancke]

Thank you. In that case: Can you please add a short sentence as it could appear in the release notes as a comment here?

[Maleware]

Sure.

Release Notes:
Apache Airflow: Allow custom arbitrary python code in webserver_config.py