This repository has been archived by the owner on Jan 29, 2024. It is now read-only.
Content for OAuth2/OIDC support for Kafka REST proxy #2237
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,66 @@ | ||
| Enable OAuth2/OIDC support for Apache Kafka® REST proxy | ||
| ======================================================== | ||
|
|
||
| Secure your Apache Kafka® resources by integrating OAuth 2.0/OpenID Connect (OIDC) with the Karapace REST proxy and enabling REST proxy authorization. This setup ensures that only authorized individuals can manage Apache Kafka resources through both token-based authentication and access control rules. | ||
|
|
||
| OAuth2/OIDC token handling | ||
| --------------------------- | ||
|
|
||
| Karapace processes the JSON Web Token (JWT) obtained from the Authorization HTTP header, specifically when employing the Bearer authentication scheme. This allows OAuth2/OIDC credentials to be supplied directly to the REST proxy, which uses the provided token to authorize requests to Apache Kafka. When a Bearer token is presented, Kafka clients configured by Karapace use the SASL OAUTHBEARER mechanism to send the JWT for validation. | ||
|
|
||
|
|
||
| Authorization enforcement | ||
| ---------------------------- | ||
|
|
||
| In the underlying Aiven for Apache Kafka® service, the default mechanism for authorization, uses the ``sub`` claim from the JWT as the username. This username is then verified against the configured Access Control Lists (ACLs) to authorize user operations on Kafka resources. | ||
|
|
||
| While the ``sub`` claim is the default identifier, this setting is configurable. You can specify a different JWT claim for authentication by adjusting the ``kafka.sasl_oauthbearer_sub_claim_name`` parameter. For more information on configuring this, see :ref:`console-authentication`. | ||
|
|
||
| To authenticate and authorize a user in Aiven for Apache Kafka, the JWT claim value and corresponding service user must be explicitly present in the ACL. This means that the identity of the user making the request with the JWT claim value must match the service user in the system. | ||
|
|
||
| Managing token expiry | ||
| ------------------------------ | ||
|
|
||
| With OAuth2/OIDC enabled, Karapace manages Kafka client connections for security and performance. It automatically cleans up idle clients and those with tokens nearing expiration, typically on a 5-minute cycle. This cleanup prevents unauthorized access with expired tokens and clears idle connections. | ||
|
|
||
| .. note:: | ||
| Before your token expires, remove any linked consumers and producers to avoid security issues and service interruptions. After removal, refresh your OAuth2 JWT tokens and reconnect with the new tokens. | ||
|
|
||
|
|
||
| Configure OAuth2/OIDC authentication | ||
| -------------------------------------------------------------- | ||
|
|
||
| To establish OAuth2/OIDC authentication for the Karapace REST proxy, complete the following prerequisites and configuration steps: | ||
|
|
||
| Prerequisites | ||
| ``````````````` | ||
| * :doc:`Aiven for Apache Kafka® </docs/products/kafka/getting-started>` service running with :doc:`OAuth2/OIDC enabled </docs/products/kafka/howto/enable-oidc>`. | ||
|
Check failure on line 37 in docs/products/kafka/karapace/howto/enable-oauth-oidc-kafka-rest-proxy.rst
|
||
| * :doc:`Karapace schema registry and REST APIs enabled </docs/products/kafka/karapace/howto/enable-karapace>`. | ||
|
Check failure on line 38 in docs/products/kafka/karapace/howto/enable-oauth-oidc-kafka-rest-proxy.rst
|
||
| * Ensure access to an OIDC-compliant provider, such as Auth0, Okta, Google Identity Platform, or Azure. | ||
|
|
||
| Configuration via Aiven Console | ||
| ``````````````````````````````````` | ||
| 1. In `Aiven Console <https://console.aiven.io/>`_, select your project and then choose your Aiven for Apache Kafka® service. | ||
| 2. On the **Overview** page, scroll down to **Advanced configuration** and select **Configure**. | ||
| 3. In the **Advanced configuration** screen, select **Add configuration options**. | ||
| 4. Look for ``kafka_rest_authorization`` parameter and set it to ``True``. | ||
|
|
||
| Configuration via Aiven CLI | ||
|
Check failure on line 48 in docs/products/kafka/karapace/howto/enable-oauth-oidc-kafka-rest-proxy.rst
|
||
| ````````````````````````````` | ||
|
|
||
| To enable REST proxy authorization, use the following command in the Aiven CLI, replacing ``SERVICE_NAME`` with your actual service name: | ||
|
|
||
| .. code:: | ||
|
|
||
| avn service update -c kafka_rest_authorization=true SERVICE_NAME | ||
|
|
||
| Disable REST proxy authorization, use: | ||
|
|
||
| .. code:: | ||
|
|
||
| avn service update -c kafka_rest_authorization=false SERVICE_NAME | ||
|
|
||
| .. warning:: | ||
| Enabling Apache Kafka REST proxy authorization can disrupt access for users if the Kafka access control rules have not been configured properly. For more information, see :doc:`Manage Apache Kafka® REST proxy authorization <../howto/manage-kafka-rest-proxy-authorization>`. | ||
|
Check failure on line 64 in docs/products/kafka/karapace/howto/enable-oauth-oidc-kafka-rest-proxy.rst
|
||
|
|
||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I tried to think how to change the text to be more specific that both are needed. Now I feel that this paragraph mixes/is not exact enough with the difference.
For example this would work.
userid-example.userid-ex*example-topicread|write|readwrite|adminIf service user would be removed from the Aiven for Kafka the authorization would start to fail. This is a bit counter intuitive as ACL should be the only item to affect authorization with OAuth2/OIDC but our system currently links the service users and ACL entries. There is a ticket to remove this link: https://aiven.atlassian.net/browse/HH-2792
So maybe something like this would be better:
To authenticate and authorize a user in Aiven for Apache Kafka a service user and ACL entry describing the permissions is needed. This means that the identity of the user making the request with the JWT claim value must match the service user in the system and must match the ACL entry.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@jjaakola-aiven I've made the following text updates. Could you check them and confirm if they're alright?
To authenticate and authorize a user in Aiven for Apache Kafka, both a service user and an ACL entry describing the permissions are required. The JWT claim value used for authentication must explicitly match the service user in the system. Furthermore, this service user must be associated with an ACL entry that outlines their permissions, ensuring that the identity of the user making the request aligns with both the service user and the ACL entry.