Brainstorm Auth Central vs Keycloak ADR

[smarru]

Let us brainstorm the feedback on ADR - https://github.com/NASA-IMPACT/veda-auth-central/blob/main/docs/architecture-decision-records/layering-on-keycloak.md

[smarru]

@j08lue @alukach please see this for reference - https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/ and a simpler explanation - https://www.osohq.com/learn/google-zanzibar. this figure nicely summerizes the need for veda auth central - https://cdn.prod.website-files.com/5f1483105c9a72fd0a3b662a/66a9566b8fac60e90f713e66_650477df621992dde19d6b2e_image3.png

[j08lue]

Thanks for setting this up, @smarru! 👌

I see your argument, that there is information / logic related to both authz and app, which fits neither into a generic authz solution (Keycloak), not into the app (too authz-specific).

Meanwhile, I am unable to verify this, since I lack the technical insight.

I can see how Keycloak must fall short for automated group assignment:

Automated Group Assignments: Unlike Keycloak, which does not have built-in support for automatic group assignments, Veda Auth Central’s custom layer introduces a visual policy enforcement interface. This interface allows administrators to automate group membership and role assignments based on user characteristics, significantly improving efficiency and reducing administrative overhead.

But for example I do not know what injecting scopes means or why a middleware is needed for that:

Although Keycloak offers advanced capabilities for authorization through scopes, the process of injecting and managing these scopes can be complex and often requires JavaScript injection, making it less ideal for straightforward integration. This highlights the need for a centralized mechanism to administer authorization policies effectively, ensuring consistency and security across the board.

It would help me to see the more complex requirements towards our auth solution listed out on their own, with an indication of whether they could be achieved with Keycloak or another off-the-shelf solution. Then it would be easier to gauge whether their value justifies the costs of developing our own middleware. I see you mentioned these:

application-based user subscriptions, scope inheritance, and detailed access control for multiple environments (dev, staging, production)

But I do not want this discussion to be an eternal blocker either...

[j08lue]

A challenge with the ADR and the architecture descriptions is that they focus on listing the features of a particular solution - the VEDA Auth Central middleware - rather than describing the authz needs more independently first and considering the options more or less impartially. This way, it is hard to see how the extra layer is a consequence of the decision drivers and pros and cons of the options, not an a priory assumption.

Also, we are missing some decision drivers in the ADR, which are important to keep in mind:

1. Use an open-source solution that fits into VEDA as an ecosystem of community-maintained open source platform components
2. Rely on existing open source solutions as much as possible to minimize maintenance costs
3. Minimize the amount of solution-specific logic that the applications need to implement (cc @yuvipanda re 2i2c JupyterHub)
4. Provide a GUI for user and group administration

(the above are not really orthogonal and may need a bit of refinement)

[j08lue]

I started a PR to bring the above criteria into the ADR:

• Review and refine ADR about Keycloak middleware #87

[alukach]

@j08lue @alukach please see this for reference - https://research.google/pubs/zanzibar-googles-consistent-global-authorization-system/ and a simpler explanation - https://www.osohq.com/learn/google-zanzibar. this figure nicely summerizes the need for veda auth central - https://cdn.prod.website-files.com/5f1483105c9a72fd0a3b662a/66a9566b8fac60e90f713e66_650477df621992dde19d6b2e_image3.png

@smarru Beyond the image, is there anything else to highlight with regards to the Zanzibar white paper? I have done some research into Zanzibar in the past but am not immediately seeing its relation to VEDA Auth Central. As I understand it, Zanzibar is a system providing fine grained access control to data objects. However, VEDA Auth Central is primarily focused on providing service-level authorization. i.e. in the above Venn diagram, VEDA Auth Central is entirely focused on the authz data 🔴, it has no knowledge of the app data.

[yuvipanda]

I too would like to know how Zanzibar relates to VEDA's needs. My current understanding is that the scale of complexity of Google's needs that required building something as complex as Zanzibar is 2-3 orders of magnitude more than what VEDA needs.

Is the suggestion that all applications be treated as microservices that take a strong dependency on Custos (or VEDA Auth central, although I'm not clear on the relationship between these two) to do authorization choices? I'm not sure that's the right call for all applications (since that'll restrict usage outside VEDA), but even if it is, how is that different from using something like the UMA standard (https://en.wikipedia.org/wiki/User-Managed_Access) which has multiple available implementations (including in keycloak)?

[j08lue]

Another question: The ADR describes the need for some software in addition to Keycloak.

Are there off-the-shelf alternatives to building our own, which we could at least acknowledge as options? How about Auth0's OpenFGA (please excuse my ignorance, if this is a completely different solution).

And would there be an option where we do not proxy all Keycloak interactions, but only implement some kind of services / hooks, e.g. for automatic group assignment, but keep using Keycloak native endpoints and UI for most things?

[alukach]

Are there off-the-shelf alternatives to building our own, which we could at least acknowledge as options? How about Auth0's OpenFGA (please excuse my ignorance, if this is a completely different solution).

I believe that this is comparing apples to oranges. At the risk of over-explaining something that you may already know: KeyCloak and Zanzibar-inspired solutions are targeting different things.

IAM

OAuth2 implementations (e.g. Keycloak, Auth0, AWS Cognito, Ory Hydra) are Identity and Access Management solutions. A non-exhaustive list of functionality of these types of IAM services are:

1. Identity: Determine who a user is (ie AuthN). I believe that all of these modern tools provide Single Sign On via OIDC, permitting users to authenticate with credentials from third-party identity providers.
2. Access Management: Delegate which identities are able to access which applications. This is an implementation of the OAuth2 protocol. These tools generate credentials that the user may use when connecting with protected services. These credentials may include information about the user, such as what the user is permitted to do within an application.

FGA

Zanzibar implementations (e.g. OpenFGA, Ory Keto, SpiceDB, Permify, Warrant) are Fine Grained Authorization solutions, making use of Relationship Based Access Control (ReBAC). They are optimized databases that manage the relationships between objects and can approve/deny access to these objects via custom relational policies. To my knowledge, they are solely focused on authorization and do not attempt to solve problems around identity (ie not authentication). Moreso, they provide APIs that your services can query prior to fulfilling a user's request, first determining whether a user is permitted to perform the requested operation.

[j08lue]

I believe that this is comparing apples to oranges

Sorry about the 🍏 🍊 and thanks for the explainer! 🙏 💡

My thought was actually different: is there a solution in addition to Keycloak that can provide the additional functionality we need?

We need to justify that we are building our own, if other OSS solutions exist that can cover the need.

[yuvipanda]

We need to justify that we are building our own, if other OSS solutions exist that can cover the need.

This is the core issue at hand I think, especially because it means a brand new UI project has to be built from scratch instead of being able to adopt Keycloak's UI.

[alukach]

Although Keycloak offers advanced capabilities for authorization through scopes, the process of injecting and managing these scopes can be complex and often requires JavaScript injection, making it less ideal for straightforward integration.

source

I'm not sure that this is accurate. When looking at auth tooling for a VEDA-related project (eoAPI), I put together an example of using KeyCloak to manage access to scopes via Groups + Roles (no JS): https://github.com/alukach/keycloak-fastapi-playground

[DImuthuUpe]

@alukach thanks for the example. Assigning groups or roles to scopes can be easily done through the keycloak and if we just have that requirement, we should use only Keycloak. I think we need to think of the problem at a higher level. There are several key usability + technical aspects that we need to pay close attention to. I am picking a few concrete examples but I can keep adding more

1. Once a user is authenticated through an IDP, how do we map that user into an existing group or role in Keycloak. We can always manually assign them using the Keycloak dashboard but we need to think of the scalability of the problem. There are script providers that we can use to do that in a semi-automated way but it is hardcoded into the code. There is no option to configure it rather than keep injecting code. Specifically, if you want to undo some changes, there is no audit trail of code changes that we have to do. I am not saying that this is Keycloak's fault. It's just very hard to generalize authorization policies for all the use cases and Keycloak is not built for that. Keycloak is an authentication broker with some general-purpose authorization salt added. That's why many companies use third-party tools like this for authentication and implement their own authorization logic in the application logic.

2. Last mile integrations: SCIM Support (Just an example): Security is not a straight line / happy day path solution. Once we put Auth Central in production, I believe we have to handle 80 - 90% edge case scenarios out of production issues (based on my experience on any distributed system). SCIM will be one of those and Keycloak only has one paid SCIM support https://scim-for-keycloak.de/. Whether we like it or not, we need to have our version of SCIM handling solution. Also, have a close look at Keycloak issues for these advanced use cases SCIM support keycloak/keycloak#13484. They also recommend using an external wrapper to handle that as the requirement is very broad and specific to the federated authentication we use.

3. Application-specific policies: This might provide some sort of an answer for your next post but I agree that it is a loaded topic. I think if we look at the UI designs for the Auth Central dashboard, this will be cleared out. In general, the idea is, how restrictive should we be for an non-geeky-Keycloak-user to manipulate authorization policies for a VEDA app in auth central as an App admin? One might argue that it should be just the dashboard's job but in reality, those restrictions should be enforced in the backend. For example, people should not be allowed to assign the veda hub's policies to the STAC catalog's policy table by mistake and that should be enforced at the data model level.

Sorry, this is very technical but I tried to be as much general as possible. Please ask follow-up questions and I am very happy to clarify

[yuvipanda]

Thank you for this response, @DImuthuUpe. I've some questions

There are script providers that we can use to do that in a semi-automated way but it is hardcoded into the code.

Can you expand a little more on what you mean by 'hardcoded into the code'? Do the script providers not have the ability to make external network calls (for example)? Or more specifically, what kind of assignment are you thinking of that would not be possible to do with Keycloak?

Specifically, if you want to undo some changes, there is no audit trail of code changes that we have to do.

Given that the method to deploy these scripts is by deploying a JAR file, wouldn't any custom code be managed in a git repository and deployed via CI/CD? That would create an audit trail via git, right? Keycloak no longer offers the ability to upload code via the UI primarily for this reason, so there is an external mechanism for an audit trail.

[yuvipanda]

For example, people should not be allowed to assign the veda hub's policies to the STAC catalog's policy table by mistake and that should be enforced at the data model level.

Would this not be an operational task, and hence managed via infrastructure as code mechanisms such as https://github.com/adorsys/keycloak-config-cli or similar? I would imagine such config changes would happen via pull request, and be tracked via a continuous deployment mechanism rather than be something that's handled either in code (which means policy changes require Java code changes now) or via the keycloak UI (very prone to error, as you say).

[alukach]

Drivers

• Supports Complex Authorization Needs: The custom layer addresses specific authorization needs that are not natively supported by Keycloak, such as application-based user subscriptions, scope inheritance, and detailed access control for multiple environments (dev, staging, production).

I admit that I don't fully know what "application-based user subscriptions, scope inheritance, and detailed access control for multiple environments" means. Can we add more concrete descriptions and examples of these needs? For example, with regards to "scope inheritance", I do know that KeyCloak supports group hierarchy and that scopes can be "inherited" through that, however I'm not sure if that is what we are talking about. Additionally, with regards to "multiple environments", I know that KeyCloak is built as a multi-tenant system which would lend well to multiple environment system, but I again am not certain that's what we're talking about.

[yuvipanda]

+1! Can a PR be made clarifying what these things mean? I left basically a replica of this comment in https://github.com/NASA-IMPACT/veda-auth-central/pull/87/files#r1732091212. It's much much easier to have discussions related to specific pieces of text and clarify what they mean in PR form than quoting and having it as a GitHub discussion, as it can get meandering and off-topic easily.