How to implement argument validation via a custom directive

[dkbarn]

I'm confused about the use of directives vs validation rules when it comes to arguments. Ariadne's documentation doesn't seem to cover this particular case in great detail.

Suppose I wanted to implement regex-based validation for certain arguments in my schema:

directive @regex(pattern: String!) on ARGUMENT_DEFINITION

type Query {
    hello_world(
        arg1: String! @regex(pattern: "^[A-Z0-9]+$")
    ): String!
}

I'm defining this as a directive rather than a validation rule because it's a re-usable concept that I want the ability to easily apply to various arguments throughout the SDL.

But how would you actually implement such a directive in code? The documentation only provides an example for a FIELD_DEFINITION directive, in which the given field's resolve function is customized: https://ariadnegraphql.org/docs/schema-directives#example-datetime-format

There doesn't appear to be any similar hook on a GraphQL argument definition for attaching a validation function:

from ariadne import SchemaDirectiveVisitor

class RegexDirective(SchemaDirectiveVisitor):
    def visit_argument_definition(self, argument, field, object_type):
        ???

Taking a look at the query_cost validator in ariadne.validation.query_cost, it appears that this is both a directive and a validation rule. It defines a directive for use in the SDL, but the actual implementation does not subclass SchemaDirectiveVisitor. Instead it subclasses ValidationRule and then inspects the directives attribute on field.ast_node: https://github.com/mirumee/ariadne/blob/master/ariadne/validation/query_cost.py#L110

This is awkward for multiple reasons:

1. In order for this to work, it requires that you supply both the directive definition to make_executable_schema and the validation rule to the graphql() function.
2. The ValidationRule is going to be executed for every single argument, not just the ones with the directive applied.

Is there no way to implement validation for an argument using an ARGUMENT_DEFINITION directive and only subclassing SchemaDirectiveVisitor?

[rafalp]

Ariadne's documentation doesn't seem to cover this particular case in great detail.

It's not covered because the goal for documentation is to document good patterns and practices, and using directives for data validation is not one of those 😄

Problems with this approach:

• Data validation logic reports errors to result.error which is troublesome to work with: it both increases complexity in client's error handling logic and takes away benefits from smart cache sync that libs like apollo-client offer.
• Those validators can't be async. Need to check if given order id exists in database? Now you need to put that logic into resolver or write another directive that wraps resolver in extra logic just for this field. Either way now you have two stages of validation.
• Its far more complex to implement and maintain than using one of many data validation libraries that already exist from your resolver.
• Adding to complexity, you will need to duplicate bunch of Query executor's logic just to retrieve argument's value from query's AST or variables list.

Is there no way to implement validation for an argument using an ARGUMENT_DEFINITION directive and only subclassing SchemaDirectiveVisitor?

Arguments don't hold any logic that is being executed at query time, only argument definition that validation rules and query executor access to find out:

• where's argument's value located (is it in variables? is it in query's AST?)
• which scalar's parse method to use to parse variable/AST to Python value.
• by what name is argument passed to resolver.

If I was determined to put validation declaration in schema, I would use directives as dumb metadata holders, and implement Python util that would construct pydantic model from field's definition using field's declaration available from resolver's info argument.

[dkbarn]

Let me take a step back to explain why I am attempting to do data validation via directives in the first place, and maybe you can point out a better way of approaching this.

When it comes to error handling, from the client's perspective there is a basic need to distinguish between "the problem is the client's fault" and "the problem is the server's fault", so that the client can make informed decisions, such as whether it's worth retrying the failed request.

We return an HTTP 400 to indicate a client-based error (they supplied bad data in the query) and an HTTP 500 to indicate a server-based error (the query itself passed validation, but something unexpectedly went wrong during execution of that query).

According to the documentation, the graphql() function returns a GraphQLResult object which is a tuple of (success, response) where success=False should correspond to an HTTP 400: https://ariadnegraphql.org/docs/types-reference#graphqlresult

A success=False only occurs when something goes wrong in the parsing/validating of the query. Once you are at the resolver stage, any exceptions raised will be captured in response["errors"], but success will be True. Therefore, we use the presence of response["errors"] to indicate an HTTP 500, because it happens when something unexpectedly went wrong in the resolvers.

If we do as you suggest, and move data validation into the resolver functions, then how can we distinguish between client-based errors and server-based errors so that we can choose HTTP 400 vs 500 to return to the client?

[rafalp]

We return an HTTP 400 to indicate a client-based error (they supplied bad data in the query) and an HTTP 500 to indicate a server-based error (the query itself passed validation, but something unexpectedly went wrong during execution of that query).

This is wrong assumption because it goes against how both GraphQL and HTTP clients come to understand those status codes. HTTP 400 indicates that HTTP request wasn't valid GraphQL payload and server didn't execute query while 500 means that server error happened.

HTTP 400 from GraphQL server means that client-server communication is done incorrectly, and developer working on either of those sides needs to correct their codebase. You can produce error like this by requesting non-existing fields. In your case this would be error 500 which in HTTP means unhandled error on server side that prevented request from being served. Is wrong query syntax server's fault? I don't think so.

When it comes to error handling, from the client's perspective there is a basic need to distinguish between "the problem is the client's fault" and "the problem is the server's fault", so that the client can make informed decisions, such as whether it's worth retrying the failed request.

The simple and proven approach to distinguishing between those two is to look if errors are in result.errors or result.data.myMutation.errors.

I've answered very similar question earlier today, and there's also Ariadne's guide on how to approach it, here. :)

I'm thinking its time to update that doc with more examples and explanation of reasoning. A lot of people starting out with GraphQL fall into trap of trying to use HTTP status codes or result.errors for error communication only to realize sooner or later that they've engineered themselves into the corner and a lot of otherwise easy or free things like rollbacking optimistic updates, detecting validation errors or retrieving updated state after mutation completion now require them to invent custom boilerplate.

[dkbarn]

HTTP 400 indicates that HTTP request wasn't valid GraphQL payload and server didn't execute query while 500 means that server error happened. HTTP 400 from GraphQL server means that client-server communication is done incorrectly, and developer working on either of those sides needs to correct their codebase.

Right, exactly. I agree. That's why I'm trying to do data validation with a directive. A basic check like whether an argument matches a regex should be classified in the same category of errors as a String argument being given an Integer value. They should both be HTTP 400 errors, which means handled either by a directive or a validation rule, but not by a resolver.

This is why Apollo wrote the graphql-constraint-directive library: https://www.apollographql.com/blog/graphql-validation-using-directives-4908fd5c1055/ This provides basic argument validation via a @constraint directive. The types of validation are things like pattern matching and maxLength. Very basic checks that should happen before resolver execution, and should be reported to the client as HTTP 400, because the client is providing invalid queries and needs to correct their codebase.

[rafalp]

This is why Apollo wrote

Thats is incorrect. Please note that is a guest post from 2018 for a third party library. ;)

You are still free to work on your approach, but I will keep arguing that both HTTP 400 and using extending query validation will cause you more problems than its worth it in the long run.

You've been warned ;)