Props for additional validation (sealed Codecs getting in the way)

[simonpetty]

My team would like to start making use of additional JSON properties to document extra validation.

The idea is that producers can use these to document validation they support, but consumers can also define them in addition to what the producer schema has. Eventually, we see this as a way to introduce data contracts.

For example:

• Min/Max size strings
• Regex for strings
• Min/Max length of collections

Confluent recently introduced a set of features in this space, but these are alongside the schemas not within, and we aren't using the Confluent platform in any case.

Adding the props is relatively easy, as we can use this method for properties OF a record:

final def record[A](
    name: String,
    namespace: String,
    doc: Option[String] = None,
    aliases: Seq[String] = Seq.empty,
    props: Props = Props.empty
  )

And the below allows us to define props for each of the field IN a record:

sealed abstract class FieldBuilder[A] {
    def apply[B](
      name: String,
      access: A => B,
      doc: Option[String] = None,
      default: Option[B] = None,
      order: Option[Schema.Field.Order] = None,
      aliases: Seq[String] = Seq.empty,
      props: Props = Props.empty
    )(implicit codec: Codec[B]): FreeApplicative[Field[A, *], B]
  }

Passing in props to the field builder results in a Schema like this:

{
 "type":"record",
 "name":"name",
 "namespace":"namespace",
 "fields":[
   {"name":"name","type":"string"},
   {"name":"age","type":"int"},
   {"name":"example","type":"string","maxLength":10}
 ]
}

The problem comes when we try to parse these props when decoding.

Originally i thought we could use codec: Codec[B] and generate a new codec by calling imapError and return a Left(AvroError) if something failed validation. But imapError hides the writer schema.

Ultimately i think the only way of doing what I want is to use this deprecated method:

instance[AvroType0, A](
    schema: Either[AvroError, Schema],
    encode: A => Either[AvroError, AvroType0],
    decode: (Any, Schema) => Either[AvroError, A]
  )

I can see a similar issue was raised here: #504 which they resolved by adding something to the library.

Do you have any suggestions if there's something else I can true, or perhaps this is something of a feature request?

[simonpetty]

closing as I may have found a workaround and didn't want to waste anyone's time, but may reopen again :)