diff --git a/docs/codecs.md b/docs/codecs.md new file mode 100644 index 000000000..f6e5d25a6 --- /dev/null +++ b/docs/codecs.md @@ -0,0 +1,32 @@ +--- +id: codecs +title: "Codecs" +--- + +Once we have our schema, we can combine it with a codec. A codec is a combination of a schema and a serializer. Unlike codecs in other libraries, a codec in ZIO Schema has no type parameter: + +```scala +trait Codec { + def encoder[A](schema: Schema[A]): ZTransducer[Any, Nothing, A, Byte] + def decoder[A](schema: Schema[A]): ZTransducer[Any, String, Byte, A] + + def encode[A](schema: Schema[A]): A => Chunk[Byte] + def decode[A](schema: Schema[A]): Chunk[Byte] => Either[String, A] +} +``` + +It basically says: +- `encoder[A]`: Given a `Schema[A]` it is capable of generating an `Encoder[A]` ( `A => Chunk[Byte]`) for any Schema. +- `decoder[A]`: Given a `Schema[A]` it is capable of generating a `Decoder[A]` ( `Chunk[Byte] => Either[String, A]`) for any Schema. + +Example of possible codecs are: + +- CSV Codec +- JSON Codec (already available) +- Apache Avro Codec (in progress) +- Apache Thrift Codec (in progress) +- XML Codec +- YAML Codec +- Protobuf Codec (already available) +- QueryString Codec +- etc. diff --git a/docs/combining-different-encoders.md b/docs/combining-different-encoders.md new file mode 100644 index 000000000..980905848 --- /dev/null +++ b/docs/combining-different-encoders.md @@ -0,0 +1,42 @@ +--- +id: combining-different-encoders +title: "Combining Different Encoders" +--- + +Let's take a look at a round-trip converting an object to JSON and back, then converting it to a protobuf and back. This is a simple example, but it shows how to combine different encoders to achieve a round-trip. + +```scala +object CombiningExample extends zio.App { + import zio.schema.codec.JsonCodec + import zio.schema.codec.ProtobufCodec + import ManualConstruction._ + import zio.stream.ZStream + + override def run(args: List[String]): UIO[ExitCode] = for { + _ <- ZIO.unit + _ <- ZIO.debug("combining roundtrip") + person = Person("Michelle", 32) + + personToJson = JsonCodec.encoder[Person](schemaPerson) + jsonToPerson = JsonCodec.decoder[Person](schemaPerson) + + personToProto = ProtobufCodec.encoder[Person](schemaPerson) + protoToPerson = ProtobufCodec.decoder[Person](schemaPerson) + + newPerson <- ZStream(person) + .tap(v => ZIO.debug("input object is: " + v)) + .transduce(personToJson) + .transduce(jsonToPerson) + .tap(v => ZIO.debug("object after json roundtrip: " + v)) + .transduce(personToProto) + .transduce(protoToPerson) + .tap(v => ZIO.debug("person after protobuf roundtrip: " + v)) + .runHead + .some + .catchAll(error => ZIO.debug(error)) + _ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString) + _ <- ZIO.debug("old person: " + person) + _ <- ZIO.debug("new person: " + newPerson) + } yield ExitCode.success +} +``` diff --git a/docs/getting-started.md b/docs/getting-started.md new file mode 100644 index 000000000..98503ef63 --- /dev/null +++ b/docs/getting-started.md @@ -0,0 +1,146 @@ +--- +id: getting-started +title: "Getting Started" +--- + +To get started, first we need to understand that a ZIO Schema is basically built-up from these three +sealed traits: `Record[R]`, `Enum[A]` and `Sequence[Col, Elem]`, along with the case class `Primitive[A]`. Every other type is just a specialisation of one of these (or not relevant to get you started). + +We will take a look at them now. + +## Basic Building Blocks + +### Schema + +The core data type of ZIO Schema is a `Schema[A]` which is **invariant in `A`** by necessity, because a Schema allows us to derive operations that produce an `A` but also operations that consume an `A` and that imposes limitations on the types of **transformation operators** and **composition operators** that we can provide based on a `Schema`. + +It looks kind of like this (simplified): + +```scala +sealed trait Schema[A] { self => + def zip[B](that: Schema[B]): Schema[(A, B)] + + def transform[B](f: A => B, g: B => A): Schema[B] +} +``` + +### Records + +Our data structures usually are composed of a lot of types. For example, we might have a `User` +type that has a `name` field, an `age` field, an `address` field, and a `friends` field. + +```scala +case class User(name: String, age: Int, address: Address, friends: List[User]) +``` + +This is called a **product type** in functional programming. The equivalent of a product type in ZIO Schema is called a record. + +In ZIO Schema such a record would be represented using the `Record[R]` typeclass: + +```scala +object Schema { + sealed trait Record[R] extends Schema[R] { + def fields: Chunk[Field[_]] + def construct(value: R): Chunk[Any] + def defaultValue: Either[String, R] + } +} + +``` + +### Enumerations + +Other times, you might have a type that represents a list of different types. For example, we might have a type, like this: + +```scala +sealed trait PaymentMethod + +object PaymentMethod { + final case class CreditCard(number: String, expirationMonth: Int, expirationYear: Int) extends PaymentMethod + final case class WireTransfer(accountNumber: String, bankCode: String) extends PaymentMethod +} +``` + +In functional programming, this kind of type is called a **sum type**: +- In Scala 2, this is called a **sealed trait**. +- In Scala3, this is called an **enum**. + +In ZIO Schema we call these types `enumeration` types, and they are represented using the `Enum[A]` type class. + +```scala +object Schema extends SchemaEquality { + sealed trait Enum[A] extends Schema[A] { + def annotations: Chunk[Any] + def structure: ListMap[String, Schema[_]] + } +} +``` + +### Sequence + +Often you have a type that is a collection of elements. For example, you might have a `List[User]`. This is called a `Sequence` and is represented using the `Sequence[Col, Elem]` type class: + +```scala +object Schema extends SchemaEquality { + + final case class Sequence[Col, Elem]( + elementSchema: Schema[Elem], + fromChunk: Chunk[Elem] => Col, + toChunk: Col => Chunk[Elem] + ) extends Schema[Col] { + self => + override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = Traversal[Col, Elem] + override def makeAccessors(b: AccessorBuilder): b.Traversal[Col, Elem] = b.makeTraversal(self, schemaA) + override def toString: String = s"Sequence($elementSchema)" + } +} +``` + +### Optionals + +A special variant of a collection type is the `Optional[A]` type: + +```scala +object Schema extends SchemaEquality { + + final case class Optional[A](codec: Schema[A]) extends Schema[Option[A]] { + self => + + private[schema] val someCodec: Schema[Some[A]] = codec.transform(a => Some(a), _.get) + + override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = + (Prism[Option[A], Some[A]], Prism[Option[A], None.type]) + + val toEnum: Enum2[Some[A], None.type, Option[A]] = Enum2( + Case[Some[A], Option[A]]("Some", someCodec, _.asInstanceOf[Some[A]], Chunk.empty), + Case[None.type, Option[A]]("None", singleton(None), _.asInstanceOf[None.type], Chunk.empty), + Chunk.empty + ) + + override def makeAccessors(b: AccessorBuilder): (b.Prism[Option[A], Some[A]], b.Prism[Option[A], None.type]) = + b.makePrism(toEnum, toEnum.case1) -> b.makePrism(toEnum, toEnum.case2) + } + +} +``` + +### Primitives + +Last but not least, we have primitive values. + +```scala +object Schema extends SchemaEquality { + final case class Primitive[A](standardType: StandardType[A]) extends Schema[A] { + type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = Unit + + override def makeAccessors(b: AccessorBuilder): Unit = () + } +} +``` + +Primitive values are represented using the `Primitive[A]` type class and represent the elements, +that we cannot further define through other means. If we visualize our data structure as a tree, primitives are the leaves. + +ZIO Schema provides a number of built-in primitive types, that we can use to represent our data. +These can be found in the `StandardType` companion-object. + diff --git a/docs/motivation.md b/docs/motivation.md new file mode 100644 index 000000000..77470f3da --- /dev/null +++ b/docs/motivation.md @@ -0,0 +1,97 @@ +--- +id: motivation +title: "Understanding The Motivation Behind ZIO Schema" +--- + +ZIO Schema is a library used in many ZIO projects such as _ZIO Flow_, _ZIO Redis_, _ZIO Web_, _ZIO SQL_ and _ZIO DynamoDB_. It is all about reification of our types. Reification means transforming something abstract (e.g. side effects, accessing fields, structure) into something "real" (values). + +## Reification: Functional Effects + +In functional effects, we reify by turning side-effects into values. For example, we might have a simple statement like; + +```scala +println("Hello") +println("World") +``` + +In ZIO we reify this statement to a value like + +```scala +val effect1 = Task(println("Hello")) +val effect2 = Task(println("World")) +``` + +And then we are able to do awesome things like: + +```scala +(Task(println("Hello")) zipPar Task(println("World"))).retryN(100) +``` + +## Reification: Optics + +In Scala, we have product types like this case class of a `Person`: + +```scala +final case class Person(name: String, age: Int) +``` + +This case class has two fields: + +- A field `name` of type `String` +- A field `age` of type `Int` + +The Scala language provides special support to access the fields inside case classes using the dot syntax: + +```scala +val person = Person("Michelle", 32) +val name = person.name +val age = person.age +``` + +However, this is a "special language feature", it's not "real" like the side effects we've seen in the previous example (`println(..) vs. Task(println(...))`). + +Because these basic operations are not "real," we are unable to create an operator that we can use, for example, we cannot combine two fields that are inside a nested structure. + +The solution to this kind of problem is called an "Optic". Optics provide a way to access the fields of a case class and nested structures. There are three main types of optics: +- `Lens`: A lens is a way to access a field of a case class. +- `Prism`: A prism is a way to access a field of a nested structure or a collection. +- `Traversal`: A traversal is a way to access all fields of a case class, nested structures or collections. + +Optics allow us to take things which are not a first-class **concept**, and turn that into a first-class **value**, namely the concept of +- drilling down into a field inside a case class or +- drilling down into a nested structure. + +Once we have a value, we can compose these things together to solve hard problems in functional programming, e.g. +- handling nested case class copies, +- iterations down deep inside on elements of a nested structure or collections + +For more information on optics, refer to the [ZIO Optics](https://zio.dev/zio-optics/) documentation. + + +## Reification: Schema + +So far we've looked at how to +- reify side-effects into values (ZIO) +- how to reify accessing and modifying fields inside case classes or arbitrary structures by turning these operations into values as well (Optics) + +**ZIO Schema** is now about how to **describe entire data structures using values**. + +The "built-in" way in scala on how to describe data structures are `case classes` and `classes`. + +For example, assume we have the `Person` data type, like this: + +```scala +final case class Person(name: String, age: Int) +``` + +It has the following information: + +- Name of the structure: `Person` +- Fields: `name` and `age` +- Type of the fields: `String` and `Int` +- Type of the structure: `Person` + +ZIO Schema tries to reify the concept of structure for datatypes by turning the above information into values. + +Not only for case classes, but also for other types like collections, tuples, enumerations etc. + diff --git a/docs/our-first-schema.md b/docs/our-first-schema.md index b8d20903f..6a2b7eb53 100644 --- a/docs/our-first-schema.md +++ b/docs/our-first-schema.md @@ -3,12 +3,11 @@ id: our-first-schema title: "Our First Schema" --- -ZIO Schema provides macros to help you create `Schema`s out of your data types. But before using the macros, -we should take a look at how to do this the manual way. +ZIO Schema provides macros to help you create `Schema`s out of our data types. But before using the macros, we should take a look at how to do this the manual way. -### The Domain +## The Domain -Like in [Overview](index.md), we define our example domain like this: +As described in the [Overview](index.md) section, we define the example domain as follows: ```scala object Domain { @@ -28,11 +27,9 @@ object Domain { ### Manual construction of a Schema -This part is similar to other libraries that you might know, e.g. for JSON processing. -Basically, you create a `Schema` for every data type in your domain: +This part is similar to other libraries that we might know, e.g. for JSON processing. Basically, we create a `Schema` for every data type in our domain: ```scala - object ManualConstruction { import zio.schema.Schema._ import Domain._ @@ -85,7 +82,6 @@ object ManualConstruction { extractField2 = c => c.paymentMethod ) } - ``` ### Macro derivation @@ -104,11 +100,9 @@ object MacroConstruction { } ``` -## Applying it to our domain - -### Json example +## Applying it to Our Domain -Lets put this all together in a small sample: +Let's put this all together in a small sample: ```scala object JsonSample extends zio.App { @@ -129,77 +123,7 @@ object JsonSample extends zio.App { ``` When we run this, we get our expected result printed out: + ```json {"name":"Michelle","age":32} ``` - -### Protobuf example - -```scala -object ProtobufExample extends zio.App { - import zio.schema.codec.ProtobufCodec - import ManualConstruction._ - import zio.stream.ZStream - - override def run(args: List[String]): UIO[ExitCode] = for { - _ <- ZIO.unit - _ <- ZIO.debug("protobuf roundtrip") - person = Person("Michelle", 32) - - personToProto = ProtobufCodec.encoder[Person](schemaPerson) - protoToPerson = ProtobufCodec.decoder[Person](schemaPerson) - - newPerson <- ZStream(person) - .transduce(personToProto) - .transduce(protoToPerson) - .runHead - .some - .catchAll(error => ZIO.debug(error)) - _ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString) - _ <- ZIO.debug("old person: " + person) - _ <- ZIO.debug("new person: " + newPerson) - } yield ExitCode.success -} -``` - - -### Combining different encoders - -Let's take a look at a roundtrip converting an object to JSON and back, then converting it to a protobuf and back. -This is a simple example, but it shows how to combine different encoders to achieve a roundtrip. - -```scala -object CombiningExample extends zio.App { - import zio.schema.codec.JsonCodec - import zio.schema.codec.ProtobufCodec - import ManualConstruction._ - import zio.stream.ZStream - - override def run(args: List[String]): UIO[ExitCode] = for { - _ <- ZIO.unit - _ <- ZIO.debug("combining roundtrip") - person = Person("Michelle", 32) - - personToJson = JsonCodec.encoder[Person](schemaPerson) - jsonToPerson = JsonCodec.decoder[Person](schemaPerson) - - personToProto = ProtobufCodec.encoder[Person](schemaPerson) - protoToPerson = ProtobufCodec.decoder[Person](schemaPerson) - - newPerson <- ZStream(person) - .tap(v => ZIO.debug("input object is: " + v)) - .transduce(personToJson) - .transduce(jsonToPerson) - .tap(v => ZIO.debug("object after json roundtrip: " + v)) - .transduce(personToProto) - .transduce(protoToPerson) - .tap(v => ZIO.debug("person after protobuf roundtrip: " + v)) - .runHead - .some - .catchAll(error => ZIO.debug(error)) - _ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString) - _ <- ZIO.debug("old person: " + person) - _ <- ZIO.debug("new person: " + newPerson) - } yield ExitCode.success -} -``` diff --git a/docs/protobuf-example.md b/docs/protobuf-example.md new file mode 100644 index 000000000..5e7591aa5 --- /dev/null +++ b/docs/protobuf-example.md @@ -0,0 +1,31 @@ +--- +id: protobuf-example +title: "Protobuf Example" +--- + +```scala +object ProtobufExample extends zio.App { + import zio.schema.codec.ProtobufCodec + import ManualConstruction._ + import zio.stream.ZStream + + override def run(args: List[String]): UIO[ExitCode] = for { + _ <- ZIO.unit + _ <- ZIO.debug("protobuf roundtrip") + person = Person("Michelle", 32) + + personToProto = ProtobufCodec.encoder[Person](schemaPerson) + protoToPerson = ProtobufCodec.decoder[Person](schemaPerson) + + newPerson <- ZStream(person) + .transduce(personToProto) + .transduce(protoToPerson) + .runHead + .some + .catchAll(error => ZIO.debug(error)) + _ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString) + _ <- ZIO.debug("old person: " + person) + _ <- ZIO.debug("new person: " + newPerson) + } yield ExitCode.success +} +``` diff --git a/docs/sidebars.js b/docs/sidebars.js index 102859d51..3d3940693 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -6,9 +6,14 @@ const sidebars = { collapsed: false, link: { type: "doc", id: "index" }, items: [ - 'use-cases', - 'our-first-schema', - 'understanding-zio-schema', + "use-cases", + "our-first-schema", + "motivation", + "getting-started", + "transforming-schemas", + "codecs", + "protobuf-example", + "combining-different-encoders" ] } ] diff --git a/docs/transforming-schemas.md b/docs/transforming-schemas.md new file mode 100644 index 000000000..40d1263e8 --- /dev/null +++ b/docs/transforming-schemas.md @@ -0,0 +1,33 @@ +--- +id: transforming-schemas +title: "Transforming Schemas" +--- + +Once we have a `Schema`, we can transform it into another `Schema` by applying a `Transformer`. In normal Scala code this would be the equivalent of `map`. + +In ZIO Schema this is modelled by the `Transform` type class: + +```scala + final case class Transform[A, B](codec: Schema[A], f: A => Either[String, B], g: B => Either[String, A]) + extends Schema[B] { + override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = codec.Accessors[Lens, Prism, Traversal] + + override def makeAccessors(b: AccessorBuilder): codec.Accessors[b.Lens, b.Prism, b.Traversal] = + codec.makeAccessors(b) + + override def serializable: Schema[Schema[_]] = Meta(SchemaAst.fromSchema(codec)) + override def toString: String = s"Transform($codec)" + } +``` + +In the previous example, we can transform the `User` Schema into a `UserRecord` Schema, which is a record, by using the `transform` method, which has to be an "isomorphism" (= providing methods to transform A to B _and_ B to A): + +```scala +/** + * Transforms this `Schema[A]` into a `Schema[B]`, by supplying two functions that can transform + * between `A` and `B`, without possibility of failure. + */ +def transform[B](f: A => B, g: B => A): Schema[B] = + Schema.Transform[A, B](self, a => Right(f(a)), b => Right(g(b))) +``` + diff --git a/docs/understanding-zio-schema.md b/docs/understanding-zio-schema.md deleted file mode 100644 index 0dd6e7999..000000000 --- a/docs/understanding-zio-schema.md +++ /dev/null @@ -1,335 +0,0 @@ ---- -id: understanding-zio-schema -title: "Understanding ZIO Schema" ---- - -ZIO Schema is a library used in many ZIO projects such as _ZIO Flow_, _ZIO Redis_, _ZIO Web_, _ZIO SQL_ and _ZIO DynamoDB_. -ZIO is all about reification of your types. Reification means transforming something abstract (e.g. side effects, accessing fields, structure) into something "real" (values). - -## Reification: Functional Effects - -In functional effects, we reify by turning side-effects into values. - -E.g. you might have a simple statement like -```scala -println("Hello") -println("World") -``` -and in ZIO we reify this statement to a value like - -```scala -val effect1 = Task(println("Hello")) -val effect2 = Task(println("World")) -``` - -and then are able to do awesome things like: -```scala -(Task(println("Hello")) zipPar Task(println("World"))).retryN(100) -``` - -## Reification: Optics - -In scala we have product types like this case class of a Person: -```scala -final case class Person(name: String, age: Int) -``` -This case class has two fields: -- a field "name" of type `String` -- a field "age" of type `Int` - -The Scala language provides special support to access the fields inside case classes using the dot syntax: - -```scala -val person = Person("Michelle", 32) -val name = person.name -val age = person.age -``` - -However, this is a "special language feature", it's not "real" like the side effects we've seen in the previous example ( `println(..) vs. Task`println(...)))` ). - -Because these basic operations are not "real", we're unable to create an operator that we can use to -e.g. combine two fields that are inside a nested structure. - -The solution to this kind of problem is called an "Optic". Optics provide a way to access the fields of a case class and nested structures. -There are three main types of optics: -- `Lens`: A lens is a way to access a field of a case class. -- `Prism`: A prism is a way to access a field of a nested structure or a collection. -- `Traversal`: A traversal is a way to access all fields of a case class, nested structures or collections. - -Optics allow us to take things which are not a first-class **concept**, and turn that into a first-class **value**, -namely the concept of -- drilling down into a field inside a case class or -- drilling down into a nested structure. - -Once we have a value, we can compose these things together to solve hard problems in functional programming, e.g. -- handling nested case class copies, -- iterations down deep inside on elements of a nested structure or collections - -For more information on optics, refer to the [ZIO Optics](https://zio.github.io/zio-optics/docs/overview/overview_index) documentation. - - -## Reification: Schema - -So far we've looked at how to -- reify side-effects into values (ZIO) -- how to reify accessing + modifying fields inside case classes or arbitrary structures by turning these operations into values as well (Optics) - -ZIO Schema is now about how to describe entire data structures using values. - -The "built-in" way in scala on how to describe data structures are `case classes` and `classes`. - -E.g. the following data type: -```scala -final case class Person(name: String, age: Int) -``` - -Has the following information: -- name of the structure: "Person" -- fields: "name" and "age" -- type of the fields: String and Int -- type of the structure: Person - -ZIO Schema tries to reify the concept of structure for datatypes by turning the above information into values. - -Not only for case classes, but also for other types like collections, tuples, enumerations etc. - -## Getting started - -To get started, first you need to understand that a ZIO Schema is basically built-up from these three -sealed traits: -- `Record[R]` -- `Enum[A]` -- `Sequence[Col, Elem]` - and the case class `Primitive[A]`. Every other type is just a specialisation of one of these (or not relevant to get you started). - -We will take a look at them now. - -### Basic Building Blocks - -#### Schema - -The core data type of ZIO Schema is a `Schema[A]` which is **invariant in `A`** by necessity, because a Schema allows you to -derive operations that produce an `A` but also operations that consume an `A` and that imposes limitations on the types of -**transformation operators** and **composition operators** that we can provide based on a `Schema`. - -It looks kind of like this (simplified): -```scala -sealed trait Schema[A] { - self => - type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] - - /** - * A symbolic operator for [[optional]]. - */ - def ? : Schema[Option[A]] = self.optional - - def makeAccessors(b: AccessorBuilder): Accessors[b.Lens, b.Prism, b.Traversal] - - /** - * Transforms this `Schema[A]` into a `Schema[B]`, by supplying two functions that can transform - * between `A` and `B`, without possibility of failure. - */ - def transform[B](f: A => B, g: B => A): Schema[B] = - Schema.Transform[A, B](self, a => Right(f(a)), b => Right(g(b))) - - def transformOrFail[B](f: A => Either[String, B], g: B => Either[String, A]): Schema[B] = - Schema.Transform[A, B](self, f, g) - - def zip[B](that: Schema[B]): Schema[(A, B)] = Schema.Tuple(self, that) -} -``` - -#### Records - -Your data structures usually are composed from a lot of types. For example, you might have a `User` -type that has a `name` field, an `age` field, an `address` field, and a `friends` field. - -```scala -case class User(name: String, age: Int, address: Address, friends: List[User]) -``` - -This is called a "product type" in functional programming. -The equivalent of a product type in ZIO Schema is called a record. - -In ZIO Schema such a record would be represented using the `Record[R]` typeclass: - -```scala -object Schema { - sealed trait Record[R] extends Schema[R] { - def structure: Chunk[Field[_]] - def annotations: Chunk[Any] = Chunk.empty - def rawConstruct(values: Chunk[Any]): Either[String, R] - } -} - -``` - -#### Enumerations - -Other times, you might have a type that represents a list of different types. For example, you might -have a type, like this: -```scala - sealed trait PaymentMethod - - object PaymentMethod { - final case class CreditCard(number: String, expirationMonth: Int, expirationYear: Int) extends PaymentMethod - final case class WireTransfer(accountNumber: String, bankCode: String) extends PaymentMethod - } -``` - -In functional programming, this kind of type is called a "sum type". -In Scala2, this is called a "sealed trait". -In Scala3, this is called an "enum". - -In ZIO Schema we call these types `enumeration` types and they are -represented using the `Enum[A]` type class. - -```scala -object Schema ... { - sealed trait Enum[A] extends Schema[A] { - def annotations: Chunk[Any] - def structure: ListMap[String, Schema[_]] - } -} -``` - -#### Sequence - -Often you have a type that is a collection of elements. For example, you might have a `List[User]`. -This is called a `Sequence` and is represented using the `Sequence[Col, Elem]` type class: - -```scala -object Schema ... { - ... - - final case class Sequence[Col, Elem]( - elementSchema: Schema[Elem], - fromChunk: Chunk[Elem] => Col, - toChunk: Col => Chunk[Elem] - ) extends Schema[Col] { - self => - override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = Traversal[Col, Elem] - override def makeAccessors(b: AccessorBuilder): b.Traversal[Col, Elem] = b.makeTraversal(self, schemaA) - override def toString: String = s"Sequence($elementSchema)" - } - ... -} -``` - -#### Optionals - -A special variant of a collection type is the `Optional[A]` type: - -```scala -object Schema ... { - - final case class Optional[A](codec: Schema[A]) extends Schema[Option[A]] { - self => - - private[schema] val someCodec: Schema[Some[A]] = codec.transform(a => Some(a), _.get) - - override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = - (Prism[Option[A], Some[A]], Prism[Option[A], None.type]) - - val toEnum: Enum2[Some[A], None.type, Option[A]] = Enum2( - Case[Some[A], Option[A]]("Some", someCodec, _.asInstanceOf[Some[A]], Chunk.empty), - Case[None.type, Option[A]]("None", singleton(None), _.asInstanceOf[None.type], Chunk.empty), - Chunk.empty - ) - - override def makeAccessors(b: AccessorBuilder): (b.Prism[Option[A], Some[A]], b.Prism[Option[A], None.type]) = - b.makePrism(toEnum, toEnum.case1) -> b.makePrism(toEnum, toEnum.case2) - } - -} -``` - -#### Primitives - -Last but not least, we have primitive values. - -```scala -object Schema ... { - ... - final case class Primitive[A](standardType: StandardType[A]) extends Schema[A] { - type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = Unit - - override def makeAccessors(b: AccessorBuilder): Unit = () - } - ... -} -``` - -Primitive values are represented using the `Primitive[A]` type class and represent the elements, -that we cannot further define through other means. If you visualize your data structure as a tree, -primitives are the leaves. - -ZIO Schema provides a number of built-in primitive types, that you can use to represent your data. -These can be found in the `StandardType` companion-object. - -### Transforming Schemas - -Once we have a `Schema`, we can transform it into another `Schema` by applying a `Transformer`. -In normal Scala code this would be the equivalent of `map`. - -In ZIO Schema this is modelled by the `Transform` type class: - -```scala - final case class Transform[A, B](codec: Schema[A], f: A => Either[String, B], g: B => Either[String, A]) - extends Schema[B] { - override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = codec.Accessors[Lens, Prism, Traversal] - - override def makeAccessors(b: AccessorBuilder): codec.Accessors[b.Lens, b.Prism, b.Traversal] = - codec.makeAccessors(b) - - override def serializable: Schema[Schema[_]] = Meta(SchemaAst.fromSchema(codec)) - override def toString: String = s"Transform($codec)" - } -``` - -In the example above, we can transform the `User` Schema into a `UserRecord` Schema, which is a record, -by using the `transform`-method, which has to be an "isomorphism" (= providing methods to transform A to B _and_ B to A): - -```scala - /** - * Transforms this `Schema[A]` into a `Schema[B]`, by supplying two functions that can transform - * between `A` and `B`, without possibility of failure. - */ - def transform[B](f: A => B, g: B => A): Schema[B] = - Schema.Transform[A, B](self, a => Right(f(a)), b => Right(g(b))) -``` - -#### Codecs - -Once you have your schema, you can combine it with a codec. -A codec is a combination of a schema and a serializer. -Unlike codecs in other libraries, a codec in ZIO Schema has no type parameter. - -```scala - -trait Codec { - def encoder[A](schema: Schema[A]): ZTransducer[Any, Nothing, A, Byte] - def decoder[A](schema: Schema[A]): ZTransducer[Any, String, Byte, A] - - def encode[A](schema: Schema[A]): A => Chunk[Byte] - def decode[A](schema: Schema[A]): Chunk[Byte] => Either[String, A] -} - -``` - -It basically says: -`encoder[A]`: Given a `Schema[A]` it is capable of generating an `Encoder[A]` ( `A => Chunk[Byte]`) for any Schema. -`decoder[A]`: Given a `Schema[A]` it is capable of generating a `Decoder[A]` ( `Chunk[Byte] => Either[String, A]`) for any Schema. - - -Example of possible codecs are: - - - CSV Codec - - JSON Codec (already available) - - Apache Avro Codec (in progress) - - Apache Thrift Codec (in progress) - - XML Codec - - YAML Codec - - Protobuf Codec (already available) - - QueryString Codec - - etc. diff --git a/docs/use-cases.md b/docs/use-cases.md index ba7f69df1..099585690 100644 --- a/docs/use-cases.md +++ b/docs/use-cases.md @@ -4,15 +4,15 @@ title: "ZIO Schema Use cases" sidebar_label: "Use cases" --- -ZIO Schema allows you to create representations of your data types as values. +ZIO Schema allows us to create representations of our data types as values. -Once you have a representation of your data types, you can use it to - - serialize and deserialize your types - - validate your types - - transform your types - - create instances of your types +Once we have a representation of our data types, we can use it to + - Serialize and deserialize our types + - Validate our types + - Transform our types + - Create instances of your types -You can then use one of the various codecs (or create your own) to serialize and deserialize your types. +We can then use one of the various codecs (or create our own) to serialize and deserialize your types. Example of possible codecs are: @@ -39,8 +39,8 @@ Example use cases that are possible: - Creating diffs from arbitrary data structures - Creating migrations / evolutions e.g. of Events used in Event-Sourcing - Transformation pipelines, e.g. - 1. convert from protobuf to object, e.g. `PersonDTO`, - 2. transform to another representation, e.g. `Person`, - 3. validate - 4. transform to JSON `JsonObject` - 5. serialize to `String` + 1. Convert from protobuf to object, e.g. `PersonDTO`, + 2. Transform to another representation, e.g. `Person`, + 3. Validate + 4. Transform to JSON `JsonObject` + 5. Serialize to `String`