Skip to content

Commit

Permalink
refactor docs.
Browse files Browse the repository at this point in the history
  • Loading branch information
khajavi committed Jul 5, 2023
1 parent 651ad37 commit 9733f83
Show file tree
Hide file tree
Showing 10 changed files with 408 additions and 433 deletions.
32 changes: 32 additions & 0 deletions docs/codecs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
id: codecs
title: "Codecs"
---

Once we have our schema, we can combine it with a codec. A codec is a combination of a schema and a serializer. Unlike codecs in other libraries, a codec in ZIO Schema has no type parameter:

```scala
trait Codec {
def encoder[A](schema: Schema[A]): ZTransducer[Any, Nothing, A, Byte]
def decoder[A](schema: Schema[A]): ZTransducer[Any, String, Byte, A]

def encode[A](schema: Schema[A]): A => Chunk[Byte]
def decode[A](schema: Schema[A]): Chunk[Byte] => Either[String, A]
}
```

It basically says:
- `encoder[A]`: Given a `Schema[A]` it is capable of generating an `Encoder[A]` ( `A => Chunk[Byte]`) for any Schema.
- `decoder[A]`: Given a `Schema[A]` it is capable of generating a `Decoder[A]` ( `Chunk[Byte] => Either[String, A]`) for any Schema.

Example of possible codecs are:

- CSV Codec
- JSON Codec (already available)
- Apache Avro Codec (in progress)
- Apache Thrift Codec (in progress)
- XML Codec
- YAML Codec
- Protobuf Codec (already available)
- QueryString Codec
- etc.
42 changes: 42 additions & 0 deletions docs/combining-different-encoders.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
id: combining-different-encoders
title: "Combining Different Encoders"
---

Let's take a look at a round-trip converting an object to JSON and back, then converting it to a protobuf and back. This is a simple example, but it shows how to combine different encoders to achieve a round-trip.

```scala
object CombiningExample extends zio.App {
import zio.schema.codec.JsonCodec
import zio.schema.codec.ProtobufCodec
import ManualConstruction._
import zio.stream.ZStream

override def run(args: List[String]): UIO[ExitCode] = for {
_ <- ZIO.unit
_ <- ZIO.debug("combining roundtrip")
person = Person("Michelle", 32)

personToJson = JsonCodec.encoder[Person](schemaPerson)
jsonToPerson = JsonCodec.decoder[Person](schemaPerson)

personToProto = ProtobufCodec.encoder[Person](schemaPerson)
protoToPerson = ProtobufCodec.decoder[Person](schemaPerson)

newPerson <- ZStream(person)
.tap(v => ZIO.debug("input object is: " + v))
.transduce(personToJson)
.transduce(jsonToPerson)
.tap(v => ZIO.debug("object after json roundtrip: " + v))
.transduce(personToProto)
.transduce(protoToPerson)
.tap(v => ZIO.debug("person after protobuf roundtrip: " + v))
.runHead
.some
.catchAll(error => ZIO.debug(error))
_ <- ZIO.debug("is old person the new person? " + (person == newPerson).toString)
_ <- ZIO.debug("old person: " + person)
_ <- ZIO.debug("new person: " + newPerson)
} yield ExitCode.success
}
```
146 changes: 146 additions & 0 deletions docs/getting-started.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
---
id: getting-started
title: "Getting Started"
---

To get started, first we need to understand that a ZIO Schema is basically built-up from these three
sealed traits: `Record[R]`, `Enum[A]` and `Sequence[Col, Elem]`, along with the case class `Primitive[A]`. Every other type is just a specialisation of one of these (or not relevant to get you started).

We will take a look at them now.

## Basic Building Blocks

### Schema

The core data type of ZIO Schema is a `Schema[A]` which is **invariant in `A`** by necessity, because a Schema allows us to derive operations that produce an `A` but also operations that consume an `A` and that imposes limitations on the types of **transformation operators** and **composition operators** that we can provide based on a `Schema`.

It looks kind of like this (simplified):

```scala
sealed trait Schema[A] { self =>
def zip[B](that: Schema[B]): Schema[(A, B)]

def transform[B](f: A => B, g: B => A): Schema[B]
}
```

### Records

Our data structures usually are composed of a lot of types. For example, we might have a `User`
type that has a `name` field, an `age` field, an `address` field, and a `friends` field.

```scala
case class User(name: String, age: Int, address: Address, friends: List[User])
```

This is called a **product type** in functional programming. The equivalent of a product type in ZIO Schema is called a record.

In ZIO Schema such a record would be represented using the `Record[R]` typeclass:

```scala
object Schema {
sealed trait Record[R] extends Schema[R] {
def fields: Chunk[Field[_]]
def construct(value: R): Chunk[Any]
def defaultValue: Either[String, R]
}
}

```

### Enumerations

Other times, you might have a type that represents a list of different types. For example, we might have a type, like this:

```scala
sealed trait PaymentMethod

object PaymentMethod {
final case class CreditCard(number: String, expirationMonth: Int, expirationYear: Int) extends PaymentMethod
final case class WireTransfer(accountNumber: String, bankCode: String) extends PaymentMethod
}
```

In functional programming, this kind of type is called a **sum type**:
- In Scala 2, this is called a **sealed trait**.
- In Scala3, this is called an **enum**.

In ZIO Schema we call these types `enumeration` types, and they are represented using the `Enum[A]` type class.

```scala
object Schema extends SchemaEquality {
sealed trait Enum[A] extends Schema[A] {
def annotations: Chunk[Any]
def structure: ListMap[String, Schema[_]]
}
}
```

### Sequence

Often you have a type that is a collection of elements. For example, you might have a `List[User]`. This is called a `Sequence` and is represented using the `Sequence[Col, Elem]` type class:

```scala
object Schema extends SchemaEquality {

final case class Sequence[Col, Elem](
elementSchema: Schema[Elem],
fromChunk: Chunk[Elem] => Col,
toChunk: Col => Chunk[Elem]
) extends Schema[Col] {
self =>
override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = Traversal[Col, Elem]
override def makeAccessors(b: AccessorBuilder): b.Traversal[Col, Elem] = b.makeTraversal(self, schemaA)
override def toString: String = s"Sequence($elementSchema)"
}
}
```

### Optionals

A special variant of a collection type is the `Optional[A]` type:

```scala
object Schema extends SchemaEquality {

final case class Optional[A](codec: Schema[A]) extends Schema[Option[A]] {
self =>

private[schema] val someCodec: Schema[Some[A]] = codec.transform(a => Some(a), _.get)

override type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] =
(Prism[Option[A], Some[A]], Prism[Option[A], None.type])

val toEnum: Enum2[Some[A], None.type, Option[A]] = Enum2(
Case[Some[A], Option[A]]("Some", someCodec, _.asInstanceOf[Some[A]], Chunk.empty),
Case[None.type, Option[A]]("None", singleton(None), _.asInstanceOf[None.type], Chunk.empty),
Chunk.empty
)

override def makeAccessors(b: AccessorBuilder): (b.Prism[Option[A], Some[A]], b.Prism[Option[A], None.type]) =
b.makePrism(toEnum, toEnum.case1) -> b.makePrism(toEnum, toEnum.case2)
}

}
```

### Primitives

Last but not least, we have primitive values.

```scala
object Schema extends SchemaEquality {
final case class Primitive[A](standardType: StandardType[A]) extends Schema[A] {
type Accessors[Lens[_, _], Prism[_, _], Traversal[_, _]] = Unit

override def makeAccessors(b: AccessorBuilder): Unit = ()
}
}
```

Primitive values are represented using the `Primitive[A]` type class and represent the elements,
that we cannot further define through other means. If we visualize our data structure as a tree, primitives are the leaves.

ZIO Schema provides a number of built-in primitive types, that we can use to represent our data.
These can be found in the `StandardType` companion-object.

97 changes: 97 additions & 0 deletions docs/motivation.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
---
id: motivation
title: "Understanding The Motivation Behind ZIO Schema"
---

ZIO Schema is a library used in many ZIO projects such as _ZIO Flow_, _ZIO Redis_, _ZIO Web_, _ZIO SQL_ and _ZIO DynamoDB_. It is all about reification of our types. Reification means transforming something abstract (e.g. side effects, accessing fields, structure) into something "real" (values).

## Reification: Functional Effects

In functional effects, we reify by turning side-effects into values. For example, we might have a simple statement like;

```scala
println("Hello")
println("World")
```

In ZIO we reify this statement to a value like

```scala
val effect1 = Task(println("Hello"))
val effect2 = Task(println("World"))
```

And then we are able to do awesome things like:

```scala
(Task(println("Hello")) zipPar Task(println("World"))).retryN(100)
```

## Reification: Optics

In Scala, we have product types like this case class of a `Person`:

```scala
final case class Person(name: String, age: Int)
```

This case class has two fields:

- A field `name` of type `String`
- A field `age` of type `Int`

The Scala language provides special support to access the fields inside case classes using the dot syntax:

```scala
val person = Person("Michelle", 32)
val name = person.name
val age = person.age
```

However, this is a "special language feature", it's not "real" like the side effects we've seen in the previous example (`println(..) vs. Task(println(...))`).

Because these basic operations are not "real," we are unable to create an operator that we can use, for example, we cannot combine two fields that are inside a nested structure.

The solution to this kind of problem is called an "Optic". Optics provide a way to access the fields of a case class and nested structures. There are three main types of optics:
- `Lens`: A lens is a way to access a field of a case class.
- `Prism`: A prism is a way to access a field of a nested structure or a collection.
- `Traversal`: A traversal is a way to access all fields of a case class, nested structures or collections.

Optics allow us to take things which are not a first-class **concept**, and turn that into a first-class **value**, namely the concept of
- drilling down into a field inside a case class or
- drilling down into a nested structure.

Once we have a value, we can compose these things together to solve hard problems in functional programming, e.g.
- handling nested case class copies,
- iterations down deep inside on elements of a nested structure or collections

For more information on optics, refer to the [ZIO Optics](https://zio.dev/zio-optics/) documentation.


## Reification: Schema

So far we've looked at how to
- reify side-effects into values (ZIO)
- how to reify accessing and modifying fields inside case classes or arbitrary structures by turning these operations into values as well (Optics)

**ZIO Schema** is now about how to **describe entire data structures using values**.

The "built-in" way in scala on how to describe data structures are `case classes` and `classes`.

For example, assume we have the `Person` data type, like this:

```scala
final case class Person(name: String, age: Int)
```

It has the following information:

- Name of the structure: `Person`
- Fields: `name` and `age`
- Type of the fields: `String` and `Int`
- Type of the structure: `Person`

ZIO Schema tries to reify the concept of structure for datatypes by turning the above information into values.

Not only for case classes, but also for other types like collections, tuples, enumerations etc.

Loading

0 comments on commit 9733f83

Please sign in to comment.