Data contracts and transformations with io-ts

Introduction

TypeScript established itself as the tool to achieve type checking in JavaScript projects. Many developers love the safety it provides by preventing the annoying TypeError: Cannot read property 'bar' of undefined errors (or for Firefox users: TypeError: foo is undefined) and the like. The compile-time approach perfectly works within the TypeScript world, but there are still situations where this safety may be undermined. For example when receiving data from an API that does not match our assumptions or when reading a configuration file.

A common solution to this problem is schema validation. There are libraries like joi, ajv or validate.js that do this. io-ts on the other hand (the library I’m going to depict in this article), is in a way related to these validation libraries but with a somehow different twist:

• It colludes perfectly with TypeScript.
• It has a focus on decoding and encoding data.
• It strongly uses functional programming paradigms.

The latter aspect certainly comes from the fact that its author, Giulio Canti, is an italian mathematician who also built the functional programming library fp-ts. You’ll be interfering with fp-ts‘ algebraic data types when using io-ts.

It is important to understand, that io-ts does not prevent any of the described errors, but it pushes them out to the runtime boundaries – as close to its root as possible. This allows to fail in a controlled fashion with an error description that is very clear about the actual problem.

Define io-ts types

Disclaimer: All of the following code examples are written for io-ts/fp-ts >= 2.0.0.

Let’s jump right in and define an io-ts run-time type for a user model:

import * as t from 'io-ts'

const User = t.type({
  id: t.number,
  name: t.string,
  username: t.string,
  email: t.string
});

The common TypeScript developer should be very familar with this declaration. Its corresponding Typescript definition would almost look the same, except for the type keyword/function and the t.’s.

io-ts types may also be composed:

const Address = t.type({
  street: t.string,
  city: t.string,
  zipcode: t.string
});

const User = t.type({
  id: t.number,
  name: t.string,
  username: t.string,
  email: t.string,
  address: Address
});

And as you’d expect, there is a whole range of types and type combinators such as t.boolean, t.array and t.union([...]) (the io-ts documentation contains a full list of the implemented types).

Derive TypeScript types

Having defined a run-time type, we can now derive the TypeScript type from it:

// io-ts type
const User = t.type({
  id: t.number,
  name: t.string,
  username: t.string,
  email: t.string
});

// TypeScript type
type UserType = t.TypeOf<typeof User>;

That’s a huge win, since we don’t have to maintain the type information twice.

As a best practice, I’d suggest to name the run-time and TypeScript types equally and create an overloaded export:

const User = t.type({ /* Type definition */ });
type User = t.TypeOf<typeof User>;
export { User };

Like this, you can use the exported type very comfortably either as TypeScript type or as JavaScript value (depending on the context), without having to think about it:

import { User } from './user.model';

// Use as TypeScript type
const user: User = { /* Some data */ };

// Use as JavaScript value
const codec = User;

Note: There are io-ts types that cannot be represented in TypeScript (t.recursion, t.brand, t.Int, t.exact and t.strict), so you want to avoid these.

Decoding data

The process of decoding data with io-ts consists of two steps:

• Check that the given data complies with the type definition.
• Transform (or deserialize) the given data if desired.

We are going to focus on the second aspect in the next section.

Let us decode some data we’ve received from the backend, by using the decode function of the io-ts type:

import { User } from './user.model';

const json: unknown = { /* Some data */ };
const result = User.decode(json);

That function call returns an Either type, which is an algebraic data type from the fp-ts library. It represents a value of one of two possible types. In our case it is either a failure if the data does not conform with the given io-ts type, or the successfully decoded and transformed value. These two values may also be referred as the left value (the failure) and the right value (the successful value) to follow functional programming lingo.

At this point you may keep on working with the algebraic data types and their monadic functions. Or – as we’ll do for the sake of simplicity – convert the result to an Observable and follow probably more familiar paths (at least for the Angular-seasoned reader).

Convert the Either to an Observable

At the risk of provoking a „How to draw an owl“ moment, let me introduce a decode utility function that decodes some data and returns an Observable which emits the value if successful, or an error otherwise:

import * as t from "io-ts/lib/index";
import { pipe } from "fp-ts/lib/pipeable";
import { fold, left } from "fp-ts/lib/Either";
import { PathReporter } from "io-ts/lib/PathReporter";
import { Observable, throwError, of } from "rxjs";

export class DecodeError extends Error {
  name = "DecodeError";
}

function decode<C extends t.Mixed>(
  codec: C
): (json: unknown) => Observable<t.TypeOf<C>> {
  return json => {
    return pipe(
      codec.decode(json),
      fold(
        error =>
          throwError(
            new DecodeError(PathReporter.report(left(error)).join("\n"))
          ),
        data => of(data)
      )
    );
  };
}

😲😕🤔 Confused? The returned function decodes the given data and basically unboxes the value of the Either type and wraps it into an Observable. In case of an error, the built-in PathReporter is used to generate a human readable error string.

But look at the following example to see how straightforward the usage of this utility is:

import { decode } from './utils/decode';
import { User } from './user.model';

const data: unknown = { /* Some data */ };
decode(User)(data).subscribe(
  user => console.log(user),
  error => console.error(error)
);

Or even in the context of an Angular service that fetches data over HTTP:

import { Injectable } from "@angular/core";
import { HttpClient } from "@angular/common/http";
import { Observable } from "rxjs";
import { switchMap } from "rxjs/operators";
import { decode } from "./utils/decode";
import { User } from "./user.model";
@Injectable({
  providedIn: "root"
})
export class UserService {
  constructor(private http: HttpClient) {}
  get(id: number): Observable {
    return this.http
      .get(`/api/users/${id}`)
      .pipe(switchMap(decode(User)));
  }
}

Easy, right? 😎

Data transformations

io-ts gives the possibility to create custom types. These so called codecs can implement custom type guards and type transformations (i.e. de-/serializations). A typical use case is when dealing with dates, which often are served by the API as ISO 8601 date strings. The following is an example from the io-ts documentation that converts date strings to Date objects and vice versa:

import { either } from 'fp-ts/lib/Either'

// Converts a Date from an ISO string
const DateFromString = new t.Type<Date, string, unknown>(
  'DateFromString',
  (u): u is Date => u instanceof Date, (u, c) =>
    either.chain(t.string.validate(u, c), s => {
      const d = new Date(s);
      return isNaN(d.getTime()) ? t.failure(u, c) : t.success(d);
    }),
    a => a.toISOString()
);

You can use a custom type just like any other io-ts type:

const User = t.type({
  id: t.number,
  name: t.string,
  lastLogin: DateFromString
});

Before writing a custom type, I’d suggest to check out the io-ts-types project. It already provides a whole bunch of custom types, such as DateFromISOString or NumberFromString.

A custom type can also be a nice way to work around awful quirks of 3rd party APIs.

Encoding data

We’ve seen how to decode data received from an API, let’s now encode data that we want to send to the API. Based on the previous Angular service example, we can call the io-ts type’s encode function as follows:

import { Injectable } from "@angular/core";
import { HttpClient } from "@angular/common/http";
import { Observable } from "rxjs";
import { switchMap } from "rxjs/operators";
import { User } from "./user.model";
@Injectable({
  providedIn: "root"
})
export class UserService {
  constructor(private http: HttpClient) {}
  create(user: User): Observable {
    return this.http
      .post("/api/users", User.encode(user))
      .pipe(switchMap(decode(User)));
  }
}

No rocket science so far… 🚀

Create a generalized Angular REST service

Finally, let’s condense all aspects from the previous sections into an abstract Angular service, that can be initialized with an io-ts type and then use this codec to decode the data received from an API:

import { HttpClient } from "@angular/common/http";
import { Observable } from "rxjs";
import { switchMap } from "rxjs/operators";
import * as t from "io-ts/lib/index";
import { decode } from "./utils/decode";

export abstract class RestService<T extends t.Mixed> {
  constructor(
    protected http: HttpClient,
    protected codec: T,
    protected resourcePath: string
  ) {}

  get(id: number): Observable<t.TypeOf<T>> {
    return this.http
      .get(`${this.baseUrl}/${id}`)
      .pipe(switchMap(decode(this.codec)));
  }

  getList(): Observable<ReadonlyArray<t.TypeOf<T>>> {
    return this.http
      .get(this.baseUrl)
      .pipe(switchMap(decode(t.array(this.codec))));
  }

  protected get baseUrl(): string {
    return `/api/${this.resourcePath}`;
  }
}

A concrete implementation of this service (using the User type from the previous sections) only takes a few lines of code:

import { Injectable } from "@angular/core";
import { HttpClient } from "@angular/common/http";
import { RestService } from "./rest.service";
import { User } from "./user.model";
@Injectable({
  providedIn: "root"
})
export class UserRestService extends RestService {
  constructor(http: HttpClient) {
    super(http, User, "users");
  }
}

You can check out a full CRUD version of the above RestService on GitHub.

Conclusion

We are very happily using io-ts in a project where we rely on a 3rd party API with vague specifications. It gives the confidence that unexpected data from the API will definitely be noticed very early on and eliminates hard to detect bugs that would be caused by such data.
Additionally, we’ve been able to work around some issues of the API by using data transformations to keep the internal interface clean.

Before closing, there is one more best practice I’d suggest: only be as strict as necessary. That means, your io-ts types should only include properties your application relies on, to avoid decoding errors of properties that are not even used. The „untyped“ properties are in fact still available on the decoded object, but TypeScript prevents you from accessing them.

Happy hacking!