Building an API

Peng Ying12/18/2023 ⋅ 8 min read

Overview

So you want to create a new world class API that differentiates you from your competitors. I hope that I can shed a bit of light on the topic. But first, let's take a step back. An API is just a piece of the puzzle it's a definition of the contract between yourself and your integrators but there's more than just the contract, there's documentation, sample code, client libraries, tools and more that can help differentiate your product. I like to think of it as an end to end integration developer experience.

Let's get into it and discuss all of the attributes of a great developer experience, API design and define a framework for creating APIs. This series of blog posts will focus on web APIs. Building a local API like interfacing with hardware or hooks for a platform framework have a similar but separate set of considerations. Lets start with what makes a great developer experience to define our goal then jump into the framework.

So what is a great developer experience? I think the easiest way to describe many of the attributes of a great experince is with a few examples. My personal gold standards(for different reasons) are Stripe, MDN, and Flutter.

Developer Experience Examples

Stripe

Stripe has democratized accepting card payments online. While on the surface card payments may seem simple, enabling it can actually be incredibly complex. There's a number of rules around securing card data, what can be stored where. A number of card networks beyond the big 4. There's both customer and merchant fraud and chargebacks and many more complexities. Stripe has simplifed that mostly down to copy and paste snippets.

  const session = await stripe.checkout.sessions.create({
    line_items: [
      {
        // Provide the exact Price ID (for example, pr_1234) of the product you want to sell
        price: '{{PRICE_ID}}',
        quantity: 1,
      },
    ],
    mode: 'payment',
    success_url: `${YOUR_DOMAIN}/success.html`,
    cancel_url: `${YOUR_DOMAIN}/cancel.html`,
  });

  res.redirect(303, session.url);

1. A great developer experience(and product) starts with simplifing something complex.

In addition Stripe innovated in documentation creating the 3 column layout. They understood the pain points of the existing documentation experience and improved upon it eg examples of requests in reference documentation instead of just a description of the field. Stripe docs are a night and day difference from my starting documentation format Javadocs (I still <3 GSON, worked with Joel and Inderjeet while in the payments org at Google).

Stripe's 3 column layout

2. Understanding developer needs and pain points and making it simple to understand the product.

The last major Stripe superlative for me is they have great tooling for simulation and testing. As an integrator you can test various card types, trigger specific flows, and test all of the edge cases that your implementation needs to handle. Imagine how painful it would be if you couldn't test some functionality of your product until you saw the case occur in production.

3. Provides testing and simulation tools to trigger all relevant use cases and flows.

Mozilla

Mozilla is my defacto reference for web APIs. Their breadth and depth of web documentation in my mind is unparalleled. In my experience with vanilla web developement I don't think that there's ever been a case where I couldn't find reference documentation or a guide on MDN. And many of the guides include multiple cases and related functionality. It means that I don't have to waste time looking through sample code, Stack Overflow or Reddit to understand how to do use something.

4. High percentage of well organized documentation coverage for APIs

And each of the examples or snippets of code work or at least state compatability with browser version. I don't have to deal with figuring out if it's an issue with my configuration or the API.

MDN Async Documentation

5. Documentation is maintained and up to date with minimal errors on examples

Flutter

Flutter is my dream experience as a developer. It's meant to be a write once compile to web, Android, iOS platform. In the past, I feel like getting setup correct was a large hurdle, setting up the Android SDK, defining run targets, defining debug config etc. Flutter has a helper to check your system config and plugins for the most popular IDEs. This means that I can easily get my config right and focus on coding and debugging. Anecdotally I tried to first build this blog on Gatsby. Maybe I'm a newb but I could not for the life of me figure out how to configure debugging for Gatsby + Typescript. The inability to debug forced me to rewrite in Next.js where the VScode source maps worked correctly.

7. Thoughtful, integrated experience and tooling to simplify debugging

There are some other aspects of a developer experience that given the right circumstances may be important as well such as:

Thriving communities can supplement insufficient documentation and drive viral adoption
Client libraries are usually expected and can simplify implementation vs autogenerating from schema

Framework

Awesome, now we have an idea of where we want the developer experience to go let's break down the components of an API / Developer Experience and figure out what questions we need to answer to get there. I would define a component as a touch point for a customer. Examples of components could be the API itself, client libraries, documentation or webhooks. All of the pieces of an API that as a customer I would interact with as part of integration (or building / maintaining the API).

I'll start with a high level overview of components then dig into each component and considerations for components in subsequent blog posts along with a end to end practical example.

Schema Definition

What are some of the best practices for structuring an API? How do you define an API? There are a few schema defination languages (SDL) like OpenAPI, GraphQl, GRPC, JsonRPC, json:api. A SDL and it's associated tooling can have huge impacts on your documentation, client libraries, engineering experience. How do you govern API design across an engineering org.

Flexibility vs Simplicity

What stage is your company and what do you want to optimize for? Flexibility means using the API like lego building blocks. But accomplishing a task may take more steps. Simplicity means focuing the API and minimizing steps for a task. Which direction should you skew and when?

Authentication & Permissions

Authentication is usually dependent on how you expect your customer to use your API. It could be a public key, jwt, bearer token, or basic auth user and password. Can a customer create multiple scoped API keys? For example, can an API key access all resources? Can it read and write or only read? How do you handle permissioning for API Keys?

Versioning / Interoperability / Deprecation

You will likely need to iterate on your API as you better understand your customer needs and develop your product. There are a few different strategies for handling breaking changes and deprecating old versions. Do you want to optimize for minimizing integrator pain or eng team maintenance pain?

Idempotency, Retries, Rate Limits & Timeouts

Does your API need some sort of recovery mechanism if the network does to prevent duplicate requests? What happens when you retry the same request?

Governance

How do you get multiple eng teams to produce an API surface that feels like a single product and not N different products? Do you need a specific release process?

Webhooks

How can you notify the customer's services when an event doesn't initiate from one of their requests? How does the customer subscribe to those events. How is security handled? What if their services are down when you want to try to notify them?

Onboarding

How does a customer sign up for your service and get API keys? What are the steps to get access to production?

Documentation & Samples

How do you educate your integrators on the integration process? What type of content do you need to provide? Who's writing the documentation? How much content do you need?

Environments, Testing, Debugging & Tooling

Should you offer a sandbox? How do your customers test their implementation? If an error occurs how do they understand and fix the issue? Would a VScode extension simplify the developer experience?

Client Libraries

Do you provide client libraries or does your customer generate client libs from the schema? What languages should you support?

Monitoring & Alerts

What should you track to be aware of the health of your API? What is downtime? Is it latency > a limit? Is it 500 responses? Does a single 500 count as down time? What about intermitten 500 responses? How can you be notified of these issues?

Next Steps

In this series of posts, I'll dig into each of these components and try to provide insight into each of these questions.