API Design: Empathetic Design for Technical Consumers

Published by

Danny Baggett
June 1, 2020

Design is important in many aspects of development which is especially true for development that drives user experience [1]. We’ve learned that API design can have a profound impact on user interfaces and thus user experiences. Poorly designed APIs can lead to awkward, unnatural or inefficient workflows within a user’s experience while a well-designed API can mitigate these issues or at least make it clear to consumers what all is technically possible. Ultimately, user interfaces are a representation of the underlying API.

Here are five design, implementation, performance and security best practices to keep in mind when producing APIs.

Best Practice: Make your API a Priority

APIs should be designed and developed with simplicity, consistency and ease of use in mind. Accomplishing these things may be easy in a silo but the consumer’s view may be drastically different since their needs or wants were likely not taken into account. It’s always important to design and iterate closely with consumers and/or clients before producing any long-term implementation. The best way to do this is by practicing API-First Design which is a design methodology focused on collaboration with stakeholders. Continuing with the alternative may result in an API that conforms to the existing system or simply serves as a conduit to the underlying database which will almost always ignore the client’s workflows. A great analogy exists in the construction world – you wouldn’t build a house and then draft blueprints.

Additionally, by leading with API design, it’s possible to identify API specification format and tooling from the beginning. The API specification should be described in an established format such as OpenAPI, API Blueprint or RAML. An established format is likely to have sufficient tooling that clients are familiar with like Apiary, Redocly or Swagger Hub. Depending on the time gap between API Design and client development, it may be appropriate to consider mocking functionality which most established tools will have. Mocking is a good way to give prospective consumers a tour of example data while the implementation is built out.

Best Practice: Structure with Resource Oriented Design

There is a very common architectural style known as REST [2] that has become the defacto standard on API’s for some time now. APIs that are developed using REST architecture are said to be RESTful. In general, REST provides important and well-known architectural constraints [2] but lacks concrete guidance on authoring API’s. There is an expanded architectural style known as Resource Oriented Design [3], satisfying all the constraints of REST, that serves as a good API design reference. If we’re to compare REST to SQL, Resource Oriented Design provides similar normalization properties to REST as 2NF and 3NF do for SQL. Here are a few constraints to adhere to:

The API URIs should be modeled as a resource hierarchy where each node is a resource or collection resource
- Resource – a representation of some entity in the underlying application e.g. /todo-lists/1, /todo-lists/1/items/1
- Collection – a list of Resources all of the same type e.g. /todo-lists, /todo-lists/1/items
The URI path – resource path – should uniquely identify a resource e.g. /todo-lists/1, /todo-lists/56
Actions should be handled by HTTP Methods

Following these constraints where all practical will lead to a normalized API that’s consistent and predictable. Also, by leaving actionable verbiage out of URIs it’s easier to ensure that every resource path is a representation of some entity. This is why verbs are frowned upon in the URI as they are likely not a representation of some underlying entity – /activate is likely not a resource.

As far as the resources themselves, there is no universally accepted answer on the format used to represent them. JSON is widely adopted and understood by the majority of platforms, however, XML or other formats may serve consumers just as well or better in certain situations. The key thing is to represent resources in a way that is quick and easy for consumers to understand.

Representing resources in this fashion enables them to “speak” for themselves, known as self-descriptive resources. Self-descriptive resources that are documented using established tooling and open standards will build a sense of trust with the consumer. Ultimately, consumers should buy-in to what the API is selling without additional “fluff” material.

Best Practice: Semantics with HTTP and REST

In order to make an API come to life it needs to be actionable especially since it will likely be used to inform a client’s user interface – the buttons need to do something. Actions, in the context of REST, should be fulfilled by HTTP methods each with an intended purpose which is described here:

GET – query/search for resources, expected to be idempotent and thus cacheable
POST – most flexible REST semantic, any non-idempotent actions excluding deletes should be handled by this method
PUT – used to modify entire resources, yet still expected to be idempotent
PATCH – used to make partial modifications to resources, yet still expected to be idempotent
DELETE – removes a resource from the API, should be idempotent from the caller’s view

Additionally, the result of each action should be returned to the client with a proper status code as defined by the HTTP standard, not all of them most likely but most likely more than you think. Here are some common statuses that will likely be required on any API project:

Successful response (200 – 399)
- 200 – responses with a body for everything besides a creation action
- 201 – responses for creation actions
- 202 – responses for a long running process
- 204 – responses that don’t require a body
Client errors (400 – 499)
- 400 – invalid or bad request, appropriate for syntactic errors
- 401 – unauthenticated request due to missing, invalid or expired credentials
- 403 – insufficient permissions e.g. wrong Oauth scope, requires admin privileges
- 404 – resource not found e.g. represented entity does not exist in database
- 409 – resource conflict e.g. resource already exists
- 422 – request is syntactically valid but not semantically valid
Server errors (500 – 599)
- 500 – classic internal or unknown errors for modeling exceptional/unrecoverable errors, sensitive errors or errors that can’t be elaborated on
- 501 – method not implemented
- 503 – server unavailable
- 504 – timeout

For clarity, an idempotent HTTP method can be called many times with the same outcome. Consumers should be able to understand the information, relationships and operations an API is providing by the resources and methods on them alone. For example, a GET method that creates or PUT that deletes a resource will lead to an unpredictable API fraught with unexpected side-effects. Proper HTTP method and status codes, which naturally includes constraints such as idempotence, used in conjunction with self-described resources are nothing more than factual representations of the underlying business domain.

Best Practice: Maintaining Performance at the API Layer

Building a fast-performing service requires careful thought and design, often across multiple technical boundaries. The list of things to consider for performance can range from proper use of concurrency in the application on down to adequate database indexing, just depends on the requirements and needs of the business. The API can uphold performance characteristics of the underlying system in multiple ways, here’s some to consider:

Asynchronous Server Code – Resources that are an aggregate of multiple independent data sources can be built by the result of multiple async execution results.
Non-blocking Implementation – APIs that access cold I/O, are CPU intensive or just naturally slow should return with a 202 whilst processing and instruct the client where to get the result. Websockets and callbacks may be appropriate in certain situations as well.
Caching – Resources that change relatively slow can be cached, idempotent GET requests are the low-hanging fruit. Service level caching may suffice but a CDN may be needed depending on load.
Paging – Collection resources should be outfitted with paging controls for the client to limit results
Statelessness – Keeping track of state across requests can be complex and time consuming for the server. Ideally, state should be managed by the client and passed to the server. This applies to authentication/authorization as well. Credentials should be passed on each request, JSON Web Token (JWT) is a good option.

These considerations will go a long way towards meeting satisfactory performance metrics set by the business. Usually, business satisfaction is going to be directly tied to the satisfaction of its consumers – their satisfaction with the user experience and thus backing APIs. There has been plenty of research to show that a consumer’s user experience is tied to the response time of a page [4]. Network latency plays a big factor in page load time so it’s important to reduce this time as much as possible. A good rule of thumb is to keep API response time between 150 and 300 milliseconds which is the range for average human reaction time.

Best Practice: Securing your API

Personal and financial information is prevalent on the internet today. It has always been important to safeguard this information. However, there have been many notorious data breaches in recent times [5] that bump security considerations from just another thing to project non-starters without them. There are two good rules of thumbs when it comes to API security – don’t embarrass yourself and don’t reinvent the wheel. It’s best to leverage open standards such as OAuth or OpenID which both cover most authentication flows. It’s also advisable to delegate identity matters to purpose-built identity providers such as Auth0, Firebase, or Okta. Security is a hard thing to get right and the aforementioned vendors have solved this challenge plus gone the extra mile or two. Regardless of the standard and/or provider used, it’s always important to apply proper access controls to API resources. Resources that are sensitive should be locked down with appropriate credentials and a 401 should be returned when these credentials are not provided. In cases where a given user does not have adequate privileges to a resource, a 403 should be returned.

The best practices highlighted above are established practices in the industry that should help with any API project. By taking an API-First approach, you will be on the right path to fostering trust with your consumers and stakeholders. Practicing established REST structure and semantics as well as proper security will be huge in your endeavor. Maintaining performance will keep consumers and customers coming back for more. Ultimately, API design is part art and science. Each API will be different and there may be some pragmatic decisions to be made. However, it’s critical to not stray too far off the beaten path.

It’s important to remember that your data and information is what your consumers and customers are truly seeking, not your radical new API design. Following these best practices will get this information to your all-important customers while keeping your consumers informed all along the way.

[Citations] [1] NEA – Design in Startups Survey –
https://www.nea.com/blog/the-future-of-design-in-start-ups-survey-2016-results

[2] REST –
https://en.wikipedia.org/wiki/Representational_state_transfer

[3] Resource Oriented Design –
https://cloud.google.com/apis/design/resources