Structured Error Messages for HTTP APIs

Ever since I started to work on the Apache APISIX project, I've been trying to improve my knowledge and understanding of REST RESTful HTTP APIs.

For this, I'm reading and watching the following sources:

• Books. At the moment, I'm finishing API Design Patterns. Expect a review soon.
• YouTube. I'd recommend ErikWilde' channel. While some videos are better than others, they all focus on APIs.
• IETF RFCs. Most RFCs are not about APIs, but a friendly person compiled a list of the ones who are.

Today, I'd like to introduce the "Problem Details for HTTP APIs" RFC, aka, RFC 7807.

The problem(s)

REST principles mandate to use HTTP status to communicate.

For errors, HTTP defines two ranges: client errors, 4xx, and server errors, 5xx.

Imagine a banking API that allows you to make transfers. It should fail if you try to transfer more funds to your account. A couple of HTTP status codes can fit:

• 400 Bad Request: The server cannot or will not process the request due to something that is perceived to be a client error
• 402 Payment Required: The request cannot be processed until the client makes a payment. However, no standard use convention exists, and different entities use it in other contexts.
• 409 Conflict: The request conflict with the current state of the target resource

Here's the first problem: HTTP status codes were specified for human-to-machine interactions via browsers, not for machine-to-machine interactions via APIs.

Hence, selecting a status code that maps one-to-one to the use case is rarely straightforward. For the record, Martin Fowler seems to favor 409 in our case.

Whatever the status code, the second problem concerns the error payload or, more precisely, its structure. The structure is unimportant if a single organization manages the client and the API provider. Even if a dedicated team develops each of them, they can align. For example, imagine a mobile app that calls its own API.

However, issues arise when a team decides to use a third-party API. The choice of the response structure is significant in this case because it's now considered part of a contract: any change from the provider may break the clients. Worse, the structure is likely different from provider to provider.

Hence, a standardized error reporting structure:

• Provides uniformity across providers
• Increases API stability

RFC 7807

RFC 7807 aims to solve the problem by providing a standardized error structure.

The structure is the following:

The RFC describes the fields:

• "type" (string) - A URI reference [RFC3986] that identifies the problem type. This specification encourages that, when dereferenced, it provide human-readable documentation for the problem type (e.g., using HTML [W3C.REC-html5-20141028]). When this member is not present, its value is assumed to be "about:blank".
• "title" (string) - A short, human-readable summary of the problem type. It SHOULD NOT change from occurrence to occurrence of the problem, except for purposes of localization (e.g., using proactive content negotiation; see [RFC7231, Section 3.4]).
• "status" (number) - The ([RFC7231], Section 6) generated by the origin server for this occurrence of the problem.
• "detail" (string) - A human-readable explanation specific to this occurrence of the problem.
• "instance" (string) - A URI reference that identifies the specific occurrence of the problem. It may or may not yield further information if dereferenced.

-- Members of a Problem Details Object

The RFC offers the following sample when there needs to be more funds to make a banking transfer.

An example

I'll use one of my existing demo as an example.

The demo highlights several steps to ease the process of evolving your APIs.

In step 6, I want users to register, so I limit the number of calls in a time window if they aren't authenticated. I've created a dedicated Apache APISIX plugin for this. After the number of calls has reached the limit, it returns:

HTTP/1.1 429 Too Many Requests
Date: Fri, 28 Oct 2022 11:56:11 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX/2.15.0

{"error_msg":"Please register at https:\/\/apisix.org\/register to get your API token and enjoy unlimited calls"}

Let's structure the message as per RFC 7807.

Conclusion

RFC 7807 not only helps client developers.

It's a tremendous help for API implementors as it provides quick guidelines to avoid reinventing the wheel on every project.

Go further:

• RFC 7807
• Standards.REST
• HTTP Client Error 4xx

Originally published at A Java Geek on October 30^th, 2022