Understanding API Mocking - Part 1: The HTTP Request Journey

Artem Zakharchenko
author
Artem Zakharchenko
satellite dish

Understanding API Mocking Series

This is the first article in a three part series. You'll learn the basics of HTTP requests and how this knowledge affects API mocking.

Part 2
• Part 2

Request Interception Algorithms

Part 3
• Part 3

Mock Service Worker

API mocking is a technique to intercept HTTP requests and respond to them with mocked responses.

We often use API mocking during development and testing to gain control of the network, modeling it to suit our needs. And, as with many things, we rely on third-party libraries to implement request interception reliably and provide us with API mocking capabilities.

But have you ever wondered how those libraries work?

In this mini-series, you and I will be taking a deep dive into how different API mocking solutions work to see what benefits and downsides they have, as well as just broadening our understanding of HTTP requests in JavaScript.

The Steps of API Mocking

In a nutshell, API mocking consists of two sequential steps:

  1. Intercepting an outgoing HTTP request;
  2. Responding to that intercepted request with a mocked response.

There are quite a number of ways to implement either of those steps in an API mocking solution.

To build a truly great one, we need to ensure a proper balance between the code’s integrity and the amount of control this solution gives us. But, most importantly, we need to understand how requests are represented in JavaScript and know about the different stages that a request goes through before being performed.

And like with many things in life, we will learn about all that on a journey.

The Request Journey

Every story has its beginning, and ours begins from a request.

You see, request interception is rather tightly coupled with how that request is being made. If we wish to understand the difference and tradeoffs of various API mocking approaches, we should first understand how HTTP requests can be made in JavaScript.

So, make sure your headers are fastened and cookies jarred, we are going on a request journey!

The Request Client

Any request starts with the intention to read or change data.

We put that intention into code and provide it to a request client to carry it out*.* A request client is, essentially, any API (native or third-party) that concerns itself with accepting a request declaration and executing it.

For example, one of the most common browser APIs to perform requests is Fetch API:

// We have an intention of fetching all movies.
// To describe that intention, we perform a "GET" request
// to the "/movies" endpoint on the server.
fetch('/movies')

In practice, you may be using all sorts of different clients, like Axios or "got", and your choice will often depend on the kind of request you wish to describe (e.g. you may want to use a specialized GraphQL client, like Apollo, to describe GraphQL requests).

Once the request client accepts our intention (request description), it returns us the means to monitor that request execution and, eventually, handle the received response from the server.

In the example above, the fetch call would return a Promise resolving to a Response instance that we can read:

// The fetch Promise resolves to a "Response" instance
// that allows us to handle the response (e.g. get its
// status, headers, or read its body).
const response = await fetch('/movies')
console.log(response.ok, response.status)

For us, developers, the request client is often where our interaction with the request ends, but for the request itself, this is only the beginning.

While request clients provide us with a great way to perform and manage requests easier, they are just abstractions for the underlying code doing all the heavy lifting.

And that is precisely where our request is headed.

The Environment

No matter the request client, our request will inevitably reach the standard API of the environment responsible for handling HTTP requests.

While we may interact with that standard API directly, when using third-party request clients we delegate that interaction to the client, and so the construction and handling of a request becomes an implementation detail of the request client. For example, when using Axios, it will represent our requests as XMLHttpRequest in the browser and http.ClientRequest in Node.js without us even knowing.

But we are here today to go beyond third-party abstractions and learn about those environmental APIs, aren’t we?

It’s important to keep in mind that each environment implements its network module differently. As JavaScript engineers, we are mostly interested in the browser and Node.js environments.

Let’s take a look at what native APIs exist there to represent a request.

Browser

window.fetch

Fetch API is one of the most common ways to make requests on the web.

Introduced to the world in 2015, it launched as a step forward from XMLHttpRequest and, without a doubt, has changed our developer lives for the better.

fetch('https://api.example.com/users', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' },
  body: JSON.stringify({ name: 'Alice' })
})

The browser implementation for fetch is in the native C code and the window.fetch() call is the last surface that we can interact with.

window.XMLHttpRequest

XMLHttpRequest made its first public appearance as a fully-functional API to dispatch requests in 2002.

This may be surprising for some, but XHR is still shipped in the browsers to this day and is a legitimate way to make requests. Moreover, XHR exposes a few capabilities that even modern Fetch does not have, such as monitoring a request’s progress and request cancellation, which was not possible via Fetch prior to AbortController becoming a thing.

const request = new XMLHttpRequest()
request.open('POST', 'https://api.example.com/users')
request.setRequestHeader('Content-Type', 'application/json')
request.write(JSON.stringify({ name: 'Alice' })
request.send()

Similar to window.fetch, XMLHttpRequest’s roots go deep into the native browser code without any intermediate layer to attach our mocking logic.

Node.js

http.request (https.request)

Node.js provides us with a high-level API to perform requests via http and https modules. While they share similar methods, like .get() and .request(), their implementation differs as they handle different request protocols.

import https from 'https'
const request = https.request('https://api.example.com/users', {
  method: 'POST',
  headers: { 'Content-Type': 'application/json' }
})
request.write(JSON.stringify({ name: 'Alice' }))
request.end()

The http.request() API is what most of the request clients in Node.js use internally.

Perhaps it’s the verbosity of said API that gave birth to so many request clients in that environment, to begin with (my wild assumption here).

http.ClientRequest

Moving further down the call stack, the http.request represents (and handles) requests using the http.ClientRequest class.

It’s fair to mention that some libraries and polyfills utilize this class directly, circumventing the higher-level API altogether.

That being said, constructing a request via http.ClientRequest is similar to using http.request, and the difference may not even be visible to a naked eye at times:

import http from 'http'
import https from 'https'
const req = new http.ClientRequest({
  protocol: 'https:',
  host: 'api.example.com',
  pathname: '/users',
  headers: {
    'Content-Type': 'application/json',
  },
  agent: new https.Agent(),
})
req.write(JSON.stringify({ name: 'Alice' }))
req.end()

Note that it’s the same http.ClientRequest class that describes both HTTP and HTTPS requests, unlike the http.request and https.request distinction at the higher level.

net.Socket

Let’s go even deeper. Any HTTP request is, essentially, data transferred over a socket, and Node.js has a net.Socket class to describe just that.

import { Socket } from 'net'
const socket = new Socket()
socket.connect(443, 'api.example.com', () => {
  socket.write('POST /users HTTP/1.0\n')
  socket.write('Content-Type: application/json\n')
  socket.end(JSON.stringify({ name: 'Alice' }))
})

The fact that we are sending raw HTTP messages over the wire should give you a good impression of how low-level the Socket API is.

Although you are unlikely to use this API directly, it’s still an inevitable part of the request journey in Node.js, and, as a consequence, important to our choice of the API mocking approach.

Honorable mentions

Node.js 17 adds a global fetch API similar to that in the browser.

Since they comply with the same specification, the fetch in Node.js represents requests and responses using the Request and Response classes respectively, but those still act as abstractions over the low-level APIs we’ve touched on above.

Once the network module does its job, the request finally gets performed, leaving the environment and traveling down the fiber to the requested server.

The Server

The server is where requests come to die resolve.

The server is the farthest context from our request intention, as it’s a different environment often implemented in a different language than our application, having intricacies of its own. However, once the server resolves the request, it sends the response through the same journey so it would, eventually, return to the request client.

Thus, once our request reaches the server, there’s little we can do to affect it from the client-side runtime.

Conclusion

The purpose of any journey is to improve and change those courageous enough to embark on it.

And the request journey we’ve just gone through is no exception. As we’ve learned about the various ways to describe requests and what checkpoints each request undergoes, we can proceed to draft the possible ways to implement API mocking in our application.

That’s precisely what I will be covering in the next post.

Read API Mocking Part 2

Part 2
• Part 2

Request Interception Algorithms