Patching Next.js to add production grade logging

I am not a frontend engineer. Claiming that would be a disservice to all the people who write code to move pixels to perfection. However, I’ve worked closely with them and had to troubleshoot applications they wrote under the pressure of a live million per second request environment.

Recently, I’ve been involved in a project where the framework of choice was Next.js. I read on Twitter that it is a very popular framework among frontend folk. From Vercel’s website, I read:

Used by some of the world’s largest companies, Next.js enables you to create high-quality web applications with the power of React components.

Well, if its core maintainer is Vercel, a $3.25 billion company (at the time of writing) and used by many other large companies, it certainly has some merit. Indeed, the UIs I’ve seen out there are really nice – smooth and fast. I cannot speak for Next.js in terms of its ergonomics since I lack the relevant experience.

However, what I can speak about, and not just speak about but rather try to improve, is its logging system, or lack thereof.

I am not sure what you get out of the box if you deploy your Next.js app on Vercel. I was never interested in locking myself into a vendor when developing software. It is unnecessary complexity tucked into a veil of features and simplicity that becomes very expensive if you end up creating successful software. But, I can tell you what you get if you decide to host your Next application on a bare-metal server, a container, or a container orchestration platform. For me, it’s Kubernetes.

We want to integrate a log-it-all solution for Next.js, which unfortunately means creating a persistent patch for the server since the framework uses its own internal server and doesn’t expose the response and request objects.

The most sustainable way to do this, that I could think of, is using patch-package to modify node_modules/next/dist/server/lib/start-server.js automatically. The packages we are using to off-load the formatting and actual logging using JSON (because eventually, we are going to ingest these logs in an ELK stack) are pino-http and pino-pretty.

In the package.json file, add a postinstall script.

"scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint",
    "ws": "node -r esm ./websockets-server.ts",
    "postinstall": "patch-package" // this one here
  },

Then, you want to install patch-package as a dev dependency since it’s only needed at the package install time, as well as pino-http and pino-pretty as a regular dependency.

yarn add --dev patch-package
yarn add pino-http
yarn add pino-pretty

In node_modules/next/dist/server/lib/start-server.js add the following consts after the _ispostpone.

const pino = require('pino');
const pinoHttp = require('pino-http');
const pretty = require('pino-pretty');

We also want to create a new prettyStream that we can use in order to only include information that we want and in a way that makes reading the logs easier.

const prettyStream = pretty({
  colorize: true,
  translateTime: 'SYS:standard',
  ignore: 'pid,hostname'
});

We create a prettyStream that will colorize the output of the logging as well as use a standard human readable time format (SYS:standard) and ignore the pid and hostname attirbutes that we don’t care about (I don’t, you might).

Then, finally, we want to create our logger. Here’s one I wrote and it works well for my use case.

const logger = pinoHttp({
  logger: pino({
    level: 'info', 
    base: null,
  }, prettyStream),
  serializers: {
    req: (req) => {
      return {
        id: req.id,
        method: req.method,
        url: req.url,
        headers: req.headers,
      };
    },
    res: (res) => {
      return {
        statusCode: res.statusCode,
        headers: res.headers,
      };
    }
  }
});

Now, search and find the requestListener method that will give us access to the request and response objects. And the logger in the first line of the method.

async function requestListener(req, res) {
    if (!/^(\/_next\/static|\/_next\/image|\/favicon.ico|\/static\/|\/.*\.(css|js|png|jpg|jpeg|gif|svg|ico))/.test(req.url)) {
        logger(req, res);
    }
    ...

Instead of logging every request, which would quickly turn out to be extremely noisy and expensive to ingest, we filter which requests we want to log by regexing on the req.url, dropping all requests that are equesting static files.

The logging happens after res.end, so we get the full request/response objects logged. Now, we have to create the patch that is going to be applied during the package install.

npx patch-package next

patch-package will try to apply the patch even if the version of Next.js changed, and in most cases it will succeed.

This is that we’ve achieved.

It’s a 34-line diff that will help you immensely if your web application turns out to be a hit. Go patch your Next.js server and advocate in the GitHub threads for Vercel to add proper logging support.