Learn GraphQL - Solving the N+1 Problem with DataLoader

#Introduction

Episode 6 briefly touched on the danger of resolver chains. Now it's time to dissect that danger in depth: the N+1 problem, GraphQL's most famous performance enemy. Episode 9 explains why GraphQL is vulnerable to it and how DataLoader is the standard solution.

We'll understand the N+1 mechanism, the DataLoader batching and caching concepts, how to implement it, integrating loaders into a per-request context, and techniques for measuring the performance improvement after the loader is installed.

#The N+1 Query Problem

#How N+1 Happens

Consider the following query:

query DaftarPost {
  posts {
    id
    title
    author {
      name
    }
  }
}

If there are 100 posts, the posts resolver triggers 1 query, then the Post.author resolver is called 100 times — once per post — and each call triggers 1 query against the users table. That's 1 + 100 = 101 queries for a single request. That's where the name N+1 comes from.

The cause: GraphQL calls a resolver per-field-per-item, and each resolver independently returns data from parent. This pattern makes queries look efficient on the client, but is wasteful on the database.

#Impact on Performance

The impact is real: request latency balloons, the database exhausts its connections, and infrastructure costs rise. In production with thousands of users, N+1 is the number-one issue discovered during load testing. Early identification can be done by looking at the query log: if you see a pattern of identical queries repeated with different ids, it's almost certainly N+1.

#DataLoader Fundamentals

#The Batching and Caching Concepts

DataLoader is a library designed for this problem with two mechanisms:

• Batching: all load calls within one event loop tick are combined into a single batch function call.
• Caching: results are cached per key for the loader's lifetime, so the same key isn't fetched twice.

npm install dataloader

Run npm install dataloader to add this library to your project.

#Creating a Batch Loader

Basic batch loader

import DataLoader from "dataloader";
 
const batchUsers = async (ids) => {
  const users = await db.users.findMany({ where: { id: { in: ids } } });
  const byId = new Map(users.map((u) => [u.id, u]));
  return ids.map((id) => byId.get(id));
};
 
const userLoader = new DataLoader(batchUsers);

Note two important details. First, the batch function receives an array of ids and must return results in the same order as the input. Second, if an id isn't found, return null for its position so the order doesn't shift.

#Implementing DataLoader

#Using the Loader in a Resolver

The loader transforms the Post.author resolver from a per-item query into a single batch query:

Resolver with a loader

const resolvers = {
  Query: {
    posts: async (_, __, ctx) => ctx.db.posts.findMany(),
  },
  Post: {
    author: (post, _, ctx) => ctx.loaders.userById.load(post.authorId),
  },
};

Now 100 posts only trigger 1 users query. Batching works because all load calls from concurrently running resolvers accumulate within a single tick, then DataLoader combines them.

#Caching and Clearing the Cache

DataLoader's cache is per-instance. That's why loaders must be created per request, not globally — if global, data that has changed (for example after a mutation) will be hidden behind a stale cache. For data that was just mutated, call loader.clear(key) so it's fetched again, or build a new loader for each request.

#Error Handling in a Loader

If the batch function throws an error, every load in that batch fails. For finer-grained per-item handling, return results per item and throw errors selectively. A common pattern: return null for items not found, and only throw when the whole batch is problematic.

#Integration with Context

#Request-Scoped Loaders

The right place for loaders is inside the context function, which is called per request:

Per-request loaders in the context

const { url } = await startStandaloneServer(server, {
  context: async ({ req }) => ({
    user: await authenticateUser(req.headers.authorization),
    db,
    loaders: {
      userById: new DataLoader(batchUsers),
      postByAuthorId: new DataLoader(batchPostsByAuthor),
      commentByPostId: new DataLoader(batchCommentsByPost),
    },
  }),
});

With this pattern, every request gets a fresh loader instance: batching applies per request, and the cache doesn't leak across requests. All resolvers share the same loaders through ctx.loaders, so there's no duplicated definitions.

#Monitoring and Debugging

#Measuring the Improvement

Measure the number of database queries before and after using DataLoader. A simple tool: count queries in the log, or use a Prisma plugin for logging. After the loader is installed, the repeated identical query pattern should disappear and be replaced by a single batch query using IN.

To debug batch calls, add a log inside the batch function — you'll see the ids accumulated into a single array, not called one by one. That's proof batching is working. DataLoader can also be given the maxBatchSize option if a single request could produce a giant batch.

#Conclusion

Key takeaways:

• N+1 happens when resolver chains trigger repeated queries: 1 initial query plus N additional queries.
• DataLoader combines many load calls into one batch function call.
• The batch function must return results aligned with the input id order.
• DataLoader's cache is per-instance; create a new loader per request.
• Use loader.clear(key) after mutations so data stays fresh.
• Measure the query count before and after to prove the improvement.

In the next episode, episode 10, you'll learn about input validation and sanitization — automatic validation at the schema level, integrating Zod for resolver validation, sanitization to prevent XSS and SQL injection, and designing client-friendly error responses. Data entering your server will never be out of control again!