iwantcoding.com
🔥 Daily 👥 Rooms 🏆 Top Log in Sign up

DataLoader (N+1)

By · Reviewed & last updated · Part of GraphQL Tutorial

DataLoader is the canonical solution to the N+1 query problem in GraphQL. It batches lookup keys collected in a single execution tick into one backend call, and caches per-request so the same id is never fetched twice. One loader per resource (user, product, organisation) wraps the underlying repository.

Per-request DataLoader pattern with batching

EXAMPLE
// npm i dataloader
import DataLoader from 'dataloader';

// 1) Build a loader factory — fresh loaders PER REQUEST (do not share across requests)
function createLoaders(db) {
  return {
    userById: new DataLoader(async (ids) => {
      const rows = await db.users.find({ id: { $in: [...ids] } }).toArray();
      const byId = new Map(rows.map((r) => [r.id, r]));
      // IMPORTANT: return results in the SAME order as ids (DataLoader contract).
      return ids.map((id) => byId.get(id) ?? null);
    }),

    ordersByUserId: new DataLoader(async (userIds) => {
      const rows = await db.orders.find({ user_id: { $in: [...userIds] } }).toArray();
      const grouped = new Map();
      for (const id of userIds) grouped.set(id, []);
      for (const r of rows) grouped.get(r.user_id)?.push(r);
      return userIds.map((id) => grouped.get(id) ?? []);
    }),
  };
}

// 2) Attach loaders to the GraphQL context
async function context({ req }) {
  return { db, loaders: createLoaders(db), user: await authFromHeader(req) };
}

// 3) Resolvers use the loaders instead of touching the DB directly
const resolvers = {
  Query: {
    user: (_, { id }, { loaders }) => loaders.userById.load(id),
  },
  User: {
    // 100 users with .orders each -> ONE batched DB call, not 100
    orders: (parent, _, { loaders }) => loaders.ordersByUserId.load(parent.id),
  },
  Order: {
    // Re-uses the per-request cache: the same user requested twice is fetched once
    customer: (parent, _, { loaders }) => loaders.userById.load(parent.user_id),
  },
};

// 4) Common pitfalls — and the fixes

// a) Cache pollution across requests
//    BUG:  const userLoader = new DataLoader(...) at module scope
//    FIX:  build loaders inside context() so they live one request only

// b) Forgetting the order contract
//    BUG:  return rows from the DB in whatever order Mongo gave you
//    FIX:  build a Map and map(ids => map.get(id))

// c) Errors per-key are silent
//    Return Error instances at the failing position; DataLoader rejects only that key.
//    return ids.map((id) => byId.has(id) ? byId.get(id) : new Error('not found ' + id));

// d) Prime the cache when you already have the data
//    loaders.userById.prime(user.id, user);

// e) Clear stale entries after a mutation
//    loaders.userById.clear(updatedUserId);

// 5) Beyond DataLoader: prisma findMany + select is also batched at the SQL level.
//    Always start with one query per resource via DataLoader; reach for
//    smarter aggregation only when you measure a real hotspot.

Why it matters

A loader is a per-request object. Module-level loaders cache across users and across mutations, leaking data and serving stale rows — the classic "I see another user`s data" bug in production. Always build loaders inside context() so they live and die with the request.

Tip: Tweak the snippet with Try it Yourself », then sit the quiz at the bottom of the page.

Example

Example
import DataLoader from 'dataloader';
const userLoader = new DataLoader(async ids => db.users.findMany({ id: ids }));
Try it Yourself »

Discussion

Loading…