Learn Gatsby - API Integration & Caching

#Introduction

A Gatsby site almost always needs external data — product APIs, external services, or a CMS. The key question isn't "can we", but "when": at build time or at runtime. That choice determines performance and data freshness.

Episode 14 covers fetching at build time and runtime, caching strategies, incremental builds, and selective sourcing to speed up builds.

#Fetching APIs at Build Time vs Runtime

#Build Time for Data That Rarely Changes

Data that doesn't change often — like product lists or documentation — is best fetched at build time. The most common pattern is in gatsby-node.js via createPages:

Fetch an API in createPages

exports.createPages = async ({ actions }) => {
  const { createPage } = actions
  const response = await fetch("https://api.example.com/products")
  const products = await response.json()
 
  products.forEach((product) => {
    createPage({
      path: `/products/${product.slug}`,
      component: require.resolve("./src/templates/product.js"),
      context: { product },
    })
  })
}

await fetch(...) runs once per build; the result is rendered into static pages. Visitors never wait on the API — the data is already in the HTML.

#Runtime for Private and Realtime Data

For per-user or frequently changing data, fetch on the client with useEffect:

Fetch at runtime

import { useEffect, useState } from "react"
 
const KursHarian = () => {
  const [nilai, setNilai] = useState(null)
 
  useEffect(() => {
    fetch("https://api.example.com/kurs")
      .then((res) => res.json())
      .then(setNilai)
  }, [])
 
  return <p>Kurs hari ini: {nilai ?? "memuat..."}</p>
}

useEffect ensures the fetch only happens in the browser, not during the build. Combine the two approaches based on the nature of the data: static data at build, dynamic data on the client.

#Caching Strategies for Static Sites

#Local Cache for External APIs

External API calls in gatsby-node.js repeat on every build and can be slow. Gatsby provides a persistent cache API:

Cache API results between builds

exports.sourceNodes = async ({ cache, actions, createContentDigest }) => {
  const { createNode } = actions
  let data = await cache.get("produk-api")
 
  if (!data) {
    const res = await fetch("https://api.example.com/products")
    data = await res.json()
    await cache.set("produk-api", data)
  }
 
  data.forEach((item) => {
    createNode({
      ...item,
      id: `Produk-${item.id}`,
      internal: {
        type: "ProdukApi",
        contentDigest: createContentDigest(item),
      },
    })
  })
}

cache.get and cache.set store data between builds in the .cache folder. The second build won't call the API again until the cache is cleared — this strategy can cut build time drastically.

#Caching in the Browser

For runtime fetches, take advantage of the external API's HTTP caching. Make sure the API responses send the right cache headers so the browser doesn't repeat the request every time.

#Incremental Builds and Content Updates

#Only the Changed Pages

Incremental builds only build the pages that changed, not the whole site. This is available on Gatsby Cloud and some hosting platforms when using Content Sync. The impact: build time drops from minutes to seconds for large sites.

#The Content Update Flow

With incremental builds, a comfortable workflow is: content is changed in the CMS, a webhook triggers a build, and only the affected pages are regenerated. The site stays fresh without sacrificing deploy speed.

#Reducing Build Time with Selective Sourcing

#Limit the Data Sources

Large data sources slow down builds. Selective sourcing means pulling only the data you truly need: set the path folder in gatsby-source-filesystem to only the directories in use, and use filter and limit on CMS queries so unneeded nodes aren't sourced.

#Parallelize the Build

Node.js can process several jobs at once via an environment variable:

GATSBY_CPU_COUNT=4 gatsby build

GATSBY_CPU_COUNT controls how many workers Gatsby uses. Adjust it to your CI machine's core count so builds are faster without overloading memory.

#Conclusion

Episode 14 completed API integration and caching: distinguishing build-time and runtime fetches, using Gatsby's persistent cache, understanding incremental builds, and selective sourcing for faster builds.

Key takeaways:

• Static data is fetched at build time in gatsby-node.js.
• Per-user data is fetched on the client with useEffect.
• cache.get and cache.set store API results between builds.
• Incremental builds only build the pages that changed.
• Selective sourcing reduces unused nodes.
• GATSBY_CPU_COUNT controls build parallelism.

In the next episode, episode 15, we enter the advanced phase: performance optimization — analyzing bundle size and page speed, preloading and code splitting, lazy loading, and improving Core Web Vitals and Lighthouse scores.