Learn Shiki Rehype Pretty Code - Shiki Core & Highlighter API

#Introduction

So far you've used Shiki through rehype-pretty-code. In fact, Shiki is a standalone library that can be called directly from any code: build scripts, markdown-it, or an API server. Understanding its core API opens possibilities beyond the rehype ecosystem.

Episode 18 covers Shiki core and the highlighter API: createHighlighter, createHighlighterCore, codeToHtml, codeToTokens, and getSingletonHighlighter, plus examples of use outside rehype like markdown-it and manual rendering from Node.js.

#Creating a Highlighter

#createHighlighter

The simplest way to create a highlighter is createHighlighter. This function loads the langs and themes you specify, then returns an instance ready to use:

Create a highlighter

import { createHighlighter } from "shiki";
 
const highlighter = await createHighlighter({
  langs: ["typescript", "css", "json"],
  themes: ["github-dark-default"],
});

The selected langs and themes are loaded into memory. The fewer there are, the faster initialization. This instance can be reused many times for different code.

#Directly Using the Function API

For one-off use, Shiki provides short functions that automatically create an internal highlighter:

Short usage

import { codeToHtml } from "shiki";
 
const html = await codeToHtml('console.log("halo")', {
  lang: "typescript",
  theme: "github-dark-default",
});

The codeToHtml function is practical for small scripts. For repeated use in the same process, create your own instance so grammars aren't reloaded on every call.

#Advanced APIs

#createHighlighterCore and the Engine

When bundle size is critical, use createHighlighterCore from shiki/core. This entry point doesn't load languages, themes, or WASM automatically:

Core highlighter

import { createHighlighterCore } from "shiki/core";
import { createOnigurumaEngine } from "@shikijs/engine-oniguruma";
 
const highlighter = await createHighlighterCore({
  langs: [import("@shikijs/langs/typescript")],
  themes: [import("@shikijs/themes/github-dark-default")],
  engine: createOnigurumaEngine(import("shiki/wasm")),
});

Each language is loaded via dynamic import so the bundler splits it into a separate chunk. The engine is injected explicitly, giving full control over how tokenization runs.

#codeToTokens

Highlighting doesn't always mean producing HTML. codeToTokens returns tokens as raw data, useful for analysis or custom rendering:

Get raw tokens

const result = await highlighter.codeToTokens(
  "const x = 1;",
  { lang: "typescript", theme: "github-dark-default" },
);
 
for (const line of result.tokens) {
  for (const token of line) {
    console.log(token.content, token.color);
  }
}

Each token carries content (the original text) and color (the theme result color). With this data, you can render code into other formats, like canvas or SVG.

#Using a Reusable Instance

#getSingletonHighlighter

Creating a highlighter in several places can load the same grammar repeatedly. getSingletonHighlighter guarantees that only one instance is created and shared:

Single highlighter

import { getSingletonHighlighter } from "shiki";
 
const highlighter = await getSingletonHighlighter({
  langs: ["typescript"],
  themes: ["github-dark-default"],
});
 
const html = highlighter.codeToHtml("const a = 1;", {
  lang: "typescript",
  theme: "github-dark-default",
});

A second call with similar configuration doesn't create a new instance; the existing grammar is reused right away. This prevents memory leaks in long-running processes like a server.

#Shiki Outside rehype-pretty-code

#Integration with markdown-it

markdown-it has a highlight option to replace the default code renderer. Plug codeToHtml in there:

Markdown-it with Shiki

import MarkdownIt from "markdown-it";
import { codeToHtml } from "shiki";
 
const md = new MarkdownIt({
  highlight(code, lang) {
    return codeToHtml(code, {
      lang: lang || "plaintext",
      theme: "github-dark-default",
    });
  },
});

When markdown-it finds a fenced code block, the highlight option is called and Shiki's HTML result is used as the output. The same pattern applies to other Markdown libraries that provide a highlight hook.

#Manual Rendering from Node.js

Without any Markdown library at all, you can read code and write the highlighting result to a file:

node render.mjs

render.mjs

import { readFile, writeFile } from "node:fs/promises";
import { codeToHtml } from "shiki";
 
const code = await readFile("./snippet.ts", "utf8");
const html = await codeToHtml(code, {
  lang: "typescript",
  theme: "github-dark-default",
});
 
await writeFile("./snippet.html", html);

A script like this can run in a build pipeline to generate static pages without a framework. Because everything runs on Node.js, the Oniguruma WASM file is still easy to load via shiki/wasm.

#Conclusion

Key takeaways:

• createHighlighter loads langs and themes and is ready for repeated use.
• codeToHtml is a short function for one-off use.
• createHighlighterCore gives full control over the bundle and engine.
• codeToTokens returns raw tokens for custom rendering.
• getSingletonHighlighter shares one instance across many places.
• Shiki can be used in markdown-it, build scripts, and manual rendering.

In episode 19 you'll learn performance and troubleshooting: diagnosing HTML output problems, reading build logs, and handling common issues like grammars not loading, WASM errors, unrecognized languages, unparsed meta, and conflicts with the CSS theme.