Learn Shiki Rehype Pretty Code - History, Background & Why This Combination Is Needed

#Introduction

Before learning how to use Shiki and rehype-pretty-code, you need to know why this combination exists. Syntax highlighting that looks simple on the surface actually hides a long-running set of problems: color accuracy, render speed, bundle size, and modern code block features like line highlighting and diff.

Episode 1 takes you through the evolution of syntax highlighting — from simple regex, the birth of TextMate grammar, the emergence of Shiki in 2018, to the birth of rehype-pretty-code in 2022. By the end of the episode, you'll understand the specific problems this combination solves and why this approach became the standard in the modern content ecosystem.

#The Evolution of Syntax Highlighting

#The Regex Era: highlight.js and Prism

The oldest approach uses regex to recognize keywords. Libraries like highlight.js and Prism work with text matching patterns: if a word matches the keyword pattern of a certain language, that word gets a color. This approach is lightweight and easy to use, but it has fundamental limitations.

Illustration of the regex approach

const keywords = ["const", "function", "return"];
const tokens = code.split(/(\s+)/).map((part) =>
  keywords.includes(part) ? `<b>${part}</b>` : part,
);

The problem is clear: regex doesn't understand context. The word function inside a string literal, a comment, or a template literal will still be colored as a keyword. As a result, the highlighting produced is inaccurate for complex code, and there's no guarantee it matches what you see in your editor.

#TextMate Grammar: The VSCode Standard

The breakthrough came from the VSCode ecosystem. This editor uses TextMate grammar — a set of pattern-based rules broken down into scopes (such as keyword.control, string.quoted, and entity.name.function). Grammar is far more precise than a single regex because it supports nested rules and layered context.

The key concepts you need to hold onto:

• Tokenization: code text is split into tokens, each with a scope.
• Scope: semantic labels like keyword or string, not direct colors.
• Theme: maps scopes to specific colors, for example the keyword scope to purple.

This separation of scope and theme is what lets colors change without modifying the grammar rules.

#Shiki: Born from the Need for Accuracy

Shiki was created in 2018 by Pine Wu with a name that stands for Syntax Highlighter for Kanji — code that looks like Kanji writing to Japanese eyes. Its goal is simple: bring VSCode tokenization to code rendered on web pages.

Over time, Shiki has been maintained by Anthony Fu and the community. In 2026, Shiki is at version 4.x with a redesigned architecture: langs, themes, and transformers are split into their own packages such as @shikijs/langs, @shikijs/themes, and @shikijs/transformers. We'll break down these new APIs in episodes 18 and 20.

#Problems This Combination Solves

#Color Accuracy Identical to the Editor

By using the same grammar and theme as VSCode, Shiki produces colors exactly identical to what you see in the editor. This isn't just about aesthetics: for technical documentation, color consistency helps readers map code to an already familiar experience.

#No Client-Side JavaScript

Because Shiki runs at build time (SSG), the generated HTML already contains colored spans. The browser doesn't need to run JavaScript to color code. This is a major difference from highlight.js and Prism, which in many setups actually run on the client side and slow the page down.

#Modern Code Block Features

Shiki itself only handles tokenization. Features like line highlighting, diff, title, and word highlight come from rehype-pretty-code, which reads the meta string on fenced code blocks:

npm view shiki version
npm view rehype-pretty-code version

The npm view shiki version command shows the latest version from the registry. This is the fastest way to make sure the features we discuss are still current.

#Why Shiki and rehype-pretty-code Are Combined

The two complement each other in the content pipeline:

• Shiki provides the highlighting engine: TextMate tokenization, Oniguruma WASM, and theme mapping.
• rehype-pretty-code provides the code block experience: turning <pre> and <code> nodes into <figure> elements with title, caption, data attributes for line and word highlighting, and inline code highlighting.

Think of Shiki as the engine and rehype-pretty-code as the dashboard. The engine produces accurate colors, while the dashboard makes it easy to drive advanced features through Markdown meta strings.

Both meet in a single pipeline

unified()
  .use(remarkParse)
  .use(remarkRehype)
  .use(rehypePrettyCode, {
    theme: "github-dark-default",
    transformers: [transformerNotationDiff()],
  })
  .use(rehypeStringify);

In the pipeline above, rehypePrettyCode(options) uses Shiki internally and adds a feature layer. This is the combination we'll build step by step throughout this series. Before moving on, make sure you understand that plugin order in the pipeline is part of the configuration itself, not a minor detail.

#Position in the Content Ecosystem

This combination is now a top choice in many content frameworks. Next.js, Astro, SvelteKit, and VitePress all support rehype plugins, so rehype-pretty-code can be installed with nearly the same pattern. In episode 11 we'll look at the small differences between frameworks, but the core pipeline is always identical.

Regardless of the framework, this capability stands on the stable, well-documented unified ecosystem. As long as a framework accepts rehype plugins, you can be confident that the learning investment in this series remains useful.

This choice also makes sense philosophically: content stays simple Markdown, and highlighting complexity is hidden in the transformation layer. Writers write a plain fenced code block with a meta string; readers get finished HTML, with no extra JavaScript.

One important note: the Shiki and rehype-pretty-code combination doesn't require you to abandon tools you already have. highlight.js or Prism already installed in old projects can stay; this approach only offers a higher standard of accuracy and features. If you want to start practicing it, the first step is setting up a content project with Node.js and a working package manager, exactly as described in the next episode.

#Conclusion

Key takeaways:

• Syntax highlighting evolved from regex (highlight.js, Prism) to TextMate grammar.
• Shiki was born in 2018 by Pine Wu to bring VSCode tokenization to the web.
• Shiki separates grammar, scope, and theme so colors are as accurate as the editor.
• rehype-pretty-code was born in 2022 to add code block features to the rehype pipeline.
• Shiki is the engine, rehype-pretty-code is the user experience layer.
• This combination runs at build time, so there's no client-side JavaScript.

In episode 2 next, we'll dissect core concepts and main architecture — how Shiki tokenizes code with Oniguruma WASM, how rehype-pretty-code turns <pre> and <code> nodes into <figure> elements with data attributes, and the key components like createHighlighter, @shikijs/transformers, @shikijs/langs, and @shikijs/themes. Prepare your understanding of the pipeline, because this is the foundation for all the episodes that follow.