Learn Backstage - Search & Discovery

#Introduction

In episode 15, you secured the deployment from the network down to the cookies. Episode 16 puts that security to work for the goal developers feel most: search & discovery. The main problem of a developer portal isn't a lack of information — it's scattered information: documentation in one place, code in another, teams somewhere else. Search in Backstage pulls all those sources into a single search box, so developers find entities, documentation, and context in one query.

#The Search Architecture in Backstage

Search in Backstage consists of three major layers:

• Backend search — collects, processes, and indexes documents from various sources.
• Search engine — stores the index and executes queries (as touched on in episode 12).
• Frontend search — the search box that presents results to users.

These layers exchange documents and queries through separate APIs, so you can swap the search engine or add new sources without touching the frontend.

#Indexing Pipeline: Collators, Decorators, Indexers

The indexing process runs in a pipeline with three roles:

1. Collators — gather documents from a source (collate), for example reading all entities from the catalog or all pages from TechDocs.
2. Decorators — enrich documents with additional metadata, such as linking a document to its owning entity so results can be filtered by team.
3. Indexers — write the finished documents into the search engine.

search:
  backend:
    engine: postgres
    auth:
      enableProtectedResult: true

One collator can produce many documents, and several collators can run in parallel. This pipeline runs in the background, continuously syncing the index with changes in the sources.

#Running and Verifying Indexing

The indexing pipeline runs automatically in the backend, but when you first set up search, you need to make sure a collator is actually pulling data:

curl -s http://localhost:7007/api/search/index -H "Authorization: Bearer ${BACKSTAGE_TOKEN}"

The response shows the indexed documents per collator. If this number doesn't grow after new data is added to the catalog, check the backend logs for errors in the related collator or indexer. Also schedule periodic synchronization — a stale index only misleads developers.

If you run several backend instances, make sure only one instance handles indexing (or use a task scheduler) so collators don't duplicate work.

#Search Types

Each collator tags its documents with a search type — an identity that determines how results are rendered and filtered. Common built-in types include software-catalog for entities, techdocs for documentation, and types from other collators for their respective sources. When the frontend receives results, this type is used to pick the right display — entity results show kind and owner, TechDocs results show the documentation path.

#The Query API

The frontend talks to the backend search through the query API. A query combines a search term, type filters, and pagination options; the backend search forwards it to the engine and returns scored results. The relative URL used is /api/search/query — calling it from a frontend plugin is the same single call for every source type, keeping the search experience uniform.

On the security side, remember the lesson from episode 15: search results can contain sensitive information, so make sure protected results don't leak to unauthorized users. Backstage provides a protected results mechanism that filters documents by permission — enable this feature before search is used widely.

#Comparing Search Providers

Official providers are available as search backend modules, for example for Stack Overflow and GitHub issues. The important part: every provider only needs a collator and an indexer — the engine and query API are shared.

#Custom Collators

When a source has no official provider, you write a custom collator — a function that produces search documents from any source Backstage can access. Collator modules are registered in @backstage/plugin-search-backend-node:

const myCollator = {
  async execute() {
    const documents = [];
    const services = await myInternalApi.listServices();
    for (const service of services) {
      documents.push({
        title: service.name,
        text: service.description,
        location: `/service/${service.slug}`,
        metadata: { owner: service.owner },
      });
    }
    return documents;
  },
};

The produced documents are then registered as a new search type and indexed through the same indexer. With this pattern, any collator — from an internal registry to a company wiki — can enter Backstage's single search box.

#Conclusion

Episode 16 delivered search & discovery: a three-layer architecture, the indexing pipeline with collators, decorators, and indexers, search types for controlling rendering and filtering, a uniform query API, built-in providers for the catalog, TechDocs, Stack Overflow, and GitHub issues, and custom collators for internal sources.

The key takeaways:

• Collators produce, decorators enrich, indexers store — three roles in one pipeline.
• Search type determines the result experience — not just filters, but also how results render.
• One query API for all sources — the frontend doesn't care where documents come from.
• Custom collators open any source — from internal registries to wikis.

In the next episode, episode 17, you enter Phase 5: frontend plugin development — how to build plugins with the New Frontend System, from createFrontendPlugin and extension points to React hooks like useEntity, styled with MUI.