Learn Backstage - Advanced Catalog & Data Model

#Introduction

In episode 5, you learned about ingestion and processing: catalog-info.yaml files become entities. In episode 10, you created entities through the scaffolder. Episode 11 unites both from the data side: Advanced Catalog & Data Model. So far entities have looked like standalone cards — this episode shows that those cards are actually connected. Ownership, partOf, dependsOn, and providesApi form a web that explains who owns what, which components depend on what, and which system shelters them all. By the end of the episode, you'll also keep that web healthy.

#Entity Relations

#Relations Connect Entities

An entity almost never stands alone. Backstage models relationships between entities through relations, declared in the spec section of catalog-info.yaml. The most fundamental relations to understand:

#Relations in One File

All of the relations above can be expressed in a single catalog-info.yaml:

apiVersion: backstage.io/v1alpha1
kind: Component
metadata:
  name: payment-api
  description: API pembayaran internal
spec:
  type: service
  owner: group:platform-team
  system: payments-system
  dependsOn:
    - resource:payment-database
    - component:notification-service
  providesApis:
    - payment-api

Reading the block above: payment-api is owned by group:platform-team, is part of payments-system, depends on payment-database and notification-service, and provides the API payment-api. The catalog translates each of these lines into a two-way relation — from payment-api you can see what it owns, and from payment-database you can see who depends on it.

#Modeling Systems, Domains, and Resources

#Four Kinds Working Together

Entity relations would feel pointless without the kinds that shelter them. These four kinds are the backbone of modeling:

• Domain — the largest business area, for example payments or logistics.
• System — a collection of components working together to fulfill one goal, the shelter for many partOf relations.
• Resource — non-software dependencies like databases, clusters, or buckets.
• Component — real software units: services, libraries, websites.

apiVersion: backstage.io/v1alpha1
kind: System
metadata:
  name: payments-system
  description: Sistem pembayaran terpadu
spec:
  owner: group:platform-team
  domain: fintech-domain

A System defines the owner and domain that shelter it, while components link themselves through system: payments-system. As a result, the catalog can answer hierarchical questions: which domain shelters this system, what systems live inside it, and what resources all of its components use.

#Resources as Dependencies

Resources capture the fact that a component depends on something that isn't a component. A managed database, a Kafka cluster, or an S3 bucket are each represented as a Resource entity with spec.type like database, message-queue, or object-store. Components that use them write dependsOn to that resource — and from there the catalog draws a visible dependency graph.

#Catalog Filters

#Filtering Entities in the API and UI

A large catalog can hold thousands of entities. Catalog filters narrow the view by kind, type, tag, or owner. In the UI, filters are picked from dropdowns; in the API, filters are sent as query parameters — multiple filter values are OR-ed, while values within one filter are AND-ed.

curl "http://localhost:7007/api/catalog/entities?filter=kind=component,metadata.tags=backend"

The query above fetches entities with kind=component and the tag backend. The filter format is very useful for automating entity discovery — for example pulling all services owned by one group for a dashboard or an ownership report.

#Entity Validation and catalog-validator in CI

#Validating Before It Enters

Catalog quality starts with the source files. The @backstage/catalog-model package provides validateEntity to check that an entity meets the rules: required fields present, correct data types, and valid structure. Running this validation in CI means a pull request carrying a broken catalog-info.yaml is rejected before it ever touches the catalog.

import { validateEntity } from '@backstage/catalog-model/validation';
import { parseEntityYaml } from '@backstage/catalog-model';
 
export async function validateCatalogFile(filePath: string) {
  const entity = parseEntityYaml(filePath);
  await validateEntity(entity);
}

validateEntity throws an error when an entity is invalid, so a CI script can fail the build immediately. It's simple but effective: the whole team gets the same feedback within seconds, not after the entity has entered the catalog.

#catalog-validator as a CI Step

Add this validation as a pipeline step — that's what catalog-validator in CI means. The script reads all catalog files changed in the pull request, validates them, and only continues if everything passes. The more catalog-info.yaml files are declared manually, the more important this step becomes.

yarn validate:catalog

The validate:catalog command is a script that calls validateEntity for every catalog-info.yaml file in the repository. With this step in CI, structural errors are caught at the earliest and cheapest point to fix.

#Unprocessed Entities

#The Queue Waiting for a Processor

When a new location is registered, the entities it finds aren't ready right away. Each is an unprocessed entity — entering a queue and waiting for a catalog processor (like the GitHub, URL, or file processor) to turn it into a full entity. This process runs periodically, and its results trigger a refresh until the entity is ready to display.

Unprocessed entities appear in the catalog with a status indicating they haven't been processed. If an entity waits in this state for a long time, its source is most likely unreachable or the processor failed — and that's where debugging starts: check whether the target URL is reachable, whether the access token is still valid, and whether the file format is correct.

#Stale Entity Removal

#Cleaning Up What No Longer Exists

An entity's source can disappear: a repository is deleted, a catalog-info.yaml file is removed, or a location is revoked. An entity whose source no longer exists is called a stale entity. The catalog runs periodic cleanup to detect entities like this and remove them, so the catalog doesn't show cards for things that are no longer alive.

This process compares registered entities against their sources. When a source no longer exists, the entity is flagged and eventually removed. This automation matters: a clean catalog is a trustworthy catalog — because every card displayed represents something genuinely running in your organization.

#Conclusion

In this episode 11, you understood the catalog data model completely: ownership, partOf, dependsOn, and providesApi relations, modeling System, Domain, and Resource alongside components, catalog filters in the API and UI, entity validation with validateEntity and catalog-validator in CI, an understanding of unprocessed entities, and periodic removal of stale entities.

The key takeaways:

• Relations are a web, not labels — ownership, partOf, dependsOn, and providesApi connect entities and produce an explorable graph.
• Domains and systems shelter components — hierarchical modeling lets the catalog answer questions from business down to resources.
• Quality starts in CI — validateEntity in the pipeline prevents broken catalog-info.yaml files from entering the catalog.
• The catalog needs maintenance — unprocessed entities signal a problematic source, and stale entities must be cleaned up periodically.

In the next episode, episode 12, we uncover the layer that has been storing all catalog and pipeline data: Persistence & Databases — how Backstage stores its data in SQLite for development and PostgreSQL for production, the role of Knex and migrations, and Redis cache and search index stores at scale.