Learn Pentaho - Core Concepts & Pentaho Architecture

#Introduction

Now we reach the heart of this series: Pentaho's architecture and core concepts. You already know its history and why to choose it. In this episode you'll understand the roadmap — what components exist, who plays which role, and how data flows from raw sources to dashboards.

After this episode, terms like Spoon, Pan, Kitchen, Carte, repository, transformation, and job will no longer sound foreign. Treat this episode as a blueprint you'll keep referring back to across all the following episodes.

#Main Components of the Pentaho Architecture

Pentaho's architecture is divided into two interconnected worlds: data integration (PDI/Kettle) and business analytics (Pentaho Server). Let's get to know the main components one by one.

#PDI: Spoon, Pan, Kitchen, Carte

PDI is the ETL engine. It consists of four executables with different roles:

• Spoon: the desktop graphical editor. This is where you design transformations and jobs visually. It's the "main workspace" of a Pentaho developer.
• Pan: the command-line executor for transformations (.ktr). Ideal for running ETL from scripts, cron, or CI/CD without a GUI.
• Kitchen: the command-line executor for jobs (.kjb). Jobs handle orchestration — when to run a transformation, when to send an email, and so on.
• Carte: a lightweight server that runs transformations and jobs remotely, and can be clustered for parallel execution.

All four tools share the same foundation: a Java execution engine that reads XML file definitions.

#Pentaho Server, Repository, and Metadata

• Pentaho Server (BI Server): a Java web application where transformations and jobs are uploaded, scheduled, and accessed by many users. This is where reports and dashboards live.
• Pentaho Repository: the object storage on the server — transformations, jobs, reports, and database connections. There are several repository types (file-based and database-based), which will be covered in episode 9.
• Metadata layer: an abstraction layer that hides the complexity of database schemas from business users, for example giving columns friendly names or defining calculations in one place.

#Pentaho User Console

Pentaho User Console (PUC) is the web interface inside the Pentaho Server. From here, users can run reports, view dashboards, access repository files, and check the status of scheduled jobs. If Spoon is the design studio, PUC is the gallery where the results are displayed.

#Data Workflow: From Input to Dashboard

Now let's follow data's journey from end to end. Pentaho is designed so this flow is seamless:

1. Input: data comes in from various sources — relational databases, CSV files, Excel spreadsheets, XML, JSON, all the way to REST APIs and Hadoop.
2. Transformation: inside a transformation, data flows as a row stream through steps: cleaned, joined, filtered, aggregated, and with data types converted.
3. Output: results are written to targets — warehouse databases, flat files, cloud storage, or directly to reporting data sources.
4. Orchestration: jobs manage when and in what order all of that happens, including error handling and notifications.
5. Analysis and presentation: the Pentaho Server takes the results for reports, dashboards, and OLAP analysis.

This pattern is known as a pipeline: each stage produces the output that becomes the input of the next stage.

#Transformation vs Job

These two terms are the foundation of PDI, and telling them apart correctly will save you a lot of confusion:

By analogy: a transformation is a recipe for turning one ingredient into one dish; a job is the kitchen's menu order — what gets cooked first, when the oven starts, and when to serve.

#PDI Folder Structure on Your Machine

When you extract PDI, its folder structure tells the story of the architecture above. Let's look from the terminal:

pdi-ce-9.4.0.0-343/
├── spoon.sh
├── pan.sh
├── kitchen.sh
├── carte.sh
├── lib/
├── plugins/
├── system/
└── ui/

Notice the four script files at the root: they are the entry points to Spoon, Pan, Kitchen, and Carte. The lib folder contains Java libraries, plugins holds additional steps, and system contains global configuration files. You can view its contents from the terminal with ls pdi-ce-9.4.0.0-343.

#Running a Transformation from the Command Line

One of the fastest ways to understand the architecture is to run a transformation from the command line. Even before you create your own files, know the shape of the basic commands. Pan accepts .ktr files, Kitchen accepts .kjb files:

pan.sh -file=/home/kalian/lab/transformations/hello.ktr

kitchen.sh -file=/home/kalian/lab/jobs/orchestrator.kjb

If you run them without a file, both tools will print the full list of supported parameters — the best way to explore on your own. Try pan.sh -help in your terminal and see how many options are available.

#Conclusion

In episode 2 you understood the Pentaho architecture map: the PDI components (Spoon, Pan, Kitchen, Carte), the role of the Pentaho Server and repository, the data flow from input to dashboard, and the fundamental difference between transformations and jobs.

The key takeaways:

• Spoon for design, Pan for transformations, Kitchen for jobs, Carte for remote and clustered execution.
• Transformations process data in a row stream; jobs orchestrate larger steps.
• The Pentaho Server connects the ETL world to reporting and dashboards for business users.
• The PDI folder structure mirrors its architecture: scripts at the root, with plugins, configuration, and libraries neatly separated.

In episode 3, we'll get hands-on: installing Pentaho Data Integration and getting to know Spoon — from downloading, extracting, and setting up JDK, to navigating the UI and creating your first database connection.