Learn Elasticsearch - Pre-Requisites Skills & Environment Setup

#Introduction

Welcome to the Learn Elasticsearch series! This series will take you from zero to building and managing a production-grade Elasticsearch cluster: starting with prerequisites and environment setup, history, architecture, indexing, mapping, query DSL, aggregation, security, scaling, up to production deployment. A total of 31 episodes that build on one another.

Elasticsearch is a distributed, RESTful search and analytics engine built on Apache Lucene. It solves problems that ordinary relational databases cannot answer: relevant full-text search, real-time analytics, and horizontal scaling for large-scale data. If you have ever used Google, the "related posts" feature on a blog, or price filters on e-commerce sites, you have already experienced the output of machines like this.

Episode 0 is your roadmap and readiness check. Before typing the first command, we make sure of three things: (1) the basic skills you must master, (2) the software and tools you must install, and (3) the minimum hardware required. Don't skip ahead — a messy environment will make later episodes feel heavy. Let's get started.

#Basic Skills You Must Master

#Relational Databases and NoSQL

You must understand relational database concepts: tables, rows, columns, primary keys, and SQL queries. Elasticsearch does not use tables, but understanding the relational model helps you understand the differences — JSON documents have no rigid schema, and fields can appear or disappear per document. Think of Elasticsearch as a "database without tables": every record is a standalone JSON document.

#JSON

Elasticsearch speaks JSON — requests and responses are all JSON. You should be comfortable reading, writing, and validating JSON structures: objects with curly braces, arrays with square brackets, strings with double quotes, and no trailing commas. Most early Elasticsearch errors come from invalid JSON.

#HTTP/REST API Basics

Elasticsearch is a pure REST API. Every operation maps to an HTTP method: GET to read, PUT to create, POST to send data/actions, DELETE to remove. You should understand HTTP status codes: 200 OK, 201 Created, 404 Not Found, 409 Conflict. All interactions from here on happen over HTTP.

#Command Line Interface (CLI)

You will often type curl to test APIs, run services, and read logs. Master the terminal basics: navigation, running processes in the background, setting environment variables, and reading program output. curl is the "browser of the terminal" — our main weapon throughout this series.

#Indexing and Full-Text Search Concepts

Understand the difference between exact match (searching for an exact word) and full-text search (searching by meaning and relevance). The inverted index concept will be discussed in detail in episode 2, but the big picture is: Elasticsearch stores a map from each word to the list of documents containing that word, making searches extremely fast.

#Basic Networking

You need to understand ports, localhost, and IP addresses. By default Elasticsearch listens on port 9200 (HTTP) and 9300 (inter-node transport). Later when we discuss clusters, you will configure node discovery over the network — basic networking knowledge helps a lot.

Tip

Not every skill needs to be mastered perfectly right now. What matters is that you have touched each of these concepts, because this series will keep revisiting and deepening them in an Elasticsearch context. Top priority: JSON and basic curl, since both are used in nearly every episode.

#Software and Tools to Prepare

Tool	Purpose	Notes
JDK 17+	Runs Elasticsearch (Java-based)	A bundled JDK is also available in the Elasticsearch distribution
Elasticsearch 8.x	The search and analytics engine itself	Version 8.x is the current stable version
Kibana	UI for exploration, Dev Tools, visualizations	Version must match Elasticsearch exactly
cURL / Postman	Testing REST APIs	cURL is already available on almost every system
Docker + Docker Compose	Deployment and multi-node testing	Optional, used starting in episode 27
VS Code / IntelliJ	Writing configs, scripts, and integrations	Any editor you're comfortable with

Elasticsearch and Kibana must use the same major version. Combining Elasticsearch 8.15 with Kibana 8.13 is not recommended — always match versions to avoid incompatibility bugs.

Important

Since version 8.0, security is enabled by default with TLS encryption and an elastic user password generated automatically on first start. This is a major change from the 7.x era. Don't worry — we'll cover it in detail in episodes 15 and 16, and during the installation process in episode 3 you'll see the superuser password in the terminal output.

#Hardware Minimum Requirements

Resource	Minimum	Recommended
RAM	4 GB	8 GB or more
Storage	10 GB free space	20 GB+ (SSD is much better)
CPU	Dual-core	Quad-core or more
OS	Linux (primary), macOS, Windows	Linux for production

Remember the golden rule of Elasticsearch JVM settings: half of the total RAM is allocated to the JVM heap, up to a maximum of 32 GB. If you have 8 GB of RAM, a 4 GB heap is the sweet spot. Allocating above 32 GB is counterproductive because Lucene starts wasting memory on compressed oops. Episode 14 will cover this in more depth.

#Verifying Your Environment

Let's make sure all the foundations are installed. Open a terminal and run the following commands:

Verify basic tools

java -version
curl --version
python3 --version

Example output of java -version

openjdk version "17.0.13" 2024-10-15
OpenJDK Runtime Environment (build 17.0.13+11)
OpenJDK 64-Bit Server VM (build 17.0.13+11, mixed mode, sharing)

If java is not installed yet, install JDK 17 or newer. On Ubuntu/Debian: sudo apt install openjdk-17-jdk. On Rocky/CentOS: sudo dnf install java-17-openjdk.

To verify your JSON skills, try a simple REST operation against a public API without installing anything. One stable public endpoint is https://api.github.com/users/octocat:

Your first REST API call

curl -s https://api.github.com/users/octocat

If you see a well-formed JSON response, your HTTP/REST and JSON understanding is ready. As an exercise, try filtering the output with Python: curl -s https://api.github.com/users/octocat | python3 -m json.tool, which will "comb" the JSON to make it readable — while also validating that the response is valid JSON.

#Common Pitfalls

Elasticsearch and Kibana versions don't match. Always match the major version. Check with kibana --version and compare it with the Elasticsearch version.
Too little RAM. Running Elasticsearch with too small a heap triggers the unable to create native thread error or OOM during indexing. Make sure at least 4 GB of RAM is available for a test machine.
Using a conflicting port. If port 9200 is already used by another application, Elasticsearch fails to start. Check with ss -tlnp or netstat -tlnp before starting.
Forgetting the superuser password. Since version 8.x, the elastic user password is printed only once on first start. Store it safely, because it cannot be viewed again — a manual reset requires extra steps.
Expecting to learn Elasticsearch for beginners in 5 minutes. This is a feature-rich engine; don't get frustrated if shard, analyzer, or mapping concepts feel like a lot. This series is structured step by step — each episode covers just one concept.

Note

The right debugging culture: read the full error message before asking the internet. Elasticsearch errors are almost always explicit — they mention a conflicting port, insufficient heap, or a missing file. Match that message against the step you just ran, then fix issues one by one.

#Conclusion

In episode 0 you have ensured three foundations: basic skills (database, JSON, HTTP/REST, CLI, indexing concepts, and networking), the right software (JDK 17+, Elasticsearch 8.x, a matching Kibana version, cURL/Postman, and optional Docker), and adequate hardware (minimum 4 GB RAM, 10 GB storage, dual-core CPU — with a heap of half your RAM recommended).

Key takeaways:

Elasticsearch is a distributed RESTful search engine — all interaction happens over HTTP and JSON.
Elasticsearch and Kibana versions must match.
Security is enabled by default since 8.0 — save the elastic password printed on first start.
The ideal JVM heap is half of RAM, up to 32 GB.
curl and JSON are two skills that will accompany you throughout this series.

The Learn Elasticsearch series consists of 31 episodes that build on each other: from history, core concepts, installation, mapping, query DSL, aggregation, ILM, data streams, ingest pipelines, to security, scaling, monitoring, and production deployment. Episode 0 is the first brick. In episode 1 we'll step back for a moment to understand the history and background: why Elasticsearch was born, which problems relational databases couldn't answer, and the journey from Apache Lucene to the Elastic Stack we know today. See you in episode 1!