Learn Apache Flink (a stateful stream processing framework for real-time data) from the fundamentals to production-grade: prerequisite skills & environment setup, background history & why choose Flink, core concepts & architecture, installation & running Flink jobs, DataStream API & core transformations, event time watermarks & windowing, state management & checkpointing, error handling & debugging stream jobs, source & sink integrations, Table API & SQL, complex event processing, job configuration & deployment, security authentication & authorization, data governance & observability, scaling & resource management, performance tuning, savepoints upgrades & migration, advanced streaming patterns, custom connectors & extensions, operational readiness & runbooks, real-time analytics & use cases, ecosystem & tooling, to future-proofing your Flink skills across 23 episodes.
Before you touch Apache Flink, you need to master the concepts of batch versus stream data processing, distributed systems, and the JVM plus JVM languages such as Java or Scala. In this episode you'll also set up JDK, Docker, and install Apache Flink to run your first local cluster.

This episode explores the history and background of Apache Flink, the evolution of stream processing from batch to real-time, and its comparison with Spark Streaming, Kafka Streams, and Beam. You'll also understand Flink's ideal use cases: stateful stream processing, event-driven analytics, CEP, and real-time pipelines.

This episode dissects Flink's core concepts: streams, the DataStream API, the Table API, and SQL. You'll understand the differences between event time, processing time, and ingestion time, as well as the concepts of stateful computation with state backends, checkpointing, and savepoints. Finally, the execution architecture is explained through the JobManager, TaskManager, task slots, and parallelism.

This episode takes you through installing a standalone Flink cluster hands-on and running your first job with the flink run command. You'll understand the Flink directory structure, configuration in config.yaml, reading logs, and get to know the job lifecycle and web dashboard on port 8081.

This episode is the most fundamental one for writing pipelines: you create your first Flink job with Java and use basic transformations like map, flatMap, filter, keyBy, window, and reduce. You'll also understand stream partitioning, parallelism, and key-based operations.

This episode dissects the most important time concept in Flink: event time versus processing time. You'll generate watermarks to handle late-arriving data, get to know the tumbling, sliding, and session window types, and configure window triggers, allowed lateness, and side outputs for late events.

This episode dissects how Flink stores state: keyed state with ValueState, ListState, MapState, and AggregatingState, as well as operator state. You'll configure checkpointing with exactly-once guarantees, choose a state backend, and understand restart strategies for fault tolerance.

This episode teaches how to deal with failures in streaming pipelines: catching exceptions in operators and sources, separating corrupt data into a side output, monitoring job status and logs, using savepoints for recovery, and debugging with a local cluster and flink run -d.

This episode connects Flink to the outside world: Kafka, Kinesis, RabbitMQ, and file sources as data sources, plus Kafka, databases, object storage, and Elasticsearch as sinks. You'll also understand the connector ecosystem and the JSON, Avro, Protobuf, and CSV data formats.

This episode introduces Flink's declarative path: the Table API and SQL. You'll use TableEnvironment and catalogs, write DDL for sources and sinks, and apply windowing, joins, and aggregations with Flink SQL. It also covers temporal tables and CDC patterns for changing data.
