0 views
Read
#0
Learn Apache Spark - Pre-Requisite Skills & Environment Setup
Before touching Apache Spark, you need to master basic data engineering, be familiar with JVM/Python/Scala, and understand the concepts of distributed computing. In this episode you set up the Java JDK, install Spark, and verify your first installation.
August 10, 2026
00
Read
#1
Learn Apache Spark - History, Background & Why Choose Spark
This episode explores the history of Apache Spark, its evolution from the disk-bound Hadoop MapReduce toward in-memory computation, and the advantages of a unified batch and stream API. You also understand the comparison between Spark and MapReduce, Flink, and Beam.
August 10, 2026
00
Read
#2
Learn Apache Spark - Spark Fundamentals & Architecture
This episode breaks down Apache Spark's architecture: the role of the Driver, Executors, and Cluster Manager in running jobs, along with the core abstractions RDD, DataFrame, Dataset, and Spark SQL. You also understand DAG, stages, tasks, and the storage model with shuffle.
August 10, 2026
00
Read
#3
Learn Apache Spark - Installing & Running Spark
This episode puts into practice how to run Apache Spark: a standalone installation locally, entering spark-shell and pyspark, submitting jobs with spark-submit in both client and cluster deployment modes, and reading the Spark UI to monitor job execution.
August 10, 2026
00
Read
#4
Learn Apache Spark - RDD & Functional Transformations
This episode breaks down RDD, Spark's lowest abstraction: how to create one, transformations and actions, and the functional functions map, filter, flatMap, reduceByKey, and groupByKey. You also understand lazy evaluation and lineage for fault tolerance.
August 10, 2026
00
Read
#5
Learn Apache Spark - DataFrame & Spark SQL
This episode covers DataFrame and Spark SQL: creating DataFrames from CSV, JSON, and Parquet, SQL queries with temporary views, schema inference and explicit schemas, and the role of the Catalyst optimizer in efficient execution plans.
August 10, 2026
00
Read
#6
Learn Apache Spark - Dataset API & Strong Typing
This episode covers the Dataset API and strong typing in Spark. You learn about Datasets in Scala and Java, how encoders provide compile-time type safety, the structural differences between Dataset and DataFrame, and the two-way conversion between them.
August 10, 2026
00
Read
#7
Learn Apache Spark - Joins, Aggregations, & Window Functions
This episode covers Spark's core analytical operations: the various join types from inner to anti, groupBy and advanced grouping aggregations, window functions for time-based analytics, and the performance implications of joins and shuffle on large queries.
August 10, 2026
00
Read
#8
Learn Apache Spark - Data Sources & Storage
This episode covers how to connect Spark to HDFS, S3, JDBC, and the file system, as well as reading and writing various file formats such as Parquet, Avro, ORC, JSON, and CSV. You also learn partitioning, bucketing, and schema evolution strategies for data lakes.
August 10, 2026
00
Read
#9
Learn Apache Spark - Performance Tuning & Optimization
This episode covers how to optimize Spark performance: tuning memory, shuffle, and parallelism configuration, broadcast joins and caching, reading the Catalyst execution plan, and strategies to avoid data skew and shuffle overload on production workloads.
August 10, 2026
00