Complete the Roadmap —
Live Sessions, Step by Step

Data structures, comprehensions, generators and patterns used in real pipelines

Read/write CSV, JSON, Parquet and Avro with robust error handling

Cleaning, reshaping, merging and aggregating large datasets

requests, pagination, rate limiting, saving responses to files and databases

Functions, classes, config files and project structure used in production

try/except, logging module, retry logic and alerting patterns

venv, requirements.txt, .env secrets — keeping projects reproducible

Running scripts as jobs, passing arguments, exit codes and scheduling hooks

Curated questions covering all topics — solve during and after the session

ROW_NUMBER, RANK, DENSE_RANK, LAG, LEAD, running totals, moving averages

WITH clauses, chained CTEs, recursive hierarchy traversal

INNER, LEFT, FULL OUTER, CROSS joins; UNION, INTERSECT, EXCEPT

Multi-level grouping, GROUPING SETS, ROLLUP, CUBE

Scalar subqueries, EXISTS, IN, correlated subqueries and when to avoid them

EXPLAIN, indexing strategies, partition pruning, statistics

Dialect differences, Delta Lake SQL, partitioned table queries

Star schema, slowly changing dimensions, normalisation vs denormalisation

Real-style questions from Deloitte, KPMG, Fractal, EXL — solved live

Driver, executors, DAG, stages, tasks, shuffle, spill and memory model

When to use each, transformations, actions and lazy evaluation

map, filter, flatMap vs groupBy, join, repartition — shuffle cost

Sort-merge join, broadcast join, skew join, repartition vs coalesce

Parquet, Delta, ORC, CSV — schema inference, partition discovery

Broadcast variables, AQE, partition tuning, caching, persist levels

Salting technique, broadcast small tables, custom partitioners

Incremental loads, MERGE, idempotent pipelines and error handling

Coding and conceptual questions asked at top DE interviews

Choosing a realistic dataset, defining Bronze/Silver/Gold layers

Pull source data via API or file, validate schema, write to Bronze layer

Clean, deduplicate and aggregate using SQL on the Silver layer

Read Bronze, apply transformations, write partitioned Parquet to Silver

Bronze raw, Silver cleaned, Gold analytics — folder structure and naming

Structured logs, try/except, retry and failure alerting patterns

Folder layout, .gitignore, requirements.txt and meaningful commit messages

Architecture diagram, tech stack, how to run, and results summary

Walk-through template: problem → design → implementation → trade-offs → results