reference · tools

Tools

A minimal set for each job — flagged for learning vs production.

For learning (zero-install)

The bare minimum to start today: SQL, Python, dashboards and a portfolio — all runnable in the browser or with one command, no setup hassle.

DuckDB

SQL locally · zero-install

A SQL engine that reads CSV and Parquet straight off disk — no server to set up. The best way to start learning SQL: grab a dataset and start querying right away (Level 1).

Google Colab + pandas

Python in the browser · free

Python notebooks right in the browser — nothing to install. Paired with pandas it's your main tool for cleaning data, EDA and your first API calls (Level 2).

ClickHouse Playground

SQL online · zero-install

Write SQL against big real-world datasets right in the browser, with hundreds of ready-made example queries next to them to learn from. Perfect for drilling SQL before you install anything locally (Level 1).

Metabase

BI · no-code

A friendly BI tool: connect a database and build dashboards with almost no code. Being able to present metrics cleanly stands out in a portfolio (Level 3+).

Tip: When dashboards start piling up, there's a beefier alternative — Apache Superset.

Git / GitHub

portfolio · free

Every project is a repo with a README (question → data → what you found). In tech this is your living résumé — employers look here first, so start one from day one, even if it's messy.

Data engineering

The building blocks of a real pipeline: what loads the data, what transforms it, where it lives, and what conducts the whole thing.

dlt

ingestion (EL) · Python

A Python library that loads data from APIs, databases and files into your warehouse: it infers the schema, flattens nested JSON and handles incremental merge-by-key. Replaces a pile of brittle ETL scripts with a few lines (Level 3).

Tip: It ships verified sources (Stripe, GitHub, SQL DBs) and destinations — swapping DuckDB for BigQuery is a one-line change.

dbt

transformations as code · SQL

The industry standard for the transformation layer (the T in ELT): you write models in SQL and dbt builds the dependency graph, runs them in the right order, tests the data and generates docs with lineage (Level 2).

Tip: A rising alternative is SQLMesh: it understands SQL more deeply and gives you virtual data environments. Worth knowing, but dbt is the easier start.

PostgreSQL

relational DB · all-rounder

A rock-solid, all-purpose relational database. Not the fastest for heavy analytics, but simple, familiar and more than enough to start — a great home for your dbt marts in a pet project.

ClickHouse

columnar · big data

A blazing-fast columnar database for analytics at scale and in real time: it chews through billions of rows and loves event data. The workhorse of serious product analytics.

Parquet

columnar format · standard

A compressed columnar format — the de facto standard for analytical data: far smaller than CSV and faster to read because it loads only the columns you need. Rule of thumb: store data as Parquet between pipeline steps, not CSV.

Airflow

orchestration · industry standard

The most widely used orchestrator: pipelines are Python DAGs, with a huge ecosystem of integrations and the most job-market demand. Heavier to set up — but if you learn one grown-up orchestrator, make it this one (Level 4).

Prefect

orchestration · Pythonic

A lighter, more Pythonic orchestrator: plain functions become tasks and flows via @task/@flow decorators, with retries and caching built in. Reach for it on your first pipeline to feel orchestration without the pain (Level 4).

Tip: Guide's advice: start with Prefect, learn Airflow second — it's the one interviews ask about.

Statistics

What you compute with: a flexible Python stack (like on the job) and no-code apps to start — click your way through first, then redo it in code.

scipy.stats

Python · tests

The base kit of tests and distributions: t-test, chi-square, correlations, plus the normal and other distributions. The first thing you import once you get to hypothesis testing (Level 3).

statsmodels

Python · regression

Regressions and models with grown-up output: coefficients, p-values, confidence intervals, R². What you want for relationships, regression and any serious analysis (Level 4).

pingouin

Python · friendly

A high-level library: one-line tests that already return the effect size and confidence interval — the bits scipy makes you compute separately. Beginner-friendly.

Tip: If scipy feels too low-level, pingouin hands you effect size and CIs out of the box — less manual work.

seaborn

Python · charts

Distributions and relationships visualized in one line: histograms, boxplots, scatter with a regression line. Plus built-in datasets (penguins, tips, titanic) that load instantly — perfect for the 'draw the data first' rule.

JASP

no-code · free

A free point-and-click app: t-tests, ANOVA, regression, correlations and even Bayesian versions — without a single line of code. Great for grasping the method itself without fighting syntax.

jamovi

no-code · free

A JASP-style alternative: a friendly interface for the core tests and models, and a comfy bridge from Excel to real statistics. If one doesn't click, try the other — they're similar.

AI layer

The layer on top of analytics that makes a pet project 'smell like a startup': models, semantic search and language models over your own data.

Hugging Face

AI hub · models & datasets

The main AI hub: thousands of datasets and models (text, images, audio) in one place, all pulled into Python with a one-line load_dataset(). Grab a ready sentiment model or embeddings for your project here.

Chroma

vector DB · for AI

The simplest vector database for your first semantic search: drop in texts and query by meaning, not exact words. When you reach the AI layer, start here.

Tip: Chroma is simpler than Pinecone or Weaviate — start with it and leave the heavy vector DBs for later.

Ollama

local LLMs · free

Runs language models locally on your own machine with one command — no cloud, no keys, no per-token bill. Handy for experimenting with models privately while you bolt an AI layer onto your analytics.

LLM API

language model · the brains

Wire up a language model over an API and build a tool on top: summarization, classification, 'a question in plain language → SQL over your dataset → an answer from the data'. That's basically a product.

Tip: Order matters: the AI layer sits on top of analytics, it doesn't replace it. Learn to question data first (Levels 1–3), then bolt on the models.