data-platform

A Dagster + dbt data platform, managed with uv and deployed via Docker Compose.

Stack

Layer	Tool
Orchestration	Dagster (webserver + daemon)
Transformation	dbt-core + dbt-postgres
Storage	PostgreSQL 16
Package/venv	uv
Secrets	.env file

Project layout

data_platform/      # Dagster Python package (assets, definitions)
dbt/                # dbt project (models, seeds, tests)
  profiles.yml      # reads credentials from env vars
dagster_home/       # dagster.yaml + workspace.yaml
Dockerfile          # single image used by both dagster services
docker-compose.yaml # postgres + dagster-webserver + dagster-daemon
.env.example        # copy to .env and fill in credentials
pyproject.toml      # uv-managed dependencies

Getting started

# 1. Install uv (if not already)
curl -Lsf https://astral.sh/uv/install.sh | sh

# 2. Clone and enter the project
cd ~/git/data-platform

# 3. Create your credentials file
cp .env.example .env
# Edit .env with your passwords

# 4. Install dependencies into a local venv
uv sync

# 5. Generate the dbt manifest (needed before first run)
uv run dbt parse --profiles-dir dbt --project-dir dbt

# 6. Start all services
docker compose up -d --build

# 7. Open the Dagster UI
#    http://localhost:3000

Local development (without Docker)

uv sync
source .venv/bin/activate

# Run the Dagster UI locally
DAGSTER_HOME=$PWD/dagster_home dagster dev