9. Project structure & packaging

Intermediate · 7 min read

A notebook is great for experiments; a project is what you deploy, test and hand to a team.

9.1 A layout that scales

rag-assistant/
├── pyproject.toml          # project metadata + dependencies + tool settings
├── .env.example            # the settings needed, with dummy values (commit this)
├── .env                    # real secrets (never commit — add to .gitignore)
├── README.md
├── src/
│   └── rag_assistant/
│       ├── __init__.py
│       ├── config.py       # settings loaded from the environment
│       ├── ingest.py       # load → clean → chunk → index
│       ├── retrieve.py     # search the index
│       ├── llm.py          # the only module that talks to the LLM API
│       └── api.py          # web endpoints (FastAPI / Flask)
└── tests/
    ├── test_ingest.py
    └── test_retrieve.py

Keep one module per responsibility, and put all LLM calls behind llm.py so you can swap providers or fake them in tests in one place.

9.2 Settings in one place

config.py

import os
from dataclasses import dataclass

@dataclass(frozen=True)              # frozen: settings can't be changed by accident
class Settings:
    model: str = os.getenv("LLM_MODEL", "gpt-4o-mini")
    temperature: float = float(os.getenv("LLM_TEMPERATURE", "0.2"))   # env vars are strings
    top_k: int = int(os.getenv("RAG_TOP_K", "4"))

settings = Settings()                # import this object everywhere else
print(settings.top_k)                # → 4

Larger projects often use pydantic-settings, which does the same with validation.

9.3 pyproject.toml

pyproject.toml

[project]
name = "rag-assistant"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "openai>=1.40,<2",
    "rank-bm25>=0.2",
    "fastapi>=0.110",
]

[project.optional-dependencies]
dev = ["pytest>=8", "ruff>=0.5"]

[tool.ruff]
line-length = 110

[tool.pytest.ini_options]
testpaths = ["tests"]

pip install -e ".[dev]" installs your project in editable mode plus the dev tools, so imports like from rag_assistant.retrieve import search work everywhere.

9.4 Code quality tools

Tool	What it does	Command
ruff	Finds bugs and style problems; can also format	ruff check . · ruff format .
mypy	Checks your type hints	mypy src
pytest	Runs tests	pytest
pre-commit	Runs the above automatically before each commit	pre-commit install

Why it matters for GenAI

Interviewers and clients judge GenAI projects on engineering, not just the demo. A clean layout, config from the environment, tests and a pyproject.toml signal "production-ready".

Practice

• Reorganise a single-file script into config.py, retrieve.py, llm.py and a tests/ folder.