Tuned a retrieval layer across 400k documents, then treated quality as an engineering problem: chunking strategy, reranking, and explicit refusal behaviour all tracked as a regression suite.
I build the parts of model systems that decide whether they survive contact with production — retrieval, evaluation, and the unglamorous plumbing between a demo and something a team can rely on.
Tuned a retrieval layer across 400k documents, then treated quality as an engineering problem: chunking strategy, reranking, and explicit refusal behaviour all tracked as a regression suite.
Made agent behaviour measurable: recorded real runs, scored tool selection and task completion, and gated merges on the score so regressions surfaced before users found them.
Compressed and served a small model for offline use, trading accuracy against latency and memory until it ran acceptably on hardware with no GPU and no reliable connection.
Built the boring half of fine-tuning: dataset construction, leak checks, and a held-out set that changed how we judged every subsequent checkpoint.
Turned prompt edits into reviewable diffs with pass/fail cases, so changing a system prompt stopped being an act of faith.
Most model work fails for ordinary reasons — bad retrieval, no measurement, costs nobody watched. That is the part I take seriously.
Building the actual product around a model: tool use, structured output, retries, fallbacks, and guardrails that fail loudly instead of quietly.
Ingestion, chunking, embeddings, hybrid search, reranking — and knowing which of those actually moved the number before adding the next one.
Datasets that reflect real traffic, scoring that a teammate can argue with, traces that explain a bad answer instead of just recording it.
Serving, caching, batching, quantization and routing. Latency and spend treated as product decisions, not as surprises on an invoice.