Model Evaluation workshop
Build an LLM Evaluation Harness
Stop judging your GenAI feature by demos. Leave with an automated eval suite.
- 2 days
- Hands-on in AI Governance Platform
- On your own system
The problem
Most teams ship prompt and model changes on gut feel, so quality silently regresses. This workshop turns your use case into a repeatable, scored test suite.
Who it's for
ML engineers, data scientists, and product owners shipping an LLM, RAG, or agent feature.
What to bring
One live or near-live LLM use case and 20 to 50 real user questions.
The course side
What your team learns
- 1
Define what “good” means
Success criteria and CAL targets (consistency, accuracy, latency) for your use case.
- 2
Build a golden dataset
Positive, negative, boundary, and adversarial scenarios from your own data.
- 3
Score automatically
Assertion, rubric, and classifier-based scoring rolled up to a success rate.
- 4
Gate releases
Wire the suite into CI so a prompt or model change that degrades quality is blocked.
Builds on these course lectures
- Accuracy Evaluation
- GenAI Model Accuracy / Consistency Challenges - Experiments
- Domain Specific Benchmarks - LegalBench vs CorpFin
The platform side
What your team builds in AI Governance Platform
- Register the model or feature in the AI inventory
- Attach the golden dataset and eval results as assurance evidence
- Set the release gate: the score a change must hold to ship
Also used: Promptfoo, Giskard, Classifier-based scoring
About AI Governance Platform →Your team leaves with
- • A golden dataset for your use case
- • A running eval suite with a baseline score
- • Go/no-go release criteria
Book this workshop
Request Build an LLM Evaluation Harness
It's pre-selected in the form. Add others to combine them. Contact us for pricing: fill in your details and we'll reply within one business day.