João Pedro Matias, home

COURSES

WHAT I'M STUDYING, IN PUBLIC

ABOUT

Short, visual courses I build while learning a new tool or concept, with animated diagrams, practice questions and milestones I only mark done after shipping real work. Open any lesson and try the practice yourself. Each course is generated with my open-source learn skill for Claude Code.

2TOPICS
9LESSONS

2026.10.19 · 5 LESSONS

Building Reliable AI Agents

Publish a reproducible TypeScript benchmark that compares two versions of my agent

MILESTONES0/5 MASTERED
  • Measure
  • Dataset
  • Trace
  • Gate
  • Benchmark
  1. 01Why LLM outputs are hard to testreplace an exact-match test with property checks over many trials and read pass@k vs pass^k→
  2. 02Designing an evaluation datasetwrite outcome-graded, solvable, balanced tasks and check a judge with TPR and TNR→
  3. 03Tracing and debugging agent failuresfind the first wrong step in a trace and turn failures into a counted taxonomy→
  4. 04Ship an evaluation pipelinecompare versions with a paired confidence interval and gate pull requests on it→
  5. 05Final: publish the benchmarkhold a benchmark write-up to a reviewer's standard: reproducible, comparable, honest→
OPEN COURSE →

2026.10.15 · 4 LESSONS

Jev (TypeSafe AI)

Use Jev’s typed decisions in products and agent workflows, and know when it beats an LLM.

MILESTONES0/4 MASTERED
  • Pick Jev vs LLM
  • Write solid calls
  • Tune thresholds
  • Ship one decision
  1. 01Jev: the smart if statementpick the right primitive and read confidence bands→
  2. 02Writing questions Jev answers wellspot and fix the six question-writing mistakes→
  3. 03Tuning your bands on real datapick the lowest line that meets an accuracy target, per action→
  4. 04Ship one decisionmake a Jev call fail safe, pin its model, and gate an agent loop→
OPEN COURSE →