Lesson 4 · final lesson · ~12 minutes

Ship one decision

You can now write a good Jev question and tune its bands. This lesson makes it safe in production and shows where Jev fits inside an AI agent.

Your win for today: write a Jev-backed function that stays correct when Jev is wrong, slow, down or upgraded, and know the three places Jev fits in an agent loop.

The production question isn't "is Jev right?"

Lesson 3 measured how often Jev is right. Production adds three other ways things go wrong: the call is slow, the call fails, or the model changes underneath you. TypeSafe's design guide gives the principle that handles all three: "Code handles deterministic work and owns the control flow. The model appears only where the system needs programmable common sense."1

Mental model: every Jev call has four exits, and you design all of them. Confident leads to act. Unsure leads to a human or a bigger model. Error or timeout leads to a safe default. And every exit is logged, so you can re-tune. Where the model breaks down: in an agent, the "safe default" can be "stop and ask", which is a valid exit too.

The log lines and the re-tune output in this scene are illustrative. APITimeoutError is a real error class in the JS SDK.2

The hidden default that can sink you

The SDK's defaults suit a batch job, not a request that a user is waiting on:34

SettingDefaultWhy it matters
defaultModeljev-latestYour tuned lines silently stop applying when a new version ships.
timeout10 000 ms per attemptThere's no total budget across retries.
retry.maxRetries2, with 500 ms backoff doubling up to 5 sRetries also fire on timeouts, 429s and 5xx errors.

So in the worst case, one "100 ms" decision can block for three 10-second attempts plus backoff, which is over 30 seconds. In a user-facing path, set a tight budget and fail safe instead.

The minimal reference model

// decide.ts: one production-safe Jev decision
import { choice, TypeSafeClient } from '@typesafe-ai/sdk'

const MODEL = 'jev-1.13.0'                    1
const LINES = { act: 0.8, confirm: 0.5 }      2

const client = new TypeSafeClient({
  timeout: 1500,                              3
  retry: { maxRetries: 1 },
})

type Decision = {
  action: 'assign' | 'suggest' | 'triage'
  team?: string
  reason: string
}

export async function routeTicket(ticket: Ticket) {
  let decision: Decision
  let answer: { choice: string; confidence: number } | undefined
  try {
    const { answers } = await client.systemOne({
      model: MODEL,
      state: {                                                4
        ticket: { subject: ticket.subject, body: ticket.body },
      },
      questions: {
        team: choice('Which team should handle `ticket`?', TEAMS),
      },
    })
    answer = answers.team
    const { choice: team, confidence: c } = answer
    decision =
      c >= LINES.act     ? { action: 'assign', team, reason: 'sure' } :
      c >= LINES.confirm ? { action: 'suggest', team, reason: 'unsure' } :
                           { action: 'triage', reason: 'low conf' }
  } catch (err) {
    const name = (err as Error).name   // e.g. APITimeoutError
    decision = { action: 'triage', reason: `jev error: ${name}` } 5
  }
  const entry = { ticket: ticket.id, model: MODEL,
                  ...answer, ...decision }
  console.log(JSON.stringify(entry)) 6
  return decision
}
  1. Pin the model. systemOne({ model }) overrides the client default, which is jev-latest.53 Upgrade on purpose, after re-running Lesson 3's table.
  2. Keep the lines as reviewed constants in one place, taken from your Lesson 3 table. TypeSafe's agent-skill guide also recommends centralizing question and threshold constants for human review.6
  3. Set a latency budget. timeout is per attempt and retry accepts a partial RetryPolicy.34 Here the worst case is about 3.5 s, not 30 s or more. Tune the budget to your path.
  4. Send minimal state. Only the fields the question needs (Lesson 2, rule 3).
  5. Fail safe, never fail open. An error takes the same exit as low confidence, so a Jev outage degrades to "a human looks at it" instead of a wrong automatic action.
  6. Log every exit, with the model version, choice and confidence. Human corrections on suggested and triaged tickets become free labels for re-tuning.

My recommendation, not from TypeSafe's docs: roll out in shadow mode first. Run routeTicket and log its decision, but let humans keep doing the work. After a few days, compare the two. That gives you real-traffic accuracy before Jev acts on anything.

Jev inside an agent loop

TypeSafe's design guide warns that agent loops "introduce repeated failure opportunities."1 Every extra LLM step is another chance to go wrong. Jev's role is to put fast, typed gates around the loop, so code decides what happens next:

user ① guardnouls in LLM agentgenerates ② toolchoice + args ③ verifyescalate? reply? unsure → bigger model / human Jev gates (orange) are cheap and fast. The LLM does only the generating.
GateWhat Jev asksOfficial recipe
① GuardrailA noul per hazard (jailbreak, harmful request, medical advice, self-harm) plus a severity score, on input and output. Two lines per hazard: review (for example 0.35) and act (for example 0.70).7LLM guardrails
② Tool choiceThe function name and its arguments as choices. The call's confidence is the least certain judgement, "since one wrong argument is enough to spoil the result."8Function calling
③ Verify / cascadeA cheap model extracts, then Jev asks nouls such as "is this value unsupported by the source?" If any signal crosses the line (0.7 in the example), escalate to a reasoning model.9SDE cascade

To build this with Claude Code, TypeSafe ships an agent skill that gives your coding agent the full API context:6

claude plugin marketplace add typesafe-ai/skills
claude plugin install typesafe@typesafe-ai

Its guide also says to review the agent's plan before implementation and to check its assumptions rather than take them at face value.6


Practice: from memory

A · Fill the faded example

const client = new TypeSafeClient({
  : 1500,
  retry: { maxRetries: 1 },
})
try {
  const { answers } = await client.systemOne({
    : 'jev-1.13.0',
    state, questions,
  })
  /* … bands … */
} catch (err) {
  decision = {
    action: '', reason: 'jev error',
  }
}

B · Quick quiz

1. The Jev call throws APITimeoutError in a user-facing path. What should the code do?

2. Why pin model instead of using the default jev-latest?

3. In the verify cascade, when does a field escalate to the reasoning model?

4. For tool choice with several arguments, the call's confidence is…

5. (Lesson 3) Your target is 95%. At 0.6 accuracy is 93%, at 0.7 it's 96%, at 0.8 it's 98%. Which line?

6. (Lesson 2) A guardrail asks noul('Is this a jailbreak or a request for medical advice?'). Which rule does it break?

🏆 Final milestone challenge: Ship one decision

  1. Put one Jev decision into a real project or agent. Pin the model, set a timeout and retry budget, fail safe, and log every exit. Use lines from your Lesson 3 table.
  2. Run it on real traffic for a few days, in shadow mode or live.
  3. Send me the code (or a link), about 20 log lines, the share of traffic taking each exit, and anything that surprised you.

I'll review it like a production code review. When all four milestones show ✓ on your dashboard, the course is complete.

Go deeper

Stuck or curious? Ask me follow-up questions in the terminal. I'm your teacher for this, and nothing is too basic to ask.