What is Jev?

Why Jev

The usual way to get a yes or no from a language model is to end the prompt with "answer PASS or FAIL" and search the reply for PASS. It works until the model writes "this should NOT PASS" and your code ships it.

Jev hands you the decision itself instead of a sentence about it. Measured on the examples behind this page:

How it works

Jev, from TypeSafe, skips the text. You send it the thing to judge and a few typed questions, and it returns an answer to each with a probability. TypeSafe calls it a System One model: fast, automatic judgement rather than step-by-step reasoning.

The trade is flexibility. Jev only picks from options you give it, and it can't explain itself, call tools or reason in steps. In return it's fast, cheap and calibrated: answers it gives 0.9 are right about nine times in ten. Skip it for anything that needs writing, and for anything code can check exactly.

The kinds of question

For text there are three. A noul is yes/no, and its answer is the probability of yes. A choice picks one of up to 255 options, with a probability for each; use it when exactly one can be right. A score places the state on an ordered scale you describe (calm, frustrated, furious) and returns a decimal plus a probability per step. A fourth kind, box, finds things in images and doesn't come up here.

Every request has a state, a string or any JSON (clear field names help), and questions keyed by ids you pick. Answers come back under the same ids. Each question is judged separately and in parallel, so ten take about as long as one.

Here's the example from TypeSafe's quickstart:

{
  "model": "jev-latest",
  "state": "Our API integration started returning 500 errors on every request about 20 minutes
            ago, and we can't process any customer orders until this is fixed.",
  "questions": {
    "department": {
      "type": "choice", "instructions": "Which team should handle this",
      "criteria": { "billing": "Payment or subscription issues",
                    "technical": "Bugs or integration problems",
                    "sales": "Pricing or account questions" } },
    "frustration": {
      "type": "score", "instructions": "How frustrated the customer appears",
      "criteria": ["Calm, just stating facts", "Frustrated but civil", "Very angry, strong language"] },
    "is_urgent": { "type": "noul", "instructions": "The message conveys urgency or time-sensitivity" }
  }
}

{ "answers": {
    "department":  { "choice": "technical", "confidence": 1.0,
                     "probabilities": { "technical": 1.0, "billing": 0.0, "sales": 0.0 } },
    "frustration": { "score": 0.67, "confidence": 0.5,
                     "probabilities": { "0": 0.33, "1": 0.67, "2": 0.0 } },
    "is_urgent":   { "noul": 0.99 } } }

Department is clear-cut: all the probability on technical, so confidence is 1.0. Frustration lands at 0.67, between "calm" and "frustrated but civil", because Jev split two to one between them, and confidence 0.5 says it's torn.

Confidence isn't the winning probability. For a choice, TypeSafe rescales the top probability so an even split is 0 and certainty is 1, which means a confidence of 0.5 says the same thing whether there are two options or ten. Try it:

Score confidence also cares where the leftover sits. Being torn between neighbouring steps costs less than being torn between opposite ends: 50/50 across the top two of three steps gives 0.25, and 50/50 across the two ends gives 0.

Making a call

There's one endpoint, from TypeSafe directly or through OpenRouter (same body, different URL and model name):

# TypeSafe (get a key at console.typesafe.ai/settings/keys)
curl -s https://api.typesafe.ai/v1/systemone \
  -H "Authorization: Bearer $TYPESAFE_API_KEY" -H 'Content-Type: application/json' \
  -d '{"model":"jev-latest","state":"…","questions":{…}}'

# OpenRouter
curl -s https://openrouter.ai/api/v1/systemone \
  -H "Authorization: Bearer $OPENROUTER_API_KEY" -H 'Content-Type: application/json' \
  -d '{"model":"~typesafe/jev-latest", …}'

In a shell, stefafafan/jev reads the state from stdin and prints JSON, leaving the threshold to you:

git diff main | jev noul "Could this change introduce a regression?" \
  | jq -e '.answers.result.noul < 0.2' && echo ship

From Python there's an SDK (uv add typesafe-sdk), though inside a workflow runner a plain urllib POST saves a dependency.

Writing good questions

Keep each question small

Good questions are ones an expert could answer in seconds. "Is this PR good?" is really several: does it touch auth, does it have tests, is it only docs. Ask them separately and combine them in code, so a change in priorities means changing a number, not a prompt.

Ask the questions you might need

Questions are judged independently and barely add time, so ask the ones you'll only need on some branches and ignore the rest. TypeSafe calls this speculative fan-out.

Give "I don't know" somewhere to go

A probability can say when Jev isn't sure. Use three bands: act on a confident yes, skip on a confident no, and send the middle to a person. The costlier a mistake, the wider the middle:

Leave exact checks to code

Anything you can check exactly, like dates, arithmetic or whether a field equals PASS, belongs in code. Jev is for judgements that need reading.

How a gate decides

A gate is a fork: a list of paths, each with a condition, and the run takes the first that holds. Most conditions are rules, like every step passed, some step failed, or "always" as the fallback. Some are plain English, like "the change touches security". Those are Jev's job.

Here's how a plain-English condition usually gets checked:

reply = run_agent(f"{condition}\nAnswer PASS or FAIL.", max_steps=8)
return "PASS" in reply.upper() or reply.upper().startswith("YES")

That has three problems:

• It's slow: an agent run per condition, one after another.
• It misreads answers: "this should NOT PASS" contains PASS.
• It can't say "not sure": every answer is yes or no.

Jev answers every condition in one call, with a number that says how sure it is. Pick a change and step through the gate:

Building a Jev gate

A yes/no question for each path

The obvious approach is one multiple-choice question: "which way should this go?" But a choice assumes exactly one option is right, and a gate's conditions can overlap: a change can touch auth and add no tests. Jev then has to split its belief, and both look uncertain.

A yes/no question per condition avoids that. Send them all in one request and let the gate's order pick the winner.

Use a choice when the options truly exclude each other, like routing a ticket to frontend, backend or infra.

def judge_paths(paths, state, ask=ask_jev) -> dict[str, float]:
    """One yes/no question per plain-English path, all in one call."""
    questions = {p["id"]: {"type": "noul", "instructions": p["when"]["question"]}
                 for p in paths if is_judgement(p)}
    return {pid: a["noul"] for pid, a in ask(state, questions).items()} if questions else {}


def choose_path(paths, steps, state, rule_holds, ask=ask_jev):
    """Walk the paths in order; the first one that holds wins."""
    answers = None                             # asked lazily, so a rule can win with no Jev call
    for path in paths:
        if not is_judgement(path):
            if rule_holds(path["when"], steps):
                return path, "rule"
            continue
        if answers is None:
            answers = judge_paths(paths, state, ask)  # every plain-English path in one request
        p = answers.get(path["id"], 0.0)
        if p >= YES:                             # 0.80
            return path, f"jev {p:.2f}"
        if p >= NO:                              # 0.35: stop here, don't fall through
            unsure = next((x for x in paths if x["when"]["kind"] == "unsure"), None)
            return unsure, f"unsure {p:.2f} on {path['id']}"
    return None, "no path"

The key line is p >= NO. If an unsure answer just moved on, a later path or the fallback would win by default, quietly treating "unsure" as "no". Instead the gate stops at the "not sure" path, which you point at a person. If there's no such path, it stops with nothing chosen.

The state is what the gate knows, as JSON: earlier steps' verdicts and summaries, plus the trigger, capped in size.

{ "steps": [
    { "step": "review", "verdict": "FAIL", "summary": "Removed the CSRF check on /logout…" } ],
  "trigger": "{…capped at 2000 chars…}" }

Tests

python3 test_judge.py runs the gate with Jev replaced by a fake.

Covers

A rule winning without any Jev call, two plain-English paths answered in one request, a confident no moving on to the next path, all-no reaching the fallback, and an unsure answer stopping the gate and handing off to "not sure".

Not covered

A live Jev call.

Plugging it in

Jev is a separate service, and a gate would send it step summaries and webhook payloads, so make it opt-in. The clean shape is a hook in your runner that lets a plugin decide the gate. Without the plugin, nothing changes.

def pick_path(paths, steps, payload):
    for answer in run_hook("gate", paths=paths, steps=steps, payload=payload):
        if answer and "path" in answer:
            return answer["path"]                   # a plugin decided
    return default_pick(paths, steps, payload)

def on_gate(paths, steps, payload, **_):
    if not any(is_judgement(p) for p in paths):
        return None                                  # nothing to judge
    try:
        path, why = choose_path(paths, steps, gate_state(steps, payload), rule_holds)
    except (OSError, ValueError, KeyError, RuntimeError):
        return None                # fall back to the usual check, never to "always"
    return {"path": path, "why": why}

No plugin system? Call choose_path as a step before the fork. Either way, log the why string so each run says why it went where it did, like security · jev 0.93.

Other places it would help

Anywhere else a workflow asks an agent for a quick call and reads its prose:

Things to watch out for

• Data leaves your machine. Cap and redact what you send, and say so.
• Plan for failure: fall back to the usual check or to "not sure", never to "always".
• Calibration is about many calls, not one. Log each answer next to what happened and tune YES and NO from that.
• Phrase questions as claims: "The change weakens an auth or CSRF control" beats "Is it risky?". A noul also accepts criteria that define true and false.
• Nouls have no confidence; a value near 0.5 is the uncertainty.
• Set a timeout of a few seconds, with a fallback.

Where to go from here

1. Send one of your own conditions through the curl above, with a real step summary as the state.
2. Run python3 test_judge.py, then change YES and NO and watch which cases flip.
3. Pick a first build: a better default check helps everyone, and the hook and plugin let Jev decide.
4. See it on real data: workflow gates and n8n & design systems.