Internal Product · LLM-as-Judge

Juri — the LLM prompt evaluation platform for QA & Dev teams

Batch-test prompts against CSV test cases, score responses automatically with configurable judge rubrics, and put a mandatory human in the loop before anything counts as "final." Juri replaces ad-hoc console testing with a repeatable, auditable evaluation workflow.

What Juri does

Manually eyeballing whether an LLM's response is "good enough" doesn't scale past a handful of examples. Juri turns that judgment call into a structured pipeline: define a prompt once as a versioned Script, attach one or more Judge Configurations (rubric + scoring model), run it against a CSV of test cases, and get consistent, comparable, human-reviewed results every time.

🧪

Repeatable testing

Every prompt, every generation parameter, and every judge rubric is versioned. A run always points back to the exact script version that produced it — no more "which prompt was this?"

⚖️

LLM-as-judge scoring

Uses the Claude Agent SDK to both generate candidate responses and score them against configurable rubrics — accuracy, relevance, completeness, clarity, or fully custom dimensions.

👤

Human-in-the-loop, always

No run is ever "final" purely from automated judging. Every row is reviewed, overridable, and requires a reviewer comment — a durable, resumable, auditable trail.

How it works

The core loop, from prompt to finalized, exportable report.

1. Configure

Create a Script — system prompt, generation model, temperature/max tokens — and attach one or more Judge Configurations with rubric + pass threshold.

2. Upload CSV

Upload test cases: input prompt, optional expected answer, test case ID, category/tag.

3. Run

Juri generates a candidate response per row and scores it against every attached judge config. Partial row failures don't abort the batch.

4. Review

Every row enters mandatory pending review. Reviewers accept, override scores/verdicts, and must comment.

5. Compare & export

Finalize the run, view the multi-judge comparison report, and export as CSV, JSON, or HTML.

Functionality supported

What's shipped and usable today, grouped by area.

Scripts & versioning

Named, versioned prompt bundles with generation params (model, system prompt, temperature, max tokens) and full version history per run.

Judge configurations

1..N judge configs per script, each with its own judge model, rubric text, scoring dimensions, scoring scale (1–5, 1–10, pass/fail), and optional custom output schema. Inherits from an org-level default unless overridden.

CSV batch runs

Row-by-row generate → judge loop with live progress ("87/200 processed") and per-row failure isolation.

Mandatory review workspace

Side-by-side prompt, candidate response, and all judge scores/justifications. Accept, override any dimension or the overall verdict, comment required.

Comparison reports

Per-row side-by-side scores across judge configs, disagreement rows surfaced prominently, and aggregate averages to spot systematically harsh or lenient configs.

Export

CSV and JSON for raw data, HTML for shareable comparison/full reports. Print-to-PDF from the browser covers PDF needs — no lock-in.

Dashboards & drift

Run-to-run comparison and judge drift tracking to see how scoring shifts as prompts or rubrics evolve over time.

Review assignment

Pending runs start in a shared team queue. Reviewers can claim; admins can assign or unassign. Assigned runs are visible only to the assignee and admins.

Sampling review modes

Per-script review policy: full, random sample, borderline-margin, or failures-only — snapshotted per run so mid-run edits don't change that batch.

Teams & roles

One deployment, multiple teams within a single organization. Tool admins manage the org-level default judge; teams manage their own scripts and runs.

Prompt Lab

A workspace for iterating on prompts and judge rubrics before committing them to a versioned script.

Integration API

A narrow, scoped HTTP API (/api/v1/integrations/{app_id}/…) for evaluate-only runs where a caller supplies its own candidate response — for teams wiring Juri's judging into an existing pipeline without adopting it as a full CI product.

Use cases

Concrete scenarios Juri is built to support for QA and Dev teams working with LLM-backed features.

Regression testing for prompts

Catch quality regressions before a prompt change ships

A team updates a customer-support system prompt. Before merging, they re-run the existing CSV of 200 support questions against both the old and new script version, and use the run-to-run comparison to confirm the new prompt doesn't quietly regress accuracy or tone.

Judge calibration

Decide which judge rubric to trust

Two judge configurations — one scoring strictly on factual accuracy, one on a broader helpfulness rubric — are attached to the same script. The comparison report highlights rows where they disagree, so the team can tune thresholds or retire the rubric that's miscalibrated.

Model migration validation

Validate a switch to a new or cheaper model

Before moving a production prompt from one model to another, a script is duplicated with the new generation model and run against the same test CSV, letting the team compare pass rates and judge scores side by side prior to cutover.

Compliance & audit trail

Keep an auditable record of human-approved outputs

Every reviewed row carries a mandatory comment and a persisted human decision. For teams in regulated environments, the finalized run plus HTML export becomes the evidence that a human confirmed AI-generated output before it was trusted.

Third-party integration scoring

Score responses generated by an external pipeline

A team already generates candidate responses in their own CI job and just needs consistent scoring. They register an Integration App, post candidate_response values to Juri's evaluate-only API, and get judged results back with an optional signed callback.

Review workload management

Distribute review work across a QA team

A 500-row run lands in the shared pending-review queue. Reviewers claim batches of rows, an admin reassigns a stuck batch to a teammate going on leave, and the run only finalizes once every required row has a human decision.

Sample data

Illustrative CSV test-case input and the kind of run output Juri produces. Not real customer data.

Sample input CSV — support_qa_testset.csv

test_id,category,input_prompt,expected_answer
TC-001,billing,"How do I update my payment method?","Guide the user to Account > Billing > Payment Methods"
TC-002,billing,"Why was I charged twice this month?","Explain proration and offer to check the invoice"
TC-003,technical,"The app crashes when I export a report","Ask for steps to reproduce, offer workaround, escalate if unresolved"
TC-004,account,"How do I delete my account permanently?","Explain the deletion process and 30-day recovery window"
TC-005,technical,"API returns a 429 error intermittently","Explain rate limits and suggest exponential backoff"

Sample run output

Test IDCategoryJudge: AccuracyJudge: ToneVerdictReview status
TC-001billing9 / 109 / 10PassReviewed — accepted
TC-002billing6 / 108 / 10BorderlineReviewed — overridden to Fail
TC-003technical8 / 107 / 10PassReviewed — accepted
TC-004account4 / 106 / 10FailReviewed — accepted
TC-005technical9 / 109 / 10PassPending review

Sample judge configuration (JSON)

{
  "name": "strict-accuracy-v2",
  "judge_model": "claude-sonnet",
  "scoring_scale": "1-10",
  "dimensions": ["accuracy", "completeness", "tone"],
  "pass_threshold": 7,
  "rubric_prompt": "Score the candidate response strictly on factual correctness against the expected answer. Penalize hedging."
}

Review policy modes

Set per script, snapshotted per run, so a policy change mid-run doesn't affect a batch already in flight.

ModeBehavior
fullEvery row requires human review before the run can finalize.
sampleA random percentage of rows require review; failed and rate-limited rows are always included.
borderlineOnly rows scoring within a configurable margin of the judge's pass threshold require review.
failures_onlyOnly rows a judge marked as fail (plus failed/rate-limited rows) require review.

Architecture at a glance

A pragmatic, single-organization internal tool — not built for multi-tenant SaaS scale, and deliberately so.

Phases

Juri's build has progressed through three broad phases.

Phase 1 — MVPShipped

Default judge, scripts, CSV runs, mandatory review, multi-judge comparison report, CSV/JSON export.

Phase 2 — Post-MVPShipped

Dashboards, run-to-run compare, judge drift, sampling review modes, user admin, review assignment (claim/assign), judge config history with version diffs and pass/override rates.

Phase 3 — Scale-outShipped

Teams, advanced judge types, optional Celery workers for concurrent processing (docker compose --profile scale up).

Integration AppsCurrent focus

Narrow, scoped HTTP API for evaluate-only runs where a caller supplies its own candidate response — deliberately not a full CI/CD product.

Getting started

Local smoke test via Docker Compose.

cp .env.example .env
# set SECRET_KEY and MySQL passwords
docker compose up -d --build
docker compose exec app python -m app.cli create-admin
# open http://localhost:8000

For a full blank-machine runbook (Docker install, git clone, MySQL, host Claude OAuth mounts), see docs/INSTALL.md in the repository. Without an ANTHROPIC_API_KEY, generation and judging fall back to the Claude Agent SDK's ambient claude login OAuth session.