A free GitHub project directory for the TypeSafe Jev / System One model.Explore official SDKs, agent tools, open-source alternatives and Jev news.

Data updated

All tags

Projects tagged with “evaluation”

Browse Jev GitHub projects tagged with evaluation. Filter, sort and explore their source repositories.

  • Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.

    openlayer-ai/jevals · Python

    12013
    ai-agentsevaluationguardrailsllmllm-evaluationragas
  • Probability-aware evaluation for typed decision models: calibration, selective risk, latency, and reproducible benchmarks.

    AbdelStark/jev-benchmarks · Python

    285
    pythonbenchmarkingcalibrationevaluation
  • Fast, typed, calibrated evaluations for LLM and agent outputs, powered by Jev — with simple, framework-agnostic Python APIs

    TrustifAI/typed_evals · Python

    191
    ai-agentscrewaievaluationevaluation-frameworkevaluation-metrics
  • Decision harness for TypeSafe Jev — confidence gates, shadow mode, recipes, and evals. Claude CLI 48.9s → Jev 1.3s on the same row-filter job.

    AntonioCoppe/jev-harness · TypeScript

    152
    ai-agentsconfidencedecisionevaluationharnessllm
  • What can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.

    keta1930/what-the-jev · Python

    131
    benchmarkdecision-modelevaluationllm-evaluation
  • Build and regression-test conversational state machines powered by Jev. Inspect decisions, capture failing conversations as tests, and export runnable TypeScript for your app.

    priyankark/jev-state · TypeScript

    101
    ai-agentscode-generationconversational-aideveloper-toolsevaluationllm-evaluationlocal-first
  • First independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs

    instax-dutta/sysone-bench · Python

    63
    benchmarkcalibrationdecision-modelevaluationlayallm-eval
  • A test harness for TypeSafe's Jev :Confidence audit, Hallucination judge, Jev for RAG

    mity-prodgen/jev-test · Python

    51
    evaluationmodelragpython
  • Evaluation tools for Jev-compatible decision models, covering accuracy, calibration, hidden instructions, padding and consistency.

    mertkayacs/jevoss · Python

    51
    benchmarkcalibrationdecision-modelevaluationllm-evaluationprompt-injectionpython
  • Ten runnable Jev examples for agent decisions: memory conflicts, tool-result checks, recovery, context selection, and handoffs. JavaScript, zero dependencies.

    ReallyArtificial/jev-by-example · JavaScript

    41
    agent-memoryai-agentsevaluationexamplesgood-first-issuehacktoberfesthelp-wanted
  • Three measured experiments on RAG hallucination: quote-checking, TypeSafe's Jev, and IBM's STAIR. 850+ graded questions, raw responses included.

    aryanchauhanoffical/no-hallucination · Python

    40
    benchmarkevaluationhallucinationllmragretrieval
  • Fast and cheap agent evals. jev as judge.

    memovai/openevals · TypeScript

    40
    ai-agentsevaluationlangchainlangfuseobservabilitytypescript
  • A reproducible, self-hosted benchmark for comparing JEV and LLM decision-making on your own datasets, models, and evaluation policies.

    Tenkei/jev-decision-bench · Python

    30
    ai-agentsanthropicbenchmarkdeepseekevaluationllmopenai
  • Standalone TypeSafe AI/Jev semantic-evidence adapter for bounded Agent-Workflow decisions, explicit uncertainty, compatibility evaluation, and host-owned policy.

    ngallodev-software/agent-workflow-typesafe-ai · Python

    10
    agentic-aiai-agentsdecision-supportdeveloper-toolsevaluationpythonsemantic-reasoning
  • Local playground and test bench for TypeSafe's Jev System One model: typed questions, calibrated-probability answers, and side-by-side comparison with an LLM baseline.

    creativoma/here-we-go-jev · TypeScript

    10
    buncalibrationclassificationevaluationllmopenrouter
  • Benchmark TypeSafe's Jev (System One model) vs Claude on structured decisions: accuracy, latency, calibration, cost

    stern9/jev-bench · Python

    10
    anthropicbenchmarkclaudeevaluationllmpython
  • An 11-level crash course on Jev, TypeSafe AI's System One decision model — runnable examples against the real API, plus a capstone project with unit tests and evals. Works with any LLM provider via LiteLLM.

    nadeemcite/jev-crash-course · Python

    10
    ai-agentscrash-courseevaluationlitellmllmpython
  • YAML-based evaluation runner for recorded LLM outputs, with TypeSafe Jev API integration, offline mock mode, and CI reports.

    00200200/jev-judge · Python

    10
    ai-agentsci-cdevaluationllm-as-a-judgeragtesting
  • Hands-on research lab for TypeSafe's Jev (System One model): reproducible benchmarks of Noul/Choice/Score primitives, confidence gating, fan-out latency, agent control — plus a living audit of the Jev ecosystem.

    llt22/jev-lab · Python

    10
    ai-agentsbenchmarkconfidence-calibrationevaluationllmllm-evaluationllm-observability
  • Reproducible riichi mahjong benchmark for Jev, GPT, Mortal, and hybrid agents using MJAI and RiichiEnv.

    hamakyo/jev-mahjong-bench · TypeScript

    10
    benchmarkevaluationgame-aigpthybrid-agentsmahjongmahjong-ai
  • Applying Jev to Snips Natural Language Understanding Benchmark

    will-rice/jev-snips-benchmark · Python

    11
    benchmarkevaluationfew-shotintent-classificationnatural-language-understandingnluslot-filling
  • Preregistered, bias-corrected test of TypeSafe Jev's calibration under human disagreement (ChaosNLI, 100 labels per item)

    GautamTalksDev/jevbench · Python

    10
    calibrationchaosnlievaluationpreregistrationpython
  • A hands-on educational lab for DeepEval + Jev: inspect real recorded judgments, explore release gates and drift playback, and run fresh evaluations on your own workloads.

    shyamsridhar123/JudgeJev · Python

    10
    deepevaldrift-detectionenterprise-aievaluationllm-evaluationllmopspython
  • 1488nocap
    No evidence, no answer. 🧢 System One evidence gate for RAG & AI agents: Jev / Laya / any LLM judges before your LLM speaks. MCP server, Python SDK, CI route tests.

    xi029/nocap · Python

    10
    ai-agentsclaude-codedecision-modeldeepseekevaluationguardrailshallucination
  • Drop-in /v1/systemone proxy that shadows Jev with local decision models (Kev, Laya) and reports whether you can swap

    ReallyArtificial/stuntdouble · JavaScript

    10
    ai-agentsevaluationgood-first-issuehelp-wantedjavascriptkevlaya
  • Find where an API router confuses look-alike operations, rewrite the wording, and rerun. Runs offline in mock mode; live mode uses TypeSafe's Jev.

    Chaddacus/crosstalk-lab · TypeScript

    00
    data-visualizationevaluationllmopenapitool-routingtypescript
  • Jev (TypeSafe AI) as a System One decision layer for a multilingual EN/UZ/RU RAG helpdesk, with an independent benchmark

    rajantripathi/fastgate-jev · Python

    00
    ai-agentsevaluationllm-routingmultilingualraguzbek
  • Classify a business's description of operations into one of 1,012 NAICS 2022 industries, or hold it for a person. No LLM writes anything: Postgres search and code-owned gates do the work, and TypeSafe Jev only picks from a closed list. The hybrid: 0 wrong codes on its eval sets.

    jmurray10/jev_naics · Python

    00
    ai-agentscensusdockerevaluationfastapifull-text-searchindustry-classification
  • A learning scaffold for TypeSafe AI's System One models: eval harness plus a measured, plain-language comparison of the Jev decision model vs an LLM stand-in on 24 real operational decisions. All numbers reproducible from committed run files.

    andreaserradev-gbj/jev-access-day · TypeScript

    00
    benchmarkdecision-supportevaluationllmtype-safety
  • Evaluate typed AI decisions on labeled French-language cases.

    gbesse/jev-banc-francais · JavaScript

    00
    calibrationevaluationfrancejavascript
  • Testing TypeSafe Jev (a decision-only model) on MLB judgement tasks against simple baselines

    yasumorishima/jev-baseball · Python

    00
    baseballevaluationllmmlbpython
  • An 11-level crash course on Jev, TypeSafe AI's System One decision model — runnable examples against the real API, plus a capstone project with unit tests and evals. Works with any LLM provider via LiteLLM.

    nadyth/jev-crash-course · Python

    00
    ai-agentscrash-courseevaluationlitellmllmpython
  • Decision Arena: TypeSafe's Jev vs open-source Laya playing highway-env, Snake and Blackjack with zero training, plus benchmarks and a Claude Code watchdog

    nadeem4/jev-demo · TypeScript

    00
    arenabenchmarkclaude-codedecision-modelevaluationhighway-env
  • LLM-as-a-judge evaluation harness: a confidence-gated cascade (fast typed judge → Claude) that scores LLM outputs and gates CI on a regression diff. TypeScript, real recorded transcripts, real benchmark numbers.

    laszloblum/jev-eval-harness · TypeScript

    00
    anthropicciclaudeeval-harnessevaluationllmllm-as-a-judge
  • A pre-registered field trial of Jev (TypeSafe's judgment model) on a second brain and Claude Code history: 20 tests, bars written first, failures included, and the tools to repeat it.

    NaluKicks-808/jev-field-trial · Python

    00
    ai-agentsclaude-codeevaluationsecond-brainpython
  • Can a decision model replace an LLM as an eval judge? Same rubrics, scored by Jev and by an LLM, side by side.

    nadheesh/jev-llm-judge · Python

    00
    agent-evaluationevaluationllm-as-a-judgellm-evaluationpython
  • Evidence-gated registry of Jev question packs — curated questions, golden cases and measured evidence for Jev-compatible decision endpoints

    dtduc-git/jev-packs · Python

    00
    ai-testingcalibrationevaluationgolden-datasetllm-evaluationquestion-packspython
  • Terminal-based Jev intent triage with typed decisions, explicit simulation mode, review policy and JSON exports.

    suprkco/jev-serp-opportunity-lab · Python

    00
    evaluationpythonseostructured-output
  • Applying Jev to zero-shot tool calling on the Berkeley Function Calling Leaderboard (BFCL)

    will-rice/jev-tool-calling · Python

    01
    benchmarkbfclevaluationfunction-callingllm-agentstoken-classificationtool-calling

1–43 of 43