Projects tagged with “benchmark”
Browse Jev GitHub projects tagged with benchmark. Filter, sort and explore their source repositories.
- Weekly builds on Jev (TypeSafe AI), benchmarked honestly enough to publish. Week 01: inbound lead triage, 90% routing at 366ms and $0.04 per thousand leads.
deepdave98/jev-playground · Python
333benchmarkgtmlead-scoringllmrevopshtml - Independent, evidence-based map of when TypeSafe's Jev actually holds up vs. breaks down — real API-call receipts, not a leaderboard. 中文為主的雙語 repo。
Zaious/jev-capability-atlas · Python
306ai-agentsbenchmarkcalibrationllm-evaluationmachine-learningzh-tw - Evidence-backed index of real-world Jev (TypeSafe AI System One) use cases: repos, patterns, benchmarks, and measured results
anandi1989/awesome-jev-usecases
195ai-agentsawesome-listbenchmarkbrowser-agentscalibration - Jev vs DeepSeek 医疗任务对比:12 类场景、96 项测试,效果、耗时与费用一览。 | Jev vs DeepSeek on 96 healthcare tasks across 12 scenarios: quality, latency, cost and open experiment data.
JuneYaooo/jev-healthcare-lab · Python
131benchmarkchinese-nlpclinical-nlpclinical-trialsdatasetsdeepseekevidence-based-medicine - 401what-the-jevWhat can Jev actually do? Reproducible experiments and research reports exploring its capabilities and limits.
keta1930/what-the-jev · Python
131benchmarkdecision-modelevaluationllm-evaluation - Independent, reproducible benchmark: a decision model (jev), eight constrained LLMs, and deterministic baselines on typed decisions - accuracy, calibration, latency, cost, failure modes
nibzard/decision-model-benchmark · HTML
121benchmarkcalibrationdecision-modelllm-evaluationreproducibilityhtml - 410jev_fsdA driving simulator on real OpenStreetMap streets where TypeSafe's Jev model drives the car, with a decision inspector and a benchmark.
BrendanH18/jev_fsd · JavaScript
123benchmarkdriving-simulatorjavascriptopenstreetmappythonself-driving - 422carve-jevOpen-source Jev-like model family with one-token Qwen3.8 decisions, TypeSafe API, one-command setup, and reproducible benchmarks.
FuturixAI-and-Quantum-Works/carve-jev · Python
110ai-agentsbenchmarkclassificationdecision-modeljev-likellm-inferencemodel-family - Real-world Jev use cases, open-source projects, benchmarks and criticism — what people actually build with TypeSafe AI's Jev, and where it fails. Machine-readable, updated daily.
Jessie-QingYu/jev-in-the-wild · JavaScript
104ai-agentsawesome-listbenchmarkclassificationllmtyped-decisions - 528Awesome-JevCurated Jev and System One papers, models, datasets, benchmarks and SDKs, with versioned sources and transparent verification levels. English / 中文.
reacher-z/Awesome-Jev · HTML
71ai-agentsawesomeawesome-listbenchmarkcalibrationcurated-listdecision-model - 601sysone-benchFirst independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs
instax-dutta/sysone-bench · Python
63benchmarkcalibrationdecision-modelevaluationlayallm-eval - Learn decision models by building with them. TypeSafe Jev and Microsoft Decision-1 route support tickets in under a second for about $0.02 per 1,000 calls; Claude handles the hard cases. Interactive site, benchmark, calibration tests and docs.
shashikantdev3/decision-models-lab · TypeScript
50ai-agentsanthropicbenchmarkcalibrationclaudedecision-modelhuman-in-the-loop - 645jevossEvaluation tools for Jev-compatible decision models, covering accuracy, calibration, hidden instructions, padding and consistency.
mertkayacs/jevoss · Python
51benchmarkcalibrationdecision-modelevaluationllm-evaluationprompt-injectionpython - Three measured experiments on RAG hallucination: quote-checking, TypeSafe's Jev, and IBM's STAIR. 850+ graded questions, raw responses included.
aryanchauhanoffical/no-hallucination · Python
40benchmarkevaluationhallucinationllmragretrieval - A reproducible, self-hosted benchmark for comparing JEV and LLM decision-making on your own datasets, models, and evaluation policies.
Tenkei/jev-decision-bench · Python
30ai-agentsanthropicbenchmarkdeepseekevaluationllmopenai - Putting TypeSafe's Jev to work on robots, fleets, and edge hardware — real measured numbers, honestly caveated.
robokrunch/jev-physical-ai · Python
30benchmarkedge-aiphysical-airoboticspython - Free English RAG benchmark for TypeSafe Jev 1.13 — frozen candidate pools, calibration, paired bootstrap CIs, $0 runs. Compared with OpenJev and NVIDIA cross-encoders.
emretheus/jev-rag-benchmark · Python
30benchmarkcalibrationinformation-retrievalllm-evaluationnvidiaopenjevrag - Tool search for LLM agents: BM25 vs embeddings vs rerankers vs Jev on 525 real MCP tools, plus an experimental Jev search engine
kachar/jev-tool-search · TypeScript
30ai-agentsai-sdkbenchmarkbm25llmmcptool-search - 910calibreMeasure when to use Jev and other models on your data, then route accordingly.
FirasSX914/calibre · Python
20pythonbenchmarkcalibrationconfidence - Jev vs GPT-4.1 as synthetic survey respondents on Twin-2K-500. How you ask mattered more than which model you used.
jjd-lab/jev-synthetic-survey · Python
20behavioral-economicsbenchmarkcalibrationdigital-twindigital-twinsllmllm-evaluation - 1013jevextractGrounded information extraction that cannot hallucinate: code proposes spans, Jev decides. An open-source alternative to LangExtract, with a bilingual benchmark and paper.
gabazureus/jevextract · Python
21benchmarkgroundinghallucinationinformation-extractionlangextractllmnamed-entity-recognition - 1096what-is-jevIndependent, source-linked research on TypeSafe AI's Jev (System One), with 947 rubric-scored public repositories, recurring patterns, datasets, and bilingual documentation.
g0runmezadam/what-is-jev · Python
20ai-agentsbenchmarkbilingualclassificationdatasetdecision-modelllm - Website for reacher-z/Awesome-Jev: bilingual Jev research papers, typed decision models, benchmarks and source-backed guides.
AwesomeJev/AwesomeJev.github.io · HTML
10ai-agentsawesomeawesome-listbenchmarkcalibrationcurated-listdecision-model - 1163ezjevOpen 4B typed-decision model (Jev-style /v1/systemone): calibrated per-option probabilities from one prefill. Training, data generation and eval pipeline on HF Jobs.
everettjf/ezjev · Python
10benchmarkcalibrationdecision-modelfine-tuninghuggingfacellmlora - 1232jev-battleshipBattleship against Jev, a model that answers in probabilities instead of text. Web game plus a CLI arena that plays it against general-purpose LLMs on identical fleets.
sah1l/jev-battleship · JavaScript
10battleshipbenchmarkllmprompt-engineeringverceljavascript - 1233jev-benchBenchmark TypeSafe's Jev (System One model) vs Claude on structured decisions: accuracy, latency, calibration, cost
stern9/jev-bench · Python
10anthropicbenchmarkclaudeevaluationllmpython - 1235jev-bias-benchA benchmark for Jev's biases, built from people who differ in one attribute at a time.
Fox-Islam/jev-bias-bench · PHP
10ai-fairnessalgorithmic-biasaudit-studybenchmarkbias-detectioncounterfactualdiscrimination - Benchmarking TypeSafe's Jev (System One model) on real IMDB reviews — sentiment, spoilers, and quality in one API call
rahulbansalc6414/jev-imdb-benchmark · Python
10benchmarkimdbpythonsentiment-analysis - 1321jev-labHands-on research lab for TypeSafe's Jev (System One model): reproducible benchmarks of Noul/Choice/Score primitives, confidence gating, fan-out latency, agent control — plus a living audit of the Jev ecosystem.
llt22/jev-lab · Python
10ai-agentsbenchmarkconfidence-calibrationevaluationllmllm-evaluationllm-observability - Reproducible riichi mahjong benchmark for Jev, GPT, Mortal, and hybrid agents using MJAI and RiichiEnv.
hamakyo/jev-mahjong-bench · TypeScript
10benchmarkevaluationgame-aigpthybrid-agentsmahjongmahjong-ai - 1339jev-measuredMeasured cost, latency, accuracy and raw output: TypeSafe's Jev and 12 other decision models (OpenAI Decisions API, Clef, Decider, d1…) — reproducible
WallerChen/jev-measured · Python
10benchmarkdecision-modelllm-routingopenai-decisions-apipython - 1369jev-simJev-compatible /v1/systemone server reading typed decisions from LLM logits, benchmarked against TypeSafe's Jev on the same items via JevBench
dashbi1/jev-sim · Python
10benchmarkcalibrationclassificationllmlogprobspython - Empirical zero-shot benchmark of TypeSafe Jev (v1.13.0) on low-resource Sinhala and Singlish NLP benchmarks (SOLD, SinhalaMMLU, NSINA)
Azri-Muhsin/jev-sinhala-review · Python
10benchmarkdecision-modelindic-nlpnlpsinhala-nlppython - Applying Jev to Snips Natural Language Understanding Benchmark
will-rice/jev-snips-benchmark · Python
11benchmarkevaluationfew-shotintent-classificationnatural-language-understandingnluslot-filling - 1390jev-testPre-registered benchmark: can a 2B local model (Gemma 4 E2B) answer web questions without making things up when a decision model (TypeSafe Jev) makes every call? SearXNG for search, MemPalace for verbatim memory, seven arms including open local judges. Spec and thresholds fixed before any run.
clduab11/jev-test · Python
10benchmarkgemmahallucinationmempalaceragsearxngsmall-language-models - 1416jevals-dataIndependent benchmark data for TypeSafe's Jev (System One model) vs LLMs: accuracy, calibration, cost. Boards + per-decision logs, CC-BY-4.0
Jevals/jevals-data
12benchmarkcalibrationdatasetleaderboardllm-evaluation - 1505pi-Jev-browserBrowser and macOS desktop agent for pi: Jev (TypeSafe System One) chooses each action from a structured observation in a bounded, surface-agnostic loop. Isolated Playwright tools, an allow-listed accessibility-tree tool, deterministic selectors, four-tier benchmarks.
laihenyi/pi-Jev-browser · TypeScript
13accessibilityai-agentsbenchmarkbrowser-automationmacospi-extensionplaywright - Recorded comparison of Jev, Terra, and Opus on 100 synthetic security-triage cases, five passes each, with a static inspectable dashboard
Robertzu43/system-one-security-triage · TypeScript
10ai-securitybenchmarkllm-evaluationsecuritytypescript - 1552taskpennyFrontier prices only for frontier work: each task goes to the cheapest model that does it well, a request is split only when that pays, and every result is checked. Decisions by Jev.
cintocasals/taskpenny · Python
10ai-agentsbenchmarkclaudecost-optimizationgeminillmllm-routing
1–50 of 66