All projects

typed-decision-bench

4nt0ineB/typed-decision-bench

Bench of typed decision models: Jev vs OpenJev vs Laya, small local LLMs and cheap hosted LLMs on the same zero-shot classification tasks, in English and French.

Python
Stars
0
Forks
0

Why it is listed

This small benchmark compares Jev with OpenJev, Laya, and local/hosted models on shared zero-shot English/French classification tasks; the authors limit it to one dataset and question type, and the metrics were not reproduced.

Related projects

  • jev-rag-benchmark

    Free English RAG benchmark for TypeSafe Jev 1.13 — frozen candidate pools, calibration, paired bootstrap CIs, $0 runs. Compared with OpenJev and NVIDIA cross-encoders.

    Shared topics: benchmark, calibration, llm-evaluation

  • sysone-bench

    First independent head-to-head benchmark of System One decision models (Laya vs Jev) on byte-identical inputs

    Shared topics: benchmark, calibration, laya

  • system-one-playground

    Run the new wave of decision models (Laya, Decider, Kev, Jev) side by side on your Mac. Structured input in, calibrated probabilities out, with honest accuracy, calibration, latency and memory numbers.

    Shared topics: benchmark, calibration, laya