deterministic-testing

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

Deterministic runtime for agent evaluation

KayaDB distributed key-value storage engine

Safe Rust metaprogramming and adversarial testing: typed code generation with receipts, proc macros, generated trials, mutation, replay, and reproducible evidence.

Local-first C# project for deterministic prompt versioning, A/B evaluation, and evidence-based promotion using structured scoring.

Reliability testing for MCP servers: deterministic faults, bounded recovery, and executable regressions.

Direct gives browser agents repeatable app states that open by URL, with your real interface running on fixture data.

A Raft consensus library built from scratch in Rust, with a deterministic network simulator for testing leader election, log replication, and partition recovery.

Deterministic UI testing framework for TS/JS — canvas-first. Virtual time, scene contracts, model-driven discovery.

Local-first AI agent evaluation, tracing, deterministic replay, human review, and self-hosted production quality — TypeScript + Python.

Deterministic, zero-dependency static SAP S/4HANA OData and ERP workflow simulator.

Record normalized DeepSeek Harness tool results and replay them offline with strict matching and zero tool-body execution.

The deterministic heap groomer for C/C++ memory debugging.

Reproduce race conditions in real PostgreSQL application code and keep the failing order as a regression test.

Deterministic net.Conn/net.Listener fault injection for Go — the testing/synctest philosophy applied to the network layer.

WordleBench — Deterministic AI Wordle benchmark. Compare 34+ LLMs (GPT-5, Claude 4.5, Gemini, Grok, Llama) head-to-head on accuracy, speed, and cost across 50 standardized words.

Compile native Playwright GUI test scripts before the GUI exists. Run them with zero AI.

Reproduce OpenAI Agents SDK runtime bugs without API keys — real Runner, scripted models, verifiable certificates.

Document-driven E2E test harness with AI-assisted compilation and deterministic Hurl execution

Deterministic Rust testing utility for simulation and stochastic workflows