Deterministic Test Verification Engine
Deterministic verification for AI-generated test suites.
When code and tests are generated together, test suites pass by construction. Line coverage increases while assertions rely on shallow checks (.toBeDefined(), .toBeTruthy()) and miss domain error branches. Pramāṇa parses abstract syntax trees and executes synthetic mutations to prove which regressions bypass CI.
- Turnaround
- 72 hours from repo access
- Deliverables
- Assertion Ledger + Hardened PR
- Verification
- AST analysis & synthetic fault injection
- Commercial
- £2,500 fixed fee (authorized audits)
Empirical Proof
Surviving mutants and unasserted error branches in production repositories.
Pramāṇa executes deterministic AST parsing and mutation testing against target codebases. It records reproduction commands, exact diffs, and exit codes for every surviving mutant.
Security specification · Webhook signature verification
standard-webhooks/standard-webhooks
All negative tests assert only generic WebhookVerificationError. Signature bypasses and expired timestamps pass CI under the same green assertion.
Metric: 30/100
E-commerce platform · Cart & order calculation
medusajs/medusa
Mutating tax addition to subtraction in order calculation passed the full test suite with 100% green status. A financial reversal escaped CI.
Metric: Surviving Mutant
Communications API · Batch email transmission
resend/resend-node
Core batch sending tests assert only that network calls and headers 'exist'. Malformed or empty payloads continue to pass green.
Metric: 13 Vacuous Matchers
Downstream Risk & Business Impact
Why this matters: The Green CI Illusion.
When teams generate code and tests together with AI tools, test suites pass green by construction. Line coverage metrics increase to 80% or 90%, giving engineering leadership false confidence while assertions check only that objects exist. When critical logic breaks, CI remains green and defects escape directly to production.
Coverage Without Fault Resistance
AI test generators create tautological assertions: checking .toBeDefined() on null checks or verifying HTTP status codes without payload validation. CI reports green while the test suite catches zero breaking changes.
Silent Downstream Failures
A surviving mutant in billing reverses tax addition to subtraction. In authentication, an unasserted error branch allows expired tokens to pass. Failures escape directly to production as customer outages and revenue leaks.
30 Seconds vs 12 Hours
Catching an unasserted error branch before merge takes 30 seconds. Fixing an escaped bug in production requires 4–12 hours of senior incident triage, emergency database patching, customer credits, and audit compliance reviews.
Deliverables
Four verified audit deliverables in 72 hours.
01
Static Assertion Integrity Ledger
Every vacuous matcher (.toBeDefined(), .toBeTruthy()) identified with exact file coordinates, masked failure modes, and production risk ratings.
02
Domain Error Branch Gap Analysis
A complete census of error, exception, and status branches in your code that have zero assertions in your test suite, with trigger conditions.
03
Fault Resistance Proof (Surviving Mutants)
Deterministic synthetic fault injection across critical logic (billing, auth, state transitions) proving which regressions slip past CI undetected.
04
Assertion Hardening Pull Request
An engineer-ready git patch replacing shallow matchers with strict domain invariants and adding tests for unasserted error branches.
Enterprise Infrastructure Integration
Connect your existing LLM infrastructure.
Pramāṇa standardizes on the universal OpenAI-compatible REST protocol (/v1/chat/completions). Connect Azure OpenAI, AWS Bedrock via gateway, self-hosted vLLM, or local Ollama instances directly to the verification engine without proprietary SDK dependencies.
Auto-Discovered Environment
Pramāṇa reads standard CI environment variables (OPENAI_BASE_URL, AZURE_OPENAI_ENDPOINT, OLLAMA_HOST) in GitHub Actions, GitLab CI, and Jenkins runners automatically.
Strict VPC Boundaries
Select your compliance mode: strict-airgap restricts traffic to RFC-1918 private subnets and Unix domain sockets; enterprise-vpc permits internal corporate domains; client-managed connects to your explicit cloud endpoint.
CI Resilience Guaranteed
External AI latency or rate limits never halt your build. If the private LLM endpoint times out, Pramāṇa automatically falls back to deterministic AST rules and completes verification without breaking CI.
Core Operating Rule
The model proposes. The machine proves.
Pramāṇa places zero trust in raw model opinions. All proposals from your internal LLM are treated as unverified candidate hypotheses until physically executed in an isolated child process sandbox and verified by native process exit codes (exit 0 = FAIL / surviving mutant, exit 1 = PASS / killed mutant).
Direct Audit Intake
Request a test suite audit for your repository.
Submit your repository coordinates below. Pramāṇa analyzes assertion invariants, detects unasserted error branches, and executes synthetic mutations in an isolated sandbox. You receive the Static Assertion Ledger and hardening patch within 72 hours.
Audit Intake
Audit your test suite before regressions reach production.
Provide read access to one repository or package. You receive the Static Assertion Ledger, Error Branch Matrix, Fault Resistance Proof, and Assertion Hardening PR within 72 hours. Fixed fee: £2,500, payable by corporate invoice (Net-14 terms) via bank wire or invoice link.
