​

Verifiable Distributed LLM Work: The State of the Art and a Solution

​

The Problem

1. Publish work = (prompt, required model)
2. Workers execute = run prompt with specified model
3. Submit results = output + cryptographic proof that the specified model produced it
4. Verification = anyone can confirm without re-running

​

The Three Families of Approaches

​

1. Zero-Knowledge Proofs (zkML) — Perfect but Slow

​

2. Trusted Execution Environments (TEEs) — Fast but Breakable

​

3. Optimistic + Economic Verification — Practical and Scalable

• Deterministic inference is now solved. Thinking Machines Lab showed that batch-invariant CUDA kernels produce 1000/1000 identical outputs for Qwen-3-235B at temperature 0 (1.6x slowdown). SGLang (LMSYS, Sep 2025) ships this. Verification becomes a byte-equality check.
• EigenAI (Jan 2026): Workers stake capital. Results are tentative for a challenge window. Challengers re-execute deterministically inside TEEs. Disagreement = slashing. 100% bit-identical across 10,000 runs on same hardware.
• Hyperbolic PoSP: Game-theoretic Nash equilibrium. If challenge probability > (fraud gain / slash amount), honest computation is the dominant strategy. <1% overhead. Adaptive sampling per node reputation.
• VeriLLM: Workers commit Merkle root of hidden-state tensors. VRF-based random sampling of intermediate states. Statistical tests distinguish hardware rounding from model substitution. ~1% verification cost.

​

The Solution: Layered Verification Protocol (LVP)

                        COST
                         ^
                         |
    Layer 4: zkML proof  |  ####  (minutes, cryptographic certainty)
                         |
    Layer 3: Deterministic|  ###   (seconds, re-execution)
             re-execution |
                         |
    Layer 2: Intermediate |  ##    (milliseconds, Merkle proofs)
             state audit  |
                         |
    Layer 1: Commitment + |  #     (zero, economic deterrent)
             stake        |
                         +------------------------------> TRUST

​

How It Works

WorkUnit {
    prompt:       "Implement the user authentication module..."
    model:        "llama-3.1-70b"
    model_hash:   SHA384(weights_file)      // commitment to exact weights
    quant:        "fp16"                     // required precision
    seed:         0x7f3a...                  // deterministic seed
    max_tokens:   4096
    reward:       0.50 USDC
    stake_req:    10.00 USDC                 // worker must stake
    escalation:   [commit, reexec, zkml]     // verification layers
}

1. Worker stakes stake_req
2. Downloads model weights, verifies SHA384(weights) == model_hash
3. Runs inference with batch-invariant kernels + fixed seed (deterministic)
4. Computes Merkle tree over hidden states at each transformer layer
5. Submits:

Result {
    output:       "Here's the implementation..."
    merkle_root:  0xabc123...                // commitment to all intermediates
    output_hash:  SHA384(output)
    signature:    sign(worker_key, output_hash || merkle_root)
}

​

Model Identity: The Key Innovation

• model_hash = SHA384(canonical_weights_file)
• A public registry maps model names to weight hashes (think: Hugging Face + content-addressable storage)
• Worker must demonstrate they loaded the exact weights
• Deterministic execution proves the committed model produced the output

• The API provider signs responses: sign(provider_key, prompt_hash || response || model_version || timestamp)
• Token-DiFR fingerprinting: regenerate with same seed, >98% token match confirms the claimed model
• Provider reputation + legal accountability replaces cryptographic proof
• Future: providers run inside TEEs with attestation (Phala already does this with DeepSeek on OpenRouter)

​

Why This Is “Unstoppable”

1. No single point of trust. Hardware can be compromised (TEE.Fail). Software can be buggy. But the combination of cryptographic commitments + economic stakes + deterministic re-execution + zkML escalation has no single attack vector that defeats all layers.
2. Economically rational honesty. At Layer 1, if challenge_probability * slash_amount > fraud_gain, the Nash equilibrium is honesty. No cryptography needed — just game theory.
3. Cryptographic fallback exists. If you truly need mathematical certainty, Layer 4 (zkML) is available. ZKTorch can prove GPT-J 6B in 23 minutes today. GPU acceleration will bring this to minutes. For individual disputed operations, it’s seconds.
4. Deterministic inference is production-ready. SGLang ships it. Thinking Machines proved it at 235B scale. The “outputs are non-deterministic” objection is no longer valid.
5. Works for any model size. The default path (Layer 1-2) has <1% overhead regardless of model size. You only pay the zkML cost if someone actually disputes AND you can’t do byte-equality re-execution.

​

What Exists Today vs. What Needs Building

​

Sources

• ZKTorch (arXiv) — 23-min proof for GPT-J 6B
• zkLLM (CCS 2024) — LLaMA-2 13B proving
• Definitive Guide to ZKML 2025
• NVIDIA H100 Confidential Computing
• TEE.Fail Attack — broke TEE attestation with $1K hardware
• EigenAI (arXiv) — deterministic optimistic verification
• Hyperbolic PoSP — game-theoretic verification
• VeriLLM — commit-reveal with Merkle proofs
• SGLang Deterministic Inference
• Thinking Machines: Defeating Nondeterminism
• Gensyn RepOps — bitwise reproducible GPU ops
• SPEX Statistical Proofs — handles non-determinism via LSH
• Phala Network GPU TEE — TEE inference in production
• Token-DiFR Fingerprinting — 98% token match for model ID
• Inference Labs on EigenLayer
• Chutes Confidential Compute
• Tolerance-Aware Verification — 0.3% overhead, no TEE needed