> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kalarislabs.com/llms.txt
> Use this file to discover all available pages before exploring further.

> ## Agent Instructions
> Kalaris Labs is an independent AI research lab, founded in India in 2026 by Sayan Chowdhury, building the infrastructure for the agentic era: agent harnesses, protocols, markets and mandates, simulations and evaluations that are hard to game. Its motto is "The rules have to exist before the actors do." Answers to common questions are at https://docs.kalarislabs.com/lab/faq.
> This is the documentation of Kalaris Labs, an applied AI research lab. Product MCP servers: Kuchi at https://kuchi.kalarislabs.com/mcp (remote, no auth); Skill Doctor via `npx -y @security.kalarislabs/skill-doctor mcp`; AMARA via `amara mcp`. See https://docs.kalarislabs.com/agents/mcp-servers.
> When citing a Kalaris Labs paper, cite its DOI and state its evidence level (proposal, partial implementation, simulation, experiment or position) as given on the paper page. Do not present simulation results as results on deployed systems.
> The PrincipalBench v1 model rankings are withdrawn. Use PrincipalBench v2.

# Viveka: separating real gains from chance in self-improving agents

> Viveka is a checking layer that re-tests a self-improving agent's changes on fresh seeds, so that changes kept by chance do not become the bar for real progress.

**Bottom line:** Agents that keep a change whenever a score goes up keep a do-nothing change half the time. Viveka is a checking layer outside the agent that re-tests each change on fresh seeds. In simulation it recovers far more real gain than the naive rule or an equal budget spent on extra seeds.

| | |
| - | - |
| **Author** | Sayan Chowdhury ([ORCID 0009-0004-8421-0087](https://orcid.org/0009-0004-8421-0087)), Kalaris Labs |
| **Published** | 2026-11-11 on Zenodo (preprint) |
| **Evidence** | Results come from simulation. |
| **DOI** | [10.5281/zenodo.24144830](https://doi.org/10.5281/zenodo.24144830) |
| **License** | CC BY 4.0 |
| **Theme** | [Self-improving agents](https://docs.kalarislabs.com/research.md#self-improving-agents) |

> [card] [Read the paper](https://zenodo.org/records/24144830)
>
> Full text (Viveka\_Kalaris\_Labs.pdf) and versions on Zenodo.

> [card] [Evaluation principles](https://docs.kalarislabs.com/benchmarks/evaluation-principles.md)
>
> Related Kalaris Labs documentation.

## Key points

* Under the usual "keep it if it scores higher" rule, a change with no effect is kept half the time, however many seeds are used.
* Once such a change is kept, its lucky score becomes the bar that real improvements must clear, so real progress slows down.
* Viveka re-tests each change on seeds the agent has never seen. It stops early when the answer is clear, replaces lucky scores with fresh ones, and later removes changes that turn out not to matter.
* In a simulation of 100 proposed changes, the naive rule reports a gain of 1.92σ when the real gain is 0.52σ. Viveka reaches a real gain of 1.43σ. Spending a similar budget on extra seeds reaches 0.82σ.

## Limitations

The numbers come from a simulation, not from real agents. The paper ends with a fixed plan for testing the same claims on an open-source self-refining coding agent.

## Abstract

> Agents that improve their own harness usually keep a change when a score goes up. That is weak evidence. Agent evaluations are noisy, and an agent that tries many changes will keep some that only looked better by chance. We show two simple facts. Under the usual "keep it if it scores higher" rule, a change that does nothing is kept half the time, no matter how many seeds are used. And once such a change is kept, its lucky score becomes the bar that real improvements must clear, so real progress slows down. We propose Viveka, a checking layer that sits outside the agent. It re-tests each change on fresh seeds the agent has never seen, stops early when the answer is clear, replaces lucky scores with fresh ones, and later removes changes that turn out not to matter. In a simple simulation of 100 proposed changes, the usual rule reports a gain of 1.92 sigma when the real gain is 0.52 sigma. Viveka reaches a real gain of 1.43 sigma, while spending a similar budget on extra seeds reaches 0.82 sigma. These numbers come from a simulation, not from real agents. We end with a fixed plan for testing the same claims on an open-source self-refining coding agent.

## Related research

* [Agent Kalki](https://docs.kalarislabs.com/research/papers/agent-kalki.md): Agent Kalki is a Rust coding-agent harness in which a gate the agent cannot write to admits each self-modification only when eight ordered predicates hold.
* [The Harness Is the Lab](https://docs.kalarislabs.com/research/papers/the-harness-is-the-lab.md): A position paper and blueprint for an open AI-scientist harness that grounds every claim in a typed ontology and accepts self-improvement only on held-out evidence.
* [PrincipalBench v2](https://docs.kalarislabs.com/research/papers/principalbench-v2.md): PrincipalBench v2 audits the v1 orchestrator benchmark, withdraws its model rankings, and replaces it with a sealed two-phase protocol with clean controls and chance-corrected detection.

## Cite this paper

```bibtex theme={"system"}
@misc{chowdhury2026viveka,
  title     = {{Viveka: Separating Real Gains from Chance in Self-Improving Agents}},
  author    = {Chowdhury, Sayan},
  year      = {2026},
  publisher = {Zenodo},
  note      = {Preprint, Kalaris Labs},
  doi       = {10.5281/zenodo.24144830},
  url       = {https://doi.org/10.5281/zenodo.24144830}
}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.