What
A bot made of many AI agents. Each one researches a question online and gives a probability; a trained model combines them into a single forecast.
All projectsIn progress · core team
Silicon Crowd
We're building a swarm of AI agents that forecast real-world events, and teaching it which agents to trust.
In short01 02 03
Why
Prediction markets show that many independent guesses can beat a single expert. We want to see if the same holds for AI agents, on real questions nobody knows the answer to yet. Inspired by Vitalik Buterin's "From prediction markets to info finance".
Where
In FutureEval, Metaculus' tournament for forecasting bots: hundreds of questions on politics, economics, science and technology, scored once they resolve.
Bots have nearly caught up with the pros
173
Bots in the Spring 2026 tournament, on 297 questions. The gap with a team of pro forecasters is no longer statistically significant.
Metaculus, FutureEval Spring 2026~10
Beyond about ten bots, combining more forecasts stopped helping: teams of 30 or more did clearly worse. How you combine matters more than how many.
Metaculus, FutureEval Spring 2026More voices, weighed well.
Averaging many bots is easy. Knowing how much to trust each one is the hard part, and that's where the machine learning comes in.
- 01
A diverse swarm
Agents differ in model, research source and prompt, so their mistakes don't all point the same way.
- 02
A trained aggregator
A model learns from resolved questions how much to trust each agent, and corrects calibration: for example, it pushes towards 0 or 1 when the agents agree.
- 03
Controlled experiments
We measure what actually helps: many small models or one big one, research or no research, from 1 to 10 agents.
Questions it should answer
- Does a trained aggregator beat the simple median of the agents?
- Are five small models worth more than one big one, at the same cost?
- How much does web research add over reasoning alone?
- How many agents are enough before adding more stops helping?
From one question to one forecast
- QuestionTaken from the Metaculus API
- SwarmN agents research it and give a probability with a reason
- AggregatorA trained model combines the answers
- ForecastSubmitted to Metaculus automatically
- LogEvery forecast saved, to learn from
Brier score: the average squared gap between the forecast probability and the outcome (1 if it happened, 0 if not). Lower is better. It is how we compare aggregators.
First milestone
- A bot that forecasts and submits on its own
- A log of every agent’s forecast and sources
- Brier score of the baseline on resolved questions
- Swarm with median and log-odds averaging
Stack
- Python and APIs of several LLMs
- Web and news research
- scikit-learn and PyTorch for the aggregator
- GitHub Actions for daily runs
Built by the DeLTA core team: no applications for now.
Timeline
- Nov – Dec 2026Setup: the bot runs on its own, first test rounds
- Dec 2026Single-agent baseline, with every forecast logged
- Early 2027Swarm live in the tournament
- Feb – Mar 2027Trained aggregator, compared with the median
- End of seasonResults, open-source write-up, and a talk for the DeLTA community
Measured, not claimed
Every result is compared with a simple baseline.
Open source
Code, logs and experiments public at the end of the season.
We say what didn't work
Failed experiments go in the write-up too.
Costs under control
Small models first; a big one only if it pays off.
Follow the build.
The core team is building this one, so there are no applications. Follow us on Instagram for the results: we'll present them at a DeLTA event.
Follow on Instagram