Into:iota
Pretraining as a swarm, not a fortress.
As of · Jun 4, 10:37 UTC
Macrocosmos rebuilt 9 as a pipeline parallel training network. The pitch: pretrain frontier-scale models on a swarm of unreliable, internet-connected GPUs instead of a billion-dollar datacenter.
What is iota
iota is a training subnet that splits a large language model into pieces and trains it across many independent machines on the open internet. The IOTA acronym stands for Incentivized Orchestrated Training Architecture. The team has been running subnet 9 since 2024, originally as a pure pretraining benchmark, and relaunched the subnet around the IOTA architecture in 2025.
The simple version: Imagine training a single huge AI model by stringing together a thousand consumer GPUs in different countries, with each machine handling one slice of the model and passing intermediate results along the chain.
Centralized equivalent: OpenAI or Anthropic running pretraining inside one operator's datacenter. iota is the decentralized version: same training objective, distributed across permissionless participants.
How it works:
- are assigned a slice of the model by an orchestrator, process activations passed to them by upstream miners, periodically upload their local weights to a shared S3 bucket, and merge with peers using a Butterfly All-Reduce variant
- Validators spot-check miner work, watch for anomalies, score activation quality, and can trigger rollbacks to a prior stable if the run is corrupted
Why This Matters
Other research from the same neighborhood of the network.
