Training infrastructure

Train frontier models on 100x clusters.

Nonlocal builds semantic & contextual (neuro symbolic) training clusters with a hundred times the effective throughput per rack, so the next generation of models trains in days, not quarters.

The problem

Scaling models has outrun the clusters that train them.

Frontier runs now span tens of thousands of accelerators. Communication, memory bandwidth and power, not raw FLOPs, decide how fast a model learns. We rebuilt the cluster around those limits.

0effective training throughput per rack
0compute density versus GPU racks
0parameters resident in cluster memory
0sustained scaling efficiency at 100k nodes
Technology

A cluster designed as one machine & context layer.

Three layers work together to remove the bottlenecks that stall large training runs.

01

Unified reasoning fabric

A flat, non-blocking interconnect keeps every node one hop from every other, so gradient sync no longer dominates step time.

02

Semantic alignment

Data shards, model partitions and communication patterns are organized around meaning, so related tokens and parameters sit close together and traffic stays local.

03

Semantic expansion

Compact representations are expanded on demand across the cluster, giving a model far more context and capacity than any single node holds.

Performance

Weeks of training, compressed to hours.

Time to train a 1T-parameter model on a fixed token budget, relative to a conventional GPU cluster of equal power.

Conventional
1x
Optimized GPU
2.6x
Nonlocal
100x
MetricConventional clusterNonlocal
Training throughput per rack1x100x
Energy per training token1x0.01x
Scaling efficiency at 100k nodes~55%94%
Recovery from node failureMinutes to hoursSeconds

Train the next model in days.