Importance 2 of 5Covered by 1 source
Paper explores recursive self-improvement in pre-training, teases 801-30B-A3B-Base model
A new paper examines whether recursive self-improvement can be applied to frontier pre-training, not just post-training. The authors say a 30B-parameter mixture-of-experts base model, 801-30B-A3B-Base, is coming soon.
Why it matters
Applying self-improvement to pre-training is early research, but it could change how base models are built.