Anthropic has announced a multibillion-dollar partnership with Google giving Anthropic access to one million Google Tensor Processing Units and more than one gigawatt of compute power by 2026.
It might not be unrealistic to expect the other leading model suppliers to keep pace, as the evolution of model generations has seemed to require an order of magnitude increase in power capability with each succeeding generation of models.
Planned/Under Construction Mega Data Centers |
Project | Organization | Total Power Capacity | Investment | Timeline |
Stargate (Total) | OpenAI/Oracle/SoftBank | 10 GW | $500B | 2025-2029 |
Stargate Abilene | OpenAI/Oracle | 1-2 GW | $10B+ | 2025-2026 |
Stargate Phase 2 | OpenAI/Oracle | 4.5 GW | $300B+ | 2026-2028 |
Stargate Phase 3 | OpenAI/SoftBank | 1.5 GW | $50B+ | 2026-2027 |
UAE Stargate | G42/OpenAI | 5 GW | TBD | 2026+ |
Meta Louisiana | Meta | 2-4 GW | $10B+ | 2026-2028 |
Microsoft Wisconsin | Microsoft | 3-5 GW | $15B+ | 2026-2029 |
CoreWeave Pennsylvania | CoreWeave | 0.5-1 GW | $6B | 2025-2027 |
Compute Capability of Current LLM Models |
Model | Organization | Release Date | Training Compute (FLOPs) | Estimated Cost |
GPT-4 | OpenAI | March 2023 | ~2×10²⁵ | ~$100M |
Gemini Ultra | Google | Dec 2023 | ~10²⁵ | ~$100M |
Claude 4 (Opus/Sonnet) | Anthropic | May 2025 | ~10²⁵ | ~$100M |
GPT-4o | OpenAI | 2024 | ~10²⁵ | ~$100M+ |
GPT-4.5 | OpenAI | 2025 | ~10²⁵ - 10²⁶ | ~$100M+ |
Gemini 2.5 Pro | Google | March 2025 | ~10²⁵ - 10²⁶ | ~$100M+ |
Grok 3 | xAI | Feb 2025 | 10²⁶ | ~$500M-$1B |
Grok 4 | xAI | July 2025 | ~10²⁶ - 10²⁷ | ~$1B |
Llama 3 | Meta | 2024 | ~10²⁵ | ~$100M |
At least up to this point, each generation requires approximately 10 times more compute than the previous generation to achieve significant capability improvements.
Expected Compute for Coming Models |
Model | Organization | Expected Release | Training Compute (FLOPs) | Estimated Cost |
GPT-5 | OpenAI | Late 2025-2026 | 10²⁶ - 10²⁷ | $1B+ |
Claude Next | Anthropic | 2026 | ~10²⁵ - 10²⁶ | $1B+ |
Gemini 3.0 | Google | 2026 | 10²⁶ - 10²⁷ | $1B+ |
Llama 4 | Meta | 2025-2026 | 10²⁶ | $500M-$1B |
No comments:
Post a Comment