DeepSeek Cost Advantages Will Only Drive Competitors to Reduce Their Own Costs

To the extent that  DeepSeek’s approach proves widely effective, algorithmic efficiency and hardware versatility rather than brute-force scaling with expensive GPUs  is among the possible outcomes for other competitors in the language model business.

That is why public firm valuations in the AI ecosystem have been challenged recently, including Nvidia, Alphabet and Microsoft, for example. 

The response by model builders will seek to narrow any perceived DeepSeek advantages in cost of model training and inference operations. 

Every leading model builder will seek to further optimize their own models, to achieve cost reductions in training and inference operations. Existing trends to reduce reliance on high-end Nvidia graphics processing units will probably intensify. 

That might include moves to use custom tensor processor units, custom application-specific integrated circuits, or lower-cost GPUs.

More focus on lightweight, fine-tuned models for enterprise use also are possible. 

Companies such as Meta, which are already pushing open-source models, may accelerate their efforts to develop cost-efficient open models that can be trained with less computational power.

From a technical standpoint, competitors will likely pursue several approaches. Algorithms might be changed to reflect:

• Sparse Computation: More use of sparse mixture-of-expert (MoE) models, which activate only a fraction of the network during inference, reducing compute demands.

• Efficient Transformer Variants: Exploration of architectures like FlashAttention, Linformer, and Mamba (state-space models) to reduce memory and computational overhead.

• Gradient and Weight Quantization: Further refinement of low-bit precision training (e.g., 4-bit, 8-bit training) to allow high-performance training on less-powerful hardware.

Different choices of hardware also will be explored, but that is an existing trend. 

Training methods are almost certain to be tweaked, with model builders possibly looking to:

• Fine-grained parallelism, a way of optimizing model sharding and tensor parallelism to reduce bottlenecks in distributed training.

• Developing training approaches that leverage many lower-powered chips across cloud and edge devices instead of relying on a few high-end GPUs.

• Adaptive computation scaling that dynamically adjusts model complexity based on hardware constraints.

4. Training Data and Pre-training Strategies

• Synthetic Data Utilization: Since compute-efficient models may rely on less high-quality training data, competitors might advance synthetic data generation to enhance pretraining efficiency.

• Progressive Training: Layer-wise or curriculum-based training methods that reduce compute-heavy early-stage processing.

5. Inference and Deployment Efficiency

• Model Distillation: Greater emphasis on distilling large models into smaller, more efficient versions with minimal performance loss.

• Token-Level Pruning: Techniques that dynamically adjust inference complexity based on task difficulty, reducing unnecessary computation.

• Adaptive Inference Pipelines: Using lower-bit or pruned versions of models for simpler queries while reserving full-scale computation for complex tasks.