Saturday, June 22, 2024

Moore's Law Slowing is Counterbalanced by Other Developments

It is possible to argue that Moore’s Law (which suggests a doubling of transistor density about every 12 to 18 months, or about every two years in practice) has only slowed, and not stopped, since the mid-1960s. 


Time Period

Doubling Time (Years)

1965-1975

1

1975-2000

2

2000-2010

2-3

2010-Present

                          3+


On the other hand, consider the transistor densities we now have to “double.” As we push the boundaries of how closely together transistors and pathways can be spaced, it becomes more difficult to manufacture the chips. 


Year

Processor Model

Transistor Count

Doubling Time (Years)

1971

Intel 4004

2,300

-

1974

Intel 8080

6,000

3

1978

Intel 8086

29,000

4

1982

Intel 80286

134,000

4

1985

Intel 80386

275,000

3

1989

Intel 80486

1,200,000

4

1993

Pentium

3,100,000

4

1997

Pentium II

7,500,000

4

1999

Pentium III

9,500,000

2

2000

Pentium 4

42,000,000

1

2006

Core 2 Duo

291,000,000

6

2010

Core i7

1,170,000,000

4

2014

Core i7 (Haswell)

1,400,000,000

4

2018

Core i9

2,000,000,000

4

2022

Apple M1

16,000,000,000

4


Similar slowing can be seen in accelerator and graphics processing chips. 


Year

GPU Model

Transistor Count

Performance Improvement

Doubling Time (Years)

2012

Kepler (GK110)

7.1 billion

Baseline

-

2016

Pascal (GP100)

15.3 billion

2x

4

2018

Turing (TU102)

18.6 billion

1.2x

2

2020

Ampere (GA102)

28.3 billion

1.5x

2

2024

Blackwell (B200)

208 billion

30x

4


The other issue is that transistor counts are not the only important variables. Parallel processing is an architectural shift that prioritizes throughput over raw clock speed.


Accelerator chips are designed for specific tasks like AI or video processing and their task-specific metrics arguably are more important than simple clock speed.


Heterogeneous computing combines CPUs, GPUs, and accelerators for optimal performance across different workloads, meaning overall system performance is more relevant than individual component speeds.


No comments:

DIY and Licensed GenAI Patterns Will Continue

As always with software, firms are going to opt for a mix of "do it yourself" owned technology and licensed third party offerings....