John L. Gustafson
Most computer users assume the issue of how to store numbers on computer is a long-solved problem. With every system now becoming severely bandwidth-bound, the AI and HPC communities are realizing that this is not the case; there is enormous waste and inefficiency in the way we represent real numbers in particular, and therein lies an opportunity for improvement by increasing information per bit. For example, of the 2^64 possible bit patterns in IEEE 753 double-precision floating point, only about 2% are ever used. Modern floating-point was not designed; it evolved from human-friendly principles, not from what was mathematical and optimal for hardware design.
A clean-slate design called posit format is rapidly being adopted because it is more accurate, better fits the dynamic range, saves memory, bandwidth, energy, power, and money. Low-precision posits beat low-precision floats in dozens of studies, and are now in use by companies like Meta. The posit format has been refined recently to take less silicon and energy than floats of the same precision, and commercial processors are now shipping that use posit instead of float arithmetic.
How did we get here, and where might this lead? I will present a survey that goes back to the earliest electronic computers like the ABC and ENIAC, and show the fascinating history of computer numbers. The performance of a broad range of applications can be more than doubled by increasing information-per-bit, like another turn of Moore’s law but without needing to shrink transistors.
Bio: Dr. John L. Gustafson is a computer scientist and pioneer in high-performance computing. He is best known for formulating Gustafson’s Law, the foundational alternative to Amdahl’s Law, which transformed the world’s understanding of massively parallel scalability and earned him the inaugural Gordon Bell Prize in 1988. In industry, he has held senior and executive roles at Intel (Director and Lab Leader for energy-efficient and extreme-scale computing), AMD (Chief Product Architect and Senior Fellow), ClearSpeed Technology (CTO), and Massively Parallel Technologies (CEO). He created the unum and posit number systems, alternatives to IEEE floating-point that improve accuracy and energy efficiency. He has authored influential books (e.g., Every Bit Counts) and numerous highly cited papers on parallel performance, computer arithmetic, and HPC productivity. He holds applied mathematics degrees from Caltech and Iowa State University. Gustafson currently serves as Visiting Scholar at Arizona State University and Chief Scientist/Co-founder of Vq Research.