Credit: AI-generated image
A team of engineers from AI inference technology company BitEnergy AI reports a method to reduce the energy requirements of AI applications by 95%. The group published a paper describing their new technique on the arXiv preprint server.
As applications of AI have become more widespread, their use has increased significantly, leading to a notable increase in energy requirements and costs. LLMs such as ChatGPT require a lot of computing power, which means a lot of electricity is needed to run them.
As an example, ChatGPT now requires approximately 564 MWh per day, enough to power 18,000 US homes. As the science continues to advance and these applications become more popular, critics have suggested that AI applications could use around 100 TWh per year in just a few years, a level comparable to Bitcoin mining operations .
In this new effort, the BitEnergy AI team claims to have found a way to significantly reduce the amount of computation required to run AI applications, without causing a reduction in performance.
The new technique is basic: instead of using complex floating-point multiplication (FPM), the method uses integer addition. Applications use FPM to handle extremely large or small numbers, allowing applications to perform calculations using them with extreme precision. This is also the most energy-intensive part of AI number crunching.
16-bit and 8-bit floating point numbers defined in IEEE 754 and on various hardware for tensor calculations, as well as the 16-bit integer. MSB means most significant bit and LSB means least significant bit. Credit: arXiv (2024). DOI: 10.48550/arxiv.2410.00907
The researchers call their new method Linear Complexity Multiplication: it works by approximating FPMs using integer addition. They say tests carried out so far have shown the new approach reduces electricity demand by 95%.
The only downside is that it requires different hardware than currently used. But the research team also notes that the new type of hardware has already been designed, built and tested.
However, it is still unclear how such hardware would be permitted: currently, GPU maker Nvidia dominates the AI hardware market. How they respond to this new technology could have a major impact on the pace of its adoption, if the company’s claims are verified.
More information:
Hongyin Luo et al, Addition is all you need for energy-efficient language models, arXiv (2024). DOI: 10.48550/arxiv.2410.00907
arXiv
© 2024 Science X Network
Quote: Integer addition algorithm could reduce AI energy needs by 95% (October 12, 2024) retrieved October 12, 2024 from
This document is subject to copyright. Except for fair use for private study or research purposes, no part may be reproduced without written permission. The content is provided for informational purposes only.
