Nvidia is expanding its presence in the AI industry beyond GPU sales by incorporating its AI-specific software development kit into more applications. The company recently announced that it will add support for its TensorRT-LLM SDK to Windows and models such as Stable Diffusion. Through this move, Nvidia aims to enhance the performance of large language models (LLMs) and related tools. TensorRT optimizes inference, the process of utilizing pretrained information to calculate probabilities and produce results, such as generating a newly created Stable Diffusion image. With this software, Nvidia strives to play a bigger role in the inference aspect of generative AI. By leveraging TensorRT-LLM, which accelerates LLMs on Nvidia's H100 GPUs, the company enables faster running of LLMs, including Meta's Llama 2 and Stability AI's Stable Diffusion. Nvidia claims that this acceleration significantly enhances the experience of more advanced LLM usage, like writing and coding assistants. In simpler terms, Nvidia aims to provide not only the GPUs for training and running LLMs but also the software that enables these models to operate more efficiently, thus discouraging users from seeking cost-effective alternatives for generative AI. The company announced that TensorRT-LLM will be publicly available to anyone interested in using or integrating it, with access to the SDK on its website.

Nvidia already dominates the market for powerful chips used in training LLMs, such as GPT-4, where multiple GPUs are typically required. The demand for its H100 GPUs has skyrocketed, with estimated prices reaching $40, 000 per chip. In response to this increasing demand, Nvidia announced that a newer version of its GPU, the GH200, would be released next year. However, new approaches have emerged to run LLMs without heavily relying on expensive GPUs, as the world of generative AI evolves quickly. Microsoft and AMD, for example, plan to develop their own chips to reduce dependence on Nvidia. Additionally, companies like SambaNova offer services that facilitate running models more easily. While Nvidia currently holds the hardware leadership position in generative AI, it seems to be envisioning a future where users will not have to rely on purchasing a large number of its GPUs. From our sponsor.