From Nvidia:

Millions of people use chatbots powered by NVIDIA GPU-based cloud servers, now available for Windows PCs with NVIDIA RTX for local, fast, custom generative AI. The tech demo Chat with RTX is free to download, letting users personalize with their own content and accelerated by a NVIDIA GeForce RTX 30 Series GPU with 8GB VRAM.

Chat with RTX uses retrieval-augmented generation (RAG), TensorRT-LLM software, and NVIDIA RTX acceleration for generative AI capabilities. Users can quickly connect local files on a PC as a dataset to a large language model like Mistral or Llama 2 for contextually relevant answers. The tool supports various file formats.

Rather than searching through notes or content, Chat with RTX allows users to integrate information from YouTube videos and playlists for contextual queries. Results are fast, and user data stays on the device for processing sensitive information on a local PC without sharing it with a third party or internet connection.

To develop and deploy their own RAG-based applications for RTX, programmers can use the TensorRT-LLM RAG developer reference project, available on GitHub. Enter the NVIDIA Generative AI on NVIDIA RTX developer contest to win prizes such as a GeForce RTX 4090 GPU or a full, in-person conference pass to NVIDIA GTC. Learn more about Chat with RTX.



Read more: Chat with RTX Now Free to Download