Nvidia has just announced Chat With RTX, an AI chatbot that runs on a personal computer. It offers features similar to ChatGPT and adds several more. To use Nvidia's new AI chatbot, all you need is an Nvidia RTX GPU.

What is Nvidia Chat with RTX?

Nvidia Chat With RTX is an artificial intelligence application that lets you deploy a large language model (LLM) on your own computer. That means you can use this AI chatbot offline whenever you want, instead of having to go online as you do with other AI chatbots.

The Chat With RTX app uses TensorRT-LLM technology to boost RTX performance, along with the Mistral 7B LLM through quantization, delivering fast responses and quality on par with other online AI chatbots. The app also offers Retrieval-Augmented Generation (RAG), which lets the chatbot access your files and generate personalized answers based on the data you provide. This helps you tailor the chatbot to give an experience that fits you personally.

To use Chat With RTX, you just download and install the app like any other software. However, this AI chatbot requires some minimum specs to install and run effectively.

RTX 30-Series or 40-Series GPU
16GB of RAM
100GB of free storage space
Windows 11

How to use Nvidia Chat With RTX

Although you can use Nvidia's Chat With RTX like an ordinary online AI chatbot, it's worth looking at its RAG feature. This feature lets you shape its output based on the files you give it access to.

Step 1: Create a RAG folder

To start using RAG in Chat With RTX, create a new folder to hold the files you want the AI to analyze.

Once it's created, put your data files in the folder. The data you store can cover many topics and personal file types, such as documents, PDF files, text and video. However, you should limit the number of files in this folder so performance isn't affected. More data to search means Chat With RTX will take longer to answer specific questions (though this also depends on your hardware).

Now that your database is ready, you can set up Chat With RTX and start using that database to answer your questions.

Step 2: Set up the environment

Under Dataset, make sure the Folder Path option is selected. Now click the edit icon (the pen icon) and choose the folder containing all the files you want Chat With RTX to read. You can also change the AI model if other options are available (for now there is only Mistral 7B, and that may be updated in the future).

Step 3: Ask questions with Chat with RTX!

There are several ways to query Chat With RTX. The first is to use it as an ordinary AI chatbot. Because this AI has RAG capability, you can also use it as a personal AI assistant.

There are many ways to put Chat With RTX's RAG to work for you. For example, you can use the tool to summarize information from legal documents, generate code for a software project you're developing, pick out the highlights of a video you don't have time to watch, and much more.

Step 4: Extended features

Besides a local data folder, you can use Chat With RTX to analyze YouTube videos. To do so, under Dataset, change Folder Path to YouTube URL, then copy the YouTube URL you want to analyze and paste it below the drop-down menu.