If you aren’t satisfied with the current AI chatbot offerings, Nvidia recently released a new one that works a little differently from the rest – ‘Chat with RTX’ is available right now as a free demo that runs locally on your Windows PC.
Instead of using cloud-based LLM (large language model) services like OpenAI’s ChatGPT or Microsoft’s Copilot, Nvidia says Chat with RTX allows users to quickly and easily “connect local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2.”
The examples shown in Nvidia’s demo include things like asking for the name of a restaurant that someone recommended. Chat with RTX produced the answer with links to the relevant files as references.
Chat with RTX instead of humans
It supports common file formats like .txt, .pdf, .doc or .docx, and .xml and will even support URLs of YouTube videos and playlists. Seeing as it only runs locally, it doesn’t require an internet connection (unless you want it to watch YouTube, presumably) so it won’t share your data with Nvidia or any other third-party servers making it a more personable and secure AI chatbot.
It might work differently from other AI chatbots but that doesn’t mean it is immune to the same bugs. It is still a free demo, so don’t expect a perfectly polished product. It also comes with its own limitations and hardware requirements.
Instead of using Nvidia-powered cloud servers like most of its AI cousins, Chat with RTX “uses retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software and NVIDIA RTX acceleration” to turn your GPU into something equivalent. That’s no easy task so you’ll need to have an Nvidia GeForce RTX 30 or 40 series GPU with at least 8GB of VRAM and running Windows for it to work.
