Enlarge (credit: Nvidia)

On Tuesday, Nvidia released Chat With RTX, a free personalized AI chatbot similar to ChatGPT that can run locally on a PC with an Nvidia RTX graphics card. It uses Mistral or Llama open-weights LLMs and can search through local files and answer questions about them.

Chat With RTX works on Windows PCs equipped with NVIDIA GeForce RTX 30 or 40 Series GPUs with at least 8GB of VRAM. It uses a combination of retrieval-augmented generation (RAG), NVIDIA TensorRT-LLM software, and RTX acceleration to enable generative AI capabilities directly on users' devices. This setup allows for conversations with the AI model using local files as a dataset.

"Users can quickly, easily connect local files on a PC as a dataset to an open-source large language model like Mistral or Llama 2, enabling queries for quick, contextually relevant answers," writes Nvidia in a promotional blog post.

A screenshot of Chat With RTX, which runs in a web browser window. (credit: Benj Edwards)

Using Chat With RTX, users can talk about various subjects or ask the AI model to summarize or analyze data, similar to how one might interact with ChatGPT. In particular, the Mistal-7B model has built-in conditioning to avoid certain sensitive topics (like sex and violence, of course), but users could presumably somehow plug in an uncensored AI model and discuss forbidden topics without the paternalism inherent in the censored models.

Also, the application supports a variety of file formats, including .TXT, .PDF, .DOCX, and .XML. Users can direct the tool to browse specific folders, which Chat With RTX then scans to answer queries quickly. It even allows for the incorporation of information from YouTube videos and playlists, offering a way to include external content in its database of knowledge (in the form of embeddings) without requiring an Internet connection to process queries.

Rough around the edges

We downloaded and ran Chat With RTX to test it out. The download file is huge, at around 35 gigabytes, owing to the Mistral and Llama LLM weights files being included in the distribution. ("Weights" are the actual neural network files containing the values that represent data learned during the AI training process.) When installing, Chat With RTX downloads even more files, and it executes in a console window using Python with an interface that pops up in a web browser window.

Several times during our tests on an RTX 3060 with 12GB of VRAM, Chat With RTX crashed. Like open source LLM interfaces, Chat With RTX is a mess of layered dependencies, relying on Python, CUDA, TensorRT, and others. Nvidia hasn't cracked the code for making the installation sleek and non-brittle. It's a rough-around-the-edges solution that feels very much like an Nvidia skin over other local LLM interfaces (such as GPT4ALL ). Even so, it's notable that this capability is officially coming directly from Nvidia.

On the bright side (a massive bright side), local processing capability emphasizes user privacy, as sensitive data does not need to be transmitted to cloud-based services (such as with ChatGPT). Using Mistral 7B feels slightly less capable than ChatGPT-3.5 (the free version of ChatGPT), which is still remarkable for a local LLM running on a consumer GPU. It's not a true ChatGPT replacement yet, and it can't touch GPT-4 Turbo or Google Gemini Pro/Ultra in processing capability.

Nvidia GPU owners can download Chat With RTX for free on the Nvidia website.

Enlarge / A sample of the RTX Remix user interface being used to develop the enhanced Portal with RTX. (credit: Nvidia )

In the waning days of 2022, when Nvidia was preparing to release the impressive, ray tracing-enabled Portal with RTX , we were already looking toward a future when "the Nvidia RTX Remix modding platform used to remaster Portal will also be released to the general public at some point, making it easier to create updated versions of old DirectX 8 and DirectX 9 games with AI-upscaled textures and modern lighting effects," as we wrote at the time. That "at some point" future has fully arrived this week with Nvidia's open beta launch of its RTX Remix modding tools .

This isn't the wider modding community's first taste of RTX Remix's upscaling and lighting tools. Nvidia released an alpha version of the RTX runtime last April, offering "capture and replacement" modules that could upgrade older game assets and add modern graphical features like DLSS3 at playback. ModDB lists dozens of older games with an RTX.conf file that offers some level of RTX-powered graphical enhancement.

But this week's official open beta launch gives "experienced modders" new tools to easily create and insert these kinds of updated graphical effects and models in classic titles. That includes "generative AI texture tools" that Nvidia says use "our own proprietary model trained on our in-house dataset" to automatically upscale low-res textures to up to four times the original resolution. It also means the ability to add "physically accurate dynamic lights" that work with ray-tracing-capable hardware and a variety of open source models and material maps for modders to play with.

Enlarge / Nvidia's Eye Contact feature automatically maintains eye contact with a camera for you. (credit: Nvidia)

Nvidia recently released a beta version of Eye Contact , an AI-powered software video feature that automatically maintains eye contact for you while on-camera by estimating and aligning gaze. It ships with the 1.4 version of its Broadcast app, and the company is seeking feedback on how to improve it. In some ways, the tech may be too good because it never breaks eye contact, which appears unnatural and creepy at times.

To achieve its effect, Eye Contact replaces your eyes in the video stream with software-controlled simulated eyeballs that always stare directly into the camera, even if you're looking away in real life. The fake eyes attempt to replicate your natural eye color, and they even blink when you do.

So far, the response to Nvidia's new feature on social media has been largely negative. "I too, have always wanted streamers to maintain a terrifying level of unbroken eye contact while reading text that obviously isn't displayed inside their webcams," wrote The D-Pad on Twitter.

Enlarge / Concept art for Control 2 . (credit: Remedy Entertainment)

Control 2 , the sequel to the 2019 third-person shooter that showcased many high-end gaming PCs, has been announced by developer Remedy Entertainment in a blog post .

It's just entering pre-production, and it has no release date, but the announcement sets up some high expectations. The game's "initial" development budget is 50 million euros (currently around $52 million), suggesting that this will be a triple-A, big-budget action game at a large scale. For comparison, the first game's budget was just 30 million euros. Budgets for games like this often expand over the course of development, so it's possible Control 2 will cost double or more than Control to make.

Control 2 's target platforms are Windows PCs, PlayStation 5, and Xbox Series X|S. Remedy will self-publish the game on the PC, but 505 Games will act as the publisher for the console releases.

Enlarge / Time to bust out the checkbook again, GPU lovers. The RTX 4090 is here (and it's not alone). (credit: Nvidia)

After weeks of teases, Nvidia's newest computer graphics cards, the "Ada Lovelace" generation of RTX 4000 GPUs, are here. Nvidia CEO Jensen Huang debuted two new models on Tuesday: the RTX 4090, which will start at a whopping $1,599, and the RTX 4080 , which will launch in two configurations.

The pricier card, slated to launch on October 12, occupies the same highest-end category as Nvidia's 2020 megaton RTX 3090 (previously designated by the company as its "Titan" product). The 4090's increase in physical size will demand three slots on your PC build of choice. The specs are indicative of a highest-end GPU: 16,384 CUDA cores (up from the 3090's 10,496 CUDA cores) and 2.52 GHz of boost clock (up from 1.695 GHz on the 3090). Despite the improvements, the card still performs within the same 450 W power envelope as the 3090. Its RAM allocation will remain at 24GB of GDDR6X memory.

This jump in performance is fueled in part by Nvidia's long-rumored jump to TSMC's "4N" process, which is a new generation of 5 nm chips that provides a massive efficiency jump from the previous Ampere generation's 8 nm process.