##Getting Started With Local LLMs Optimized for RTX PCs
Many users want to run large language models (LLMs) locally for more privacy and control, and without subscriptions, but until recently, this meant a trade-off in output quality. Newly released open-weight models, like OpenAI’s gpt-oss and Alibaba’s Qwen 3, can run directly on PCs, delivering useful high-quality outputs, especially for local agentic AI.
This opens up new opportunities for students, hobbyists and developers to explore generative AI applications locally. NVIDIA RTX PCs accelerate these experiences, delivering fast and snappy AI to users.
NVIDIA has worked to optimize top LLM applications for RTX PCs, extracting maximum performance of Tensor Cores in RTX GPUs.
One of the easiest ways to get started with AI on a PC is with Ollamaan open-source tool that provides a simple interface for running and interacting with LLMs. It supports the ability to drag and drop PDFs into prompts, conversational chat and multimodal understanding workflows that include text and images.
NVIDIA has collaborated with Ollama to improve its performance and user experience. The most recent developments include:
- Performance improvements on GeForce RTX GPUs for OpenAI’s gpt-oss-20B model and Google’s Gemma 3 models
- support for the new Gemma 3 270M and EmbeddingGemma3 models for hyper-efficient retrieval-augmented generation on the RTX AI PC
- Improved model scheduling system to maximize and accurately report memory utilization
- Stability and multi-GPU improvements
Ollama is a developer framework that can be used with other applications. Such as, AnythingLLM – an open-source app that lets users build their own AI assistants powered by any LLM – can run on top of Ollama and benefit from all of its accelerations.
Enthusiasts can also get started with local LLMs using LM Studioan app powered by the popular llama.cpp framework.The app provides a user-kind interface for running models locally, letting users load different LLMs, chat with them in real time and even serve them as local request programming interface endpoints for integration into custom projects.
A simple way to do this is with AnythingLLMwhich supports document uploads, custom knowledge bases and conversational interfaces.This makes it a flexible tool for anyone who wants to create a customizable AI to help with research, projects or day-to-day tasks. And with RTX acceleration,users can experience even faster responses.
By loading syllabi, assignments and textbooks into AnythingLLM on RTX PCs, students can gain an adaptive, interactive study companion. They can ask the agent, using plain text or speech, to help with tasks like:
- Generating flashcards from lecture slides: “Create flashcards from the Sound chapter lecture slides. Put key terms on one side and definitions on the other.”
- Asking contextual questions tied to their materials: “Explain conservation of momentum using my physics 8 notes.”
- Creating and grading quizzes for exam prep: “Create a 10-question multiple choice quiz based on chapters 5-6 of my chemistry textbook and grade my answers.”
- Walking through tough problems step by step: “Show me how to solve problem 4 from my coding homework, step by step.”
Beyond the classroom, hobbyists and professionals can use AnythingLLM to prepare for certifications in new fields of study or for other similar purposes. And running locally on RTX GPUs ensures fast,private responses with no subscription costs or usage limits.
Project G-Assist Can Now control Laptop Settings
project G-Assist is an experimental AI assistant that helps users tune,control and optimize their gaming PCs through simple voice or text commands – without needing to dig through menus. Over the next day, a new G-Assist update will roll out via the home page of the NVIDIA App.
Building on its new, more efficient AI model and support for the majority of RTX GPUs released in Augustthe new G-Assist update ad
NVIDIA AI Updates: G-Assist, windows ML Acceleration, and Nemotron advancements
NVIDIA continues to push the boundaries of AI development and deployment with several recent updates, including enhancements to its G-Assist tool, accelerated Windows ML performance, and the ongoing development of the Nemotron open-source AI collection.
🚀 G-Assist v0.1.18 Now available
the latest version of NVIDIA’s G-Assist, v0.1.18,is now available for download through the NVIDIA App. This update introduces new commands specifically designed for laptop users and aims to improve the overall quality of responses generated by the AI assistant.
⚙️ Faster AI Inference with Windows ML and NVIDIA TensorRT
Microsoft has released Windows ML with NVIDIA TensorRT for RTX acceleration, bringing up to a 50% performance increase to AI inference on Windows 11 PCs equipped with RTX GPUs. This integration streamlines the deployment of Large Language Models (LLMs), diffusion models, and other AI workloads.
🌐 NVIDIA Nemotron Fuels AI Innovation
NVIDIA’s Nemotron – a collection of open models, datasets, and techniques – is driving innovation across a wide range of AI applications. From generalized reasoning to specialized industry solutions, Nemotron provides developers with the tools to build and deploy cutting-edge AI systems. The project includes a 2.9B model with Flash Attention enabled by default and CUDA kernel optimizations.
Stay Connected:
* NVIDIA AI PC: Facebook, Instagram,TikTok, X (formerly Twitter). Subscribe to the RTX AI PC newsletter.
* NVIDIA Workstation: LinkedIn, X (formerly Twitter).
* Terms of Service