Technology

68066 readers

3817 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 2 years ago

MODERATORS

[email protected]

Nvidia creates gaming-centric AI chatbot that runs on your GPU, locally. (www.nvidia.com)

submitted 6 days ago by [email protected] to c/[email protected]

14 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 43 points 5 days ago (1 children)

Can you help me improve my framerate?

Sure I can! First try not to run an LLM on your GPU while gaming. Hope that helped!

[–] [email protected] 5 points 5 days ago (1 children)

It sounds like it only needs to consume resources (at least significant resources, I guess) when answering a query, which will already be happening when you're in a relatively "idle" situation in the game since you'll have to stop to provide the query anyway. It's also a Llama-based SLM (S = "small"), not an LLM for whatever that's worth:

Under the hood, G-Assist now uses a Llama-based Instruct model with 8 billion parameters, packing language understanding into a tiny fraction of the size of today’s large scale AI models. This allows G-Assist to run locally on GeForce RTX hardware. And with the rapid pace of SLM research, these compact models are becoming more capable and efficient every few months.

When G-Assist is prompted for help by pressing Alt+G — say, to optimize graphics settings or check GPU temperatures— your GeForce RTX GPU briefly allocates a portion of its horsepower to AI inference. If you’re simultaneously gaming or running another GPU-heavy application, a short dip in render rate or inference completion speed may occur during those few seconds. Once G-Assist finishes its task, the GPU returns to delivering full performance to the game or app. (emphasis added)

[–] [email protected] -1 points 5 days ago* (last edited 5 days ago) (1 children)

So they literally agree not using an LLM would increase your framerate. Also what would the machine know that the Internet couldn‘t answer as or more quickly while using fewer resources anyway? I‘m just saying this is an actually bad use case for AI precisely because it relies on the very hardware you‘re likely allocating to something else when you need it while there are better alternatives already. 8B models are fairly limited and still won‘t run very quickly or at all on slightly less beefy graphics cards than their horrendously expensive flagships of recent years while you‘re running a recent game. There just aren‘t all that many resources you can snatch from the process without crashing it.

[–] [email protected] 6 points 5 days ago

So they literally agree not using an LLM would increase your framerate.

Well, yes, but the point is that at the time that you're using the tool you don't need your frame rate maxed out anyway (the alternative would probably be alt-tabbing, where again you wouldn't need your frame rate maxed out), so that downside seems kind of moot.

Also what would the machine know that the Internet couldn‘t answer as or more quickly while using fewer resources anyway?

If you include the user's time as a resource, it sounds like it could potentially do a pretty good job of explaining, surfacing, and modifying game and system settings, particularly to less technical users.

For how well it works in practice, we'll have to test it ourselves / wait for independent reviews.