LocalLLaMA

2268 readers

2 users here now

Community to discuss about LLaMA, the large language model created by Meta AI.

This is intended to be a replacement for r/LocalLLaMA on Reddit.

founded 1 year ago

MODERATORS

SkySyrup

pax

noneabove1182

NVIDIA Chat With RTX (www.nvidia.com)

submitted 9 months ago by [email protected] to c/localllama

6 comments fedilink hide all child comments

This is an interesting demo, but it has some drawbacks I can already see:

It's Windows only (maybe Win11 only, the documentation isn't clear)
It only works with RTX 30 series and up
It's closed source, so you have no idea if they're uploading your data somewhere

The concept is great, having an LLM to sort through your local files and help you find stuff, but it seems really limited.

I think you could get the same functionality(and more) by writing an API for text-gen-webui.

more info here: https://videocardz.com/newz/nvidia-unveils-chat-with-rtx-ai-chatbot-powered-locally-by-geforce-rtx-30-40-gpus

you are viewing a single comment's thread
view the rest of the comments

[–] [email protected] 6 points 9 months ago (1 children)

It works with windows 10 as well; it didn't like a lot of the older pdfs that were in my documents; and it didn't work offline. The last bit is probably the python dependencies trying to update themselves, but it doesn't fail gracefully into a working state when the network isn't available.

[–] [email protected] 6 points 9 months ago

Thanks for the info. I'm not on Windows, so I can't try it. Doesn't work offline? That's what I was afraid of.

It's a great proof of concept. Hopefully we'll see more text-gen-webui extensions soon.