That will certainly be amazing, but for now it's actually not bad to use either oobabooga web UI or koboldcpp to run the inferencing and provide a rest endpoint, cause you can trick basically any program into treating it as if it's OpenAI and use it the same way
Yeah I'm using it with home assistant :)
Basically I'm using oobabooga for inference and providing an API endpoint as if it were OpenAI, and then plugging that into Microsoft's guidance, which I then give a tool. The tool takes as input the device and the state, and then calls my home assistant rest endpoint to execute the command!
This is a great write up, thank you so much! Subscribed to your Lemmy too :)
I'm really intrigued by Llongma and super curious to see if it actually does anything vs scaling llama2, on llama 1 going from 2k->3k was pretty trivial (eventually), so I wonder what llama2s trivial scaling would be.
Also curious to see if it combats the U shaped attention in any way, I imagine little, but ready to be delightfully surprised!
Thanks again for making this, love contributions like this for our community!!
I'm trying to find a way to use it with Guidance to control my smart home, actually really doable with only a 13b model
I guess it depends on what you mean by usable, I think people have had success with ROCM, it's not as solid as CUDA of course but it's been more than usable
Security patches are out monthly less than a week after Google releases, OS updated are slower but have been getting better, only major downside is the lack of commitment to more than 2 years of OS, real kick in the shins for such an expensive phone but alas I'm a sucker for all it's other offerings
Camera is nice and especially unique so that is a bonus, but for me it's the physical fingerprint scanner, SD card slot, headphone jack, front firing speakers, and no notch in a flagship from a reputable company, could drop several of those requirements and not find any options and this has all of them
I'm devoted to Sony, can't switch to anything else.. they're the only one watching all the features I care about
There's a few things, one of the biggest is the commercial license and the 4k context
they may have added that recently, cause i wanna say it didn't but on my last phone (2 months ago) it did
yup especially with all these quadratic scalings, we need to break away from it
100%, this number is skewed by the fact that tesla will basically "recall" for any minor issue because it's a simple software update, I imagine a lot of companies try to avoid recalls as aggressively and for as long as possible because it's a significantly bigger burden on them
I say this as someone who drives a Tesla but is still extremely judgemental of Tesla