TootGuitar

You didn’t define “free speech.”

This is false: Mistral small 24b at q4_K_M quantization is 15GB. q8 is 26GB. A 3090/4090/5090 with 24GB or two cards with 16GB (I recommend the 4060 Ti 16GB) will work fine with this model, and will work in a single computer. Like others have said, 10Gbe will be a huge bottleneck, plus it’s just simply not necessary to distribute a 24b model across multiple machines.

This is false when it comes to me to PCIe, as mentioned elsewhere in this thread.

Most motherboards have cutouts on one end of the PCIe x1/x4 slots, for exactly this situation. If not, and you want to be adventurous, you can cut the plastic of the slot and it’ll work fine.

If the card is PCIe 3.0 x4, and the slot is PCIe 4.0 x1, the card will run at PCIe 3.0 x1. But it’ll work.

This isn’t really true — a lot of the newer MoE models run just fine on a CPU coupled with gobs of RAM. Yes, they won’t be quite as fast as a GPU, but getting 128GB+ of VRAM is out of reach of most people.

You can even run Deepseek R1 671b (Q8) on a Xeon or Epyc with 768GB+ of RAM, at 4-8 tokens/sec depending on configuration. A system supporting this would be at least an order of magnitude cheaper than a GPU setup to run the same thing.

Yeah I definitely get your point (and I didn’t downvote you, for the record). But I will note that ChatGPT generates text way faster than most people can read, and 4 tokens/second, while perhaps slower than reading speed for some people, is not that bad in my experience.

It depends on what you mean by “relative responsiveness”, but you can absolutely get ~4 tokens/sec of performance on R1 671b (Q4 quantized) from a system costing a fraction of the number you quote.

It is indeed called a refund by the IRS and all tax professionals. The person(s) attempting to correct your use of “refund” are wrong, but they were probably trying to make the point that giving a lot of extra money to the government interest-free is not a smart financial idea.

Quit letting politics ruin our collective ability to drive by suggesting to people that Volkswagen is now an evil company. They support Hitler because they think his business policies will benefit their company. True or false as that may be their company is still great at making cars and we shouldn't be infighting about that.

Yeah, that's a good point. I'm not counting on sideloading bringing any benefit to me, but if it does I'll be pleasantly surprised.

you are on the privacy lemmy, i hope you realize that

Yes.

iPhones are as anti-privacy as possible

I'm not even sure I know what "as anti-privacy as possible" actually means, but this is a garbage statement, and I say this as someone who thinks that both Android and iOS are flaming piles of shit. Did you see that the OP mentioned that they'd consider switching to an iPhone?

Just my two cents on this topic: I used to use an Android phone with LineageOS (this was before Graphene was a thing), and struggled with similar bugs/issues from time to time.

I got an iPhone and never looked back.

Don't get me wrong, as you suggested here, iPhones are objectively worse in a lot of ways. But mostly, it. Just. Works. And, rather than fight the OS on things like VPN configs, ad blocking, browser usage, etc, I've found that I simply use my phone less, and tether my phone to a real computer more often. Paired with a small chromebook or other laptop running Linux, or (gasp) even MacOS, I just don't use my phone as much as I used to.

On the plus side, iPhones are supported for a long time, have a secure lockdown mode which you can enable if you're extra paranoid, and have "don't need to think about it" full-device encryption including full phone backup support. If your phone ever dies or you want to upgrade, you can load a full backup/image from your old device on to your new one with close to zero fuss (just gotta deal with USB 2.0 speeds on all but the newest phones :)

One final note, you don't need to sign in to an account to use iOS as far as I'm aware. You lose out on the sync/iCloud stuff that Apple provides, but it sounds like you don't care much about that anyway.

You evidently don't know enough about logic and logical fallacies to grasp what I'm saying. I don't think it's worth spending any more time on. Take care.