• 1 Post
  • 13 Comments
Joined 9 months ago
cake
Cake day: March 22nd, 2024

help-circle

  • But I am grateful for independent journalism, which is now my main hope for the future.

    Well guess who’s in control of eyeballs on those journalists?

    Social media companies, who have clear incentives to deprioritize such content and have repeatedly shown they do.

    Let’s reclaim music from the technocrats. They have not proven themselves worthy of our trust.

    While I agree with the article, I have issue with this line. These are not technocrats, they are “leaders” willing to make companies and their products objectively worse in the name of short term profits. These aren’t ‘technical experts put in charge,’ they are greedy, spineless pigs.


  • They’re kinda already there :(. Maybe even worse than raspberry pies.

    Intel has all but said they’re exiting the training/research market.

    AMD has great hardware, but the MI300X is not gaining traction due to a lack of “grassroots” software support, and they were too stupid to undercut Nvidia and sell high vram 7900s to devs, or to even prioritize its support in rocm. Same with their APUs. For all the marketing, they just didn’t prioritize getting them runnable with LLM software


  • B580 24GB and B770 32GB

    They would be incredible, as long as they’re cheap. Intel would be utterly stupid for not doing this.

    With OpenAPI being backed by so many big names, do you think they will be able to upset CUDA in the future or has Nvidia just become too entrenched?

    OpenAI does not make hardware. Also, their model progress has stagnated, already matched or surpassed by Google, Claude, and even some open source chinese models trained on far fewer GPUs… OpenAI is circling the drain, forget them.

    The only “real” competitor to Nvidia, IMO, is Cerebras, which has a decent shot due to a silicon strategy Nvidia simply does not have.

    The AMD MI300X is actually already “better” than Nvidia’s best… but they just can’t stop shooting themselves in the foot, as AMD does. Google TPUs are good, but google only, like Amazon’s hardware. I am not impressed with Groq or Tenstorrent.


  • Its complicated.

    So there’s Intel’s own project/library, which is the fastest way to run LLMs on their IGPs and GPUs. But also the hardest to set up, and the least feature packed.

    There’s more than one Intel compatible llama.cpp ‘backend,’ including the Intel-contribed SYCL one, another PR for the AMX support on CPUs, I think another one branded as ipex-llm, and the vulkan backend that the main llama.cpp devs seem to be focusing on now. The problem is each of these backends have their own bugs, incomplete features, installation quirks, and things they don’t support, while AMD’s rocm kinda “just works” because it inherits almost everything from the CUDA backend.

    It’s a hot mess.

    Hardcore LLM enthusiasts largely can’t keep up, much less the average person just trying to self-host a model.

    OneAPI is basically a nothingburger so far. You can run many popular CUDA libraries on AMD through rocm, right now, but that is not the case with Intel, and no devs are interested in changing that because Intel isn’t selling any “3090 class” GPU hardware worth buying.









  • So what IS their strategy now?

    Some of Pat’s initiatives were good (stay the course with Xe and fabs, which take a long time to pan out), but they kept delaying everything!

    Yet Intel is kind of screwed without good graphics or ML IP.

    If they spin off the fabs, I feel like they are really screwed, as they will be left with nothing but shrinking businesses and no multi year efforts to get out of it.

    Like… Even theoretically, I dont know what I would do to right Intel as CEO unless they can fix whatever is causing consistent delays, and clearly thats not happening. What is their path?


  • As a fervent AI enthusiast, I disagree.

    …I’d say it’s 97% hype and marketing.

    It’s crazy how much fud is flying around, and legitimately buries good open research. It’s also crazy what these giant corporations are explicitly saying what they’re going to do, and that anyone buys it. TSMC’s allegedly calling Sam Altman a ‘podcast bro’ is spot on, and I’d add “manipulative vampire” to that.

    Talk to any long-time resident of localllama and similar “local” AI communities who actually dig into this stuff, and you’ll find immense skepticism, not the crypto-like AI bros like you find on linkedin, twitter and such and blot everything out.