“The future ain’t what it used to be.”

-Yogi Berra

  • 3 Posts
  • 114 Comments
Joined 2 years ago
cake
Cake day: July 29th, 2023

help-circle




  • Think of it more as a “distro” bazooka.

    Its linux, yes. But what it allows you to do is take old machines that you might have left retired, and create “pools” of compute resource, that you can then deploy whatever image floats your boat onto any size machine you like.

    Say for example, like me, you have an old System76 Serval. Its a good processor (6 cores, 32gb ram (ddr4), and a 2070. I haven’t used it as a daily driver since 2022.

    I put put proxmox on it, and stuck it next to my NAS, close to my router. Then I took 2 cores and 6gb, and installed Ubuntu LTS on it, and then installed coolify. Using coolify I spun up jellyfin, some home assistant stuff, and a postgres for one of my work projects.

    Then I took the other 4 cores and 28 gb of RAM and put PopOS to use as a development machine for that work project. It can stay alive as long as the project is going, and then when I’m finished, I can give that compute back to the pool and redeploy it.

    It also makes it incredibly easy and fast to test different versions of Linux (or I think you can do windows from an image this way too, but don’t quote me).









  • I’ll give it a shot, but tbh, it’s been a bit of a slog. I’m on the new Z13, the 128gb variant.

    I can’t find an “it just works” variant where both ollama and rocm play nice on the hardware AND the mediatek card works correctly. It’s either I’m able to self host fullsize llms (and do the rest of my ml work) OR I get fully functional wifi.

    I’ve got the whole install process for ollama + rocm + openwebui all set on Ubuntu, but the wifi card is barely getting 20 mbps. But access to rocm (and I assume it will be the same in pytorch) is buttery smooth and I can run medium models in the range of hundreds of tokens per second locally.

    When I throw on bazzite I’m hitting 350 mbps down but it doesn’t seem like it’s got the right rocm/ driver/ kernel/ ollama combo because I’m not even able to get 5 tps.



  • Pop_OS I s a great first “it just works” experience.

    But also, don’t be af aid to be a bit of a distro slut. I’ve been distro hopping lately and it’s very liberating.

    If you want to try another, “it just works” experience, I highly recommend bazzite. It doesn’t exactly work for me because of the immutability, and I run high end hardware in weird configurations, Ill need to hop in and wrench on things from time to time. But I installed it in my exploration last week and found it immensely pleasurable.

    If anyone wants to provide some guidance for how to overcome some of the issues immutability creates (I need specific versions of ollama and rocm), I could really use the help.