“The future ain’t what it used to be.”

-Yogi Berra

  • 3 Posts
  • 121 Comments
Joined 2 years ago
cake
Cake day: July 29th, 2023

help-circle











  • Think of it more as a “distro” bazooka.

    Its linux, yes. But what it allows you to do is take old machines that you might have left retired, and create “pools” of compute resource, that you can then deploy whatever image floats your boat onto any size machine you like.

    Say for example, like me, you have an old System76 Serval. Its a good processor (6 cores, 32gb ram (ddr4), and a 2070. I haven’t used it as a daily driver since 2022.

    I put put proxmox on it, and stuck it next to my NAS, close to my router. Then I took 2 cores and 6gb, and installed Ubuntu LTS on it, and then installed coolify. Using coolify I spun up jellyfin, some home assistant stuff, and a postgres for one of my work projects.

    Then I took the other 4 cores and 28 gb of RAM and put PopOS to use as a development machine for that work project. It can stay alive as long as the project is going, and then when I’m finished, I can give that compute back to the pool and redeploy it.

    It also makes it incredibly easy and fast to test different versions of Linux (or I think you can do windows from an image this way too, but don’t quote me).









  • I’ll give it a shot, but tbh, it’s been a bit of a slog. I’m on the new Z13, the 128gb variant.

    I can’t find an “it just works” variant where both ollama and rocm play nice on the hardware AND the mediatek card works correctly. It’s either I’m able to self host fullsize llms (and do the rest of my ml work) OR I get fully functional wifi.

    I’ve got the whole install process for ollama + rocm + openwebui all set on Ubuntu, but the wifi card is barely getting 20 mbps. But access to rocm (and I assume it will be the same in pytorch) is buttery smooth and I can run medium models in the range of hundreds of tokens per second locally.

    When I throw on bazzite I’m hitting 350 mbps down but it doesn’t seem like it’s got the right rocm/ driver/ kernel/ ollama combo because I’m not even able to get 5 tps.