Bye Apple and hi NVIDIA

#6
by mitkox - opened
open/ acc org

I've been all about on-device AI, cozying up with Apple silicon since the early days of 2023. But this week, I've switched gears to NVIDIA, making it my new tech steed.

Meet my new workhorse, the HP Z8 Fury G5: armed with 4 NVIDIA A6000-48GB warriors, 192GB of GPU VRAM, a hefty 512GB RAM, the Xeon w7, and a vast 9TB of local SSD space.

This week's mission is to stabilize the tech fortress for on-device AI. I'll be fine-tuning some DS SLMs, LoRA-tuning LLMs, synth data gen from the secret vaults of private data seeds, engineering a local/remote LLM platform, dabbling in multimodal AI, and, for the grand finale, acclimating to the Linux as a desktop jungle.

Make sure you own your AI. AI in the cloud is not aligned with you; it's aligned with the company that owns it.

1732006129862.jpeg
1732000882830.jpeg

open/ acc org

Wow @mitkox - that's amazing! What model are you most excited to run on-device? ๐Ÿ”ฅ

open/ acc org

I'm more excited about the complete on device platform as the model is only a cog in the machine. IIf I focus more on the model's only side, then the Qwen family is my current favorite. I love the underdogs from Alibaba.

open/ acc org

Wow, what a build, congrats! I didn't know HP had a workstation so customizable, that's amazing. Sounds like the journey will be a lot of fun!

open/ acc org

A6000! Wow!

Sign up or log in to comment