$200 Nvidia AI GPU with server “socket” hacked into a PCIe card with custom PCB and 3D printed cooling – the modified Tesla V100 SMX data center GPU runs AI LLMs and is more efficient at AI inference than many modern midrange offerings

Running LLMs locally on your GPU requires a lot of VRAM, which can drive up the cost of the rig exponentially these days. Amid the ongoing AI boom, the best value lies in older, often forgotten and still powerful processors, and that’s exactly what YouTuber is HHardware haven found. He took an Nvidia V100 server GPU with an SMX interface, which is similar to using a socket processor, and converted it into a standard PCIe bus that plugged into a consumer motherboard. In the end, it performed quite well for its size (and cost), even compared to modern SKUs.

The device starts with an Nvidia Tesla V100 AI GPU that uses the SMX2 socket and is designed for rack-scale use. The SMX interface is a mezzanine connector that mounts GPUs flat on a special motherboard, similar to a CPU socket, and the GPU is then screwed to the motherboard. The host was able to purchase this GPU for just $100, and the accompanying SMX to PCIe x16 adapter also cost around $100, making the total…