WEKA software accelerates AI inference across long contexts in Oracle’s public cloud

WEKA NeuralMesh and Augmented Memory Grid software, when used with Oracle Cloud Infrastructure (OCI), provides 10x more token throughput, 10x more concurrent users, and 7x more tokens per GPU than DRAM-OCI alone.

WEKAs Expanded storage grid enables AI models to extend GPU server memory for inference to external neural mesh storage, use it as a KV cache with microsecond latencies and multi-TBP bandwidth, and provide up to additional petabytes of memory address space capacity. It supports Nvidia’s SX KV caching architecture. NeuralMesh is WEKA’s powerful AI file system software. The joint test results were validated on a nine-node OCI bare metal H100 cluster with 100,000 token context windows.