Thursday, July 16th 2026

Bosgame Demonstrates Scalable Private AI with 7-Node AI MAX+ 395 Cluster

Bosgame, a technology-based electronic product brand, has demonstrated distributed inference of the DeepSeek-V3.1 671B-parameter model on its AI MAX+ 395 7-Node Cluster. Built from seven Bosgame M5 AI Mini PCs connected through USB4 Direct Connection, the system presents a scalable approach to private AI deployment and local large language model (LLM) inference.

As AI models continue to expand in parameter size and complexity, a single AI PC may lack the memory and computing resources required for advanced workloads. While traditional AI servers can provide the necessary computing power, their high deployment costs and infrastructure complexity can limit adoption. Enterprises and developers are therefore exploring scale-out architectures that support larger models while providing greater flexibility than conventional AI server deployments.
Against this backdrop, the Bosgame AI Cluster offers a modular, cost-effective approach that grows with users' needs.

The Bosgame demonstration shows how seven M5 AI Mini PCs can operate as a unified AI Cluster. By distributing the inference process across multiple nodes, the cluster supports models that may exceed the memory capacity of a standalone system. Its ability to run the DeepSeek-V3.1 671B model highlights the potential of distributed computing for locally deployed generative AI applications.

Across seven nodes, the cluster provides 896 GB of Unified Memory, with up to 672 GB allocated as Unified VRAM, creating a substantial resource pool for memory-intensive AI workloads. USB4 Direct Connection enables the nodes to work together without requiring a traditional centralized server architecture.

Designed around a modular scale-out architecture, the Bosgame AI Cluster allows organizations and developers to begin with one M5 AI Mini PC and expand capacity as project requirements evolve. This flexible expansion model supports workload-based growth and offers a simpler alternative to traditional AI servers while aligning infrastructure investment with actual computing needs.

Beyond AI inference, the platform also supports local AI development through a local API service powered by Llama.cpp. The cluster can be used for software development, debugging, Kubernetes operations, enterprise AI initiatives, and edge AI computing applications. By keeping workloads and data on local infrastructure, organizations can maintain greater control over data management and deployment environments.

The demonstration reflects growing demand for scalable private AI infrastructure capable of supporting increasingly large models. Through distributed inference, modular expansion, and local deployment, Bosgame aims to provide flexible computing options for enterprises, developers, and edge AI applications.
Source: Bosgame
Add your own comment

6 Comments on Bosgame Demonstrates Scalable Private AI with 7-Node AI MAX+ 395 Cluster

#1
Kelutrel
Yeah... the 128GB model costs $2800. So 7 of them cost $20,000. And for $20,000, from the benchmarks I've seen around, each one does less than 10tok/sec on a 30B dense model Q8, so nothing to call home for even multiplied by 7.
For that amount of money and ram it may be able to run large models with acceptable power efficiency, but the response speed would be unbearably slow for any real use imho.
Posted on Reply
#2
igormp
Is there any proper, public source for this piece, @TheLostSwede? All I've found were stuff from Yahoo/PR Newswire with those same contents, but only linking to Bosgame's website, which doesn't have anything mentioning such clustered setup.
KelutrelYeah... the 128GB model costs $2800. So 7 of them cost $20,000. And for $20,000, from the benchmarks I've seen around, each one does less than 10tok/sec on a 30B dense model Q8, so nothing to call home for even multiplied by 7.
Given how the mentioned deepseek model is a 671B MoE with 37B active params, such setup using pipeline parallelism would indeed be in the ~10tok/s range, which is nothing impressive but still it's somewhat viable given the price range.
However, with tensor parallelism you'd be able to actually distribute the compute load and in theory achieve ~70tok/s, which is way more usable given the model size. But that's not really feasible with AMD stack and the bandwidth limitation of USB4.
At this price range one would be way better off with a cluster of DGX Sparks, which have way better interconnects with CX7 and way better raw compute perf for way faster prefill speeds, or even getting some 8-channel platform with a single GPU to help on the PP part. Even a DDR4 platform with 8-channels at 3200MHz would theoretically give you a little bit over 200GB/s, which is not so far off from Strix Halo's 256GB/s.
If one manages a DDR5 platform it's even better, given that now you can get 4800~5200MHz at 6/8/12-channels, meaning a bandwidth of 230GB/s on the lower end, and up to 500GB/s on the higher end of the spectrum, but then you're throwing the power efficiency part out of the window.
Posted on Reply
#3
lambda
igormpIs there any proper, public source for this piece, @TheLostSwede? All I've found were stuff from Yahoo/PR Newswire with those same contents, but only linking to Bosgame's website, which doesn't have anything mentioning such clustered setup.


Given how the mentioned deepseek model is a 671B MoE with 37B active params, such setup using pipeline parallelism would indeed be in the ~10tok/s range, which is nothing impressive but still it's somewhat viable given the price range.
However, with tensor parallelism you'd be able to actually distribute the compute load and in theory achieve ~70tok/s, which is way more usable given the model size. But that's not really feasible with AMD stack and the bandwidth limitation of USB4.
At this price range one would be way better off with a cluster of DGX Sparks, which have way better interconnects with CX7 and way better raw compute perf for way faster prefill speeds, or even getting some 8-channel platform with a single GPU to help on the PP part. Even a DDR4 platform with 8-channels at 3200MHz would theoretically give you a little bit over 200GB/s, which is not so far off from Strix Halo's 256GB/s.
If one manages a DDR5 platform it's even better, given that now you can get 4800~5200MHz at 6/8/12-channels, meaning a bandwidth of 230GB/s on the lower end, and up to 500GB/s on the higher end of the spectrum, but then you're throwing the power efficiency part out of the window.
No one should be seriously using current iteration of Strix Halo for clustering that many nodes. The hardware seems lacking and the software ecosystem still isn't mature.

Something like Intel's Crescent Island inference cards should be way better in terms of hardware but the software stack remains a problem there as well.

Man, the hardware prices suck, the so called trickle down economics ain't working for us plebs.
Posted on Reply
#4
TumbleGeorge
Is will they removing the "game" from name of company?
Posted on Reply
Nov 17th, 2026 13:14 CST change timezone

New Forum Posts

Popular Reviews

TPU on YouTube

Controversial News Posts