A single GPU can serve a Pilot phase or a small AI model just fine. But once a business moves to a larger model (for more accurate answers and better handling of complex context), or once the number of employees using the AI assistant at the same time grows, a single GPU hits its limit in two different ways: it runs out of VRAM to hold the model, or it runs out of parallel processing capacity to serve many requests at once. Combining multiple GPUs solves both limits — but the right way to combine them depends on which problem you actually have.

When the AI model is too large to fit in one GPU's memory, the solution is model parallelism: splitting the model itself into multiple parts, with each GPU holding and processing only one part. The GPUs then work together to answer a single question — one GPU finishes its part and passes the work to the next. This lets you run models many times larger than a single GPU's capacity, but it requires the GPUs to constantly exchange data with each other throughout processing.

When the problem isn't a model that's too large but too many concurrent users, a better fit is data parallelism: running multiple independent copies of the same model (each one small enough to fit in a single GPU), with each GPU handling its own group of requests. This doesn't let you run a larger model, but it significantly increases how many users can be served at the same time — scaling nearly linearly with each GPU added.

Because model parallelism requires GPUs to exchange data constantly, the speed of the connection between them matters just as much as the speed of each GPU. Inside a single server, GPUs are typically linked via NVLink — a very high-speed connection built specifically for this purpose — far faster than having GPUs on separate servers communicate over an ordinary network. This is why model parallelism works best when the GPUs sit inside the same server, while scaling out across multiple servers is a better fit for data parallelism, where GPUs don't need to exchange data with each other in real time.

DABO assesses each business's actual problem before proposing a GPU configuration — based on the model size needed and the expected number of concurrent users, rather than defaulting to "more GPUs is always better." For a business that needs a large model but has few concurrent users, DABO favors model parallelism within a single NVLink-connected server; for a business serving many employees with a moderate-sized model, DABO favors scaling out with data parallelism, combined with the SAN storage architecture and replication covered in earlier articles — so customers end up with exactly the number of GPUs they need, no more and no less.