A T-POD™ is a software-defined, workload-specific GPU cluster (also known as a slice) created on demand at any scale. This blog explains the architecture in detail - why static GPU server allocation produces low utilization, and how T-PODs change that.
There is a specific type of waste that is nearly universal in enterprise GPU infrastructure and almost never surfaces in planning documents: the gap between the GPU capacity an organization has provisioned and the GPU capacity that is doing productive work on any given day.
In most fixed-server GPU clusters, a large share of provisioned capacity sits idle - waiting for the next job, unused by the current workload, or allocated to a task that simply doesn't need it.
T-PODs are the architectural response to this problem.
T-POD (TrndX Pod) - Technical Description
A T-POD is a logically grouped cluster of compute resources consisting of virtual machines (VMs) and/or bare metal servers, designed to deliver high-performance infrastructure for AI/ML and data-intensive workloads. At its core, a T-POD abstracts compute, GPU, storage and network resources into a unified, schedulable unit. The defining characteristic of a T-POD is its use of a novel architecture to dynamically configure the needed topology, with interconnecting nodes, enabling efficient GPU-to-GPU communication across distributed workloads. This enables flexible, high-bandwidth, and low-latency communication paths that are not constrained by traditional fixed network topologies.
Why Static GPU Server Allocation Fails at Scale
To understand why T-PODs matter, it helps to trace where GPU utilization is lost in a traditional fixed-server cluster.
Loss 1: Between-job latency. When a training job completes on a server, the scheduler must detect completion, select the next job, move data if needed, and start the next job. In a busy, well-managed cluster, this might take a variable amount of time. During that time, the server's GPUs are idle and accounted for as provisioned capacity.
Loss 2: Intra-job underutilization. Many workloads use fewer GPUs than the server they are assigned to. A job that uses 6 of 8 GPUs on a server leaves 2 GPUs completely idle for the duration of the job. In fixed-server allocation, those 2 GPUs cannot be assigned to another workload - they are bound to that server. This is a structural inefficiency, not a scheduling problem.
Loss 3: Mismatched allocation. A small inference job requiring 2 GPUs arrives when no 2-GPU server is available. It is either queued until a full 8-GPU server frees up (delay), or assigned to an 8-GPU server (waste). In a fixed-server pool, there are no partial allocations - every job gets a whole server or waits.
Across a real cluster running diverse workloads - training jobs, inference services, experiment queues, batch processing - these three sources of inefficiency compound. The result is low average utilization in well-managed clusters, lower in less well-managed ones.
How T-POD Architecture Eliminates These Losses
T-PODs address all three sources of inefficiency structurally, not operationally.
Loss 1 eliminated: When a T-POD job completes, its compute, GPU, and network resources are logically released back to the pool. The orchestration software dynamically re-establishes interconnects for the next T-POD - enabling rapid reconfiguration of network topology, significantly reducing delays compared to static architectures.
Loss 2 eliminated: T-POD allocation is granular and flexible. A job requiring 6 GPUs can be allocated the required GPUs from the shared resource pool across nodes, subject to physical topology and interconnect constraints. The remaining GPUs remain available for allocation to other T-PODs, reducing idle capacity. This removes the limitation of fixed server boundaries, where unused GPUs on a host would otherwise remain idle.
Loss 3 eliminated: T-POD topology is defined in software, not by server hardware boundaries. A 4-GPU T-POD is as valid as a 128-GPU T-POD. Small and large workloads are served from the same pool simultaneously without either waiting for the other.
The result is higher average GPU utilization in TrndX clusters - and in some configurations, higher still for well-scheduled workload mixes.
What This Looks Like in Practice
Consider a 64-GPU TrndX cluster running three concurrent workloads:
A large language model fine-tuning job requires 32 GPUs. T-POD A is composed with these resources. Simultaneously, a batch inference service requires 16 GPUs. T-POD B is composed from the remaining GPU pool with RDMA paths. It runs concurrently with T-POD A.
The remaining 16 GPUs serve a T-POD C managed by the AI/ML Workbench experiment scheduler: a queue of 8 small fine-tuning experiments, each requiring 2 GPUs, running sequentially as the T-POD's GPU allocation cycles through the queue.
All 64 GPUs are in use. Workloads can run concurrently with significantly reduced waiting time compared to fixed-server architectures. When T-POD A completes, its 32 GPUs become available for new T-POD composition in minutes.
This is not an optimistic scenario. It is what makes architecture routine.
Key Characteristics
- Heterogeneous Compute Pool - Supports both VMs and bare metal nodes, allowing flexible workload placement based on performance and isolation requirements.
- Dynamic GPU Allocation - GPUs are not statically bound to a single node; instead, they can be allocated and composed across nodes using the OCS fabric.
- High-Speed Network Interconnect - NICs (e.g., RDMA-capable interfaces) are interconnected via OCS, enabling efficient east-west traffic for distributed training workloads.
- Topology-Aware Scheduling - Resource allocation strategies (e.g., Dragonfly-based algorithms) optimize placement across racks and minimize communication overhead.
- Network Isolation - Each T-POD can be isolated at the network level, ensuring secure and independent operation of workloads.
- Scalability - Designed to scale across multiple racks or datacenter segments while maintaining performance guarantees.
Typical Use Cases
- Distributed deep learning training (e.g., multi-node GPU workloads)
- High-performance computing (HPC)
- Large-scale data processing pipelines
- AI/ML experimentation environments with dynamic resource needs
To know more: Talk to an Architect →