Recommended Reading

Turn latency/throughput curves into a cost-per-token TCO model with GenAI-Perf.

The right TCO unit for inference fleets, and the framework to anchor sizing to.

Canonical primary source for training memory math (params + grads + optimizer + activations).

SXM vs NVL specs (memory, bandwidth, TDP, NVLink config), providing the numbers for sizing.

Official scale-up fabric reference: per-GPU bandwidth and NVSwitch all-to-all topology.

System-level reference for the 8×H100 / 640 GB node and SuperPOD building block.

The operator automating driver, device-plugin, toolkit, GFD and DCGM on Kubernetes.

The official DaemonSet advertising GPUs to the kubelet; the layer beneath the operator.

NUMA-aware alignment (single-numa-node) so GPUs and CPUs land on one NUMA node.

Concrete PFC + ECN lossless RoCE configuration guide.

Foundational ECN spec underpinning the congestion signaling RoCEv2/DCQCN rely on.

AI-tuned Ethernet (adaptive routing, congestion control) as the RoCE alternative to InfiniBand.

The emerging open standard (packet spraying, flexible ordering) challenging InfiniBand for AI RDMA.

Hard-partitioning a GPU into up to 7 isolated instances, serving as the basis of secure multi-tenancy.

Oversubscription via the device plugin; time-slicing (no isolation) vs MIG.

HPC GPU scheduling: --gpus syntax, device isolation, MPS/MIG-aware allocation.

Fractional GPUs, fair-share scheduling and pooled multi-tenant allocation.

Direct DMA between storage and GPU memory via cuFile, removing the CPU bounce buffer.

An NVMe parallel filesystem built for metadata-heavy, high-concurrency AI I/O.

Disaggregated Shared-Everything design decoupling performance from capacity at exabyte scale.

Standards-body source for the fabric protocol underpinning disaggregated AI storage.

Canonical ref for GPU health, diagnostics, telemetry fields and power/clock policy.

Official end-to-end exporter → Prometheus → Grafana walkthrough, incl. K8s + MIG.

Definitive XID code reference (170+ codes incl. NVLink) with recommended actions.

The ready-to-import GPU dashboard NVIDIA's docs direct you to.

The canonical Principles/Domains/Capabilities operating model for GPU cost accountability.

GPU-specific cost drivers (scarcity, token pricing, buy-vs-rent) with a Crawl/Walk/Run model.

Authoritative primary text driving data-residency, encryption and access-control decisions.

The actual regulatory text of the safeguards for ePHI, covering the infra-relevant core of HIPAA.