Mellanox (NVIDIA Mellanox) MQM9790-NS2F InfiniBand Switch in Action – Low‑Latency Interconnect Optimization for RDMA/HPC
July 13, 2026
Mellanox (NVIDIA Mellanox) MQM9790-NS2F InfiniBand Switch in Action – Low‑Latency Interconnect Optimization for RDMA/HPC/AI Clusters
As AI training clusters scale from thousands to tens of thousands of GPUs, network latency and congestion control become the decisive factors in effective compute utilization. A leading global AI research organization recently deployed the Mellanox (NVIDIA Mellanox) MQM9790-NS2F switch to underpin its next‑generation supercomputing facility, supporting large‑scale language models, molecular dynamics simulations, and distributed deep learning. This article examines the real‑world deployment, detailing how this InfiniBand switch transforms RDMA transport, HPC communication, and AI data‑parallel training fabrics.
Background & Challenges: From Ethernet Compromise to Lossless Interconnect Imperative
The organization initially relied on a 100GbE Ethernet fabric with RoCEv2. However, as their GPU nodes grew to over 1,200 servers (each with 8× NVIDIA H100 GPUs), they encountered persistent PFC deadlocks, complex ECN tuning, and occasional packet drops that caused training stalls. These issues extended average job completion times by more than 35% and consumed considerable engineering effort in network troubleshooting. The existing infrastructure could no longer deliver the deterministic microsecond‑level latency and congestion‑free throughput required for massive all‑reduce and all‑to‑all collective operations.
Solution & Deployment: The MQM9790-NS2F InfiniBand Switch as the Fabric Backbone
After evaluating multiple alternatives, the team selected the NVIDIA Mellanox MQM9790-NS2F as the core building block for a new two‑tier leaf‑spine architecture. Each leaf rack is equipped with two MQM9790-NS2F 400Gb/s NDR 64-port OSFP switches, providing 64 OSFP ports per unit and a total switching capacity of 25.6 Tb/s. The 400 Gb/s NDR speed per port, combined with cut‑through switching and adaptive routing, enables sub‑600 ns port‑to‑port latency – a critical requirement for their GPU‑intensive workloads. The deployment leveraged the MQM9790-NS2F InfiniBand switch solution integrated with NVIDIA Quantum‑2 firmware, ensuring seamless MQM9790-NS2F compatible operation with existing ConnectX‑7 adapters and BlueField‑3 DPUs.
Key deployment choices included:
- Topology: Non‑blocking fat‑tree with 64‑port OSFP to 400G optical transceivers, enabling full‑bisection bandwidth for all GPU nodes.
- Management: NVIDIA UFM (Unified Fabric Manager) for real‑time telemetry, congestion detection, and proactive path re‑routing.
- Collective offload: SHARPv3 enabled to accelerate all‑reduce operations, offloading communication from host CPUs.
To ensure a smooth transition, the team consulted the MQM9790-NS2F datasheet and MQM9790-NS2F specifications to validate power and cabling requirements. The 1U form factor and redundant power supplies fit neatly into their existing rack density, and the OSFP ports support both DAC cables and optical modules, offering flexibility for future expansion.
Measured Outcomes & Benefits – Quantifiable Gains in Performance and Efficiency
After full deployment and optimization, the organization documented significant improvements across multiple dimensions:
| Metric | Before (RoCEv2) | After (MQM9790-NS2F) |
|---|---|---|
| Average All‑Reduce Latency (1GB) | ~18 μs | < 8 μs |
| Tail Latency (99.9th %ile) | > 150 μs | < 15 μs |
| Effective GPU Utilization | ~72% | ~91% |
| Job Completion Time (avg.) | Baseline | ‑28% (1.4× faster) |
Beyond the raw numbers, the team reported near‑zero congestion‑related interruptions, greatly simplifying network operations. The built‑in telemetry and adaptive routing of the NVIDIA Mellanox MQM9790-NS2F allowed automatic load balancing across all 64 OSFP ports, eliminating manual ECN tweaking. Moreover, the MQM9790-NS2F InfiniBand switch provided deterministic performance even under 95% link utilization – a feat impossible with their previous Ethernet setup.
The organization also noted that the MQM9790-NS2F price, when amortized over the 5‑year lifecycle, was offset by reduced power consumption per port (≈1.5W per 400G port) and lower cabling costs due to higher port density. With general availability now mature, they confirmed that MQM9790-NS2F for sale through authorized channels made procurement straightforward, and the MQM9790-NS2F datasheet provided all necessary installation and compliance details.
Summary & Outlook – A Blueprint for Next‑Gen AI Infrastructures
The adoption of the Mellanox (NVIDIA Mellanox) MQM9790-NS2F in this large‑scale HPC/AI cluster demonstrates that a purpose‑built InfiniBand fabric is no longer a luxury but a necessity for organizations pushing the boundaries of AI and scientific computing. By delivering consistent sub‑microsecond latency, lossless RDMA transport, and in‑network computing acceleration, the MQM9790-NS2F 400Gb/s NDR 64-port OSFP switch empowers data centers to achieve near‑linear scalability up to tens of thousands of GPUs.
Looking forward, the team plans to expand the fabric with additional MQM9790-NS2F InfiniBand switch units to support upcoming exascale‑class workloads, including climate modeling and genomics. The MQM9790-NS2F compatible ecosystem with future NVIDIA generations ensures investment protection. For architects and IT managers evaluating similar upgrades, this real‑world case confirms that the MQM9790-NS2F InfiniBand switch solution offers a proven, cost‑effective path to high‑performance, low‑latency cluster networking – a critical enabler for the next decade of AI innovation.