HPE Slingshot 400 switch offerings

The HPE Slingshot 400 switches are designed from the silicon-level up for the highest throughput, performance and reliability while minimizing queueing latency. Optimizations on links between switches reduce packet overheads and eliminate end-to-end timeouts and retransmissions that otherwise hinder use of Ethernet for latency-sensitive HPC and AI applications. Meanwhile, adaptive routing and congestion control between switches ensure bandwidth is used extremely efficiently. The product family includes two switch models: a multipurpose Top-of-Rack (ToR) and a blade for HPE Cray EX platforms

The ToR exposes 64 400 Gbps ports via 32 QSFP double density (QSFP-DD) connectors in a 1U 19” standard rack-mount form factor chassis. Ports can be configured to connect to other HPE Slingshot switches, to edge routers including select HPE or 3^rd party enterprise Ethernet switches, and to supported NICs. Redundant N+1 hot swappable power supplies and fans ensure availability. Management and monitoring operations utilize the cluster’s out-of-band management network.

The 1U full-width direct liquid-cooled blade switch for HPE Cray EX platform has 24 QSFP-DD connectors on the faceplate to connect 48 400 Gbps to other switches and/or Ethernet NIC ports, and 16 400 Gbps ports routed internally to compute endpoints though mezzanine NIC cards operating in the EX chassis. Management and monitoring operations utilize the out-of-band management network via HPE Cray EX cabinet manager.

HPE Slingshot 400 NIC offerings

The HPE Slingshot NICs accelerate HPC and AI performance and are also based on silicon designed by the HPE Cray team. They expose RDMA and HPC optimized features to software using the Libfabric software interface. MPI messages are transmitted over the same links as data traffic, and the NICs’s MPI message matching offload capability frees up host memory bandwidth and CPU resources. The NIC reduces compute overhead for GPU-initiated communications by allowing the host CPU to set up operations that the GPU can trigger asynchronously. The efficient command-launch mechanism reduces latency by receiving and pipelining commands without reading host memory. And a reliable delivery mechanism detects packet loss and enables end-to-end retry in hardware to reduce software overhead.

The NICs deliver supercomputing-class performance at double the bandwidth of the previous generation on challenging latency-sensitive HPC codes without compromise while also running standard IP applications using native Ethernet IP protocols. Some HPE Slingshot fabric capabilities are extended to the NIC-to-switch port link when using the HPE Slingshot NIC versus industry standard NICs. This includes enhanced link resiliency and fine-grained flow control. The latter helps the automatic congestion apply stiff backpressure to the specific flow causing congestion which delivers granularity over backpressure to the entire port.

The ASIC is a dual-NIC device running 400 Gbps Ethernet links, and supports 32 lanes of Gen 5 PCIe. 400G Ethernet ports are based on 4 lanes of 100G PAM-4 that can also be configured as 4x50G PAM-4 and 4x25G NRZ for respective 200G and 100G port operations. It is available in form factors for HPE Cray EX, Cray XD and ProLiant platforms. HPE Cray XD and ProLiant nodes use the standard half-height half-length standup PCI form factor. This model has a single QSFP connector as it only makes use of 1 NIC port and 16 PCIe lanes.

The mezzanine card for HPE Cray EX platform makes full advantage of the ASIC capacity and drives two Slingshot ports to the EX switch blade through Cray EX chassis’ internal circuitry.

Ordering Information

Consult HPE Slingshot interconnect 400 reference architecture information for supported host systems.

HPE Slingshot Fabric Management Suite

The HPE Slingshot Fabric Management Suite provides the management, control, and network functions and is required to configure and operate the HPE Slingshot network. For large deployments it is recommended to install the software on a dedicated standard server that communicates with the switches via a 1 or 10 Gbps management network. It may also be installed on a virtual machine on a shared server, or on a shared server as a container. In the latter case, the Fabric Manager installation file contains all required podman-based packages to make the software deployment trivial for the administration.

The Fabric Management Suite supports comprehensive health checking, monitoring and logging, performance profiling, and diagnostics that assure high performance with high availability and effective operational processes. Event, error, and counter data is gathered from all the switches in the network. Open documented APIs and a scriptable command line interface allow for integration with cluster management and network monitoring tools. All of these APIs support AuthN based on client certificates.

The separation of data and control of planes allows for the independent reliable data transfers and management operations. All network management traffic is encrypted with switch-based certificates. Scaling, efficiency, and performance of management and operations are achieved via a hierarchical distributed control plane design. Multiple Fabric Management Suite nodes can optionally operate in an Active/Standby mode to deliver a high-availability configuration.

Ordering Information

Consult HPE Slingshot interconnect 400 reference architecture information for current details on supported host systems and recommended configurations.

HPE Slingshot Monitoring Software and HPE Slingshot AIOps Software

The HPE Slingshot Monitoring Software (SMS) is designed to generate real-time insights, offering a quick and clear synthesis of the fabric’s overall health. Real-time notifications allow customers to identify key issues on easy-to-read dashboards. They reduce root cause identification time and manual troubleshooting.

End users can improve or maintain fabric utilization, ensuring all components operate optimally while swiftly detecting and resolving any degradation or failures. Slingshot Monitoring Software is available at no cost for Slingshot Customers.

The HPE Slingshot AIOps Software is an advanced analytics toolset aimed at managing complex HPE Slingshot Fabric challenges with advanced analytics techniques. Developed to address the quantity of telemetry data collections, it leverages advanced analytics to extract actionable insights, empowering system administrators to improve application and fabric performance.

By continuously monitoring application and fabric performance the software enables administrators to act before any business impact, implement rapid remediation techniques when possible, and divert resources to boost ROI. For information about licensing please refer to the related Slingshot AIOps QuickSpecs.

Both HPE Slingshot Monitoring Software and HPE Slingshot AIOps Software rely on HPE Performance Cluster Manager (HPCM) to collect Slingshot telemetry. HPCM version 1.12 or higher is required.

Ordering Information

Consult HPE Slingshot interconnect 400 reference architecture information for current details on supported host systems and recommended configurations.

For customers requiring a “standalone” Slingshot monitoring telemetry node, a single HPCM license is sufficient. One perpetual license per node includes three years of support. Electronic and paper license options are available. The software will continue working even when the support term ends. For customers with existing, separately purchased HPCM, no additional HPCM license is required to enable SMS.

For the most up-to-date information on HPE Services, please refer to the HPE Services – Supplemental QuickSpecs, which provides a comprehensive and regularly updated overview of available services.

HPE Slingshot interconnect 400 components are available in multiple form factors for different standalone and chassis-based solutions. While large network deployments with HPE Cray EX are typically custom tuned to achieve best price/performance, small and midsize clusters with HPE Cray XD and ProLiant can be ordered with off-the-shelf offerings. HPE strongly recommends HPE supplied rack-level integration of HPE Slingshot-based clusters. Please contact your Hewlett Packard Enterprise representative for requirements to configure the solution correctly and for recent additions and details on supported configurations.

Single-switch use

Even though the HPE Slingshot interconnect technology is built to implement the largest high-performance computing machines in the world, it is also suitable for the smallest compute environments. A single top-of-rack switch (R9Y96A) can simply sit in a standard 19” rack and connect up to 64 processors and accelerators.

All-to-all Topology

HPE Slingshot 400 supports “dragonfly” network topologies that ensure great bandwidth using low-cost electrical links for most of the connections. Ports on every switch are configured either as downlinks that connect to server endpoints, “intra-group” links that connect to other switches in the dragonfly group, and “inter-group” links (or “global” links) that connect each group to all the other groups. All endpoints are networked with a maximum of 3 switch hops.

Single all-to-all group:

17 switches, with 32 endpoints attached to each, can be connected together with dual-density cables for a total of up to 544 endpoints 33 switches, with 32 endpoints attached to each, can be connected together with single-density links for a total of up to 1,056 endpoints

Two-level all-to-all:

In order to build very large clusters, only 16 endpoints may be attached to each switch. It is common to form groups of 16 switches connected in all-to-all fashion with dual-density cables. In this scenario, 9 dual-density cables are theoretically available per switch for global links. Up to 145 groups can be networked together, with 256 endpoints each for a total of 37,120 endpoints.

The same idea can be applied to form groups of 32 switches with single-density all-to-all links. With 16 endpoints attached per switch, 18 single-density links would remain available per switch such that up to 577 groups can be networked together for a theoretical maximum number of 295,424 endpoints.

Fat Tree Topology

In Fat Tree or Clos topology every leaf switch can attach up to 32 endpoints. The 16 dual-density uplinks can connect to 16 different spine switches. Using a single 64-radix switch per spine allows to network 32 leaf switches and a total of 1,024 endpoints in a simple 2-Tier architecture.

It is possible to build larger 3-Tier Fat Trees with multi-chip switch complexes or with a Middle-of-Row complex. Please check with your sales representative for availability and software support.

HPE Slingshot interconnect 400 for Cray XD and ProLiant clusters

Deployments with Cray XD and ProLiant may implement Fat Tree or Dragonfly topologies. It is possible to build an entire supercomputing fabric with the HPE Slingshot 400 switch R9Y96A and the HPE Slingshot 400 NIC R9Y95A models along with cables for each NIC-to-Switch and Switch-to-Switch link.

Please refer to this link to identify HPE Cray XD and ProLiant server nodes supporting the HPE Slingshot 400 NIC R9Y95A

HPE Slingshot interconnect 400 for Cray EX supercomputers

Deployments with Cray EX supercomputers implement the Dragonfly topology. In Cray EX cabinets, the HPE Slingshot 400 NIC R9Y94A model connects to the HPE Slingshot 400 switch blade R9Y97A.

The following L0 cables need to be configured for the NIC to switch connectivity through the EX cabinets. Cables are specific to the HPE Cray EX server nodes:

Interoperability with HPE Slingshot and Multi-Fabric Connectivity

HPE Slingshot 400 is interoperable with previous generation switches and NICs at 200 Gbps per link. For cabling recommendations please refer to the Supplemental Configuration and Integration Guidelines for Slingshot 400

For the new Slingshot Transport LAG feature and its use cases please refer to the Configuration Guide - HPE Slingshot - Multi Fabric Support and Connectivity application note.

Edge router Interoperability

HPE Slingshot 400 switches are validated with select HPE Networking and third-party enterprise Ethernet switch routers for connectivity between the HPE Slingshot cluster and external Ethernet networks without use of gateway nodes. Please refer to the Supplemental Configuration and Integration Guidelines for Slingshot 400for a list of supported edge routers.

Common Features

This section describes HPE Slingshot interconnect 400 common specifications available on any HPE Cray EX, Cray XD and ProLiant platforms

HPE Slingshot interconnect 400 for Cray XD and ProLiant Clusters

HPE Slingshot interconnect 400 for Cray EX