Table of Contents
HPE Cray XD670 System is a top-performing GPU-accelerated server, delivering a complete AI Training and Deep Learning solution to rack-scale, in a rack and roll fashion.
HPE Cray XD670 is a 5U chassis system that contains a single 2x CPU node with 8x NVIDIA® H100 or H200 Tensor Core SXM5 GPUs. It offers a complete, scalable solution for AI and HPC customers everywhere, with flexibility of fabric, memory, storage and operating system. HPE Cray XD670 provides maximum performance for large model AI training and Deep Learning. HPE Cray XD670 is available with Direct Liquid Cooling option for improved power efficiency.
Built with high-speed networking technologies, integrated storage, a robust software portfolio and management tools, HPE Cray XD670 systems can enable customers to innovate and prepare for tomorrow’s challenges.
Overview
HPE Cray XD670 Server System Key Features
- – 5U single-node chassis for air-cooled or direct cooled infrastructure.
- – GPUs: 8x NVIDIA® H100 or H200 Tensor Core SXM5 GPUs providing leadership performance for AI Training, Deep Learning and advanced HPC simulations. PCIe GPUs are not supported on HPE Cray XD670.
- – CPUs:
- Support for 4th Generation Intel® Xeon® Scalable Processors: “Sapphire Rapids” for H100.
- Support for 5th Generation Intel® Xeon® Scalable Processors: “Emerald Rapids” for H200.
- – DRAM: Support for up to 32x DDR5 5600 MT/s DIMMs.
Notes:
- – Setting the memory bus to 4800 is not supported under 2DPC configurations for DDR5-4800 MT/s memory.
- – Setting the memory bus to 5600 is not supported under 2DPC configurations for DDR5-5600 MT/s memory.
- – After selecting “Best Performance” the memory speed will need to be manually set to 4400 MT/s to allow operating with 2 DPC for both DDR5-4800 MT/s and DDR5-5600 MT/s DIMMs.
- – High-Speed Fabric: 8x PCIe Gen 5.0 Half-Height, Half-Length slots supporting Slingshot 11, InfiniBand NDR and Ethernet, providing direct switchable connections between High-Speed Fabric, GPUs, NVMe drives and CPUs.
- – Storage: Up to 8 SFF NVMe U.2 | U.3 and 2 M.2 RAID SSDs (Read-Intensive and Mixed-Use).
- – CRPS Power Supplies:
- 6x 3,000-Watt capacity per server system for H100 configurations (Air and DLC).
- 6x 3,600-Watt capacity per server system for H200 configurations (Air and DLC).
- Providing N+2 redundancy.
- – PCIe Expansion: 4x PCIe Gen 5.0 x16 expansion slots with embedded 2-port 10G Base-T (RJ45), 1-port MLAN (1 GbE), 1x VGA, 2x USB3.0, PWR Button/Reset/ID Button/Status LEDs.
NVIDIA and NVLink are trademarks and/or registered trademarks of NVIDIA Corporation in the U.S. and other countries. All third-party marks are the property of their respective owners.
Intel and Xeon are trademarks and/or registered trademarks of Intel Corporation in the U.S. and other countries. All third-party marks are property of their respective owners.
Rack Airflow Requirements
HPE Cray XD670 System
The increasing power of new high-performance processor technology requires increased cooling efficiency for rack-mounted servers. For maximum cooling, HPE racks are recommended to allow these racks to be fully loaded with servers using the latest processors. For detailed information, see HPE Cray XD670 System User Guide:
https://support.HPE.com/connect/s/search?language=en_US#q=XD6500
Notes:
− If a third-party rack is used, observe the following additional requirements to ensure adequate airflow and to prevent damage to the equipment.
− When selecting Direct Liquid Cooling (DLC), HPE racks are required for factory integration.
− When selecting Direct Liquid Cooling, select one of each per rack:
- HPE Cray XD670 CDU Extension Bracket (P67797-B21)
- HPE Cray XD670 42U/48U DLC Rack Manifold (P70519-B21)
- If using the 42U rack: HPE Cray 42U 800 mm Air Dam FIO Kit (P71226-B21)
- If using the 48U rack: HPE Cray 48U 800 mm Air Dam FIO Kit (P71228-B21)
− Always use blanking panels to fill all remaining empty front panel U-spaces in the rack. This arrangement ensures proper airflow. Using a rack without blanking panels will result in improper cooling that can lead to thermal damage.
Cooling Options
These optional cooling technologies are compatible with air cooled HPE servers.
HPE Adaptive rack Cooling System (ARCS)
This cooling solution supports up to 4 full racks (servers, networking or storage) and manages 100% of the server heat in a completely enclosed cluster. It provides cool air for inlet side and manages exhaust heat, transferring it to facility water cooling.
Learn more at www.HPE.com/psnow/doc/a00051240enw
HPE Rear Door Heat Exchanger (RDHX)
This cooling solution attaches to the back of a single rack (servers, networking or storage) and neutralizes the hot air exiting the rack by transferring the heat to facility water cooling.
Learn more at www.HPE.com/psnow/doc/a00039574enw
Cooling Options
These optional cooling technologies are compatible with both air cooled and liquid cooled HPE servers.
HPE Adaptive rack Cooling System (ARCS)
This cooling solution supports up to 4 full racks (servers, networking or storage) and manages 100% of the server heat in a completely enclosed cluster. It provides cool air for inlet side and manages exhaust heat, transferring it to facility water cooling. When servers with DLC are added to ARCS, the highest rack density is possible, with 100% heat transfer to facility water.
Learn more at www.HPE.com/psnow/doc/a00051240enw
HPE Rear Door Heat Exchanger (RDHX)
This cooling solution attaches to the back of a single rack (servers, networking or storage) and neutralizes the hot air exiting the rack by transferring the heat to facility water cooling. When servers with DLC are added to RDHX, the highest rack density is possible, with 100% heat transfer to facility water.
Learn more at www.HPE.com/psnow/doc/a00039574enw
Standard Features
Processors (Select two of the following):
| 4th or 5th Gen Intel® Xeon® Processor | Cores | Base Frequency | Max Frequency | Max Memory | Default TDP (W) | Cache | Memory |
|---|---|---|---|---|---|---|---|
| Xeon 8480+ | 56 | 2.0 GHz | 3.8 GHz | 2 TB | 350 | 105 MB | 4800 MT/s |
| Xeon 8470Q | 52 | 2.1 GHz | 3.8 GHz | 2 TB | 350 | 105 MB | 4800 MT/s |
| Xeon 8470 | 52 | 2.0 GHz | 3.8 GHz | 2 TB | 350 | 105 MB | 4800 MT/s |
| Xeon 8468 | 48 | 2.1 GHz | 3.8 GHz | 2 TB | 350 | 105 MB | 4800 MT/s |
| Xeon 8468 V | 48 | 2.1 GHz | 3.8 GHz | 2 TB | 350 | 105 MB | 4800 MT/s |
| Xeon 8462Y+ | 32 | 2.8 GHz | 3.7 GHz | 2 TB | 300 | 105 MB | 4800 MT/s |
| Xeon 8452Y | 36 | 2.0 GHz | 3.2 GHz | 2 TB | 300 | 67.5 MB | 4800 MT/s |
| Xeon 6430 | 32 | 2.1 GHz | 3.5G Hz | 2 TB | 270 | 60 MB | 4800 MT/s |
| Xeon 6530 | 32 | 2.1 GHz | 4.0 GHz | 4 TB | 270 | 160 MB | 4800 MT/s |
| Xeon 8558 | 48 | 2.1 GHz | 4.0 GHz | 4 TB | 330 | 260 MB | 5200 MT/s |
| Xeon 8568Y+ | 48 | 2.3 GHz | 4.0 GHz | 4 TB | 350 | 300 MB | 5600 MT/s |
| Xeon 8570 | 56 | 2.1 GHz | 4.0 GHz | 4 TB | 350 | 300 MB | 5600 MT/s |
| Xeon 8580 | 60 | 2.0 GHz | 4.0 GHz | 4 TB | 350 | 300 MB | 5600 MT/s |
| Xeon 8593Q | 64 | 2.2 GHz | 3.9 GHz | 4 TB | 385 | 320 MB | 5600 MT/s |
| Xeon 8592+ | 64 | 1.9 GHz | 3.9 GHz | 4 TB | 350 | 320 MB | 5600 MT/s |
| Xeon 8562Y+ | 32 | 2.8 GHz | 4.1 GHz | 4 TB | 300 | 60 MB | 5600 MT/s |
| For H200 DLC Configurations: | SKU |
| HPE Cray XD670 Direct Liquid Cooling 3600 W Configure-to-order Server | P69518-B21 |
| NVIDIA HGX H200 141 GB 8-GPU FIO Accelerator for HPE Cray Supercomputing XD670 | S3P62A |
| Notes: – Select either NVIDIA HGX H100 80 GB 8-GPU FIO Accelerator for HPE Cray (P65657-B21) or NVIDIA HGX H200 141 GB 8-GPU FIO Accelerator for HPE Cray XD670 (S3P62A). – 42U or 48U rack is required. – Add 1 per rack of the following SKUs: – P67278-B21 HPE Cray XD670 Connection kit – P67797-B21 HPE Cray XD670 CDU Extension Bracket – P70519-B21 HPE Cray XD670 42U/48U DLC rack Manifold – If using the 42U rack: HPE Cray 42U 800 mm Air Dam FIO Kit (P71226-B21) – If using the 48U rack: HPE Cray 48U 800 mm Air Dam FIO Kit (P71228-B21) | |
Memory (Select two of the following)
| Type | DDR5 Registered (RDIMM) |
| DIMM slots available | 32 slots per server for XD670 8 channels per processor, 2 DIMMs per channel |
| Minimum capacity | 1.024 GB DDR @4800 MT/s |
| Maximum capacity | Up to 4 TB DDR @4800 MT/s |
| Description | SKU |
| HPE Cray XD 32 GB (1x32GB) Dual Rank x8 DDR5-4800 Registered Standard Memory Kit | P46968-H21 |
| HPE Cray XD 64 GB (1x64GB) Dual Rank x4 DDR5-4800 Registered Standard Memory Kit | P46970-H21 |
| HPE Cray Supercomputing XD 64 GB (1x64GB) Dual Rank x4 DDR5-5600 Registered Standard Memory Kit | P70079-H21 |
| HPE Cray XD 96 GB (1x96GB) Dual Rank x4 DDR5-5600 CAS-52-52-52 R Standard Memory Kit | S4L14A |
| HPE Cray XD 128 GB (1x128GB) Dual Rank x4 DDR5-5600 CAS-52-52-52 Registered Standard Memory Kit | S4L15A |
| Notes: – Minimum memory density per platform is 1,024 GB for NVIDIA HGX H100 80 GB 8-GPU FIO Accelerator for HPE Cray (P65657-B21) and for NVIDIA HGX H200 141 GB 8-GPU FIO Accelerator for HPE Cray XD670 (S3P62A) – Mixing of x4, x8, 3DS memory is not allowed – Mixing of memory DIMMs is not allowed – Optimal performance requires 8 (1 DIMM / channel) or 16 (2 DIMMs / channel) DIMMs per processor | |
| Storage | |
| M.2 NVMe Drives - All CTO Models (System: Std 0 // Max 4) (User Selection: Min 0 // Max 4) | SKU |
| HPE Cray XD670 CMT4034 PCIe M.2 Adapter for HPC | P60995-B21 |
| GIGABYTE 4-port with RAID PCIe M.2 22110 mm Adapter for HPE | P62966-B21 |
| M.2 NVMe Drives - All CTO Models (System: Std 0 // Max 4) (User Selection: Min 0 // Max 4) | SKU |
| Samsung PM9A3 960 GB PCIe Gen4 Mixed Use M.2 22110 Self-encrypting SSD for HPE Cray XD 2000 | P48695-H21 |
| Samsung PM9A3 1.92 TB PCIe Gen4 Mixed Use M.2 22110 Self-encrypting SSD for HPE Cray XD 2000 | P48701-H21 |
| Samsung PM9A3 3.84 TB PCIe Gen4 Mixed Use M.2 22110 Self-encrypting SSD for HPE Cray XD 2000 | P55178-H21 |
| Micron 7450 960 GB NVMe Gen4 Read Intensive M.2 22110 SSD for HPE Cray XD | P60528-H21 |
| Micron 7450 1.92 TB NVMe Gen4 Read Intensive M.2 22110 SSD for HPE Cray XD | P60529-H21 |
| Micron 7450 3.84 TB NVMe Gen4 Read Intensive M.2 22110 SSD for HPE Cray XD | P60530-H21 |
| Micron 7450 960 GB NVMe Gen4 Read Intensive M.2 2280 SSD for HPE Cray XD | P69525-H21 |
| Notes: – M.2 SSDs require the selection of one PCIe M.2 Adapter – Both M.2 Adapters support up to 4x M.2 drives – Mixing of 'M.2 Drives' is not allowed | |
| SFF RAID Controllers | SKU |
| Broadcom MR 9560-16i 32-port 12 GB PCIe 4 Tri-Mode Dual Controller for HPE | P60998-B21 |
| HPE Cray Supercomputing XD670 MR9560-16i 12 GB Cache PCIe4 x8 Tri-Mode Dual Controller | P68164-B21 |
| HPE Cray XD6500 MR 9560-16i Controller Cable Kit | P62663-B21 |
| Notes: – If RAID is not required for SFF SSDs, no controllers are required – For RAID with over 4 SFF Drives, select P60998-B21 controller kit (these are two separately managed RAID controllers – RAID volumes cannot span controllers [i.e., a RAID volume cannot consist of more than 4 SSDs]). – For RAID with 4 or less SFF Drives, select P68164-B21 controller (this is a single RAID controller). – Order only 1x Cable per server. The factory will automatically populate the server with the correct number of cables – For P68164-B21, it is one Broadcom RAID card, one battery, and a bracket for the battery; it uses two PCIe slots (the bracket for the battery uses a PCIe slot) – For P60998-B21, it is two Broadcom RAID cards, two batteries, a bracket for the batteries, and it uses three PCIe slots (the bracket for the batteries uses a PCIe slot) | |
| SFF Drives - All CTO Models (System: Std 0 // Max 8) (User Selection: Min 0 // Max 8) | SKU |
| Samsung PM9A3 7.68 TB NVMe Read Intensive SFF U.2 Self-encrypting SSD for HPE Cray XD 2000 | P55175-H21 |
| Samsung PM9A3 3.84 TB NVMe Read Intensive SFF U.2 Self-encrypting SSD for HPE Cray XD 2000 | P48411-H21 |
| Samsung PM9A3 1.92 TB NVMe Read Intensive SFF U.2 Self-encrypting SSD for HPE Cray XD 2000 | P55285-H21 |
| Samsung PM9A3 960 GB NVMe Read Intensive SFF U.2 Self-encrypting SSD for HPE Cray XD 2000 | P48414-H21 |
| Micron 7450 7.68 TB NVMe Gen4 Read Intensive SFF U.3 SSD for HPE Cray XD | P60522-H21 |
| Micron 7450 1.92 TB NVMe Gen4 Read Intensive SFF U.3 SSD for HPE Cray XD | P60520-H21 |
| Micron 7450 960 GB NVMe Gen4 Read Intensive SFF U.3 SSD for HPE Cray XD | P60519-H21 |
| HPE Cray XD670 7.68 TB NVMe Read Intensive SFF U.3 Self-encrypting 7500 SSD | S4R17A |
| HPE Cray XD670 1.92 TB NVMe Read Intensive SFF U.3 Self-encrypting 7500 SSD | S4U54A |
| HPE Cray XD670 3.2 TB NVMe Mixed Use SFF U.3 Self-encrypting 7500 SSD | S4V26A |
| HPE Cray XD670 1.6 TB NVMe Mixed Use SFF U.3 Self-encrypting 7500 SSD | S4V27A |
| HPE Cray XD670 3.84 TB NVMe Read Intensive SFF U.3 Self-encrypting 7500 SSD | S4V29A |
| HPE Cray Supercomputing 15.36 TB NVMe Read Intensive SFF U.3 7450 SSD | P65872-H21 |
| HPE Cray Supercomputing 800 GB NVMe Mixed Use SFF U.3 7450 SSD | P65874-H21 |
| HPE Cray Supercomputing 1.6 TB NVMe Mixed Use SFF U.3 7450 SSD | P67579-H21 |
| HPE Cray Supercomputing 3.2 TB NVMe Mixed Use SFF U.3 7450 SSD | P67584-H21 |
| HPE Cray XD670 6.4 TB NVMe Mixed Use SFF U.3 7450 SSD | P67589-B21 |
| HPE Cray Supercomputing 12.8 TB NVMe Mixed Use SFF U.3 7450 SSD | P67594-H21 |
| HPE Cray XD670 15.36 TB NVMe Read Intensive SFF U.2 Self-encrypting PM9A3 SSD | P67897-H21 |
RAID Settings | SKU |
| HPE RAID 0 Drive 1 FIO Setting | 339777-B21 |
| HPE RAID 1 Drive 1 FIO Setting | 339778-B21 |
| HPE RAID 5 Drive 1 FIO Setting | 339779-B21 |
| HPE Raid 5 w/SP Drive 1 FIO Setting | 339780-B21 |
| HPE RAID FIO Advanced Data Guarding Option | 339781-B21 |
| HPE Customer Defined RAID Setting Service | 389692-B21 |
| Notes: – If SKU 389692-B21 is selected, then the following RAID rules do not apply:
– If SKU 389692-B21 is selected, then at least 1 hard drive must be on the order. – If SKU 389692-B21 is selected, then preinstalled OS must be on the order.
– This Customer Intent Document should include all details about the desired RAID custom configuration. (This includes drive part numbers and quantities, RAID levels desired, which drives should be applied to each RAID level, and if a preinstalled OS has been ordered - which RAID set it should be installed on). – Warning Message - HPE Recommends using below standard RAID Settings.
– General RAID rules:
– RAID levels are not allowed if ' Broadcom MR 9560-16i ‘Controller is not in the order’ | |
InfiniBand - All CTO Models (System: Std 0 // Max 12) (User Selection: Min 0 // Max 12) | |
| Description | SKU |
| HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter | P45641-H21 |
| HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter | P45641-H24 |
| HPE InfiniBand NDR200/Ethernet 200 GbE 2-port QSFP112 PCIe5 x16 MCX755106AC-HEAT Adapter | P65333-H21 |
| HPE InfiniBand NDR/Ethernet 400G QSFP112 MPO12 850nm Multi-mode 50 m APC Transceiver | P65334-B21 |
| HPE Data Processing Unit InfiniBand NDR200/Ethernet 200 GB 2-port QSFP112 FHHL B3220 Adapter | P66386-H21 |
| HPE Data Processing Unit InfiniBand NDR/Ethernet 400 GB 1-port QSFP112 HHHL B3140H Adapter | P66387-H21 |
| Notes: – The MAX PCIe Card capacity limit is 12. It is “all inclusive” and must account for ALL PCIe cards from all categories and sub-categories (except Graphic Options category and sub-category). – If configured for a HPE Cray or HPE Slingshot Solution, select the Slingshot networking card. | |
| Networking - All CTO Models (System: Std 0 // Max 12) (User Selection: Min 0 // Max 12) | SKU |
| Intel E810-CQDA2 Ethernet 100Gb 2-port QSFP28 Adapter for HPE | P21112-B21 |
| Mellanox MCX623106AS-CDAT Ethernet 100 Gb 2-port QSFP56 Adapter for HPE | P25960-B21 |
| HPE Slingshot SA210S Ethernet 200 Gb 1-port PCIe NIC | R4K46A |
| Broadcom BCM57414 Ethernet 10/25 Gb 2-port SFP28 Adapter for HPE | P26262-B21 |
| Notes: Can only be selected or configured for a HPE Cray or HPE Slingshot Solution. | |
| Power Supplies – All H100 CTO Models contain an embedded kit consisting of 6x Delta Titanium 3,000-Watt Power Supplies – All H200 CTO Models contain an embedded kit consisting of 6x Delta Titanium 3,600-Watt Power Supplies. Power Supply Cords will still need to be ordered separately for H100 configurations. Please note that the XD670 (H200) version uses SDG-400 to C-20 power cables versus the XD670 (H100) version which uses C19-C20 cables. The XD670 (H200) version can support 277 V input; however, if this is the case, please ensure that the correct power cable is selected (SDG-400 to SDG-300). | |
Interfaces
| USB Ports | 1 USB 3.2 Gen1 Type A Port (external) |
| HPE Cray XD Management Network Port | Dedicated 1 Gbps network management port |
| Health LED | 1 |
| Power | 1 |
| UID | 1 |
| Do not remove LED | 1 |
Accessories | |
| HPE XD670 FHHL Bracket FIO Kit | P66297-B21 |
Industry Standard Compliance
- – ACPI 6.3 Compliant
- – PCIe 5.0 Compliant
- – WOL Support
- – PXE Support
- – USB 3.0 Compliant (internal); USB 2.0 compliant (external ports via SUV)
- – SMBIOS 3.4
- – UEFI 2.8
- – Redfish API
HPE Cray XD6500 Server UEFI
Unified Extensible Firmware Interface (UEFI) is an industry standard that provides better manageability and more secured configuration than the legacy ROM while interacting with your server at boot time.
The UEFI System utilities is embedded in the system ROM. Its features enable you to perform a wide range of configuration activities, including:
- – Configuring system devices and installed options.
- – Enabling and disabling system features.
- – Displaying system information.
- – Selecting the primary boot controller including configuring drive arrays and partitions.
- – Configuring memory options.
- – Launching other pre-boot environments.
HPE Cray XD6500 servers with UEFI can provide:
- – Secure Boot that enables the system firmware, option card firmware, operating systems, and software collaborate to enhance platform security.
- – An Embedded UEFI Shell that provides a preboot environment for running scripts and tools.
- – Boot support for option cards that only support a UEFI option ROM.
UEFI
UEFI provides a higher level of security by protecting against unauthorized operating systems and malware rootkit attacks, validating that only authenticated ROMs, pre-boot applications, and OS boot loaders that have been digitally signed are run. Configure and boot your servers securely with industry standard Unified Extensible Firmware Interface (UEFI). The Cray XD6500 is a UEFI Class 3 solution and does not support the less secure CSM (Compatibility Support Module) BIOS.
Software Portfolio for HPE Cray XD670
Operating Systems and Virtualization Software Support for Cray XD670 Servers
- − XD670 H100:
- RHEL: 8.6, 9.0
- Ubuntu: 22.04 LTS, 24.04 LTS
- VMWare ESXi: 7.0 U3, 8.0, 8.0 U1, 8.0 U2, 8.0 U3
- SUSE: SLES15 SP4 and SLE Micro 5.3
- Other: COS (Supported but not certified)
- − Comments:
- 4th Generation Intel Xeon Scalable Processor Family requires a minimum of RHEL 8.6 or 9.0. This server is supported with VMware's base image and not supported with the HPE custom ESXi image. For 2-socket server platforms using 4th Generation Intel® Xeon® Scalable Processors - ESXi 7.0U3 support starts at ESXi 7.0 Update 3i and ESXi 8.0 support starts at ESXi 8.0a. vSAN customers are recommended to use ESXi 7.0 Update 3
- − XD670 H200:
- RHEL: 9.3+
- Ubuntu: 22.04 LTS, 24.04 LTS
- VMWare ESXi: No Support
- SUSE: SLES15 SP5
- − Comments:
- 5th Generation Intel Xeon Scalable Processor Family and NVIDIA H200 GPU require a minimum of RHEL 9.3 (not RHEL 8). This server is supported with VMware's base image and not supported with the HPE custom ESXi image.
Notes: For more information on Hewlett Packard Enterprise Certified and Supported Servers for OS and Virtualization Software and latest listing of software drivers available for your server. Open source software is not factory-installed nor is it supported by HPE.
https://www.HPE.com/us/en/servers/server-operating-systems.html
Fabric software | SKU |
| NVIDIA UFM Enterprise with Business Standard per HCA 1-year Subscription E-LTU | S1D73AAE |
| NVIDIA UFM Enterprise with Business Standard per HCA 3-year Subscription E-LTU | S1D74AAE |
| NVIDIA UFM Enterprise with Business Standard per HCA 3-year Subscription E-LTU | S1D74AAE |
Server Management
HPE Cray XD Baseboard Management Controller
Embedded, in-depth server-level monitoring and management technology offering system management, service alerting, reporting and remote management including remote console and virtual media mount.
Industry Standard Redfish
The HPE Cray XD670 supports industry standard DMTF Redfish that provides API conformance and offers simplified server management automation such as configuration and maintenance tasks based on modern industry standards.
Learn more at: https://dmtf.org/standards/redfish.
Enabling HPE Services Remote Support
- Auto-case creation for hardware failures requires HPCM version 1.9 or higher. This feature identifies failures and creates support cases automatically, reducing downtime by starting repairs and parts ordering proactively.
- Link to the configuration instructions: HPE Performance Cluster Manager Administration Guide
For clustered HPE Cray XD670 system deployments (for HPC or other emerging workloads such as AI), customers can use the following cluster management software solutions:
HPE Performance Cluster Manager
Fully integrated system management solution offering all the functionalities you need to manage your HPE Linux®-based high performance computing (HPC) clusters, all day every day.
HPE Performance Cluster Manager aggregates system metrics.
The software provides:
- – System setup
- – Hardware monitoring and management including GPU management
- – Image management and software updates
- – Power management
- – Integration with ISV and open source software solutions
Alternatively, to manage heterogeneous clusters or for customers with additional requirements, HPE also offers:
Software Development Tools (Programming languages, debuggers, libraries)
HPE Cray Programming Environment is a fully integrated software development suite offering programmers comprehensive set of tools for developing, porting, debugging, and tuning of their applications so they can shorten application development time and accelerate their performance. CPE cannot be supported on Ubuntu OS.
Notes: For more information on HPE Cray Programming Environment visit this page
Additional 3rd party software developmental tools:
- − Intel® oneAPI
- − GNU Compiler Collection
- − Perforce® TotalView®
- − Mellanox HPC-X
HPE Message Passing Interface (MPI) - an MPI development environment designed by HPE to enable optimization of high-performance computing (HPC) applications on HPE Cray XD670 systems. The HPE Performance Software—MPI leverages a scalable MPI library and boosts performance of existing applications without requiring recompilation.
Security
The HPE Cray XD670 Root-of-Trust security option enables additional security features.
Please note that for XD670 H200 configurations, TPM does not need to be selected and comes default with the chassis.
| Security Features | P60537 B21 CTO Server SKU | P60537-B22 CTO Server SKU with Root-of-Trust |
|---|---|---|
| UEFI Secure Boot | X | X |
| UEFI TPM 2.0 (Trusted Platform Module 2.0) HPE Cray XD670 supports OpenSSL 3.0 provide for secure remote connectivity session to system. TPM Module is a selectable option. | X | TPM is required to be ordered in OCA when selecting P60537-B22 HPE Cray SC XD670 RoT CTO Svr. This is the TPM part: P63379-B21 HPE XD670 CTM010 TPM 2.0. |
| Secure out-of-the-box high-entropy RMC administrator password is unique for each system. | X | X |
| Support for multiple Redfish role-based access control (RBAC) accounts ensuring that passwords need not be shared and can provide separation of duties | X | X |
| Secure Start – only boot Signed FW | | X |
| NIST SP800-193 Protect, Detect, Recover for BIOS and BMC Tamper-free updates – digitally signed and verified with SHA256 hashing with RSA2048 key | | X |
| Secure Recovery – recover critical firmware to known good state on detection of compromised firmware. Event recorded in the event log | | X |
| SPI BUS monitoring to ensure BIOS and BMC FW are always signed and secure | | X |
Service and Support
For the most up-to-date information on HPE Services, please refer to the HPE Services – Supplemental QuickSpecs, which provides a comprehensive and regularly updated overview of available services.
Configuration Information
This section lists some of the steps required to configure a Factory Integrated Model. To ensure only valid configurations are ordered, Hewlett Packard Enterprise recommends the use of a Hewlett Packard Enterprise approved configurator. Contact your local sales representative for information on configurable product offerings and requirements.
Notes: FIO indicates that this option is only available as a factory installable option.
A Mechanized Lift will be required to accompany the order due to the excessive size and weight of the HPE Cray XD670 servers. If the customer already confirms that they already have an appropriate Lift in-house, one will not need to be ordered.
Step 1: Choose a Chassis | SKU |
| HPE Cray Supercomputing XD670 H200 Configure-to-order Server | S3P61A |
| HPE Cray XD670 Direct Liquid Cooling 3600 W Configure-to-order Server | P69518-B21 |
| Notes: If DLC Option is selected, include the following: – (1) HPE Cray XD670 HGX H100 SXM5 PC GPU P65657-B21 OR (1) NVIDIA HGX H200 141 GB 8-GPU FIO Accelerator for HPE Cray XD670 (S3P62A) – 42U or 48U HPE rack is required – Add 1 per rack of the following SKUs: – P67278-B21 HPE Cray XD670 Connection kit – P67797-B21 HPE Cray XD670 CDU Extension Bracket – P70519-B21 HPE Cray XD670 42U/48U DLC rack Manifold – If using the 42U rack: HPE Cray 42U 800 mm Air Dam FIO Kit (P71226-B21) – If using the 48U rack: HPE Cray 48U 800 mm Air Dam FIO Kit (P71228-B21) – P60537-B22 and S3P61A, which are H100 and H200 air-cooled servers respectively, contain the H100 and H200 GPUs (respectively) without needing to include a separate GPU SKU (unlike the DLC servers). | |
Step 2: Select M.2 and SFF Storage Options (max 1) | SKU |
| HPE Cray XD670 CMT4034 PCIe M.2 Adapter for HPC | P60995-B21 |
Step 3: Select M.2 Storage Options (max 4) | SKU |
| Micron 7450 960 GB NVMe Gen4 Read Intensive M.2 22110 SSD for HPE Cray XD | P60528-H21 |
| Micron 7450 1.92 TB NVMe Gen4 Read Intensive M.2 22110 SSD for HPE Cray XD | P60529-H21 |
| Micron 7450 3.84 TB NVMe Gen4 Read Intensive M.2 22110 SSD for HPE Cray XD | P60530-H21 |
| Micron 7450 960 GB NVMe Gen4 Read Intensive M.2 2280 SSD for HPE Cray XD | P69525-H21 |
| Notes: – Maximum quantity of 4 M.2 SSD Drives – All selected M.2 drive quantity are of the same SKU | capacity | |
Step 4: Select a Processor (Dual Processor Configuration Only) | SKU |
| Intel Xeon-Platinum 8468 2.1 GHz 48-core 350 W Processor Kit for HPE Cray XD | P56399-B21 |
| Intel Xeon-Platinum 8468 V 2.4 GHz 48-core 330 W Processor for HPE | P49631-H21 |
| Intel Xeon-Platinum 8452Y 2.0 GHz 36-core 300 W Processor Kit for HPE Cray XD | P56408-B21 |
| Intel Xeon-Gold 6530 2.1 GHz 32-core 270 W Processor Kit for HPE Cray XD | P70083-B21 |
| Intel Xeon-Platinum 8558 2.1 GHz 48-core 330 W Processor Kit for HPE Cray XD | P70085-B21 |
| Intel Xeon-Platinum 8568Y+ 2.3 GHz 48-core 350 W Processor Kit for HPE Cray XD | P70086-B21 |
| Intel Xeon-Platinum 8570 2.1 GHz 56-core 350 W Processor Kit for HPE Cray XD | P70087-B21 |
| Intel Xeon-Platinum 8580 2.0 GHz 60-core 350 W Processor Kit for HPE Cray XD | P70088-B21 |
| Intel Xeon-Platinum 8593Q 2.2 GHz 64-core 385 W Processor Kit for HPE Cray XD | P70089-B21 |
| Intel Xeon-Platinum 8592+ 1.9 GHz 64-core 350 W Processor Kit for HPE Cray XD | P70090-B21 |
| Intel Xeon-Platinum 8562Y+ 2.8 GHz 32-core 300 W Processor Kit for HPE Cray XD | P71376-B21 |
| Notes: Certain limitations may apply to select processors, please contact your HPE sales representative | |
Step 5: Select from the following Memory options (minimum 1,024 GB) | SKU |
| HPE Cray XD 32 GB (1x32GB) Dual Rank x8 DDR5-4800 Registered Standard Memory Kit | P46968-H21 |
| HPE Cray XD 64 GB (1x64GB) Dual Rank x4 DDR5-4800 Registered Standard Memory Kit | P46970-H21 |
| HPE Cray Supercomputing XD 64 GB (1x64GB) Dual Rank x4 DDR5-5600 Registered Standard Memory Kit | P70079-H21 |
| HPE Cray XD 96 GB (1x96GB) Dual Rank x4 DDR5-5600 CAS-52-52-52 R Standard Memory Kit | S4L14A |
| HPE Cray XD 128 GB (1x128GB) Dual Rank x4 DDR5-5600 CAS-52-52-52 Registered Standard Memory Kit | S4L15A |
| Notes: Minimum System Memory of 1,024 GB Required for H100. and H200. | |
Step 6: Choose from the following SFF Storage and RAID options | |
| Hot Plug SFF SSD | SKU |
| HPE Cray XD670 7.68 TB NVMe Read Intensive SFF U.2 Self-encrypting SSD | P62560-H21 |
| HPE Cray XD670 3.84 TB NVMe Read Intensive SFF U.2 Self-encrypting SSD | P62563-H21 |
| HPE Cray XD670 1.92 TB NVMe Read Intensive SFF U.2 Self-encrypting SSD | P62566-H21 |
| HPE Cray XD670 960 GB NVMe Read Intensive SFF U.2 Self-encrypting SSD | P62569-H21 |
| HPE Cray XD670 7.68 TB NVMe Read Intensive SFF U.3 7450 SSD | P62572-H21 |
| HPE Cray XD670 3.84 TB NVMe Read Intensive SFF U.3 7450 SSD | P62575-H21 |
| HPE Cray XD670 1.92 TB NVMe Read Intensive SFF U.3 7450 SSD | P62578-H21 |
| HPE Cray XD670 960 GB NVMe Read Intensive SFF U.3 7450 SSD | P62581-H21 |
| HPE Cray XD670 7.68 TB NVMe Read Intensive SFF U.3 Self-encrypting 7500 SSD | S4R17A |
| HPE Cray XD670 1.92 TB NVMe Read Intensive SFF U.3 Self-encrypting 7500 SSD | S4U54A |
| HPE Cray XD670 3.2 TB NVMe Mixed Use SFF U.3 Self-encrypting 7500 SSD | S4V26A |
| HPE Cray XD670 1.6 TB NVMe Mixed Use SFF U.3 Self-encrypting 7500 SSD | S4V27A |
| HPE Cray XD670 3.84 TB NVMe Read Intensive SFF U.3 Self-encrypting 7500 SSD | S4V29A |
| Broadcom MR Controller | SKU |
| Broadcom MR 9560-16i 32-port 12 GB PCIe 4 Tri-Mode Dual Controller for HPE | P60998-B21 |
| HPE Cray Supercomputing XD670 MR9560-16i 12 GB Cache PCIe4 x8 Tri-Mode Dual Controller | P68164-B21 |
| HPE Cray XD6500 MR 9560-16i Controller Cable Kit | P62663-B21 |
| Notes: Order only 1x Cable Kit (P62663-B21) with either controller. The factory will automatically populate the server with the correct cable kit. | |
Step 7: Choose Factory Configuration Setting | |
| HPE Cray Compute Node Identifier (no max) | SKU |
| HPE Cray Compute Node FIO Configuration | R9H92A |
Step 8: Choose Networking Card Configuration (Max 8) | |
| Ethernet PCIe | SKU |
| Intel E810-CQDA2 Ethernet 100 Gb 2-port QSFP28 Adapter for HPE | P21112-B21 |
| Mellanox MCX623106AS-CDAT Ethernet 100 Gb 2-port QSFP56 Adapter for HPE | P25960-B21 |
| HPE Slingshot SA210S Ethernet 200 Gb 1-port PCIe NIC | R4K46A |
| InfiniBand PCIe | SKU |
| HPE InfiniBand NDR 1-port OSFP PCIe5 x16 MCX75310AAS-NEAT Adapter | P45641-H21 |
Step 9: Select Power Cables (Max 6) | |
| A power supply kit consisting of 6x Delta Titanium 3,000-Watt Supplies is already embedded within the chassis for H100 configurations [H100 Configurations Only] Select Power Cords (required - max 6) | |
| HPE C19 - C20 WW 250V 16Amp 2.5m Jumper Cord | 295633-B22 |
| HPE C19 - C20 WW 250V 16Amp Flint Gray 2.0m Jumper Cord | AF574A |
| HPE C19 - C20 WW 250V 16Amp Flint Gray 1.20m Jumper Cord | AF575A |
| HPE C19-C20 IN 250V 16Amp 2.5m Black Jumper Cord | R1C66A |
Step 10: [H200 Configurations Only] Select Power Cords (required – max 6) | |
| | SKU |
| HPE Cray XD670 SDG-400 - C20 300 V 1 m IN Power Cord | S4X17A |
| HPE Cray XD670 SDG-400 - C20 300 V 1.5 m IN Power Cord | S4X18A |
| HPE Cray XD670 SDG-400 - C20 300 V 2 m IN Power Cord | S4X19A |
| HPE Cray XD670 SDG-400 - C20 300 V 1 m WW Power Cord | S4X20A |
| HPE Cray XD670 SDG-400 - C20 300 V 1.5 m WW Power Cord | S4X21A |
| HPE Cray XD670 SDG-400 - C20 300 V 2 m WW Power Cord | S4X22A |
| HPE Cray XD670 SDG-400 - SDG-300 300 V 1 m Power Cord | S4X23A |
Step 11: Select Operating Systems | |
| Red Hat HPC Operating Systems (max 99) | SKU |
| HPE Performance Cluster Manager FIO Software | Q9V61A |
| Red Hat Enterprise Linux for HPC Head Node 1yr Subscription 24x7 Support E-LTU | R1P34AAE |
| Red Hat Enterprise Linux for HPC Head Node 3yr Subscription 24x7 Support E-LTU | R1P35AAE |
| Red Hat Enterprise Linux for HPC Head Node 5yr Subscription 24x7 Support E-LTU | R1P36AAE |
| Red Hat Enterprise Linux for HPC Head Node 1yr Subscription 9x5 Support E-LTU | R1P37AAE |
| Red Hat Enterprise Linux for HPC Head Node 3yr Subscription 9x5 Support E-LTU | R1P38AAE |
| Red Hat Enterprise Linux for HPC Head Node 5yr Subscription 9x5 Support E-LTU | R1P39AAE |
| Red Hat Enterprise Linux for HPC Compute Node 1yr Subscription E-LTU | R1P40AAE |
| Red Hat Enterprise Linux for HPC Compute Node 3yr Subscription E-LTU | R1P41AAE |
| Red Hat Enterprise Linux for HPC Compute Node 5yr Subscription E-LTU | R1P42AAE |
| Red Hat Operating Systems (max 99) | SKU |
| Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 1 Year Subscription 24x7 Support E-LTU | G3J28AAE |
| Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 1 Year Subscription 9x5 Support E-LTU | G3J29AAE |
| Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 3 Year Subscription 24x7 Support E-LTU | G3J30AAE |
| Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 3 Year Subscription 9x5 Support E-LTU | G3J31AAE |
| Red Hat Enterprise Linux Server 2 Sockets or 2 Guests 5 Year Subscription 24x7 Support E-LTU | G3J32AAE |
| Red Hat Enterprise Linux Server 2 Sockets or Guests 5 Year Subscription 9x5 Support E-LTU | G3J33AAE |
| Red Hat Enterprise Linux 8 FIO Software | R4T04A |
HPE Performance Cluster Manager | |
| For additional information, please visit HPE Performance Cluster Manager QuickSpecs here | |
| Description | SKU |
| HPE Performance Cluster Manager 1 Node 3yr 24x7 Support Perpetual E-LTU | Q9V60AAE |
| Notes: – One license per node – Includes three years of support – This is an electronic license – This is a perpetual license. The Software will continue working even when the support term ends | |
| HPE Performance Cluster Manager 1 Node 3yr 24x7 Support Perpetual LTU | Q9V60A |
| Notes: – One license per node. – Includes three years of support. – This is a perpetual license. The software will continue working even when the support term ends. | |
| HPE Performance Cluster Manager FIO Software | Q9V61A |
| Notes: – This SKU does not include the license. Please order Q9V60AAE. – Order one per node | |
| HPE Performance Cluster Manager Media Kit | Q9V62A |
| Notes: One media kit per solution. | |
HPE Power Distribution UnitsPower Distribution Units (PDUs) are an integral piece to this data center solution and HPE offers several types. Basic PDUs provide reliable power with 0U or 1U installation options. Metered PDUs have added intelligence to precisely track power usage and switched PDUs provide both local and remote power management. There are additional metered PDUs that are recommended for this solution that are not part of the mainstream PDU product offering. They are as follows: | |
| Description | SKU |
| HPE Switched 3-phase 66.5kVA/60309 5-wire 100A/277 V 21-breaker Vertical NA PDU | R8P19A |
| HPE Metered 3Ph 66.5kVA/60309 100A 5-wire 480/277 V Outlets (21) SDG23/Vertical NA PDU | 879034-B21 |
| HPE Metered 3Ph 39.9kVA/60309 60A 5-wire 480/277 V Outlets (21) SDG23/Vertical NA PDU | 880459-B21 |
| HPE Metered 3Ph 57.6kVA/60309 100A 5-wire 80A/230 V Outlets (3) C13 (18) C19/Vertical NA PDU | 880460-B21 |
| HPE Metered 3Ph 34.5kVA/60309 60A 5-wire 48A/230 V Outlets (3) C13 (18) C19/Vertical NA FIO PDU | 880461-B21 |
| HPE Cray Supercomputer 60A 415 V 3 Phase 24 CX PDU | R4N30A |
| HPE Mtrd 3P 69.1kVA 125A 96A230 V FIO PDU | 880462-B21 |
| HPE Metered 3Ph 45.1kVA/60309 63A 5-wire 63A/230 V Outlets (3) C13 (18) C19/Vertical INTL FIO PDU | 880463-B21 |
| HPE Cray Supercomputer 63A 400 V 3 Phase 24 CX PDU | R4N29A |
| HPE G2 Metered/Switched 3Ph 17.3kVA/60309 4-wire 48A/208 V Out (12) C13 (12) C19/Vertical NA/JP PDU | P9S22A |
Technical Specifications
HPE Cray XD670 Chassis / Server | ||
|---|---|---|
| Dimensions | Height | 8.75 in (22.23 cm) |
| Width | 17.63 in (44.8 cm) | |
| Depth | 37.2 in (945 mm) | |
| Shipping Dimensions | Height | 12.5 in (31.75 cm) |
| Width | 23.63 in (60 cm) | |
| Depth | 39.37 in (100 cm) | |
| Weight (approximate) | Minimum | 155 lbs. (100.0 kg) |
| Maximum | 240 lbs. (108.9 kg) | |
| System Inlet Temperature Standard Operating Support | 10° to 30°C (50° to 86°F) | At sea level with an altitude derating of 1.0°C per every 305 m (1.8°F per every 1000 ft) above sea level to a maximum of 3050 m (10,000 ft), no direct sustained sunlight. The maximum rate of change is 20°C/hr (36°F/hr). The upper limit and rate of change may be limited by the type and number of options installed. |
| Relative Humidity (non-condensing) Operating | 8% to 90% | Relative humidity (Rh), 28°C maximum wet bulb temperature, non-condensing. |
| Non-operating | 5 to 95% | Relative humidity (Rh), 38.7°C (101.7°F) maximum wet bulb temperature, non-condensing. |
| Altitude Operating | 3050 m (10,000 ft). | This value may be limited by the type and number of options installed. The maximum allowable altitude change rate is 457 m/min (1500 ft/min). |
| Non-operating | 9144 m (30,000 ft). | The maximum allowable altitude change rate is 457 m/min (1500 ft/min). |
Environmental-friendly Products and Approach – End-of-life Management and Recycling
Hewlett Packard Enterprise offers end-of-life product return, trade-in, and recycling programs, in many geographic areas, for our products. Products returned to Hewlett Packard Enterprise will be recycled, recovered or disposed of in a responsible manner.
The EU WEEE Directive (2012/19/EU) requires manufacturers to provide treatment information for each product type for use by treatment facilities. This information (product disassembly instructions) is posted on the Hewlett Packard Enterprise web site
These instructions may be used by recyclers and other WEEE treatment facilities as well as Hewlett Packard Enterprise OEM customers who integrate and re-sell Hewlett Packard Enterprise equipment.
Summary of Changes
| Date | Version History | Action | Description of Change |
|---|---|---|---|
| 04-May-2026 | Added |
| |
| Changed |
| ||
| Removed |
| ||
| 02-Feb-2026 | Added | Configuration Information section was updated, SKU added. | |
| 15-Dec-2025 | Added | Standard Features section was updated. Obsolete SKUs were deleted in Configuration Information section. | |
| 08-Sep-2025 | Added | Software Portfolio for HPE Cray XD670 was added in Standard Features section. | |
| 28-Jul-2025 | Changed | Update survey link. | |
| 07-Jul-2025 | Added | This note was added: “P60537-B22 and S3P61A, which are H100 and H200 air-cooled servers respectively, contain the H100 and H200 GPUs (respectively) without needing to include a separate GPU SKU (unlike the DLC servers)”. in the Configuration information section, under Step 1, on page 16. | |
| Removed | Obsolete SKUs were removed: P67287-B21, P65657-B21, P23666-H21, P60537-B22, P67287-B21, P23666-H21 This SKU was removed: P60992-B21 in the Standard features section under Storage M.2 Controllers on page 7 and in the Configuration information section under step 2, on | ||
| 28-Apr-2025 | Changed | Overview section was updated. | |
| 03-Feb-2025 | Changed | Standard Features section was updated | |
| 02-Dec-2024 | Changed | Standard Features section was updated | |
| 18-Nov-2024 | Changed | Standard Features section was updated | |
| 04-Nov-2024 | Changed | Standard Features and Configuration Information sections were updated. | |
| 07-Oct-2024 | Changed | Overview, Standard Features Configuration Information sections were updated. Obsolete SKU was removed. | |
| 03-Sep-2024 | Changed | Standard Features and Configuration Information sections were updated Obsolete SKU was removed. | |
| 05-Aug-2024 | Changed | Overview, Standard Features Configuration Information sections were updated | |
| 17-Jun-2024 | Changed | Standard Features section was updated | |
| 03-Jun-2024 | Changed | Overview, Standard Features Configuration Information and Technical Specifications sections were updated | |
| 06-May-2024 | Changed | Overview and Standard Features sections were updated | |
| 01-Apr-2024 | Changed | Standard Features section was updated | |
| 04-Mar-2024 | Changed | Overview and Standard Features sections were updated | |
| 05-Feb-2024 | Changed | Overview, Standard Features Configuration Information sections were updated | |
| 08-Jan-2024 | Changed | Standard Features section was updated | |
| 04-Dec-2023 | Changed | Overview, Standard Features Configuration Information and Technical Specifications sections were updated | |
| 24-Oct-2023 | Changed | Standard Features section was updated | |
| 05-Sep-2023 | Changed | Standard Features and Technical Specifications sections were updated. | |
| 15-May-2023 | Changed | Overview, Standard Features and Configuration Information sections were updated | |
| 03-Apr-2023 | Changed | Overview and Standard Features sections were updated | |
| 13-Mar-2023 | New | New QuickSpecs |
© Copyright 2026 Hewlett Packard Enterprise Development LP. The information contained herein is subject to change without notice. The only warranties for Hewlett Packard Enterprise products and services are set forth in the express warranty statements accompanying such products and services. Nothing herein should be construed as constituting an additional warranty. Hewlett Packard Enterprise shall not be liable for technical or editorial errors or omissions contained herein.
Microsoft and Windows NT are US registered trademarks of Microsoft Corporation.
Intel, the Intel logo, Xeon and Xeon Inside are trademarks of Intel Corporation in the U.S. and other countries.
AMD Opteron™ is a US registered trademark of AMD Corporation.
a50004292enw, - 16896 - Worldwide - V27 - 04-May-2026