THE CONFLUENCE OF SIMULATION AND AI CHANGES EVERYTHING
This confluence put new demands on HPC storage. And it is happening right now. A recent study of the independent analyst firm Intersect360 found out that the majority (61%) of the HPC users today already are running machine learning programs . Hyperion Research forecasts that- current course and speed - HPC storage spending in public sector organizations and enterprises will grow 40% faster than spending for HPC compute for the next three years.
+ show more
ALCF has its eye set on the Exascale Era by putting in place the infrastructure required for converged workloads in HPC, AI, analytics, modeling and simulation,” said Peter Ungaro, senior vice president and general manager, HPC and AI, at HPE. “Cray ClusterStor E1000 delivers the scalability and performance ALCF requires to unlock insights and discovery from these data intensive workloads.”
New Era of HPC Storage: The Best of Both Worlds
| | Traditional HPC cluster | New Era Converged | Traditional AI stack (POD) |
|---|---|---|---|
| Primary workload | Modeling & simulation | Both | Machine Learning/Deep Learning |
| Compute node type
| CPU nodes like HPE Apollo 2000 | Both | GPU nodes like HPE Apollo 6500 |
| Number of compute nodes | Hundreds to thousands | Hundreds to thousands | A few |
| Typical interconnect | InfiniBand | Both (plus HPE Slingshot) | Gigabit Ethernet |
| Primary I/O pattern | Write-intensive | Both | Read-intensive |
| Storage capacity measured in | Petabytes | Petabytes | Terabytes |
| Storage scalability in single namespace | Up to exabytes | Up to exabytes | Up to low double digits petabytes |
| Typical storage | HDD-based parallel file system storage | Cray ClusterStor E1000 | All Flash Enterprise NAS storage |
| Storage well suited for | Serving large files in sequential order at speeds of up to terabytes per second | Serving files of all sizes in both sequential and random order at speeds of up to terabytes per second | Serving files of all sizes in both sequential and random order at speeds of up to double digit gigabytes per second |
| Price per terabyte | $ | $$$ | $$$$$$$$$$ |
HPC storage solutions for the new era for your hot, warm, and cold data
HPE provides HPC storage solutions that span the whole storage hierarchy in order to accelerate time-to-insights while managing and protecting the valuable data in your parallel file systems in a cost-effective way.
+ show more
Leverage a new parallel HPC storage system that was purpose-engineered for the era of converged simulation and AI workloads. Get unprecedented throughput delivered to your CPU/GPU compute nodes with unprecedented efficiency. Use it with any supercomputer or HPC cluster of any vendor that supports InfiniBand (EDR/HDR), Gigabit Ethernet (100/200) or Cray Slingshot.
HPE Data management framework (DMF)
Optimize HPC storage resources by enabling a hierarchical, tiered storage management architecture. The HPE Data Management Framework optimizes storage resource utilization for lower TCO and streamlines data workflows with time proven automated data movement across the full storage hierarchy. Now with native support for IBM Spectrum Scale in addition to LustreR and HPE XFS.
Cray ClusterStor E1000 Technical White Paper
Find out how the Cray ClusterStor E1000 storage system can:
- Drive up to 80 gigabytes per second file system performance per two rack units
- Drive up to 3.3 gigabytes per second file system performance per NVMe SSD
- Pack up to 10 petabytes of usable storage capacity in just one rack
- Connect to any supercomputer or HPC cluster of any vendor as long as the compute nodes support modern 100 or 200 gigabit per second high speed networks