Storage for HPC and AI
HPC and AI require high-performance storage
High-performance computing (HPC) and artificial intelligence (AI) workloads are dramatically different than enterprise compute workloads. Instead of small file sizes and enterprise applications, HPC and AI workloads are usually batch jobs with large data sets on large distributed compute clusters. Parallel distributed file systems are typically required to provide high-speed, concurrent access to data. Without the ability to share data, expensive CPUs starve for data, delaying processing.
Key differences among HPC, AI and enterprise storage
| | Enterprise storage | HPC storage | AI storage |
|---|---|---|---|
Typical workload | Transactional | Job (Data Set) | Job (Data set) |
Performance requirement | IOPS | Bandwidth | Bandwidth |
Typical file size | Small | Large | Mixed |
Typical file access | Random | Sequential | Mixed |
Data movement | Minimal | Extensive | Moderate |
Read/write intensity | Mixed | Write-intensive | Read-intensive |
Meet the varied storage requirements for HPC and AI
Storage supports compute by providing the data needed for applications and users. HPE provides storage solutions that meet IO requirements, manage data, minimise TCO and streamline data workflows.
WekaIO Matrix
Leverage a high-performance storage solution optimised for flash technology and designed to provide the storage throughput needed for large data sets and next-generation workloads. WekaIO Matrix provides a high-performance, scalable parallel file system offering high IOPS and low-latency throughput ideal for HPC and AI workloads.
HPE Data Management Framework
Optimise storage resources by enabling a hierarchical, tiered storage management architecture. The HPE Data Management Framework optimises storage resource utilisation for lower TCO and streamlines data workflows with time proven automated data movement.
Scalable Storage for Lustre
The Scalable Storage for Lustre solution offers a custom, modular Lustre configuration that can be tailored to your workload specifications. Opt to have Lustre preinstalled in the factory for a turnkey custom Lustre solution.
Services to take your storage further
Storage consulting services
Modernise data infrastructure cost efficiencies with HPE Pointnext and deliver better business outcomes.
Big data consulting services
Operationalise data architectures for effective and adaptive analytics by leveraging experts from HPE Pointnext.
Let’s talk
We’re always happy to let you know more about our HPC storage products.
Resources
Analyst White Paper : HPE's storage solutions for the HPC sector: Meeting the unique requirements of an increasingly demanding and diverse applications space
Download the Hyperion research report and learn about the growing importance of HPC storage and how HPE is meeting the requirements that are unique to the HPC sector.
Video : HPE data management framework, leveraging a tiered data model for HPC
HPE data management framework, leveraging a tiered data model for HPC
Analyst White Paper : Technology spotlight: Flash storage trends and impacts
Learn about trends in chip technology and architectures, coupled with the flash-based solid-state disk (SSD) devices that use them particularly in high performance storage systems.