HPC Storage Software

Do you need to improve data management in your HPC and AI Linux® storage environment? The HPE Data Management Framework (DMF) provides more efficient utilization of storage infrastructure, reduced time to insight, and allows for petabyte scale backup and Point-in-Time restoration of data. A new architecture allows for extensible metadata, which allows tagging data with attributes which can be queried to allow simplified creation of data sets. Along with data set labeling, job scheduler integration and the built-in policy engine, data intensive workflows can be automated and streamlined through automatic data set creation, staging of data, and data movement for processing. This automated data management functionality allows efficient utilization of storage infrastructure by removing stale data from defined data tiers and provides a virtual storage space that appears to be unlimited in size. Needed data is automatically retrieved as needed, making storage look "bigger on the inside."

What's new

  • Updated architecture with distributed database for enhanced scalability, performance and security.
  • Enhanced data management with extensible metadata, data set labeling, and job scheduler integration.
  • Enhanced Lustre file system integration to simplify Lustre data management.

Features

Streamline Data Workflows

The HPE Data Management Framework provides extensible metadata which can be queried, simplifying data set creation and manipulation. For example, an extended metadata attribute can be used to identify the geographic location for data samples, simplifying gathering of data from that location.

Data set labeling simplifies data manipulation of a set of data files by allowing easy reference to it, and helping eliminate the need to gather that set of data.

Job scheduler integration allows DMF tasks to be automatically activated when desired.

More Efficient Storage Infrastructure Utilization

The HPE Data Management Framework can remove stale or old data to archive, freeing up resources for active data. Data costs are reduced by moving data to one of the most cost effective storage tier based on user defined policies.

Provide a virtual storage space that appears limitless through an automated and seamless transfer and retrieval of data to archive, allowing active storage resources to appear as if it generally has the required data.

Petabyte-scale Backup and Recovery

The HPE Data Management Framework (DMF) enables backup of petabytes of data by allowing "rolling" backups throughout the day (and night). At a petabyte-scale level, backups take too long to fit into typical backup windows.

DMF maintains versions of data and allows a Point-in-Time recovery to allow granular recovery of data. For example, this allows easy reproduction of previous job runs.

File systems can be created, modified, deleted, and restored, allowing files to be deleted to provide space in an existing file system. Since all file data is kept in the metadata repository, the files can be restored into the same or different file system when needed.

Enhanced Lustre Integration

The HPE Data Management Framework provides advanced data management capabilities to the Lustre file system with the ability to natively manage the Lustre file system using the Lustre persistent change log and "copy tool" via available APIs.

DMF simplifies Lustre file management as the Lustre Robinhood policy engine and separate tracking databases are no longer required.

  • Linux® is a registered trademark of Linus Torvalds.