Key features

• – High-Performance and Scalability:

  • HPE Cray SC USS leverages the full capabilities of HPE Cray hardware, software and other third party and open-source software to deliver superior performance and scalability, supporting large-scale parallel processing across thousands of nodes.
• – Optimized resource management:

  • Advanced resource management tools ensure efficient allocation and utilization of computational resources, enhancing throughput and minimizing idle times.
• – Advanced I/O and networking:

  • USS supports high-speed networking including HPE Cray Slingshot and optimizes I/O operations for handling large data intensive applications efficiently.
• – Resilience and Reliability:

  • Features for fault tolerance and automatic recovery ensure high availability and consistent operation, minimizing downtime and maintaining system stability.
• – Energy efficiency:

  • Dynamic power management features optimize energy consumption, balancing performance with cost savings and environmental sustainability.
• – Support for diverse workloads:

  • HPE Cray SC USS provides tools and functionalities to manage a wide array of workloads from scientific computing to AI and machine learning, providing flexibility for various applications.

Specific Functionalities:

• – Job Scheduling and Management:

  • Integration with job schedulers like Slurm and PBS Pro allows for sophisticated scheduling, resource allocation, and job prioritization, ensuring efficient job management.
• – Performance Monitoring and Tuning:

  • Tools for performance monitoring and analysis help identify bottlenecks and optimize application and system performance, including specialized NUMA tools and performance extensions.
• – Data Virtualization and File Systems:

  • Support for high-performance file systems like Lustre and Data Virtualization Service (DVS) simplifies data management and enhances access across the HPC environment.
• – Container Support:

  • HPE Cray SC USS supports container technologies, enabling easy deployment, scalability, and management of containerized applications, which is crucial for HPC and AI workflows.
• – Power Management Tools:

  • Features like turbo boost limiting and other power management tools such as pm_counters help control energy usage, contributing to operational efficiency and sustainability.

The HPE Cray SC User Services Software is a cornerstone of HPE Cray supercomputing solutions, designed to meet the rigorous demands of evolving workloads at the convergence of high-performance computing and artificial intelligence. By combining advanced resource management, superior performance, energy efficiency and robust security, HPE Cray SC USS empowers organizations to achieve significant scientific and engineering breakthroughs. Its ability to handle diverse workloads and its integrated, high-performance features make it an essential platform for cutting-edge HPC and AI applications.

• – Custom Lustre filesystems and Lustre network (LNet) to enable Lustre on compute nodes
• The Lustre file system is renowned for its high throughout and low latency, making it suitable for the demanding I/O operations typical in HPC workloads. It can handle petabytes of data and billions of files, supporting the scalability needs of large-scale scientific computations. It also allows for multiple compute nodes to read from and write to the storage systems simultaneously, optimizing the data flow and reducing bottlenecks.
• The Lustre file system and Lustre Network (LNet) are integral components of HPE Cray Supercomputing User Services Software, providing the performance, scalability, and reliability necessary for modern high-performance computing environments. By leveraging these technologies, HPE Cray systems can effectively manage vast amounts of data, deliver high-speed access and support the demanding requirements of scientific research and complex computational tasks. These capabilities ensure that HPE Cray supercomputers remain at the forefront of HPC innovation, enabling researchers and scientists to achieve groundbreaking discoveries and advancements.
• Some of the benefits of Lustre and LNet are:

  • The combination of Lustre parallel I/O capabilities and LNet's efficient data transport ensures that HPC applications can access and process data at high speeds, reducing wait times and increasing overall system throughput.
  • The robust design of Lustre and LNet, including data redundancy and fault-tolerant networking, ensures that HPC environments can maintain high availability and data integrity, even under heavy load or in the event of hardware failures.
  • Lustre and LNet together provide a scalable solution that can grow with the increasing demands of HPC and AI workload, ensuring that storage and network resources do not become bottlenecks.
  • With POSIX compliance and seamless integration with HPE User Services Software, Lustre and LNet provide a familiar and straightforward interface for users and administrators simplifying the deployment and management of HPC resources.

User application enhancements

The HPE Cray Supercomputing User Services Software is designed to optimize the performance of high-performance computing applications, providing a robust environment tailored for the demanding needs of scientific research, data analysis, and complex situations. To achieve this, HPE Cray SC USS includes a suite of application performance extensions that enhance the efficiency, scalability, and overall performance of HPC workloads. These extensions are essential for leveraging the full potential of HPE Cray supercomputers and ensuring that complex scientific research and applications run at maximum efficiency. The major user application enhancements included with HPE Cray SC USS are:

• – Custom NUMA (Non-Uniform Memory Access) management tools:
• NUMA tools in the HPE Cray SC USS are designed to manage and optimize memory access in systems where memory access time varies depending on the memory location relative to the processor. These tools help in minimizing memory access latency and maximizing data throughput, which are critical for the performance of HPC applications. Whether through advanced memory management policies, integration with job scheduler, or detailed performance monitoring, NUMA tools help ensure that HPC applications run smoothly and efficiently, delivering the high-performance required for scientific research, data analysis and complex simulations.
• Some of the features and benefits of integration with NUMA tools are:

  • NUMA aware scheduling ensures that jobs are placed on processors and memory nodes in a way that minimizes cross-NUMA node memory access, thereby reducing latency and improving overall performance. These tools manage CPU and memory affinity, ensuring that hardware resources are used more effectively, reducing idle times and increasing throughputs.
  • These tools minimize the latency associated with remote memory access by ensuring that memory is allocated and addressed locally wherever possible.
  • NUMA tools balance the loads across all available NUMA nodes, preventing bottlenecks and improving overall system performance.
  • They support the scalability of applications by efficiently handling memory and processor resources across numerous nodes.

• – XPMEM (Cross-Partition Memory):
• HPE Cray Supercomputing User Services Software includes XPMEM which enhances inter-process communication capabilities in HPC environments. By allowing different processes running on the node to directly access virtual memory regions, XPMEM facilitates low-latency data exchange and collaboration within an application. Some of the benefits of XPMEM are:

  • XPMEM integration streamlines communication and coordination between different processes running on the compute nodes. By providing a high-performance, kernel-level mechanism for sharing memory regions, XPMEM optimizes data transfers and reduces overhead associated with inter-process communication, improving overall system efficiency and utilization.
  • XPMEM's efficient shared memory communication capabilities enable seamless integration with parallel programming frameworks like Cray MPI and SHMEM (included in HPE Cray Programming Environment), enhancing performance and scalability of parallel applications.

• – Low Noise Mode:
• Low Noise Mode (LNM) with HPE Cray Supercomputing User Services Software is a vital feature for optimizing the performance and reliability of high-performance computing applications. By making it easy to use Linux features that reduce system noise and jitter, Low Noise Mode can create a more predictable execution environment, enhancing the performance of sensitive applications and improving overall system efficiency. This makes it an essential tool for HPC users aiming to achieve the highest levels of performance and consistency in their computational tasks. Benefits include:

  • Low Noise Mode reduces the frequency and impact of background tasks, such as system daemons and interrupts. This minimizes the interference with running applications, helping to maintain consistent performance and potentially leading to faster completion times and more efficient processing. Applications that are sensitive to system noise, such as those requiring high synchronization, benefit significantly from the predictable environment provided by Low Noise Mode.
  • Low Noise Mode enables optimizations in task scheduling to ensure that compute-bound processes are less likely preempted by lower-priority system tasks. Reducing operating system induced variability leads to more predictable application performance.
  • By reducing the system noise, Low Noise Mode helps in reducing jitter, which is the variability in execution time that can occur due to the operating system’s activities. This is crucial for applications that require precise timing and consistency.
  • This mode also helps in achieving better resource utilization by ensuring that compute resources are primarily dedicated to user applications rather than background system processes.

• – Turbo boost limiting:

• The turbo boost limiting feature included with HPE Cray SC USS is designed to control or limit the CPU turbo boost technology on compute nodes. Turbo boost is a feature present in many modern CPUs (e.g., Intel's Turbo Boost and AMD's Precision Boost) that dynamically increases the clock speed of cores to provide bursts of performance when needed. The feature includes dynamic frequency scaling of CPU cores to prevent them from exceeding certain frequency thresholds associated with turbo boost operation. Benefits include:

  • By limiting turbo boost frequencies, HPE Cray SC USS helps optimize power usage across the HPC cluster, reducing energy consumption and operational costs associated with running compute intensive workloads.
  • With turbo boost limiting, administrators can ensure more predictable performance behavior across the cluster, enabling better resource allocation, scheduling, and workload management for HPC applications.
  • By optimizing power usage and reducing energy consumption, turbo boost limiting contributes to environmental sustainability efforts by minimizing ecological footprints of HPC operations.
  • Administrators can optimize power usage and prevent overheating by controlling CPU frequencies, ensuring reliable operation of HPC clusters under varying workloads and environmental conditions.

• – Support for third party performance tools not included in standard Linux distributions:
• The HPE Cray Supercomputing User Services Software includes several performance enhancement tools designed to optimize the performance of high-performance computing applications. Among these tools are:

  • msr-safe and other utilities that provide fine-grained control over hardware performance, monitoring and optimization. These tools provide users access to performance data, who in turn can identify bottlenecks and optimization opportunities, thereby ensuring efficient utilization of system resources.
  • GPU memory copy functionality included with HPE Cray SC USS enables seamless and efficient data transfer between CPU and GPU's dedicated memory. It leverages optimized data transfer mechanisms and advanced memory techniques to minimize latency and maximize throughput, thereby improving the overall performance of GPU-accelerated applications.
  • Spindle: Integration with Spindle is provided with HPE Cray SC USS. Spindle is a high-performance, scalable tool designed to optimize the loading of dynamic libraries in large-scale, parallel computing environments. Key features of Spindle include its ability to significantly reduce the overhead associated with launching applications that use shared libraries by aggregating and distributing these libraries efficiently across multiple compute nodes. This reduces redundant data transfers and leverages collective communication mechanisms to enhance performance. Benefits of Spindle include faster application start-up times, improved overall system throughput, and reduced load on network file systems, making it particularly advantageous in high-performance computing environments.

Power Management features and tools

The power management features provided with HPE Cray SC User Services Software offer comprehensive capabilities for optimizing energy efficiency, reducing operational costs and promoting environmental sustainability in HPC environments. By leveraging dynamic voltage, frequency scaling and advanced power capping, HPE Cray SC USS along with HPE cluster management tools enable organizations to achieve optimal performance while minimizing energy consumption and maximizing the return on investment in high-performance infrastructure. With its scalability, flexibility and focus on efficiency, HPE Cray SC USS is a key enabler of energy efficient computing and sustainable HPC operations. The power management features included in HPE Cray SC USS are:

• – Node Power Management:

  • HPE Cray SC USS includes pm_counter and blade power management control daemon used to monitor and track power related metrics. It exposes applications’ power data counters provided by the node controller to the compute node. It helps to monitor power consumption, thereby identifying opportunities for optimizing energy usage and reducing operational costs.
  • AMD and NVidia GPU energy management plugins are included in HPE Cray SC USS offers significant benefits in terms of energy efficiency, performance optimization, cost savings and environmental sustainability for HPC deployment.

• – Integration with Power Monitoring and Reporting tools:

  • HPE Cray SC USS integrates with HPE system management infrastructure monitoring platforms allowing for centralized management and control of power related settings across the entire HPC infrastructure. This facilitates comprehensive energy management and optimization strategies.

• .– Integration with workload management tools:

  • HPE Cray SC USS offers integrations with popular workload management tools SLURM and PBS Pro. Both SLURM and PBS Professional offer power management features that help optimize energy usage, reduce operational costs, and promote environmental sustainability in HPC clusters. By integrating power aware scheduling algorithms, enforcing power caps, and providing power monitoring and reporting capabilities, SLURM and PBS Pro enable organizations to achieve efficient resource utilization while maintaining system stability and reliability.
  • HPE Cray SC USS also include plugins that provide per-job power statistics to users.

User Mode Container Support

User mode container support in the HPE Cray SC User Services Software provides a powerful and flexible solution for running containerized applications in high-performance computing environments. By enabling rootless container execution, enhancing security and ensuring performance optimization, HPE's container support meets the demands of ever evolving HPC and AI workloads. Tools like Singularity and Podman, along with comprehensive container management capabilities, empower users to leverage the benefits of containers without disrupting their established HPC workflows. Key container capabilities provided with HPE Cray SC USS are:

• – Compatibility with Singularity

  • Designed for running applications in unprivileged user mode.
  • Integrates with job schedulers like SLURM and PBS Professional, enabling seamless container execution within scheduled jobs.
  • optimized for performance, ensuring that containerized applications run efficiently on HPC infrastructure.
• – Compatibility with Podman

  • Enhanced security through daemon-less architecture. Users can also run containers without requiring root privileges, reducing the risk of running containerized applications with elevated permissions.
  • Adherence to OCI standards for container images and runtimes, ensuring compatibility and interoperability with other OCI compliant tools and platforms.
  • Integrates with job schedulers like SLURM and PBS Professional
  • Lightweight design ensuring minimal overhead.

Workload Management and Application Launch:

HPE Cray SC USS offers comprehensive tools and integrations for workload management and application launch, tailored to the demanding needs of high-performance computing environments. By integrating with advanced scheduling systems such as Slurm and PBS Professional with specialized tools like PALS, ATOM, etc., HPE Cray SC USS ensures efficient resource utilization, optimized application performance, and streamlined job management. These capabilities empower researchers and engineers to tackle complex computational challenges with efficiency and reliability. Below is an overview of the key workload management tool integrations and application launch tools included in HPE Cray SC USS:

• – Integration with Slurm:
• Slurm is an open-source job scheduler widely used in HPC environments. It is designed to allocate resources, schedule jobs, and manage workloads efficiently across large-scale compute clusters. It includes features such as:

  • Dynamic allocation of compute resources based on job requirements.
  • Support for various scheduling policies to optimize resource utilization.
  • Features for job checkpointing and recovery, enhancing resilience.
  • Various command-line tools and graphical interfaces for job submission and monitoring.
• HPE Cray SC USS provides plugins for integration of Slurm with the Slingshot interconnect. These plugins significantly enhance the Slurm job scheduler by integrating advanced networking features of the Slingshot interconnect. These plugins improve network performance, optimize resource utilization, and ensure scalable and efficient management of large-scale HPC environments. With features like topology aware scheduling, virtual network interface (VNIs) for workloads, QoS management, and real-time performance monitoring, these plugins not only boost system performance but also enhance the overall user experience, making them an indispensable tool for HPC administrators and users alike.

• – Integration with Altair PBS Professional:
• PBS professional is a commercial workload management system designed for complex distributed computing environments. It offers advanced scheduling capabilities and high scalability. Key features include:

  • Support for advanced scheduling policies, including priority-based scheduling and fair-share scheduling.
  • Efficient allocation of resources manages job queues, and monitors job execution.
  • Integration mechanism with other HPC and AI tools for seamless workflow management.

• – Parallel Application Launch Service (PALS):
• The parallel application launch service provided with HPE Cray SC USS is a sophisticated tool designed to streamline and optimize the launch and execution of parallel applications in high-performance computing environments. Its optimized resource allocation, scalability, high throughput, and integration with job schedulers like PBS Pro ensures efficient and reliable application launches, thereby contributing significantly to the overall performance and productivity of HPC systems. Key benefits include:

  • PALS enhances the efficiency of resource utilization by optimizing the launch process and ensuring that resources are allocated precisely based on application needs.
  • By providing a streamlined launch mechanism and reducing overhead, PALS improves the overall performance of parallel applications, enabling faster time-to-results.
  • PALS's ability to manage and launch applications across thousands of nodes ensures that it can scale to meet the demands of the largest HPC environments.
  • The user-friendly interfaces and integration with job schedulers like PBS Pro and standards like PMIx simplify the process of managing parallel applications, allowing users to focus more on scientific computations and less on operational details.
  • PALS helps integrate the Slingshot interconnect with PBS Professional and helps in configuring VNIs, traffic classes, and NIC resources for applications.

• – Application Task Orchestration and Management Service (ATOM):
• The Application Task Orchestration and Management Service (ATOM) provided with HPE Cray SC USS is a powerful and versatile solution designed to cleanup and check the health of compute nodes after each job run, thereby making sure that the nodes are healthy and ready to run further jobs. Major features and benefits include:

  • With capabilities such as configuration, compute node cleanup and node health testing, ATOM makes sure that the compute nodes are ready to run the next jobs.
  • ATOM is highly configurable and allows users to add their own tasks (command or script that ATOM runs) and configure the built-in tasks, thereby making sure that tasks are executed in an optimal sequence, reducing wait times and maximizing throughput.
  • It also provides an accounting feature which can create energy usage reports for jobs. It captures detailed metrics on resource usage, including CPU, memory, I/O and network utilization.

• – Integration with PMIx:
• PMIx is a standard designed to enhance interoperability and scalability for job scheduler and runtime environments in high-performance computing. HPE Cray SC USS provides integration with PMIx and thereby helps in facilitating efficient communication and management of parallel applications, enabling seamless integration with job schedulers to optimize resource utilization and performance. Integration with popular job schedulers like Slurm and PBS Professional provides a robust framework for job management and execution. Some of the benefits of this integration are:

  • Streamlined job launch process by providing standardized mechanism for initializing and managing job execution.
  • Efficient resource allocation based on real-time analysis of system load and job requirements, ensuring optimal utilization of computational resources.
  • Scalable job management allowing job schedulers to handle a large number of simultaneous job submissions and executions without performance degradation.
  • Real-time communication between job schedulers and runtime environments enabling rapid dissemination of job-related information and improving overall system responsiveness.

Reliability, Availability, Serviceability (RAS):

HPE Cray SC User Services Software incorporates a robust set of features aimed at maximizing reliability, availability, and serviceability in high-performance computing environments. Through advanced error detection, failover mechanisms, dynamic resource management, and proactive maintenance tools, HPE Cray SC USS along with other HPE HPC software solutions ensures that HPC systems operate smoothly, handle failures gracefully, and are easy to manage. These RAS features are critical for maintaining the high levels of performance and uptime required in demanding HPC applications. Some of the RAS features included in HPC Cray SC USS are:

• – Reliability features:

  • Continuous monitoring of hardware health (e.g., PCIe link bandwidth monitoring and node health monitoring) helps in detecting failures and allows for preemptive action
  • Advanced files systems (Lustre) included with HPE Cray SC USS include integrity checks to ensure data consistency and reliability.
• – Availability features:

  • HPE Cray SC USS supports high availability and automatic failover mechanisms in event of a failure.
  • Leverages the scalable capabilities of Linux distributions to enhance system performance.
• – Serviceability features:

  • Comprehensive logging and diagnostic capabilities
  • Rich set of command line utilities to provide fine grained control over system operations
  • Comprehensive documentation and vendor support