DVS Configuration and Use

Lists procedures for configuring, reconfiguring, and using DVS.

What Needs DVS?

DVS (cray_dvs) is required for Netroot¹ to function and for distributing the programming environment (PE) to compute and login nodes using the cray_image_binding service. However, sites are not required to make any changes to the cray_dvs service to achieve this functionality.

If DVS is used to provide access to DataWarp, some DVS configuration will be required.

What does DVS Need?

For DVS to function properly, the following services may need to be configured. The first two are required for a functional system regardless of DVS, but they are included because DVS-specific information or a consideration of how DVS will be used is necessary during their configuration.

cray_scalable_services: Defines which servers are used in the scaling of the system. When configuring the cray_scalable_services service, ensure that any DVS servers/nodes that will be used to project external file systems are NOT added to the list of tier2 servers.
cray_net: Configures key network attributes. When configuring the cray_net service, set network values to define a network interface for any DVS servers/nodes that will be used to project external file systems. Needed only for communications using anything other than the Cray Aries high-speed network (HSN).
cray_lnet: DVS uses Lustre™ Networking (LNet) to communicate on Aries networks, so even if Lustre is not used on this system, the LNet service must be configured. No DVS-specific settings are necessary; it is sufficient to ensure that the cray_lnet service is enabled.
cray_multipath: Detects and coalesces multiple paths to devices. If the node acting as the DVS server is natively mounting the file system, it has access to the storage directly, and if more than one path to the storage is desired, then the cray_multipath service needs to be enabled. This is not typical; DVS is usually a client of the file system.

Node Groups

To configure DVS, it is necessary to define at least one DVS node group that contains one or more DVS servers. Sites that are doing a fresh install will have an opportunity to define DVS node groups during the installation process. Information about defining and migrating node groups data is found in the XC™ Series Software Installation and Configuration Guide (S-2559).

See About Node Groups for more information about node groups in general.

Procedures to Configure DVS using the Configurator

DVS is one of many services that store service configuration content in CLE configuration sets (config sets) on Cray systems. DVS can be configured when config sets are created during a fresh install or major upgrade, or it can be configured/reconfigured later by updating existing config sets during normal system operation (bearing in mind that some of the DVS module parameters are best set during initial system configuration). These procedures guide site administrators and staff in entering appropriate values for DVS configuration settings using the configurator. Whether sites enter values in an interactive configurator session or enter values in a configuration worksheet for bulk import, the configurator takes the supplied values and ensures that they become part of the config set being created or updated.

Use one of the following procedures to configure or reconfigure DVS using the configurator. In all procedures, steps correspond to order of settings as encountered in an interactive session or configuration worksheet.

Configure DVS using the Configurator	General-purpose procedure, emphasis on decision support. For both initial configuration and reconfiguration Detailed description of settings No examples
Configure DVS using Worksheets	Procedure for editing the DVS worksheet, emphasis on examples. For initial configuration or major update Detailed steps and examples
Reconfigure DVS Interactively	Procedure for using the configurator interactively, emphasis on examples. For minor updates Detailed steps and examples

Caveat: The above procedures do not cover how to use cfgset and the configurator (which is invoked by cfgset). See the cfgset man page and the XC™ Series Configurator User Guide (S-2560).

Procedure to Configure DVS using Modprobe.d, Proc Files, and Simple Sync

At this time, there are some DVS configuration parameters that cannot be set using the configurator. For such cases, configuration files can be created within the config set and distributed throughout the system using the Simple Sync mechanism, part of the new Cray management system. It is important to make these changes within the config set directory structure, otherwise changes may not persist. For instructions on how to do that, use this procedure.

Configure DVS using Modprobe or Proc Files

For more information about Simple Sync, see About Simple Sync. For general information about the way Cray XC systems are configured, see Cray XC System Configuration.

Procedure to Apply Configuration Changes

After using one of the configuration procedures or making any configuration changes, validate the revised config set and run Ansible plays to propagate and apply the configuration changes.

Validate the Config Set and Run Ansible Plays

Procedures to Manage and Optimize DVS Use

Use the following procedures, as needed, during operation or for further configuration.

Quiesce a DVS-projected File System
DVS Client-side Write-back Caching can Yield Performance Gains
Force a Cache Revalidation on a DVS Mount Point
Disable DVS Fairness of Service
Reconfigure DVS for an External NFS Server
Improve Performance and Scalability of GPFS (Spectrum Scale) Mounts
For procedures to configure a DVS mount of GPFS (Spectrum Scale), see XC Series GPFS Software Installation Guide (S-2569), which is available at http://pubs.cray.com

¹ Netroot is a mechanism to enable nodes booted with a minimal, local in-memory file system to execute within the context of a larger, full-featured root file system. For more information, see XC™ Series Software Installation and Configuration Guide (S-2559).