InfiniBand and Omni-Path

Lustre Network setup for InfiniBand or Omni-Path

The network setup will include choosing a method of assigning IP addresses, configuring the IP addresses of the OSS and MDS servers, the Netmask address, the MTU, the DNS servers and the NTP servers. At any point during the setup, clicking the Cancel button will stop the configuration. However, upon next startup the wizard will pick up on the last completed page.
  1. Locate the Assign IP Addresses group for IB or Omni-Path.
  2. Enter the IP Range Start value in the address field for the cluster.
  3. Enter the IP Range End address in the next field.
    If the user enters any field values incorrectly, upon moving to the next field, the field name text with the error will change to red, indicating a problem. Return to that field and correct the error before continuing.
    As an example, the user could set the first set of addresses specifically for the primary and secondary MDS servers, then click the Plus button and enter the range of addresses for the OSS servers. The user could choose to use one set of entries and include all addresses in one range group. If choosing separate ranges, click the Plus symbol button and enter the next set of IP address ranges.
  4. Enter the Netmask address.
    To choose from the list of standard netmask addresses, click the down arrow.
  5. Enter the MTU (Maximum Transmit Unit) value in bytes.
    The default value is 65520 bytes and is the optimum setting for IP over InfiniBand (IPoIB). This value allows for jumbo frames (JF) traffic. For Omni-Path Lustre networks, accept the default setting. It is the administrator's responsibility to ensure that all machines on the network support and work with the same MTU value.
  6. Enter the DNS Server(s) IP addresses, if not already discovered.
  7. Enter the NTP Server(s) IP addresses.
    This is the address of the time server.
  8. Click the Enable Subnet Manager button to enable this Subnet Manager.
    To disable, deselect the option.
  9. Adjust the slider to establish the priority for the Subnet Manager functions.
    The values are 0-15, with 15 being the highest priority.
  10. Adjust the slider to select a value for the Maximum Operational Virtual Lanes.
    The values are 1-255.
  11. Complete the required fields in the Advanced LNET configuration section:
    1. Enter network index number.
      The default value is 0 and may be up to a 3 digit value. For example: Network Index of 100 would connect the cluster to Lustre clients using LNET o2ib100.
  12. Work with a service representative to create and upload the Optional Configuration files: lnet.conf, ip2nets, and routes.
    Do not attempt to upload arbitrary files in the Advanced LNET Configuration section, as this may cause incorrect configuration and unexpected errors. This feature should be used only in cooperation with a service representative.
  13. Click the Next button.
    The configuration wizard will advance to the Lustre Users Settings.
  14. Complete the Lustre user authentication settings, as required. For more information, see the related information for the Configure tab.
Note: For InfiniBand Lustre networks, the MTU (minimum transmission unit) will be set to the default of 65520. For Omni-Path Lustre networks, accept the default setting.

Once the steps above are complete, the next step is to complete the configuration wizard.