Configuring YARN for Control Groups

Control groups (cgroups) are a Linux kernel feature available through the LinuxContainerExecutor program that you can configure to limit and monitor the CPU resources available to YARN container processes on a node.

Configure YARN to use cgroups through properties in the yarn-site.xml file and then restart each NodeManager and ResourceManager service after you update yarn-site.xml for changes to take effect.

Important:

Control groups (cgroups) mounted by YARN are deprecated due to a security issue with the container-executor.cfg option. By default, feature.mount-cgroup.enabled=0. To enable this feature, set feature.mount-cgroup.enabled=1.
The HPE Ezmeral Data Fabric does not support YARN cgroups on RHEL 7. In RHEL 7, libcgroup is deprecated, and systemd maintains cgroups. Moving default hierarchies that systemd manages might generate unexpected results.
RHEL 8 officially supports cgroups. Starting from MEP 7.0.x with RHEL 8, you no longer have to manually unmount and mount /sys/fs/cgroup/cpu,cpuacct to make YARN cgroups work; you can use automount.

The following sections describe how to configure mount paths for cgroups when automount is enabled and disabled, how to use an alternative method for configuring a mount path, and the yarn-site.xml properties related to YARN cgroup configuration.

Enabling Automount

Before you enable automount, set feature.mount-cgroup.enabled=1 in the container-executor.cfg file.

To enable automount, set yarn.nodemanager.linux-container-executor.cgroups.mount to true, and define the mount path in the yarn-site.xml file, as shown in the following example:

<property>
  <name>yarn.nodemanager.linux-container-executor.cgroups.mount</name>
  <value>true</value>
</property>
<property>
  <name>yarn.nodemanager.linux-container-executor.cgroups.mount-path</name>
  <value>/mycgroup</value>
</property>

Create the mount path, as shown in the following example:

mkdir -p /mycgroup/cpu
chown -R mapr:mapr /mycgroup/cpu

Based on the settings, NodeManager will mount cgroups under /mycgroup/cpu automatically.

Note: If NodeManager restarts (without the node shutting down or restarting), you must manually unmount /mycgroup/cpu or set the yarn.nodemanager.linux-container-executor.cgroups.mount property to false before NodeManager restarts. However, if the NodeManager node restarts, you do not have to perform the manual unmount; the /mycgroup/cpu path is unmounted automatically during restart.

Disabling Automount

To disable automount, set yarn.nodemanager.linux-container-executor.cgroups.mount to false in yarn-site.xml.

When automount is disabled, the default mount path is /sys/fs/cgroup/. When each node running NodeManager restarts, you must manually create the hadoop-yarn directory under /sys/fs/cgroup/cpu,cpuacct, as shown in the following example:

mkdir -p /sys/fs/cgroup/cpu,cpuacct/hadoop-yarn
chown -R mapr:mapr /sys/fs/cgroup/cpu,cpuacct/hadoop-yarn

Alternative Method to Configure a Mount Path for Control Groups (cgroups)

For pre-MEP 7.0.x and RHEL 8 environments, you can unmount the default mount path and configure a new mount path.

First, set yarn.nodemanager.linux-container-executor.cgroups.mount to false in yarn-site.xml.

To unmount the default path and create a new mount path, run the following commands:

umount /sys/fs/cgroup/cpu,cpuacct
cgroup -o 'cpu' none '/opt/mapr/cgroup'
mkdir /opt/mapr/cgroup/hadoop-yarn
chown -R mapr:mapr /opt/mapr/cgroup/hadoop-yarn 
chmod 750 /opt/mapr/cgroup/hadoop-yarn

Properties in yarn-site.xml Related to YARN Control Groups

The following table lists and describes the properties related to YARN cgroups and cpu settings in yarn-site.xml:

Properties	Configuration Description
yarn.nodemanager.container-executor.class	Set this property to `org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor`.
yarn.nodemanager.linux-container-executor.resources-handler.class	Set this property to `org.apache.hadoop.yarn.server.nodemanager.util.CgroupsLCEResourcesHandler`.
yarn.nodemanager.linux-container-executor.cgroups.hierarchy	Set this property to the location of the Cgroups hierarchy. If `yarn.nodemanager.linux-container-executor.cgroups.mount` is `false` then this cgroup's hierarchy must already exist in this location. By default, this is set to `/hadoop-yarn`.
yarn.nodemanager.linux-container-executor.cgroups.mount	If the cgroup hierarchy is already configured, verify that this value is set to the default, `false`. Otherwise, set this value to `true`.
yarn.nodemanager.linux-container-executor.cgroups.mount-path	Set this property to the path where the `LinuxContainerExecutor` should attempt to mount cgroups if the hierarchy is not found. The container-executor binary will try to mount to `<mountPath>/cpu`, which must exists before the NodeManager service is started on this node.
yarn.nodemanager.linux-container-executor.group	Verify that this setting matches the `yarn.nodemanager.linux-container-executor.group` setting in `container-executor.cfg (/opt/mapr/hadoop/hadoop-2.x.x/etc/hadoop/container-executor.cfg)`.
yarn.nodemanager.resource.percentage-physical-cpu-limit	Limits the CPU resources available to container processes. Set`yarn.nodemanager.resource.percentage-physical-cpu-limit` to the percentage that you want to limit the CPU usage of all YARN containers to. For example, if you set this value to 75, YARN containers on this node cannot use more than 75% of the CPU available on the node.
yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage	Specifies whether the CPU allocation should have a hard or soft limit. If containers should not use more CPU than what was originally allocated, set `yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage` to `true`. Set `yarn.nodemanager.linux-container-executor.cgroups.strict-resource-usage` to false if containers can use additional amounts of unused, available CPU.