In addition to the YARN container resource allocation parameters, the MapReduce ApplicationMaster also considers the following container requirements when it sends requests to the ResourceManager for containers to run MapReduce applications:
| Parameter | Default | Description |
|---|---|---|
|
|
1024 | Defines the container size for map tasks in MB. |
mapreduce.reduce.memory.mb
|
3072 |
Defines the container size for reduce tasks in MB. |
mapreduce.reduce.java.opts
|
-Xmx2560m --add-opens java.base/java.lang=ALL-UNNAMED -XX:+UseParallelGC | Java options for reduce tasks. |
mapreduce.map.java.opts
|
-Xmx900m --add-opens java.base/java.lang=ALL-UNNAMED -XX:+UseParallelGC | Java options for map tasks. |
|
|
0.5 |
Defines the number of disks a map task requires. For example, a node with 4 disks can run 8 map tasks at a time. Note: If I/O intensive tasks do not run on the node, you may want to change this value. |
|
|
1.33 |
Defines the number of disks that a reduce task requires. For example, a node with 4 disks can run 3 reduce tasks at a time. Note: If I/O intensive tasks do not run on the node, you might want to change this value. |
You can use one of the following methods to change the default configuration:
mapred-site.xml file
on the node that runs the job. You can use central configuration to
change this value on each node that runs the NodeManager in the
cluster. Then, restart NodeManager on each node in the cluster. The
mapred-site.xml file for MapReduce ve applications is
located in the following directory:
opt/mapr/hadoop/hadoop-2.x.x/etc/hadoop