Every 10 seconds, the collectd service uses a HPE Ezmeral Data Fabric plugin to gather Resource Manager
metrics on the active Resource Manager. Collectd gathers metrics on the
Resource Manager JVM process, YARN applications, and nodes that are managed by the Resource
Manager. The method used to gather the metrics differs based on the metric type.
YARN Application
Metrics
Collectd gathers YARN application metrics via JMX and REST
API. The application metrics that are collected by JMX have the metric name
mapr.rm.<metric_name>. Application metrics collected via REST API have
the metric name mapr.rm_queue.<metric_name>.
Metrics Collected Using JMX
The following
metrics are collected using JMX. To filter these metrics by queue using the
rm_queue
tag, see
Configure Queue Filters for mapr.rm.<value> Metrics.
- mapr.rm.active_applications
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of active applications.
- mapr.rm.active_users
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of users with active applications.
- mapr.rm.aggregate_containers_allocated
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of allocated containers.
- mapr.rm.aggregate_containers_released
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of released containers.
- mapr.rm.allocated_MB
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The amount of memory allocated to the Resource Manager in MB.
- mapr.rm.allocated_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of CPUs allocated to the Resource Manager.
- mapr.rm.apps_completed
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of completed applications.
- mapr.rm.apps_failed
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of failed applications.
- mapr.rm.apps_killed
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of killed applications.
- mapr.rm.apps_pending
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of pending applications.
- mapr.rm.apps_running
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of running applications.
- mapr.rm.apps_submitted
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of submitted applications.
- mapr.rm.available_MB
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The amount of memory available to the Resource Manager in MB.
- mapr.rm.available_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of disks available to the Resource Manager.
- mapr.rm.available_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of CPUs available to the Resource Manager.
- mapr.rm.pending_MB
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The amount of memory, in MB, waiting to be allocated by the
Resource Manager.
- mapr.rm.pending_containers
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of containers waiting to be allocated by the Resource
Manager.
- mapr.rm.pending_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of disks waiting to be allocated by the Resource
Manager.
- mapr.rm.pending_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of CPUs waiting to be allocated by the Resource
Manager.
- mapr.rm.reserved_MB
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The amount of memory reserved by the Resource Manager in MB.
- mapr.rm.reserved_containers
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of containers reserved by the Resource Manager.
- mapr.rm.reserved_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of disks reserved by the Resource Manager.
- mapr.rm.reserved_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of CPUs reserved by the Resource Manager.
Metrics Collected Using REST API
The following YARN application metrics are collected using REST API.
- mapr.rm_queue.aggregate_containers_allocated
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of containers allocated for applications in the
default and custom queues.
- mapr.rm_queue.appmaster_used_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of disks used by the Application Master for applications
in the default and custom queues.
- mapr.rm_queue.appmaster_used_memory
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the amount of memory, in MB, used by the Application Master for
applications in the default and custom queues.
- mapr.rm_queue.appmaster_used_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of CPUs used by the Application Master for applications in
the default and custom queues.
- mapr.rm_queue.apps_pending
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of pending applications in the default and custom
queues.
- mapr.rm_queue.apps_running
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of applications running in the default and custom
queues.
- mapr.rm_queue.fairshare_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Fair Scheduler, this
parameter is the number of disks allocated to default and custom queues.
- mapr.rm_queue.fairshare_memory
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Fair Scheduler, this
parameter denotes the amount of memory, in MB, allocated to default and custom queues.
- mapr.rm_queue.fairshare_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Fair Scheduler, this
parameter denotes the number of CPUs used by applications in the default and custom
queues.
- mapr.rm_queue.used_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of disks used by applications in the default and custom
queues.
- mapr.rm_queue.used_memory
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The amount of memory, in MB, used by applications in the default
and custom queues.
- mapr.rm_queue.used_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: The number of CPUs used by applications in the default and custom
queues.
- mapr.rm_queue.max_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Fair Scheduler, this
parameter denotes the maximum number of disks available to default and custom queues.
- mapr.rm_queue.max_memory
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Fair Scheduler, this
parameter denotes the maximum amount of memory, in MB, available to default and custom
queues.
- mapr.rm_queue.max_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Fair Scheduler, this
parameter denotes the maximum number of CPUs available to default and custom
queues.
- mapr.rm_queue.user_allocated_disks
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of disks allocated to the queues.
- mapr.rm_queue.user_allocated_memory
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the amount of memory, in MB, allocated to the queues.
- mapr.rm_queue.user_allocated_vcores
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of CPUs allocated to queues.
- mapr.rm_queue.user_appmaster_used_disks
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of disks used by the queues.
- mapr.rm_queue.appmaster_used_memory
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the amount of memory used by the queues.
- mapr.rm_queue.appmaster_used_vcores
- Additional Tags:
rm_queue: Display values for a specified
queue.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of CPUs used by the queues.
- mapr.rm_queue.user_apps_pending
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of applications pending in the queues.
- mapr.rm_queue.user_apps_running
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of applications running in the queues.
- mapr.rm_queue.user_used_disks
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of number of disks used by the queues.
- mapr.rm_queue.user_used_memory
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the amount of memory, in MB, used by the queues.
- mapr.rm_queue.user_used_vcores
- Additional Tags:
rm_queue: Display values for a specified queue.rm_user: Display values for a specified user.
- Description: When queue resources are managed by the Capacity Scheduler, this
parameter denotes the number of CPUs used by the queues.
Resource Manager Node Metrics
The following are the Node metrics:
- mapr.rm_cluster.active_nodes
- The number of nodes in the cluster where containers are running.
- mapr.rm_cluster.total_nodes
- The number of nodes in the cluster.
- mapr.rm_cluster.unhealthy_nodes
- The number of nodes in the cluster that are unable to accept applications.
Resource Manager JVM Metrics
The following Resource Manager metrics are collected using JMX:
- mapr.rm.jvm.gc_count
- The number of garbage collections.
- mapr.rm.jvm.gc_count_ps_mark_sweep
- The number of parallel scavenge mark sweep collections.
- mapr.rm.jvm.gc_count_ps_scavenge
- The number of parallel scavenge collections.
- mapr.rm.jvm.gc_time_millis
- The amount of time, in milliseconds, spent on garbage collection.
- mapr.rm.jvm.gc_time_millis_ps_mark_sweep
- The amount of time, in milliseconds, spent on parallel scavenge mark sweep
collection.
- mapr.rm.jvm.gc_time_millis_ps_scavenge
- The amount of time, in milliseconds, spent on parallel scavenge collection.
- mapr.rm.jvm.log_error
- The total number of ERROR logs.
- mapr.rm.jvm.log_fatal
- The total number of FATAL logs.
- mapr.rm.jvm.log_info
- The total number of INFO logs.
- mapr.rm.jvm.log_warn
- The total number of WARN logs.
- mapr.rm.jvm.mem_heap_committed_m
- The amount of heap memory, in megabytes, committed to the Resource Manager.
- mapr.rm.jvm.mem_heap_max_m
- The maximum amount of heap memory, in megabytes, that can be committed to the Resource
Manager.
- mapr.rm.jvm.mem_heap_used_m
- The amount of heap memory, in megabytes, used by the Resource Manager.
- mapr.rm.jvm.mem_max_m
- The maximum amount of memory, in megabytes, that can be committed to the Resource
Manager.
- mapr.rm.jvm.mem_non_heap_committed_m
- The amount of non-heap memory, in megabytes, committed to the Resource Manager.
- mapr.rm.jvm.mem_non_heap_max_m
- The maximum amount of non-heap memory, in megabytes, that can be committed to the
Resource Manager.
- mapr.rm.jvm.mem_non_heap_used_m
- The maximum amount of non-heap memory, in megabytes, that can be used by the Resource
Manager.
- mapr.rm.jvm.threads_blocked
- The number of Resource Manager threads in BLOCKED state.
- mapr.rm.jvm.threads_new
- The number of Resource Manager threads in NEW state.
- mapr.rm.jvm.threads_runnable
- The number of Resource Manager threads in RUNNABLE state.
- mapr.rm.jvm.threads_terminated
- The number of Resource Manager threads in TERMINATED state.
- mapr.rm.jvm.threads_time_waiting
- The number of Resource Manager threads in TIMED_WAITING state.
- mapr.rm.jvm.threads_waiting
- The number of Resource Manager threads in WAITING state.