Increasing Log Retention

To increase log retention for specific component, you can modify the configuration file and reset the value of the properties.

Increasing Log Retention for filesystem

In the /opt/mapr/conf/mfs.conf file, increase the value for the mfs.max.logfile.size.in.mb property. The value for this property is computed using the following formula:

maxSizePerLogFile = maxLogSize / MAX_NUM_OF_LOG_FILES

Here:

maxLogSize specifies the total amount of space that filesystem log files can consume.
MAX_NUM_OF_LOG_FILES specifies the total number of filesystem log files (5 hard coded).

For example, for a value of 10 GB, there will be 5 log files (mfs.log-3 to mfs.log-3.4) of 2GB each.

Increasing Log Retention for CLDB

In the /opt/mapr/conf/log4j.cldb.properties file, modify the value for the following properties:

log4j.appender.R.MaxFileSize — specifies the size of each cldb.log before rolling over
log4j.appender.R.MaxBackupIndex — specifies the number of cldb.log* before the oldest one is deleted.

For example, suppose the following configuration:

log4j.appender.R.MaxFileSize = 1024MB
log4j.appender.R.MaxBackupIndex = 10

The CLDB log files will grow to 10GB before they are purged.

Note: This setting does not impact audit logging.

Increasing Log Retention for Hadoop Services

You can increase log retention for ResourceManager, NodeManager, HistoryServer, and TimelineServer by modifying the following properties in the /opt/mapr/hadoop/hadoop-<Version>/etc/hadoop/log4j.properties file:

hadoop.log.maxfilesize — specifies the size of each <service-name>-hostname>.log before rolling over.
hadoop.log.maxbackupindex — specifies the number of <service-name>-<hostname>.log* before the oldest one is deleted.

For example, suppose the following configuration:

hadoop.log.maxfilesize = 1024Mb
hadoop.log.maxbackupindex = 10

There will be 10 GB of total service-specific logs (1GB per file) before the oldest file is purged.