Installer Known Issues

This version of Kibana requires Elasticsearch v6.2.3 on all nodes. I found the following incompatible nodes in your cluster: v5.4.1 @ 10.10.103.231:9200 (10.10.103.231), v5.4.1 @ 10.10.102.21:9200 (10.10.102.21), v5.4.1 @ 10.10.103.230:9200 (10.10.103.230)

Workaround If Loopback NFS is running and you need to stop the cluster, you must first unmount /mapr and stop Loopback NFS on all nodes. Then, you can stop ZooKeeper and Warden. For more information, see Managing the mapr-loopbacknfs Service.

Workaround Use the Installer uninstall feature, and retry the installation from scratch. See Uninstalling Software Using the Installer Uninstall Button.

Workaround Upgrade to the latest Installer. You must use version 1.11 or later of the mapr-setup.sh script. If you cannot use Installer version 1.11 or later of the mapr-setup.sh script, install sshpass before running the mapr-setup.sh script on a SUSE cluster.

Workaround Add the Hive password to the .yaml installation file and re-run the Stanza.

Workaround To view more detailed error information when using the -v or --verbose options, check the installer-cli.log[.x] file after running the Stanza. For information about the Installer logs, see Logs for the Installer.

• diskType: io1
• installerOnitsOwn: false

Workaround Try using a diskType of gp2 (general-purpose SSD) instead of io1 (provisioned IOPs SSD), or set InstallerOnitsOwn to false instead of true. Then retry the deployment.

Workaround Ignore the Control System alarms, or upgrade to 6.1 or later, which should not generate core alarms.

Workaround Before using the probe command, update the MapR Installer. The probe command is fixed in versions 1.10 and later.

Workaround Before using the probe command, update the MapR Installer to version 1.10 or later. Or, if you must use version 1.9 or earlier, edit the probe-generated YAML file to specify the correct cluster admin user.

Workaround You can resolve the issue by manually copying mapruserticket into the /opt/mapr/conf directory of the node to be added to the cluster.

The extend cluster operation fails because keystore, truststore, and server ticket (maprserverticket) files are not present on the installer node.

If metrics monitoring is configured on the cluster, you must also copy the tickets related to Fluentd, Kibana, and Elasticsearch to the same location.

Recent packaging changes in Drill contribute to this issue. Drill 1.10 consists only of mapr-drill-1.10 (role and binaries), whereas Drill 1.15.0.0 consists of mapr-drill-1.15 (roles) and mapr-drill-internal-1.15 (binaries). During the upgrade, the mapr-drill-1.10 binaries are successfully uninstalled, but the OLD_DRILL_VERSIONS directory that is needed to back up Drill 1.10 is not created.

Workaround Avoid refreshing the browser page during an upgrade operation. If you must refresh the page, go back to the first page of the upgrade operation and start over again to ensure that the Installer has the correct parameters before it begins the Verify phase of the upgrade.

Workaround If /mapr is not mounted, check the time stamp of the /opt/mapr/conf/mapr_fstab file to see if it is older than the time stamp in the warden.log file for starting NFS. For example:

[root@atsqa4-61 logs]# ls -ld /opt/mapr/conf/fstab
rw-rr- 1 mapr mapr 39 Sep 26 11:31 /opt/mapr/conf/mapr_fstab

[root@atsqa4-61 logs]# fgrep starting warden.log | fgrep nfs
2018-09-26 11:29:33,407 INFO com.mapr.warden.service.baseservice.Service [Thread-34]: -------------Service is starting for: nfs4

ERROR: install command failed
Service mapr-data-access-gateway must be a member of a template group. Configured services require it: ['mapr-data-access-gateway']

Workaround In the MASTER services section of the sample_advanced.yaml file, add mapr-data-access-gateway to at least one of the host groups, and retry the installation.

Workaround Uninstall the software on the node manually. See Decommissioning a Node and Uninstalling Data Fabric Software from the Command-line.

file not found: /opt/mapr/elasticsearch/elasticsearch-5.4.1/etc/elasticsearch/sg/admin-usr-clientCombo.pem

Workaround Update the installer to version 1.10 or later, and retry the operation.

Workaround Try using Installer 1.9 to install the 6.0.1 cluster and Installer 1.10 to upgrade the cluster. See Updating the MapR Installer.

Workaround Re-create the cluster in Azure so that the internal DNS suffix starts with a letter and not a number.

Workaround Move any original MASTER services that caused the error back to the MASTER group. The mapr-installer.log indicates the services that need to be moved along with the Unscalable host groups found error.

If necessary, change the setting in the YAML file to match the value from the probed cluster.

Workaround Clear your browser cache or open a new browser tab whenever you need to update the Installer and perform a new installer operation.

Workaround Stop the kafka-connect service manually, and restart the service.

Workaround When the Installer prompts you for the MySQL user ID and password, enter the password that you specified when you first installed the cluster. If you did not specify a password for MySQL when you installed MapR 5.x, leave the password field blank.

• Extend Cluster
• Incremental Install
• Maintenance Update
• Shutdown
• Uninstall

Workaround: Do not use the Previous button during startup.

/opt/mapr/conf/ssl_keystore.sv.<timestamp>

To merge the files, you can use the /opt/mapr/server/manageSSLKeys.sh utility, as shown in Configuring Secure Clusters for Running Commands Remotely.

• mapr-setup.sh reload
• apt-get install mapr-installer-definitions

Workaround: When the Installer indicates that a node did not get added correctly (typically during the Installation phase), select the node and click Remove Node. Then retry adding the node.

nValueError: need more than 1 value to unpack

Workaround: Check for syntax errors in the /etc/sysctl.conf file, which can cause an Ansible error when the Installer is attempting to set various kernel parameters.

Workaround: Click Previous to go back to the Node Configuration page, and re-specify the disks that you want. Then continue with the operation.

Workaround: Do not use an IP address for node configuration with the Installer. If you already used an IP address, change the IP address in the yarn-site.xml file on all nodes. In the following example, the 10.10.10.7 IP address must be changed to a host name, such as bld73.qa.lab:

<property>
<name>yarn.timeline-service.hostname</name>
<value>10.10.10.73</value>
</property>

Workaround: Update the Installer to version 1.7 or later, and retry the operation. See Updating the Installer.

Workaround: For increased security, remove the earlier logs or change the user permissions for them.

Workaround: None.

A one-time change can prevent this issue. After updating the Installer from version 1.5 to version 1.6 or later, but before performing any Installer operations, use an Incremental Install to change your 2-digit MEP version to the equivalent 3-digit MEP version. See Updating the Installer.

Workaround: Check the installer log for a message indicating that Elasticsearch could not be secured. Use the Incremental Install feature of the Installer to retry installation of the Monitoring logging components. Alternatively, you can configure security for the logging components manually. See Step 9: Install Log Monitoring.

Workaround: Use the Control System or the maprcli service list -node command to determine if the added services are running. If not, perform a rolling restart of the nodes to which the new services were added. The rolling restart will rebalance memory across the filesystem and services. One node at a time, restart Warden on each node following the group upgrade order prescribed in Manual Rolling Upgrade Description. Use the following steps:

os.write(self.sshpass_pipe[1], to_bytes(self._play_context.password) + b'\n')
OSError: [Errno 9] Bad file descriptor

Workaround:

Workaround: If you are installing on Ubuntu 14.04, you must install Java JDK 1.8 before running the Installer. For more information, see this website. If you are installing on RHEL/CentOS or SUSE, the Installer installs Java JDK 1.8 for you.

Workaround: To proceed using the Installer, disable SSH connection reuse by including this entry underneath the [ssh_connection] property of /opt/mapr/installer/etc/ansible.cfg:

ssh_args=-o ControlMaster=no -o ControlPath=none -o ControlPersist=no

This workaround can lead to longer install times. We recommend that you resolve any network connectivity issues in your environment.

Workaround: Before upgrading, if you have Hive Metastore installed by itself, use the Incremental Install feature of the Installer to install Hive. Then proceed with the upgrade.

Performing an upgrade without doing the Incremental Install of Hive will cause the upgrade to fail. In this scenario, you will have to reinstall or rebuild the database by using Stanza commands. You can use the reset command, followed by probe, and then edit the versions in the resulting YAML file. The last step is to import the edited YAML using the import command. See Using probe and import to Generate the Installer Database.

Workaround: In the Installer Web Interface, click Restore Default.

After you use the command line to upgrade a cluster that you installed with the Installer, the Installer is not aware that the cluster runs the upgraded version. Therefore, Installer does not install nodes and ecosystem components that apply to the upgraded version.

Workaround: Use the Installer Stanzas probe and import commands to update the installer database. See Using probe and import to Generate the Installer Database.

Workaround:

1. Install JDK 1.8.
2. Use the alternatives system to set JDK 1.8 as the default Java. Or, uninstall JDK 1.7.

When this issue occurs, the Control System displays the "Hue Down Alarm."

2015-05-08 01:50:22.713: failed: 
      [command /usr/bin/apt-get install -q -y {{mapr.node.core_packages|sort|join(' ') }}] [10.10.10.162] => 
      {"changed": true,"cmd": ["/usr/bin/apt-get", "install", "-q", "-y", "mapr-cldb",
      "mapr-core","mapr-fileserver", "mapr-httpfs", "mapr-nfs", "mapr-nodemanager","mapr-resourcemanager", 
      "mapr-webserver", "mapr-zookeeper"], "delta":"0:00:05.432067", "end": "2015-05-08 01:50:22.676193", "rc": 100,
      "start":"2015-05-08 01:50:17.244126", "warnings": 
      ["Consider using apt-get modulerather than running apt-get"]}

• After installation, Spark workers may not start due to the mechanism used to start services.
• Spark worker does not start after the node that runs the worker is restarted.

/opt/mapr/spark/spark-$VERSION/sbin/start-slave.sh 1 spark://<comma-separated list of spark
      master hostname: port> 

/opt/mapr/spark/spark-$VERSION/sbin/start-slave.sh 1
     spark://host1:7077,host2:7077 

It is important to list all the spark primary node hostnames so that the worker nodes can connect to the active primary node.

maprcli node list -columns service -filter '[service==spark-master]' -json | grep
      \"hostname\": | cut -d \" -f 4