Pre-Upgrade Steps for Spark

Complete the following steps before you upgrade Spark with or without the Installer.

Parent topic: Preparing to Upgrade the Ecosystem Pack

Pre-Upgrade Steps for Spark Standalone

  1. Copy configuration files from /opt/mapr/spark/spark-<version>/conf to a location outside of the data-fabric installation directory.
    For example, if Spark SQL is configured to work with Hive, copy the /opt/mapr/spark/spark-<version>/conf/hive-site.xml file to a backup directory.
  2. Shut down the spark-master, spark-historyserver services (if the spark-historyserver is running). 
    maprcli node services -nodes <node-ip> -name spark-master -action stop
maprcli node services -nodes <node-ip> -name spark-historyserver -action stop
  3. As the mapr user, stop the secondary instances: 
    /opt/mapr/spark/spark-<version>/sbin/stop-slaves.sh

Pre-Upgrade Steps for Spark on YARN

  1. Copy configuration files from /opt/mapr/spark/spark-<version>/conf to a location outside of the installation directory.
    For example, if Spark SQL is configured to work with Hive, copy the /opt/mapr/spark/spark-<version>/conf/hive-site.xml file to a backup directory.
  2. Shut down the spark-historyserver services (if the spark-historyserver is running): 
    maprcli node services -nodes <node-ip> -name spark-historyserver -action stop

Preserving Spark Configuration

Starting from MEP 6.0.0, in case of a version update, configuration from a previously installed version of Spark is stored in a folder with an old version timestamp.