Spark SQL Thrift Server

Spark SQL Thrift (Spark Thrift) was developed from Apache Hive HiveServer2 and operates like HiveSever2 Thrift server.

Spark Thrift is supported on secure clusters. You can run the Spark Thrift server and connect to Hive versions supported by Spark 2.1.0 and later with Business Intelligence (BI) tools or the Beeline command-line tool.

Starting in the MEP 4.0 release, the Spark Thrift server is available as a separate package. To install this package, see Installing Spark Standalone or Installing Spark on YARN, depending on the type of cluster manager you are installing.

In MEP 3.0, MapR introduces additional security mechanisms for Spark with the Spark Thrift server. MapR-SASL and Kerberos are supported:

• For JDBC connections into Spark Thrift server
• Between Spark and Hive metastore

To enable these security mechanisms for the Spark Thrift server, starting in the MEP 4.0 release, for secure clusters, running configure.sh -R configures MapR-SASL security. The script modifies or creates a SPARK_HOME/conf/hive-site.xml file as follows:

• If Hive is installed in your cluster, the script copies HIVE_HOME/conf/hive-site.xml to SPARK_HOME/conf and modifies the file.
• If Hive is not installed and you are using MapR-SASL security, the script creates a new SPARK_HOME/conf/hive-site.xml file.
• Each time the script runs, if there is a pre-existing SPARK_HOME/conf/hive-site.xml file, the script saves a copy of the file in SPARK_HOME/conf/hive-site.xml.old before modifying it.

You can configure security manually by following the steps outlined in sub-topics listed on this page.

To launch the Spark Thrift server, perform the procedures required to configure Spark to use Hive.