Integrating Spark

This section includes the following topics about configuring Spark to work with other ecosystem components.

Integrate Spark-SQL (Spark 2.3.1 and later) with Avro
You integrate Spark-SQL with Avro when you want to read and write Avro data. This information is for Spark 2.3.0 or later users.
Integrate Spark-SQL (Spark 1.6.1) with Avro
You integrate Spark-SQL with Avro when you want to read and write Avro data. This information is for Spark 1.6.1 or earlier users.
Integrate Spark with HBase
Integrate Spark with HBase or HPE Ezmeral Data Fabric Database when you want to run Spark jobs on HBase or HPE Ezmeral Data Fabric Database tables.
Integrate Spark-SQL (Spark 2.0.1 and later) with Hive
You integrate Spark-SQL with Hive when you want to run Spark-SQL queries on Hive tables. This information is for Spark 2.0.1 or later users.
Integrate Spark-SQL (Spark 1.6.1) with Hive
You integrate Spark-SQL with Hive when you want to run Spark-SQL queries on Hive tables. This information is for Spark 1.6.1 or earlier users.
Integrate Spark with HPE Ezmeral Data Fabric Event Store
Integrate Spark with MapR Streams to enable Spark to query HPE Ezmeral Data Fabric Event Store for new messages at a given interval, process any new messages that are available, and also publish messages into HPE Ezmeral Data Fabric Event Store.
Integrate Spark with R
You integrate Spark with R when you want to run R programs as Spark jobs.
Integrate Spark with Kafka
From MEP-5.0.0, Spark can be integrated with Kafka-1.0. You can configure a Spark application to produce Kafka messages.

Parent topic: Spark