Configuring the HPE Ezmeral Data Fabric Database OJAI Connector for Apache Spark

Before using the HPE Ezmeral Data Fabric Database OJAI Connector for Apache Spark, you must edit the pom.xml file for your project.

Add the Spark core dependency into the pom.xml file:

Note: If all dependent JAR files are already present on the node, consider setting the scope parameter to provided. For example:

<scope>provided</scope>

Setting the scope this way reduces the size of the JAR file.

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_<scala_version></artifactId>
  <version><spark_artifact_version></version>
</dependency>

Add the Spark Maven dependency to the pom.xml file:

<dependency>
  <groupId>com.mapr.db</groupId>
  <artifactId>maprdb-spark</artifactId>
  <version><spark_artifact_version></version>
</dependency>

For example, see the dependencies for Spark 2.4.4.0 (MEP 6.3.0 release):

<dependency>
  <groupId>org.apache.spark</groupId>
  <artifactId>spark-core_2.11</artifactId>
  <version>2.4.4.0-mapr-630</version>
</dependency>
<dependency>
  <groupId>com.mapr.db</groupId>
  <artifactId>maprdb-spark</artifactId>
  <version>2.4.4.0-mapr-630</version>
</dependency>

To enable Maven to download dependencies, add the following repository information to the pom.xml file:

<repository>
  <id>mapr-releases</id>
  <url>https://repository.mapr.com/maven/</url>
  <snapshots>
    <enabled>false</enabled>
  </snapshots>
  <releases>
    <enabled>true</enabled>
  </releases>
</repository>