Configuring Object Store with S3-Compatible API

Describes how to configure Object Store with S3-Compatible API.

MEP 7.1.0 and later supports Object Store 2.1.0.

To configure the Object Store with S3-Compatible API, add the Object Store superadmin and change the deployment mode in the minio.json file. After you update and save the file, restart the objectstore service.

Global Configuration

To change the global configuration of the Object Store update the configuration file located in /opt/mapr/objectstore-client/objectstore-client-<version>/conf/minio.json.

The configuration file for Object Store 2.1.0 differs from previous versions; it includes LDAP, distributed mode, and skip secure verification configuration updates.

Object Store 2.1.0 Configuration
{ 
    "version": "2", 
    "fsPath": "/mapr//apps/s3/", 
    "deploymentMode": "S3", 
    "distributedHosts": "", 
    "accessKey": "minioadmin", 
    "secretKey": "minioadmin", 
    "oldAccessKey": "", 
    "oldSecretKey": "", 
    "port": "9000", 
    "domain": "", 
    "logPath": "/opt/mapr/objectstore-client/objectstore-client-2.1.0/logs/minio.log", 
    "logLevel": 4, 
    "insecureSkipVerify": false, 
    "ldap": { 
        "serverAddr": "", 
        "usernameFormat": "", 
        "userDNSearchBaseDN": "", 
        "userDNSearchFilter": "", 
        "groupSearchFilter": "", 
        "groupSearchBaseDn": "", 
        "lookUpBindDN": "", 
        "lookUpBindPassword": "", 
        "stsExpiry": "", 
        "tlsSkipVerify": "", 
        "serverStartTls": "", 
        "serverInsecure": "" 
    } 
}
Object Store 2.0.x Configuration
{
   "fsPath":"/mapr//apps/s3/",
   "deploymentMode":"S3",
   "accessKey":"minioadmin",
   "secretKey":"minioadmin",
   "oldAccessKey":"",
   "oldSecretKey":"",
   "port":"9000",
   "domain":"",
 "logPath":"/opt/mapr/objectstore-client/objectstore-client-2.0.0/logs/minio.log",
   "logLevel":4,
   "ldap":{
      "serverAddr":"",
      "usernameFormat":"",
      "usernameSearchFilter":"",
      "groupSearchFilter":"",
      "groupSearchBaseDn":"",
      "usernameSearchBaseDn":"",
      "groupNameAttribute":"",
      "stsExpiry":"",
      "tlsSkipVerify":"",
      "serverStartTls":"",
      "severInsecure":""
   }
}

To change the port and log path requires you to reconfigure Warden for the server node and restart the Object Store. Other properties require only an Object Store restart.

Setting the superadmin Credentials

You can configure the superadmin user name and password with properties accessKey and secretKey. If you configure the Object Store for the first time, then it is enough just to change them. If you would like to change the superadmin credentials on an already configured instance, you also have to set previous values oldAccessKey to the and oldSecretKey.

Setting the Deployment Mode

The deployment mode sets the type of authorization being used. The default deployment mode is S3. To change the deployment mode setting, you can edit the deploymentMode parameter in the /opt/mapr/objectstore-client/objectstore-client-<version>/conf/minio.json file.

The Object Store supports the following deployment modes:
Option Deployment Mode Description
1 FS Enforced by filesystem file permissions only. The S3 bucket policy is disabled. Access is granted based on the bucket owner UID and GID (not on read, write, or execute permissions).
  • Configuration maps the application key or secret to a data-fabric ID.
  • S3 policy is not used, and the policy check is skipped.
  • Data Fabric file security validates inbound mapped UID and GID to authorize read or write file permissions.

You must Set the Path to Mount the Filesystem.
2 S3 Enforced by S3 bucket policy only:
  • Access is controlled by the secret access key and the key ID.
  • Read-write permissions and the owner of the file are ignored while granting access.
Important: To prevent unauthorized access to data, upload the corresponding S3 policies after Object Store starts.

Setting the Path to Mount the Filesystem

If you use the FS deployment mode, you must set the path to mount the file system:
Node Type How to Set the Path
Server node In the minio.json file, set the fsPath parameter to the file-system mount path. By default, the path is set to /mapr/<clustername>/apps/s3/<nodename>. If you want to share existing folders with users, set the fsPath parameter to point to the directory with the folders that you want to share. All the folders in the directory to which fsPath points are accessible as buckets.
Important: All folders in the parent folder (fsPath parameter) will be accessible as buckets. Set filesystem permissions on folders to prevent unauthorized access to data.
Edge node If you completed the edge-node installation steps, no action is required. This is because you already set the path to mount the filesystem when you ran the Object Store configure.sh script with the --path parameter.

Distributed Mode

Object Store 2.1.0 supports distributed mode. Object Store 2.0.x does not support distributed mode.

Note the following points related to Object Store 2.1.0 in distributed mode:
  • In distributed mode, instances of Object Store share user and policy information only, as well as locks between nodes.
  • Distributed mode does not include proxy or load balancing in front of instances. You must configure and install proxy and load balancing manually.
  • Distributed mode only works for S3 mode.
  • Three or more nodes is recommended for Object Store in distributed mode. If only one node is active, the node will be in read-only mode.
To enable distributed mode for Object Store 2.1.0:
  1. Configure Object Store 2.1.0 on three or more nodes.
  2. Disable caching in Fuse clients.
  3. On each Object Store node, update the /opt/mapr/objectstore-client/objectstore-client-<version>/conf/minio.json file with the accessKey, secretKey, and ldap. The value for each of these properties must be identical across all the minio.json files.
  4. On all Object Store 2.1.0 nodes, verify that the distributedHosts property contains the list of nodes with mount paths, for example:
    "distributedHosts": "http(s)://HOST1:PORT1/MOUNT_PATH1 http(s)://HOST2:PORT2/MOUNT_PATH2"
    Note that the mount path on all nodes should target the same folder in the filesystem.
  5. If the cluster is secure with self-signed certificates, copy the public certificates for each host to the /opt/mapr/objectstore-client/objectstore-client-<version>/conf/certs/CAs directory. To ignore validation of the certificates, set "insecureSkipVerify": true.

Setting Virtual Hosting of Buckets

You can enable the virtual hosting of buckets by setting the domain property in the /opt/mapr/objectstore-client/objectstore-client-<version>/conf/minio.json configuration file. For more information, see the Domain section in the MinIO Server Config Guide.

Setting LDAP Integration

You can set LDAP integration in the ldap section of the /opt/mapr/objectstore-client/objectstore-client-<version>/conf/minio.json configuration file. For descriptions and examples of the properties for LDAP integration, seeMinIO LDAP documentation.

For LDAP integration information, refer to Integrating Object Store 2.1.0 with LDAP/AD.

The following example shows the LDAP configuration for Object Store 2.1.0:
"ldap": { 
"serverAddr": "localhost:389", 
"usernameFormat": "cn=%s,dc=mapr,dc=local", 
"userDNSearchBaseDN": "", 
"userDNSearchFilter": "(cn=%s)", 
"groupSearchFilter": "(&(objectclass=group)(member=%s))", 
"groupSearchBaseDn": "dc=mapr,dc=local", 
"lookUpBindDN": "", 
"lookUpBindPassword": "", 
"stsExpiry": "60h", 
"tlsSkipVerify": "on", 
"serverStartTls": "", 
"serverInsecure": "on" 
}
The following example shows the LDAP configuration for Object Store 2.0.x:
"ldap":{
      "serverAddr":"10.244.0.11:636",
      "usernameFormat":"uid=%s,ou=users,dc=example,dc=com",
      "usernameSearchFilter":"(uid=%s)",
      "groupSearchFilter":"",
      "groupSearchBaseDn":"ou=users,dc=example,dc=com",
      "usernameSearchBaseDn":"",
      "groupNameAttribute":"",
      "stsExpiry":"60h",
      "tlsSkipVerify":"on",
      "serverStartTls":"",
      "severInsecure":"off"
   }

LDAP Limitations

  • LDAP integration only works in S3 mode; it does not support FS mode.
  • LDAP/AD integration does not work with Hadoop and Spark because the HPE Ezmeral Data Fabric implementation of Hadoop and Spark does not support sessionToken.
  • In Object Store 2.0.0, the MINIO_IDENTITY_LDAP_SERVER_STARTTLS property is not supported, and its value in the configuration file is ignored.

About Credentials

In the S3 world, credentials represent the application and not the identity of the end user. The application layer is responsible for end-user verification. The S3 administrator must assign S3 credentials for the application or set of applications and, optionally, map those S3 credentials to a data-fabric identity.

As defined in the Amazon S3 documentation, the S3 REST API uses a “key” and “secret” (in a REST-like manner) as credentials to authenticate to the underlying object store and authorize access to data.

The Object Store supports a multitenant scenario in which the S3 administrator can configure one or more credentials with the appropriate data-fabric credential mapping. The S3 administrator can assign credentials to a user and, optionally, map them to a data-fabric identity.

For an overview of tenants and multi-tenancy, see Multitenancy on MapR Filesystem.

Adding Credentials and Policies

You add credentials and policies by using the MinIO Client (mc), which is located here: /opt/mapr/objectstore-client/objectstore-client-<version>/util/mc. For examples of using the MinIO Client, see the MinIO Admin Complete Guide.

The only difference is that in the Object Store, you can additionally specify the UID and GID for the user to ensure that FS mode is working. For example:
mc admin user add SERVER_ALIAS USERNAME PASSWORD UID GID

Restarting the Object Store Service

You start, restart, or stop the Object Store by using the command appropriate for the node:

Server Node
/opt/mapr/bin/maprcli node services -name objectstore -nodes <node_name> -action [ start |
          restart | stop ]

Edge Node

sudo /opt/mapr/objectstore-client/objectstore-client-<version>/bin/objectstore.sh [ start |
        stop ]
Parent topic: Object Store with S3-Compatible API