Offloading a Volume to a Tier

Explains how to offload a volume to a tier using either the Control System, the CLI, or the REST API.

At the volume level, data can be offloaded automatically by creating and associating a schedule with the tiering enabled volume or manually by triggering the offload operation. See Data Offload and Purge for more information. The following sections describe how to set up an automatic offload of data and how to trigger a one-time manual offload data at the volume level using either the Control System, the CLI, or the REST API.

Note: For a tiered volume, there can be only one running job at any given time. For example, suppose another job is running on the tiered volume, if you trigger an offload operation, the offload operation will fail.

To offload volume data, you must have one of the following permissions:

Cluster level fc permissions
Volume level fc permissions
Volume modify permissions

Note: You can also offload individual files in a tiering-enabled volume to the associated tier. See Offloading a File to a Tier Using the CLI and REST API for more information.

Important: EC volumes are automatically offloaded once they crossed the size of autooffloadthresholdgb even if they are not using the Automatic scheduler. The default size is 1024 GB (1 TB). You can modify this size as specified in Offloading a Volume to a Tier.

Setting up Automatic Offload of Data Using the Control System

Create a tier.
See Creating a Cold Tier Using the Control System or Creating a Warm Tier Using the Control System for more information.
Create a storage policy to associate with the volume.
See Creating a Storage Tier Policy Using the Control System for more information.
Create an offload schedule.
See Specifying a Schedule for Offloading Data for more information.
Create a tiering enabled volume and associate the tier, the storage policy, and schedule with the volume.
See Creating a Tiering-Enabled Volume Using the Control System for more information.

Triggering an Offload of all Data in a Volume to a Tier Using the Control System

Log in to the Control System and click Data > Volumes.
Note: The Volumes page is under the Volumes menu in the Kubernetes version of the Control System.
Select the tiered volumes to offload from the list of volumes in the Volumes pane.
Selecting a volume makes the Actions drop-down menu available.
Click Offload Data from the Actions drop-down menu to display the Offload Volume Data confirmation window.
Review the list of volumes to offload and click Offload.

If the offload fails, CLDB retries the operation after some time. See Retrying Failed Operation for more information.

Setting up Automatic Offload of Data Using the CLI and REST API

To automatically offload data:

Create a tier, a rule that contains the criteria for offloading data to the tier, and a schedule to automatically offload data to the tier.

For example:

CLI

/opt/mapr/bin/maprcli tier create -name <tier_name> -type ectier 
/opt/mapr/bin/maprcli tier create -name <tier_name> -type cold -url <tier_url> -credential <credentials>.txt -json 
/opt/mapr/bin/maprcli tier rule create -name <rule_name> -expr <expressions>
/opt/mapr/bin/maprcli schedule create -schedule <JSON>

REST

curl -k -X POST 'https://abc.sj.us:8443/rest/tier/create?name=<tier_name>&type=ectier' --user mapr:mapr
curl -k -X POST 'https://abc.sj.us:8443/rest/tier/create?name=<tier_name>&type=cold&url=<tier_url>&credential=<credential_str>' --user mapr:mapr
curl -k -X POST 'https://abc.sj.us:8443/rest/tier/rule/create?name=<rule_name>&expr=<expressions>' --user mapr:mapr
curl -k -X POST 'https://abc.sj.us:8443/rest/schedule/create?schedule=<JSON>' --user mapr:mapr

For more information, see Managing Tiers and Managing Storage Policies.

Create a tiering-enabled volume and associate the tier, rule, and schedule (that you created in step 1) with the volume.

For example, to create a volume and enable it for:

Warm tier:

CLI

/opt/mapr/bin/maprcli volume create -name <vol_name> -path <vol_mount_path> -tieringenable true -tiername <tier_name> -ecscheme <coding_scheme> -ectopology <ec_vol_topo> -tieringrule <rule_name> -offloadschedule <schedule_ID>

REST

curl -k -X POST 'https://abc.sj.us:8443/rest/volume/create?name=<vol_name>&path=<vol_mount_path>&tieringenable=true&tiername=<tier_name>&ecscheme=<coding_scheme>&ectopology=<ec_vol_topo>&tieringrule=<rule_name>&offloadschedule=<schedule_ID>' --user mapr:mapr

Cold tier:

CLI

/opt/mapr/bin/maprcli volume create -name <vol_name> -path <vol_mount_path> -tieringenable true -tiername <tier_name> -tieringrule <rule_name> -offloadschedule <schedule_ID> -recallexpirytime 2 -json

REST

curl -k -X POST 'https://abc.sj.us:8443/rest/volume/create?name=<vol_name>&path=<vol_mount_path>&tieringenable=true&tiername=<tier_name>&tieringrule=<rule_name>&offloadschedule=<schedule_ID>&recallexpirytime=2' --user mapr:mapr

You can also specify the maximum amount of data (in GB) to offload automatically for warm-tier volumes using the autooffloadthresholdgb parameter. For more information, see Working with Tiered Volumes.

Triggering an Offload of all Data in a Volume to a Tier Using the CLI and REST API

CLI

Run the following command to manually trigger an offload of all data in the volume:

maprcli volume offload -name <volume-name>

If you run the command with the ignorerule option value set to true, rules for the volume where the data resides is ignored and data is offloaded immediately. If the ignorerule option value is not specified or is set to false (default), data is offloaded based on the rules associated with the volume where the data resides.

REST

Submit a request of type POST. For example:

# curl -k -X POST 'https://abc.sj.us:8443/rest/volume/offload?name=sampleVol' --user mapr:mapr
{"timestamp":1519947659597,"timeofday":"2018-03-01 03:40:59.597 GMT-0800 PM","status":"OK","total":0,"data":[],"messages":["Successfully started offload."]}

For more information, see volume offload.

If the offload fails, CLDB retries the operation after some time. See Retrying Failed Operation for more information.