Use a Custom Image Recipe to Change a File on a Compute Node

Describes how to change a file on a compute node by using a custom recipe with post-build actions (postbuild_copy and postbuild_chroot).

This procedure describes how to change a file on a compute node by using a custom recipe with post-build actions (postbuild_copy and postbuild_chroot). For other ways to accomplish this task, see Change a File on a Compute Node.

Important: Do not directly modify a Cray-provided recipe.

List the existing recipes to determine which image recipe to include.

smw# recipe list
admin_cle_7.0.up01_sles_15_ari
cdt-base-1.0.0_sle12_ari
cdt-base_1.0.0_sles_12sp3_ari
compute-large_cle_7.0.up01_sles_15_ari
compute_cle_7.0.up01_sles_15_ari
dal_cle_7.0.up01_sles_15_ari
diags_cle_7.0.up01_sles_15
elogin-smw-large_cle_7.0.up01_sles_15_ari
elogin-smw_cle_7.0.up01_sles_15_ari
...

Create a new image recipe. This example uses the recipe name site_compute.

smw# recipe create --description \
"Example recipe for 3rd-party software on compute nodes" site_compute

Add the existing image recipe as a subrecipe. This example uses the Cray-provided recipe compute_cle_7.0.up01_sles_15_ari.
```
smw# recipe update -i compute_cle_7.0.up01_sles_15_ari site_compute
```
Add post-build actions by manually editing the image recipe in /etc/opt/cray/imps/image_recipes.d/image_recipes.local.json.
Post-build actions can add non-RPM content (files or directories) to the image or specify commands to run in the chroot environment of the image root (on the SMW). For example, the post-build actions could include copying a tar file into the image, then using chroot to run the commands to untar it and run an install script.
- In the postbuild_chroot section, add the commands to run in a chroot environment for this image root.
- In the postbuild_copy section, add the files to copy into the image.
```
smw# vi /etc/opt/cray/imps/image_recipes.d/image_recipes.local.json
"site_compute": {
        ...
        "postbuild_chroot": [
            "chroot_command1",
            ...
            "chroot_commandN"
        ],
        "postbuild_copy": [
            "/file/1",
            ...
            "/dir/2/content"
        ],
        ”recipes": { ... },
        ...
    },
```
Tip: Post-build scripts can use the following environmental variables:
- IMPS_IMAGE_NAME
- IMPS_VERSION
- IMPS_IMAGE_RECIPE_NAME
- IMPS_POSTBUILD_FILES
Validate the image recipe.
```
smw# recipe validate site_compute
INFO - Validating recipe site_compute is valid for x86_64 architecture.
INFO - Validating Image 'site_compute-validate-2018-05-31_13:31:44'
...
INFO - Building out site_compute
INFO - Calling package manager to validate Recipe 'site_compute'; this can take a few minutes.
INFO - Removed Image 'site_compute-validate-2018-05-31_13:31:44'.
INFO - Removed Image 'site_compute-validate-2018-05-31_13:31:44'.
INFO - Recipe validates.
```
This command checks that the JSON syntax of the image recipe is correct. It also validates, for the specified architecture, all repositories and package collections referenced by the image recipe, checks that all required packages are included in the recipe, and ensures that it can access any files in the postbuild_copy section.
Caveat. Recipe validation does NOT validate post-build activities, such as running scripts and copyfiles actions, because without actually installing packages, the scripts/actions cannot be run.
Build the image recipe to create the image root.
Choose a unique name for the image root. Cray recommends using the image recipe name plus the current date/time. This example uses the image root name site_compute_timestamp.

Important: If the image root name is not unique, it will overwrite an existing image root. A unique name is especially important for images that are pushed to the boot node. Do not overwrite the image root that is currently used by running nodes.

The image create command builds the image recipe starting with the package manager installation and then proceeds to step through the post-build copy and post-build chroot commands (in that order).
If the image to be created should have a different architecture than the recipe's default architecture, add the --arch ARCH to the following command, where ARCH is one of the valid architectures, such as x86_64 or aarch64.
```
smw# image create -r site_compute site_compute_timestamp
INFO - Repository 'my_sles15_repo' validates.
INFO - Recipe 'site_compute' is valid for building.
INFO - Calling Package manager to build new image root; this will take a few minutes.
INFO - Rebuilding RPM database for Image 'site_compute_timestamp'.
INFO - RPM database does not need to be rebuilt.
INFO - Running post-build scripts for Image 'site_compute_timestamp'.
INFO - Copying postbuild files to /tmp/tmpmAyzGl in Image 'site_compute_timestamp'
INFO - * Executing post-build chroot script: 'common_command1'
INFO - post-build chroot script output will be located in /tmp/site_compute_postbuild_out_20180513-15:55:11g4WA6p
INFO - Build of Recipe 'site_compute' has completed successfully.
```

(Optional) Display the build history of the image root.

smw# image show site_compute_20180521082046
site_compute_20180521082046:
  name: site_compute_20180521082046
  arch: x86_64
  dist: sles15
  created: 2018-05-21T08:20:46
  history:
    2018-05-21T08:20:57: Successful build of Recipe 'seed_common_6.0.up01_sles_15' into Image 'site_compute_20180521082046'.
    2018-05-21T08:22:53: Successful build of top level recipe 'compute_cle_7.0.up01_sles_15_ari'.
    2018-05-21T08:22:53: Successful rebuild of RPM database.
  path: /var/opt/cray/imps/image_roots/site_compute_20180521082046

Package the image root into a boot image.

smw# image export site_compute_timestamp

INFO - Copying kernel /var/opt/cray/imps/image_roots/site_compute_timestamp/boot/bzImage-3.12.28-4.6_1.0000.8685-cray_ari_c into /tmp/temp_tempfs_50LJ93/DEFAULT
INFO - Copying parameters file /var/opt/cray/imps/image_roots/site_compute_timestamp/boot/parameters-ari_c into /tmp/temp_tempfs_50LJ93/DEFAULT
 .
 .
 .
INFO - Image 'site_compute_timestamp' has been packaged into /var/opt/cray/imps/boot_images/site_compute_timestamp.cpio.

If this is a netroot image, push the image root to the boot node.

Important: Before pushing the image root, make sure that there is sufficient space on the boot node in /var/opt/cray/imps/image_roots.
```
smw# image sqpush site_compute_timestamp --destination boot
```
The image sqpush command puts a SquashFS compressed image root on the boot node. Cray recommends using this command instead of image push for better boot performance. For more information, see About Image Pushes: push versus sqpush.
Test the new boot image on a single node.
1. Assign the boot image to a node with the NIMS cnode command.
  The cnode and cmap commands replace the nimscli command, which was deprecated in CLE 6.0.UP04 and removed in CLE 6.0.UP05. Be sure to change any scripts that reference nimscli.
  This example assigns the boot image file site_compute_timestamp.cpio (in the directory /var/opt/cray/imps/boot_images/) to the compute node with the cname c0-0c0s15n3.
  - For a tmpfs image:
```
smw# cnode update -i \
/var/opt/cray/imps/boot_images/site_compute_timestamp.cpio c0-0c0s15n3
```
  - For a netroot image:
```
smw# cnode update c0-0c0s15n3 \
--set-parameter netroot=site_compute_timestamp
```
2. Warm-boot the node to test the boot image.
```
smw#  xtcli shutdown c0-0c0s15n3
.
.
.
crayadm@smw> xtbootsys --reboot \
-r "testing new boot image site_compute_timestamp" c0-0c0s15n3
```
Change the NIMS map to assign the new image to the applicable nodes.
1. Back up the current map before changing to the new image. First, identify the active map.
```
smw# cmap list | grep True
```
  The following steps use the active map name "current-map".
2. Next, clone the current map.
```
smw# cmap create --clone current-map new-map
```
  Trouble? The cmap command will first verify that the CLE config set associated with the NIMS map exists. If it does not exist, the command will fail with an error message to that effect.
  - If the config set is expected to be missing (for example, during an installation when the CLE config set has not yet been created), then repeat the cmap create command with the --no-verify flag.
  - If the config set is NOT expected to be missing, then create/locate the missing config set, set it as the default config set for the NIMS map, and repeat the cmap create command.
3. Mark the new map as the active map.
```
smw# cmap setactive new-map
```
4. Assign the new boot image to all applicable nodes. This example uses "--filter group=compute" to assign the image to all compute nodes.
  - For a tmpfs image:
```
smw# cnode update --filter group=compute \
-i /var/opt/cray/imps/boot_images/site_compute_timestamp.cpio
```
  - For a netroot image:
```
smw# cnode update --filter group=compute \
--set-parameter netroot=site_compute_timestamp
```
  Trouble? If problems occur, use this command to revert to the previous map (current-map):
```
smw# cmap setactive current-map
```
Choose when the nodes should switch to the new image.
- To immediately use the new image, warm-boot all applicable nodes with the new image. This example specifies the compute nodes as a comma-separated list of cnames; see the xtcli(8) man page for other ways of specifying multiple nodes.
```
smw# xtcli shutdown cname, cname, ... cname
.
.
.

crayadm@smw> xtbootsys --reboot -r "Booting custom image on all compute nodes" \
cname, cname, ... cname
```
- To have the workload manager (WLM) reboot the node once the current user's job finishes, see Apply Rolling Patches to Compute Nodes with cnat.
- Otherwise, wait until the next full system reboot. The nodes will boot with the new image.