Create Custom RUR Data Plugins

Details all requirements for creating an RUR data plugin.

A data plugin is comprised of a staging component and a post processing component. The data plugin staging component is called by rur-stage.py on the compute node prior to the application/job running and again after the application/job has completed. The staging component may reset counters before application/job execution and collect them after application/job completion, or it may collect initial and final values prior to and after application/job execution, respectively, and then calculate the delta values. Python functions have been defined to simplify writing plugins, although it is not necessary for the plugin to be written in Python. The interface for the data plugin staging component is through command line arguments.

Data Plugin Staging Component

All data plugin staging components must support the following arguments:

--apid=apid: Defines the application ID of the running application.
--timeout=time: Defines a timeout period in seconds during which the plugin must finish running. Set to 0 for unlimited; default is unlimited.
--pre: Indicates the plugin is being called prior to the application/job.
--post: Indicates the plugin is being called after the application/job.
--outputfile=output_file: Defines where the output data is written. Each plugin should define a default output file in /var/spool/RUR/ if this argument is not provided.
--arg=arg: A plugin-specific argument, set in the RUR config file. RUR treats this as an opaque string.

The output of an RUR data plugin staging component is a temporary file located in /var/spool/RUR on the compute node. The file name must include both the name of the plugin, as defined in the RUR config file, and .apid. The RUR gather phase will automatically gather the staged files from all compute nodes after the application/job has completed and place it in gather_dir as defined in the configuration file.

Data plugin staging component

#
# Copyright (c) 2013 Cray Inc. All rights reserved.
#
# Sample data plugin staging component
#
#!/usr/bin/env python 
import sys, os, getopt
from rur_plugins import rur_plugin_args
def main():
   apid, inputfile, outputfile, timeout, pre, post, \
			   parg = rur_plugin_args(sys.argv[1:])
   if outputfile is "":
       outputfile = "/var/spool/RUR/pluginname."+str(apid)
   if (pre==1):
       zero_counters()
   else:
       write_postapp_stateto(outputfile)

if __name__ == "__main__":
   main()

Data Plugin Post Processing Component

A data plugin also requires a post processing component that processes the data staged by the staging component and collected during the RUR gather phase. The post processing component is called by rur-post.py. The input file contains records, one node per line, of all of the statistics created by the staging component. The output of the post processing component is a file containing the summary of data from all compute nodes.

All data plugin post processing components must support the following arguments:

--apid=apid: Defines the application ID of the running application.
--timeout=time: Defines a timeout period in seconds during which the plugin must finish running. Set to 0 for unlimited; default is unlimited.
--inputfile=input_file: Specifies the file from which the plugin gets its input data.
--outputfile=output_file: Specifies the file to which the plugin writes its output data.

Data plugin post processing component

#
# Copyright (c) 2013 Cray Inc. All rights reserved.
#
# Sample data plugin post processing component
#
#!/usr/bin/env python
import sys, os
from rur_plugins import rur_args

def main():
   apid, inputfile, outputfile, timeout = rur_args(sys.argv[1:])
   if outputfile is "":
       outputfile = inputfile + ".out"

   pc = PostCompute()
   pc.process_file(inputfile)
   formated = pc.present_entries([('plugin_foo_data','sum')])
   fout=open(outputfile, 'w+')
   fout.write("energy %s" % formated)

if __name__ == "__main__":
   main()