The dws_server Data Plugin

Collects DataWarp server stats for an application

The dws_server plugin collects utilization statistics from DataWarp servers, if present. This usage information is within the scope of an application, not the scope of a job. The data is written in JSON dictionary format. DataWarp server usage information within the scope of a job is available through the dws_job_server plugin. Compute node usage of DataWarp is available through the dws plugin.

Type scratch File Systems

The following data is collected for type=scratch file systems.
Per realm:
dwtype
Type of DataWarp file system (scratch)
namespace_count
Number of namespaces within the realm
realm_id
Realm ID
server_count
Number of servers in the realm
server_hostname
Server hostname
Per fragment:
bytes_read
Number of bytes read from this fragment
bytes_written
Number of bytes written to this fragment
capacity_used
Amount of file system capacity used
capacity_max
Maximum capacity of fragment
files_created
Number of files created in this realm
fs_capacity
Capacity of file system to which this fragment belongs
max_window_write
Maximum amount of data written during a write window period of time
write_high_water
The largest amount of data written
write_limit
Maximum bytes allowed to be written per fragment
write_moving_avg
The average amount of data written during a write window period of time
Per namespace:
namespace_id
Namespace ID
bytes_read
Number of bytes read from this namespace
bytes_written
Number of bytes written to this namespace
files_create_threshold
Maximum number of files allowed to be created within this namespace
file_size_limit
Maximum size (bytes) of one file
files_created
Number of files created within this namespace
max_offset_read
Maximum byte offset read
max_offset_written
Maximum byte offset written
num_data_created
Total number of data files created on all DataWarp servers
stage_bytes_read
Number of staged bytes read
stage_bytes_written
Number of staged bytes written
stripe_size
Size of each stripe (bytes)
stripe_width
Number of stripes in this namespace
substripe_size
Size of each substripe (bytes)
substripe_width
Number of substripes in per stripe

RUR dws_server / dws_job_server output for type=scratch file systems

This example shows data as written to /var/opt/cray/log/partition-current/messages-date on the SMW:
uid: 12345, apid: 416746, jobid: 21268, cmdname: xio_p, 
plugin:dws_server {{"realm": {"server_count": 1, "fragments": [{
{"capacity_used": 128648781824, "fs_capacity": 3296920076288, 
"capacity_max": 128648781824, "max_window_write": 86400, 
"files_created": 258, "write_high_water": 3407329284614, 
"write_moving_avg": 3407329284614, "bytes_read": 3298534883328, 
"write_limit": 32985348833280, "bytes_written": 3407329284614, 
"server_hostname": "c0-0c1s1n1"], "namespaces": [{ "namespace_id": 
9324, "stripe_width": 1, "stripe_size": 8388608, "bytes_read": 
3298534883328, "substripe_width": 12, "stage_bytes_read": 0, 
"substripe_size": 8388608, "max_offset_read": 1099511627776, 
"files_created": 258, "bytes_written": 3407329284614, 
"files_create_threshold": 0, "file_size_limit": 0, 
"num_data_created": 258, "stage_bytes_written": 0, 
"max_offset_written": 1099511627776 }], "realm_id": 3704}}