sgather - Man Page

sgather [-rikCFpvV] SOURCE DEST

sgather is used to transmit a file from all nodes allocated to the currently active Slurm job. This command should only be executed from within a Slurm batch job or within the shell spawned after a Slurm job's resource allocation. SOURCE should be the fully qualified pathname for the file copy to be fetched from each node. SOURCE should be on a file system local to that node. DEST is the name of the file to be created on the current node where the source node name will be appended. Note that parallel file systems may provide better performance than sgather can provide, although performance will vary by file size, degree of parallelism, and network type.

-C,  --compress

Compress the file being transmitted.

-f,  --force

Ignore nonexistent source file.

-F number, --fanout=number

Specify the fanout of messages used for file transfer. Maximum value is currently eight.

-k,  --keep

Do not remove the source file after transmission.

-p,  --preserve

Preserves modification times, access times, and modes from the original file.

-r,  --recursive

Copy directories recursively.

-t seconds, --timeout=seconds

Specify the message timeout in seconds. The default value is 60 seconds.

-v,  --verbose

Provide detailed event logging through program execution.

-V,  --version

Print version information and exit.

Some sgather options may be set via environment variables. These environment variables, along with their corresponding options, are listed below. (Note: Command line options will always override these settings.)

SGATHER_COMPRESS

-C, --compress

SGATHER_FANOUT

-F number, --fanout=number

SGATHER_FORCE

-f, --force

SGATHER_KEEP

-k, --keep

SGATHER_PRESERVE

-p, --preserve

SGATHER_RECURSIVE

-r, --recursive

SGATHER_TIMEOUT

-t seconds, --timeout=seconds

Using a batch script, execute a program that produces /tmp/my.data on all nodes allocated to the Slurm job and then transmit these files to the batch node.

> cat my.job
#!/bin/bash
srun my.prog --output /tmp/my.data
sgather /tmp/my.data all_data

> sbatch --nodes=8 my.job
srun: jobid 12345 submitted

Copyright (C) 2013-2013 ZIH, TU Dresden, Federal Republic of Germany.

sbcast(1)