sxcp man page
sxcp — Copy files from, to and between SX clusters
sxcp can copy files and entire directories from and to Skylable SX clusters. It can also copy data between two different SX clusters. By default for each file a progress bar is displayed, which shows the copy speed and the estimated time of arrival. sxcp makes use of all the advanced features of Skylable SX, such as deduplication and transfer resuming to minimize the bandwidth usage.
- -h, --help
- Print help and exit
- Print help, including hidden options, and exit
- -V, --version
- Print version and exit
- -r, --recursive
- Recursively copy files from SOURCE to DEST directory
- -x, --one-file-system
- Don't cross filesystem boundaries when recursing. When this option is turned on sxcp will skip all mount-point directories.
- Skip all files matching the pattern argument, which is a shell wildcard pattern. This option works both for uploads and downloads, and can be specified multiple times with different patterns.
- Only process files matching the pattern argument, which is a shell wildcard pattern. This option works both for uploads and downloads, and can be specified multiple times with different patterns.
- -b, --bwlimit=RATE
- Set bandwidth usage limit in kilobytes per second. The RATE value can additionally be followed by K(-ilobytes), M(-egabytes), or G(-igabytes) suffixes (K is the default one).
- -q, --no-progress
- Don't display the progress bar
- When operating on multiple files, keep processing them even when errors occur. sxcp will display an appropriate error message for each file that failed to upload or download, and the total number of failures.
- When this option is enabled, sxcp will wait for the cluster to replicate the data across nodes, and report a problem if full replication cannot be achieved.
- -v, --verbose
- Print more details about the transfer
- -D, --debug
- Enable debug messages
- -c, --config-dir=PATH
- Path to the SX configuration directory (default: ~/.sx)
- -f, --filter-dir=PATH
- Path to the SX filter directory (default: /usr/lib/sxclient)
- Limit the total number of connections used in a session. The default limit is 5 connections.
- Limit the number of connections to a single host. The default is 2 and means that sxcp will not use more than 2 connections to a single node of an SX cluster. The value for this limit may not exceed the limit for the total number of connections.
- Set node preference level for data transfers. The argument should be a float number with a value between 0.0 and 1.0. The higher the value, the faster (from sxcp perspective) nodes will be used. For example, with the level of 1.0 sxcp will only transfer data from/to the fastest nodes, and with 0.5 it will use the fastest nodes for ~50% of transfer's time. The default is 0.0, which makes sxcp use nodes in the order returned by remote server.
- -s, --dot-size=STRING
- When the output of sxcp is redirected to a file, sxcp switches to a dot format. With this option you can control the size represented by each of the dots. The allowed values are: "short" (1 dot = 1KB), "long" (1 dot = 8KB) and "scale" (1 dot = 1 SX block size, which depends on the file size).
To recursively copy '/home/user' to the 'home' volume on the SX cluster run:
sxcp -r /home/user/ sx://cluster/home
To download the entire directory 'movies' from the volume 'data' and limit the bandwidth usage to 1MB/s run:
sxcp -r --bwlimit=1M sx://cluster/data/movies/ /home/user/