cryosparcw reference (v5.0+)

How to use the cryosparcw utility for managing CryoSPARC workers

Worker management with cryosparcw

CryoSPARC worker nodes host CryoSPARC’s compute programs and libraries used to run jobs.

Workstations or worker nodes with a cryosparc_worker installation have access to cryosparcw, a built-in command-line utility to perform advanced usage tasks.

Run all commands in this section while logged into the workstation or worker nodes where the cryosparc_worker package is installed.

Navigate to the worker installation directory and run command starting with bin/cryosparcw. For example:

cd /path/to/cryosparc_worker
bin/cryosparcw info --gpu

You may add cryosparcw to the worker's PATH by adding a line like this to the ~/.bashrc file, replacing /path/to/cryosparc_worker with the actual path to the worker installation directory:

export PATH=/path/to/cryosparc_worker/bin:$PATH

And restart your shell. Run the following command to verify that PATH was updated correctly:

which cryosparcw

And ensure the output is /path/to/cryosparc_worker/bin/cryosparcw .

For help with a specific command, run:

cryosparcw COMMAND --help

For example, for help showing worker info, run:

cryosparcw info --help

cryosparcw

Usage:

  $ cryosparcw [OPTIONS] COMMAND [ARGS]...

Options:

• --install-completion: Install completion for the current shell.

• --show-completion: Show completion for the current shell, to copy it or customize the installation.

• --help: Show this message and exit.

All available cryosparcw commands are listed and documented below.

cryosparcw version

Show CryoSPARC version.

Usage:

$ cryosparcw version [OPTIONS]

Options:

• --help: Show this message and exit.

cryosparcw update

Install a previously-downloaded CryoSPARC update at cryosparc_worker/cryosparc_worker.tar.gz

Usage:

$ cryosparcw update [OPTIONS]

Options:

• --force

• --help: Show this message and exit.

cryosparcw deps

Install Python and external dependencies. Specify --force to install even if they haven't changed.

Usage:

$ cryosparcw deps [OPTIONS]

Options:

• --force

• --help: Show this message and exit.

cryosparcw patch

Install a patch that was previously-downloaded on the master with cryosparcm patch --download.

Usage:

$ cryosparcw patch [OPTIONS]

Options:

• -f, --file FILE: Path to input file [default: cryosparc_worker/cryosparc_worker_patch.tar.gz]

• --force

• --help: Show this message and exit.

cryosparcw env

Export environment variables used by CryoSPARC. Execute the output to activate, e.g., eval $(cryosparcw env)

Usage:

$ cryosparcw env [OPTIONS]

Options:

• --help: Show this message and exit.

cryosparcw connect

Connect a managed worker to CryoSPARC. Run this command on the worker that you wish to register with CryoSPARC, or whose existing registration you wish to update.

For full details, see Connecting A Worker Node.

Usage:

$ cryosparcw connect [OPTIONS]

Options:

• --license TEXT: CryoSPARC license ID read from installed configuration file (config.sh) if not specified. [env var: CRYOSPARC_LICENSE_ID; required]

• --master TEXT: Hostname of machine running cryosparc_master. Specify "localhost" for current host. [env var: CRYOSPARC_MASTER_HOSTNAME; required]

• --port INTEGER: Base port for CryoSPARC services. Usually 61000 unless changed during installation. [env var: CRYOSPARC_BASE_PORT; default: 61000]

• --auth / --no-auth: Enable master database authentication [env var: CRYOSPARC_DB_ENABLE_AUTH; default: auth]

• --timeout INTEGER: Database connection timeout, in milliseconds [env var: CRYOSPARC_DB_CONNECTION_TIMEOUT_MS; default: 20000]

• --worker TEXT: Name of worker. Defaults to $(hostname) if not specified.

• --lane TEXT: Scheduler lane for worker (create if does not exist). [default: default]

• --sshstr TEXT: SSH login string to access worker, required if the worker's hostname or UNIX user differs when connecting from master, or to specify additional SSH flags. Defaults to "$(whoami)@worker".

• --cpus INTEGER RANGE: Number of CPU cores to enable for jobs. Enable all cores if not specified. [x>=1]

• --rams INTEGER RANGE: Number of 8GiB RAM slots to enable for jobs. Enable all RAM if not specified. [x>=1]

• --gpus TEXT: Comma-separated list of GPU device IDs, e.g., '0,1,2'. Selects all GPUs if not specified. Cannot be specified with --no-gpu.

• --gpu / --no-gpu: Do not attempt to select any GPUs. Don't specify both --no-gpu and --gpus flag. [default: gpu]

• --ssdpath DIRECTORY: Local SSD scratch path. Strongly recommended.

• --ssdquota INTEGER: Maximum amount of SSD space to use for caching, in megabytes (MB).

• --ssdreserve INTEGER: Minimum amount free space to leave on the SSD, in megabytes (MB). [default: 10000]

• --help: Show this message and exit.

Example command to connect a worker to a new lane with the same name.

cryosparcw connect \\
	--worker $(hostname -f) \\
	--master csmaster.local \\
	--port 61000 \\
	--ssdpath /scratch/cryosparc_cache \\
	--lane $(hostname -s)

cryosparcw disconnect

Disconnect a worker node. You may also use cryosparcm worker disconnect if the worker installation directory is no longer accessible.

Usage:

$ cryosparcw disconnect [OPTIONS]

Options:

• --license TEXT: CryoSPARC license ID, read from installed configuration file (config.sh) if not specified. [env var: CRYOSPARC_LICENSE_ID; required]

• --master TEXT: Hostname of machine running cryosparc_master. Specify "localhost" for current host. [env var: CRYOSPARC_MASTER_HOSTNAME; required]

• --port INTEGER: Base port for CryoSPARC services. Usually 61000 unless changed during installation. [env var: CRYOSPARC_BASE_PORT; default: 61000]

• --auth / --no-auth: Enable master database authentication [env var: CRYOSPARC_DB_ENABLE_AUTH; default: auth]

• --timeout INTEGER: Database connection timeout, in milliseconds [env var: CRYOSPARC_DB_CONNECTION_TIMEOUT_MS; default: 20000]

• --worker TEXT: Name of worker. Defaults to $(hostname) if not specified.

• --help: Show this message and exit.

cryosparcw info

Show system information.

Usage:

$ cryosparcw info [OPTIONS]

Options:

• --gpu / --no-gpu: Include GPU info [default: no-gpu]

• --format [text|json]: Output format [default: text]

• --help: Show this message and exit.

cryosparcw gpulist

Show information about available GPUs on this host.

Usage:

$ cryosparcw gpulist [OPTIONS]

Options:

• --format [text|json]: Output format [default: text]

• --help: Show this message and exit.

Use to verify that the worker is installed correctly.

cryosparcw call

Run any command with the CryoSPARC worker environment.

Usage:

$ cryosparcw call [OPTIONS]

Options:

• --help: Show this message and exit.

cryosparcw python

Run python command with the CryoSPARC worker environment.

Usage:

$ cryosparcw python [OPTIONS]

Options:

• --help: Show this message and exit.

cryosparcw ipython

Run interactive python shell with the CryoSPARC worker environment.

Usage:

$ cryosparcw ipython [OPTIONS]

Options:

• --help: Show this message and exit.