cryosparcm reference

How to use the cryosparcm utility for starting and stopping the cryoSPARC instance, checking status or logs, managing users and using cryoSPARC's command-line interface.

Access the command line utility, cryosparcm
The cryoSPARC master node hosts the web server and manages job resource allocation.
Workstations or master nodes with a cryosparc_master installation have access to cryosparcm, cryoSPARC's built-in command-line utility for all administrative, management and advanced usage tasks.
To use it, log into the machine onto which cryoSPARC was installed. Open a Terminal running a shell (such as bash) and enter any of the commands described below.
cryoSPARC's installation script adds the location of cryosparcm to the $PATH environment variable via ~/.bashrc for the <cryosparcuser> UNIX user account (see Prerequisites & System Requirements for details).
For systems that do not support this, navigate to the cryosparc_master installation directory and run ./bin/cryosparcm instead of cryosparcm
Instance Setup
cryosparcm update
See Software Updates for details.
Software Updates
cryosparcm patch
Apply the latest patches available for your installed version of cryoSPARC without doing a full version update.
Specify the --help flag to see full usage
1
$ cryosparcm patch --help
2
usage: cryosparcm patch [-h] [-f] [-y] [--check] [--download] [--install]
3
​
4
Download and apply cryoSPARC patches. Run cryosparcm patch to automatically
5
install the latest patches on master and worker nodes
6
​
7
optional arguments:
8
  -h, --help   show this help message and exit
9
  -f, --force  install latest patch again even if already installed
10
  -y, --yes    confirm patch installation without prompt
11
  --check      check to see if a patch is available
12
  --download   download master and worker patches for manual installation
13
  --install    manually install a downloaded patch file
Copied!
Frequently used commands:
cryosparcm patch: Automatically installs the latest patches on workstations or master node and connected workers. Not recommended for clusters: Use the --download and --install flags instead.
cryosparcm patch --force: Reinstalls the latest patches in case something went wrong with a previous attempt
cryosparcm patch --check: Shows information about the latest patches without installing
cryosparcm patch --download: Downloads the latest patches without installing them. Follow the resulting instructions to install the master and worker patches
cryosparcm patch --install: Run this command immediately after a --download to install the patch on the master node.
​See also cryosparcw patch​
Some patches require restarting cryoSPARC. After running cryosparcm patch, when prompted restart with cryosparcm restart.
cryosparcm cluster
Configures cluster installation. See the Download and Installation page for details.
Downloading and Installing cryoSPARC
Use with one of the following sub-commands.
cryosparcm cluster example <cluster_type>
Writes config and script template files to current working directory, cluster_info.json and cluster_script.sh respectively.
Examples are available for Portable Batch System (cryosparcm cluster example pbs) and SLURM (cryosparcm cluster example slurm) schedulers. Other systems are similar; run one of the two cluster example commands and modify the output files accordingly.
cryosparcm cluster dump <name>
Outputs the existing config and script to current working directory for the cluster with the given name.
cryosparcm cluster connect
Reads cluster_info.json and cluster_script.sh from the current directory. Connects a new or updates an existing cluster configuration using the name from cluster_info.json.
cryosparcm cluster remove <name>
Removes a cluster configuration from the scheduler.
Instance Status and Management
The following commands are only allowed to be executed by <cryosparcuser> (the user that installed cryoSPARC), and can only be executed on the master node. If these conditions are not met, you may see the following error:
Error message shown when a core command is not executed on the master node
You can mitigate errors temporarily by specifying the CRYOSPARC_FORCE_HOSTNAME variable just before calling the command:
1
[email protected]:~$ CRYOSPARC_FORCE_HOSTNAME="true"
2
[email protected]:~$ cryosparcm status
3
----------------------------------------------------------------------------
4
CryoSPARC System master node installed at
5
/home/cryosparcuser/cryosparc/cryosparc_master
6
----------------------------------------------------------------------------
7
​
8
cryosparcm process status:
9
...
Copied!
If this error message is incorrect (the hostname specified in the error message is actually the same host, just a different identifier), you can set the hostname that the management script will use to compare by adding the variable CRYOSPARC_HOSTNAME_CHECK to cryosparc_master/config.sh. You can also set CRYOSPARC_FORCE_HOSTNAME in this file to permanently suppress this error.
1
File: /home/cryosparcuser/cryosparc/cryosparc_master/config.sh                          
2
​
3
export CRYOSPARC_LICENSE_ID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
4
export CRYOSPARC_MASTER_HOSTNAME="uoft"
5
export CRYOSPARC_DB_PATH="/home/cryosparcuser/cryosparc/cryosparc_master"
6
export CRYOSPARC_BASE_PORT=61000
7
export CRYOSPARC_HOSTNAME_CHECK="uoft"
Copied!
cryosparcm status
Prints out current status of the cryoSPARC master system, including the status of all individual processes (database, webapp, command_core, etc). Also prints out configuration environment variables.
cryosparcm start
Starts the cryoSPARC instance if stopped, including the database, the command server, the web interface, etc.
All processes start in the background, including all jobs and the web interface; the command line may be closed after running start.
cryosparcm stop
Stops the cryoSPARC instance if running. This will gracefully kill all the master processes, and will cause any running jobs (potentially on other nodes) to fail.
cryosparcm restart
Equivalent to running cryosparcm stop && cryosparcm start
cryosparcm jobstatus
Show a summary of how many jobs are active
cryosparcm backup
Backs up the cryoSPARC MongoDB database using mongodump. By default, the function creates a folder named backup inside the database path specified by the CRYOSPARC_DB_PATH environment variable in cryosparc_master/config.sh (e.g., /u/cryosparcuser/cryosparc_db/backup) and saves the backup as an .archive file with the current date and time in its path (e.g., cryosparc_backup_2021_06_14_11h27.archive) inside this folder.
To change the directory the backup will be written to, specify the --dir flag. To change the name of the backup file, specify the --file flag.
1
cryosparcm backup \
2
    --dir=/path/to/backups \
3
    --file=custombackupfile
Copied!
Creating a cron job to backup the cryoSPARC database automatically on a schedule
You can create a cron job to execute the cryosparcm backup command on a schedule, ensuring you always have a recent copy of the database that you can restore in case anything goes wrong.
For example, adding the following cron job using crontab -e (more information here):
1
10 4 * * * /u/cryosparcuser/cryosparc_master/bin/cryosparcm backup --dir=/remote_volume/cryosparc_db_backup --file=database_backup_$(date +"%A").archive >> /remote_volume/cryosparc_db_backup/backuplog
Copied!
Will run the backup command every day at 4:10 AM.  The argument --file=database_backup_$(date +"%A").archive will simulate a rolling-window backup, as a backup file will be created with the day of the week in the file path (e.g., database_backup_Friday.archive). This allows you to keep a one-week buffer for the backup while limiting how much space the backups take, since future backups will overwrite a file if they are made on the same day.
cryosparcm restore
Restore your database from a backup file (see cryosparcm backup)
1
cryosparcm restore --file=/path/to/backups/backupfile
Copied!
cryosparcm changeport <port>
Change cryoSPARC's base port to something else. Use the --yes or -y flag to proceed without confirmation.
1
cryosparcm changeport 40000
Copied!
cryosparcm checkdb
Ensure the database is running with the correct host configuration
cryosparcm fixdbport
Run this command after manually changing cryoSPARC's base port number to ensure the Mongo database registers the change.
cryosparcm licensestatus
Check configured cryoSPARC license ID to see if there are any issues with verifying it.
Logs
The cryoSPARC system maintains several log files for its various processes that help with debugging any components of the cryoSPARC system that are not working.
cryosparcm log <process>
Shows the output log of the master node process denoted by <process> which can be one of the following:
command_core 
command_rtp
command_vis 
database 
webapp
app
For example
1
cryosparcm log command_core
Copied!
The log is live and automatically updates while the command-line remains open and new data is added to the log. To stop live updates and return to the shell, press control C on your keyboard, and then q.
Additional Arguments
To see cryosparcm log usage, run cryosparcm log -h:
1
Usage:
2
    cryosparcm log SERVICE
3
    cryosparcm log [--days|-d N] [--date|-D YYYY-MM-DD] [--name|-n NAME] [--func|-f FUNCTION] [--level|-l LEVEL] SERVICE
4
Where SERVICE is one of:
5
    app
6
    command_core
7
    command_rtp
8
    command_vis
9
    database
10
    webapp
11
Some flags not available for all services.
Copied!
Please note that only command_core, command_rtp and command_vis support the following arguments with the exception of database, which also supports the date filter.
--days|-d N
Show last N days worth of logs
--date|-D YYYY-MM-DD
Show only logs from the given date
--name|-n NAME
Show only logs with the given log name as a prefix (before dots). For example: cryosparcm log command_core -n COMMAND.SCHEDULER
--func|-f FUNCTION
Show only logs from the given log function. For example cryosparcm log command_core -f get_gpu_info_run
--level|-l LEVEL
Show only logs with the given level. For example: cryosparcm log command_core -l ERROR
To save the full log, redirect the output to a file
1
cryosparcm log command_core > command_core.log
Copied!
To show only the last xxx
 lines of the log, use tail. For example, to see the last 1000 lines of the log:
1
cryosparcm log command_core | tail -n 1000
Copied!
cryosparcm joblog PX JXX
Shows a live output log for job JXX in project PX. Includes the standard input and error from the python process for the job.
For example, to show the output of Job 123 in Project 3, run the following
1
cryosparcm joblog P3 J123
Copied!
joblog shows the full stdout for the job, which is more comprehensive than the job log in the web interface and is more helpful for debugging.
To stop logging, save the full log or view only the last few log lines, see instructions for cryosparcm log​
User Management
cryosparcm listusers
Outputs a table of users registered with the system, including their names, email addresses and admin status.
cryosparcm createuser
Creates a new user account for log in through the web interface. Full use:
1
cryosparcm createuser \
2
    --email "<email address>" \
3
    --password "<new password>" \
4
    --username "<login username>" \
5
    --firstname "<given name>"\
6
    --lastname "<surname>"
Copied!
cryosparcm resetpassword
Resets the password for indicated user with the new <password> provided. Full use:
1
cryosparcm resetpassword \
2
    --email "<email address>" \
3
    --password "<new password>"
Copied!
cryosparcm updateuser
Updates a user profile to change the user name or set admin privileges. Full use:
1
cryosparcm updateuser \
2
    --email "<email address>" \
3
    --password "<current password>" \
4
    --name "<new name>" \ 
5
    --admin "<'true' or 'false'>"
Copied!
The --name and --admin flags are optional.
Other than for the first-created user account, new users do not have administrative privileges applied by default. Following creation of the first-created user account, other accounts can also be created through the user interface if preferred:
Tutorial: User Management
Command-Line Interface
cryosparcm env
Prints a list of environment variables for use in a command-line shell to replicate the exact environment used to when running cryoSPARC processes.
Run this command with eval to define the variables output by the env command.
1
eval $(cryosparcm env)
Copied!
This may be used to, for example, run the Python distribution that ships with cryoSPARC.
cryosparcm cli
Runs a command using with cryoSPARC's command-line interface
1
cryosparcm cli <command>
Copied!
See the full command reference
cryosparcm cli reference
cryosparcm icli
Runs an an interactive cryosparc shell that connects to the master processes. Use this to interactively run Python commands in the same environment as cryoSPARC.
cryosparcm cli reference
cryosparcm rtpcli
Similar to cryosparcm cli, but for running interactive commands with CryoSPARC Live.
Additional details coming soon.
cryosparcm downloadtest
Download test data (subset of 20 movies from the EMPIAR-10025 dataset) to the current working directory for use with the T20S Introductory Tutorial.
cryosparcm help
Prints a help message
cryosparcm mongo
Starts a mongo shell for cryoSPARC's local mongoDB instance.
cryosparcm call <command>
Execute a shell command in cryoSPARC's shell environment, such as Python. Equivalent to calling eval $(cryosparc env) followed by another shell command.

(Optional) cryoSPARC and systemd

cryosparcm cli reference

Last modified 1mo ago

Access the command line utility, cryosparcm

Instance Setup

`cryosparcm update`

`cryosparcm patch`

`cryosparcm cluster`

`cryosparcm cluster example <cluster_type>`

`cryosparcm cluster dump <name>`

`cryosparcm cluster connect`

`cryosparcm cluster remove <name>`

Instance Status and Management

`cryosparcm status`

`cryosparcm start`

`cryosparcm stop`

`cryosparcm restart`

`cryosparcm jobstatus`

`cryosparcm backup`

Creating a cron job to backup the cryoSPARC database automatically on a schedule

`cryosparcm restore`

`cryosparcm changeport <port>`

`cryosparcm checkdb`

`cryosparcm fixdbport`

`cryosparcm licensestatus`

Logs

`cryosparcm log <process>`

Additional Arguments

`cryosparcm joblog PX JXX`

User Management

`cryosparcm listusers`

`cryosparcm createuser`

`cryosparcm resetpassword`

`cryosparcm updateuser`

Command-Line Interface

`cryosparcm env`

`cryosparcm cli`

`cryosparcm icli`

`cryosparcm rtpcli`

`cryosparcm downloadtest`

`cryosparcm help`

`cryosparcm mongo`

`cryosparcm call <command>`