Comment on page
cryosparcm reference
How to use the cryosparcm utility for starting and stopping the CryoSPARC instance, checking status or logs, managing users and using CryoSPARC's command-line interface.
The CryoSPARC master node hosts the web server and manages job resource allocation.
Workstations or master nodes with a
cryosparc_master
installation have access to cryosparcm
, CryoSPARC's built-in command-line utility for all administrative, management and advanced usage tasks.To use it, log into the machine onto which CryoSPARC was installed. Open a Terminal running a shell (such as
bash
) and enter any of the commands described below.cryosparcuser@emserver1:~$ cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/home/cryosparcuser/sw/cryosparc_master
Current cryoSPARC version: v4.3.1
----------------------------------------------------------------------------
​
CryoSPARC process status:
​
app RUNNING pid 1599310, uptime 0:30:11
app_api RUNNING pid 1599643, uptime 0:30:10
app_api_dev STOPPED Not started
app_legacy STOPPED Not started
app_legacy_dev STOPPED Not started
command_core RUNNING pid 1593955, uptime 0:30:26
command_rtp RUNNING pid 1595620, uptime 0:30:16
command_vis RUNNING pid 1594923, uptime 0:30:17
database RUNNING pid 1593553, uptime 0:30:29
​
----------------------------------------------------------------------------
License is valid
----------------------------------------------------------------------------
​
global config variables:
export CRYOSPARC_LICENSE_ID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
export CRYOSPARC_MASTER_HOSTNAME=emserver1.internal
export CRYOSPARC_DB_PATH="/home/cryosparcuser/sw/cryosparc_db_61561"
export CRYOSPARC_BASE_PORT=61560
export CRYOSPARC_DB_CONNECTION_TIMEOUT_MS=20000
export CRYOSPARC_INSECURE=false
export CRYOSPARC_DB_ENABLE_AUTH=true
export CRYOSPARC_CLUSTER_JOB_MONITOR_INTERVAL=10
export CRYOSPARC_CLUSTER_JOB_MONITOR_MAX_RETRIES=1000000
export CRYOSPARC_PROJECT_DIR_PREFIX='CS-'
export CRYOSPARC_DEVELOP=false
export CRYOSPARC_CLICK_WRAP=true
​
cryosparcuser@emserver1:~$
For systems that do not support this, navigate to the
cryosparc_master
installation directory and run ./bin/cryosparcm
instead of cryosparcm
This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Apply the latest patches available for your installed version of CryoSPARC without doing a full version update.
Specify the
--help
flag to see full usage$ cryosparcm patch --help
usage: cryosparcm patch [-h] [-f] [-y] [--check] [--download] [--install]
​
Download and apply cryoSPARC patches. Run cryosparcm patch to automatically
install the latest patches on master and worker nodes
​
optional arguments:
-h, --help show this help message and exit
-f, --force install latest patch again even if already installed
-y, --yes confirm patch installation without prompt
--check check to see if a patch is available
--download download master and worker patches for manual installation
--install manually install a downloaded patch file
Frequently used commands:
cryosparcm patch
: Automatically installs the latest patches on workstations or master node and connected workers. Not recommended for clusters: Use the--download
and--install
flags instead.cryosparcm patch --force
: Reinstalls the latest patches in case something went wrong with a previous attemptcryosparcm patch --check
: Shows information about the latest patches without installingcryosparcm patch --download
: Downloads the latest patches without installing them. Follow the resulting instructions to install the master and worker patchescryosparcm patch --install
: Run this command immediately after a--download
to install the patch on the master node.
Some patches require restarting CryoSPARC. After running
cryosparcm patch
, when prompted restart with cryosparcm restart
.This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Use with one of the following sub-commands.
Writes config and script template files to current working directory,
cluster_info.json
and cluster_script.sh
respectively.Outputs the existing config and script to current working directory for the cluster with the given name.
Reads
cluster_info.json
and cluster_script.sh
from the current directory. Connects a new or updates an existing cluster configuration using the name from cluster_info.json
.Validate the cluster configuration specified by submitting test jobs. For more information on cluster validation, see Guide: Cluster Integration Validation​
Removes a cluster configuration from the scheduler.
Verifies the instance has been correctly installed by running several tests. Provides a report upon completion. For more information, see Guide: Installation Testing with cryosparcm test​
Specify the
--help
flag to see full usagecryosparcm test install
Tests the core components of CryoSPARC (HTTP connections, licensing, workers, etc.) that are required to start running jobs and provides information on the status of the CryoSPARC instance (e.g., which version is running, whether a patch is available, etc.).
cryosparcm test workers <project_uid>
Tests workers connected to CryoSPARC to ensure they can correctly run CryoSPARC jobs by testing if the worker can launch jobs, cache particles to an SSD (if an SSD is configured), and utilize the GPU correctly.
The following commands are only allowed to be executed by
<cryosparcuser>
(the user that installed CryoSPARC), and can only be executed on the master node. If these conditions are not met, you may see the following error:
Error message shown when a core command is not executed on the master node
You can mitigate errors temporarily by specifying the
CRYOSPARC_FORCE_HOSTNAME
variable just before calling the command:[email protected]:~$ CRYOSPARC_FORCE_HOSTNAME="true"
[email protected]:~$ cryosparcm status
----------------------------------------------------------------------------
CryoSPARC System master node installed at
/home/cryosparcuser/cryosparc/cryosparc_master
----------------------------------------------------------------------------
​
cryosparcm process status:
...
If this error message is incorrect (the hostname specified in the error message is actually the same host, just a different identifier), you can set the hostname that the management script will use to compare by adding the variable
CRYOSPARC_HOSTNAME_CHECK
to cryosparc_master/config.sh
. You can also set CRYOSPARC_FORCE_HOSTNAME
in this file to permanently suppress this error.File: /home/cryosparcuser/cryosparc/cryosparc_master/config.sh
​
export CRYOSPARC_LICENSE_ID="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
export CRYOSPARC_MASTER_HOSTNAME="uoft"
export CRYOSPARC_DB_PATH="/home/cryosparcuser/cryosparc/cryosparc_master"
export CRYOSPARC_BASE_PORT=61000
export CRYOSPARC_HOSTNAME_CHECK="uoft"
app
app_api
app_legacy
command_core
command_rtp
command_vis
database
Prints out current status of the CryoSPARC master system, including the status of all individual processes (
database
, app
, command_core
, etc). Also prints out configuration environment variables.This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Starts the CryoSPARC instance if stopped, including the database, the command server, the web interface, etc.
All processes start in the background, including all jobs and the web interface; the command line may be closed after running
start
.Providing an optional service name will only start that specific service. For a list of all available services, see Available CryoSPARC Services.
In v4.0.0,
app_legacy
(the original web application, formerly "webapp
") is not started by default, but can be at any time.This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Stops the CryoSPARC instance if running. This will gracefully kill all the master processes, and will cause any running jobs (potentially on other nodes) to fail.
Providing an optional service name will only stop that specific service. For a list of all available services, see Available CryoSPARC Services.
This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Equivalent to running
cryosparcm stop && cryosparcm start
Providing an optional service name will only restart that specific service. For a list of all available services, see Available CryoSPARC Services.
Stops the CryoSPARC instance if running. This will gracefully kill all the master processes, and will cause any running jobs (potentially on other nodes) to fail.
Show a summary of how many jobs are active.
Backs up the CryoSPARC MongoDB database using
mongodump
. By default, the function creates a folder named backup
inside the database path specified by the CRYOSPARC_DB_PATH
environment variable in cryosparc_master/config.sh
(e.g., /u/cryosparcuser/cryosparc_db/backup
) and saves the backup as an .archive
file with the current date and time in its path (e.g., cryosparc_backup_2021_06_14_11h27.archive
) inside this folder.To change the directory the backup will be written to, specify the
--dir
flag. To change the name of the backup file, specify the --file
flag.CryoSPARC can be running when
cryosparcm backup
is run, but note that it will impact the performance of your running database. Not only does the command create traffic for a running database instance, it also forces the database to read all data through memory. When MongoDB reads infrequently used data, it can evict more frequently accessed data, causing a deterioration in performance for the database’s regular workload (source).cryosparcm backup \
--dir=/path/to/backups \
--file=custombackupfile
You can create a
cron
job to execute the cryosparcm backup
command on a schedule, ensuring you always have a recent copy of the database that you can restore in case anything goes wrong.10 4 * * * /u/cryosparcuser/cryosparc_master/bin/cryosparcm backup --dir=/remote_volume/cryosparc_db_backup --file=database_backup_$(date +"%A").archive >> /remote_volume/cryosparc_db_backup/backuplog
Will run the backup command every day at 4:10 AM. The argument
--file=database_backup_$(date +"%A").archive
will simulate a rolling-window backup, as a backup file will be created with the day of the week in the file path (e.g., database_backup_Friday.archive
). This allows you to keep a one-week buffer for the backup while limiting how much space the backups take, since future backups will overwrite a file if they are made on the same day.This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
CryoSPARC must be turned off before running this command.
The database directory (
CRYOSPARC_DB_PATH
found in cryosparc_master/config.sh
) must exist and be empty before running this command.cryosparcm restore --file=/path/to/backups/backupfile
This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Change CryoSPARC's base port to something else. Use the
--yes
or -y
flag to proceed without confirmation.cryosparcm changeport 40000
This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Ensure the database is running with the correct host configuration
Run this command after manually changing CryoSPARC's base port number to ensure the Mongo database registers the change.
Check configured CryoSPARC license ID to see if there are any issues with verifying it.
Stops queued jobs from running while allowing running jobs to finish. Can be used to facilitate a better user experience while CryoSPARC is undergoing maintenance, for example during restart, patch, or update. For more information, see Guide: Maintenance Mode and Configurable User Facing Messages​
Run
cryosparcm test --help
for usage information.Test all components of a CryoSPARC instance to confirm it is working properly. For more information, see Guide: Installation Testing with cryosparcm test​
The CryoSPARC system maintains several log files for its various processes that help with debugging any components of the CryoSPARC system that are not working.
Tails the output log of the master node process denoted by
<process>
which can be one of the following:command_core
command_rtp
command_vis
database
app
app_legacy
app_api
For example
cryosparcm log command_core
The log is live and automatically updates while the command-line remains open and new data is added to the log. To stop live updates and return to the shell, press
control C
on your keyboard, and then q
.To save the full log, redirect the output to a file
cryosparcm log command_core > command_core.log
To show only the last
lines of the log, use
tail
. For example, to see the last 1000 lines of the log:cryosparcm log command_core | tail -n 1000
Shows the output log of the master node process denoted by
<process>
which can be one of the following:command_core
command_rtp
command_vis
database
app
app_legacy
app_api
For example
cryosparcm filterlog command_core
The log is live and automatically updates while the command-line remains open and new data is added to the log. To stop live updates and return to the shell, press
control C
on your keyboard.To see
cryosparcm filterlog
usage, run cryosparcm filterlog -h
:Usage:
cryosparcm filterlog SERVICE
cryosparcm filterlog [--days|-d N] [--date|-D YYYY-MM-DD] [--max_lines|-m MAX_LINES] [--name|-n NAME] [--func|-f FUNCTION] [--level|-l LEVEL] [--tail|-t] SERVICE
Where SERVICE is one of:
app
app_legacy
app_api
command_core
command_rtp
command_vis
database
Some flags not available for all services.
Please note that only
command_core
, command_rtp
and command_vis
support the following arguments with the exception of database
, which also supports the date filter.--days|-d N
- Show last N days worth of logs
--date|-D YYYY-MM-DD
- Show only logs from the given date
--max_lines|-m MAX_LINES
- the maximum number of lines to return
--name|-n NAME
- Show only logs with the given log name as a prefix (before dots). For example:
cryosparcm filterlog command_core -n COMMAND.SCHEDULER
--func|-f FUNCTION
- Show only logs from the given log function. For example
cryosparcm filterlog command_core -f get_gpu_info_run
--level|-l LEVEL
- Show only logs with the given level. For example:
cryosparcm filterlog command_core -l ERROR
--tail|-t
- Will tail the log with the filters applied. To stop live updates and return to the shell, press
control C.
Shows a live output log for job
JXX
in project PX
. Includes the standard input and error from the python process for the job.For example, to show the output of Job 123 in Project 3, run the following
cryosparcm joblog P3 J123
joblog
shows the full stdout
for the job, which is more comprehensive than the job log in the web interface and is more helpful for debugging.To stop logging, save the full log or view only the last few log lines, see instructions for
cryosparcm log
​Prints the text of the processing log for job
JXX
in project PX
to stdout.For example, to write text components of the processing log for CryoSPARC job J123 in project P3 to a file in the current working directory, run:
cryosparcm eventlog P3 J123 > P3_J123_events.log
Compresses all
.log
files in the cryosparc_master/run
folder into a .tgz
file inside cryosparc_master/run
.Run
cryosparcm errorreport --help
for usage information.Create a CryoSPARC error report which includes diagnostic information and CryoSPARC instance logs. For more information, see Guide: Download Error Reports​
Outputs a table of users registered with the system, including their names, email addresses and admin status.
Creates a new user account for log in through the web interface. Full use:
cryosparcm createuser \
--email "<email address>" \
--username "<login username>" \
--firstname "<given name>"\
--lastname "<surname>" \
[--password "<new password>"]
If the
--password
argument is not specified, a silent input prompt is provided.This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Resets the password for indicated user with the new
<password>
provided. Full use:cryosparcm resetpassword \
--email "<email address>" \
[--password "<new password>"]
If the
--password
argument is not specified, a silent input prompt is provided.This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Updates a user profile to change the user name or set admin privileges. Full use:
cryosparcm updateuser \
--email "<email associated with cryoSPARC account>" \
--username "<new username for that user>" \
--firstname "<new first name(s) for that user>" \
--lastname "<new last name(s) for that user>" \
--admin "<true|false>" \
[--password "<corresponding cryoSPARC password>"]
If the
--password
argument is not specified, a silent input prompt is provided.--email
is required and can not be changed with this command.--username
, --firstname
, --lastname
are used for display and do not otherwise affect CryoSPARC function.Other than for the first-created user account, new users do not have administrative privileges applied by default. Following creation of the first-created user account, other accounts can also be created through the user interface if preferred:
This command can only be run by the UNIX user that owns the CryoSPARC installation directory.
Prints a list of environment variables for use in a command-line shell to replicate the exact environment used to when running CryoSPARC processes.
Run this command with
eval
to define the variables output by the env
command.eval $(cryosparcm env)
Runs a command using with cryoSPARC's command-line interface
cryosparcm cli <command>
See the full command reference
Runs an an interactive cryosparc shell that connects to the master processes. Use this to interactively run Python commands in the same environment as CryoSPARC.

Additional details coming soon.
See our guide on Managing a CryoSPARC Live Session from the CLI:
Download test data (subset of 20 movies from the EMPIAR-10025 dataset) to the current working directory for use with the T20S Introductory Tutorial.
Prints a help message
Execute a shell command in CryoSPARC's shell environment, such as Python. Equivalent to calling:
eval $(cryosparcm env)
followed by another shell command.