cryosparcm cli reference

Call methods described in this module directly as arguments to cryosparcm cli or referenced from the cli object in cryosparcm icli.
cli example
1
cryosparcm cli "enqueue_job(project_uid='P3', job_uid='J42', lane='cryoem1')"
Copied!
icli example
1
$ cryosparcm icli
2
​
3
connecting to cryoem5:61002 ...
4
cli, rtp, db, gfs and tools ready to use
5
​
6
In [1]: cli.enqueue_job(project_uid='P3', job_uid='J42', lane='cryoem1')
7
​
8
In [2]:
Copied!
add_project_user_access(project_uid, requester_user_id, add_user_id)
Allows the owner of a project to grant access to another user to view and edit the project
Parameters
project_uid (str) -- the uid of the project to grant access to
requester_user_id (str) -- the _id of the user requesting a new user to be added to the project
add_user_id (str) -- the uid of the user to add to the project
Raises
AssertionError
add_scheduler_lane(name, lanetype, title=None, desc='')
Adds a new lane to the master scheduler
Parameters
name (str) -- name of the lane
lanetype (str) -- type of lane ("cluster" or "node")
title (str) -- optional title of the lane
desc (str) -- optional description of the lane
add_scheduler_target_node(hostname, ssh_str, worker_bin_path, num_cpus, cuda_devs, ram_mb, has_ssd, cache_path=None, cache_quota=None, cache_reserve=10000, monitor_port=None, lane='default', title=None, desc=None)
Adds a worker node to the master scheduler
Parameters
hostname (str) -- the hostname of the target worker node
ssh_str (str) -- the ssh connection string of the worker node
worker_bin_path (str) -- the absolute path to 'cryosparc_package/cryosparc_worker/bin/cryosparcw' on the worker node
num_cpus (int) -- total number of CPU threads available on the worker node
cuda_devs (list) -- total number of cuda-capable devices (GPUs) on the worker node, represented as a list (i.e. range(4))
ram_mb (float) -- total available physical ram in MB
has_ssd (bool) -- if an ssd is available or not
cache_path (str) -- path on SSD that can be used for the cryosparc cache
cache_quota (int) -- the max size (in MB) to use for the cache on the SSD
cache_reserve (int) -- size (in MB) to initially reserve for the cache on the SSD
lane (str) -- the scheduler lane to add the worker node to
title (str) -- an optional title to give to the worker node
desc (str) -- an optional description of the worker node
Returns
configuration parameters for new worker node
Return type
dict
Raises
AssertionError
clear_job(project_uid, job_uid, nofail=False, isdelete=False)
Clear a job to get it back to building state (do not clear params or inputs)
Parameters
project_uid (str) -- the uid of the project that contains the job to clear
job_uid (str) -- the uid of the job to clear
Raises
AssertionError
clone_job(project_uid, workspace_uid, job_uid, created_by_user_id, created_by_job_uid=None)
Creates a new job as a clone of the provided job
Parameters
project_uid (str) -- uid of project
workspace_uid (str) -- the uid of the workspace
job_uid (str) -- the uid of the job to copy
created_by_user_id (str) -- the id of the user creating the clone
created_by_job_uid (str) -- the uid of the job creating the clone (default = None)
Returns
the job uid of the newly created clone
Return type
str
complete_patch()
Called automatically after a patch installation with cryosparcm patch
create_empty_project(owner_user_id, project_container_dir, title=None, desc=None, project_dir=None, export=True)
Creates a new project and project directory and creates a new document in the project collection
NOTE: project_container_dir is an absolute path that the user guarantees is available everywhere.
Parameters
owner_user_id (str) -- the _id of the user that owns the new project (which is also the user that requests to create the project)
project_container_dir (str) -- the path to the "root" directory in which to create the new project directory
title (str) -- the title of the new project to create [optional] [default = None]
desc (str) -- the description of the new project to create [optional] [default = None]
Returns
the new uid of the project that was created
Return type
str
create_new_job(job_type, project_uid, workspace_uid, created_by_user_id, created_by_job_uid=None, title=None, desc=None, do_layout=True, dry_run=False, enable_bench=False, priority=None)
Creates a new job
NOTE: This request is run in its own thread, but locks on the "jobs" lock, so only one create can happen at a time.This means that the builder functions below can take as much time as they like to do stuff in their own thread (releasing GIL like IO or numpy etc) and command can still respond to other requests that don't require this lock.
Parameters
job_type (str) -- the name of the job to create
project_uid (str) -- the uid of the project
workspace_uid (str) -- the uid of the workspace
created_by_user_id (str) -- the id of the user creating the job
created_by_job_uid (str) -- the uid of the job that created this job
title (str) -- optional title of the job
desc (str) -- optional description of the job
do_layout (bool) -- specifies if the job tree layout should be recalculated (default = True)
enable_bench (bool) -- should performance be measured for this job (benchmarking)
Returns
the uid of the newly created job
Return type
str
Raises
AssertionError
create_user(created_by_user_id, email, password, username, first_name, last_name, admin=False)
Creates a new cryosparc user
Parameters
created_by_user_id (str) -- identity of user who is creating the new user
email (str) -- email of user to create
password (str) -- password of user to create
admin (bool) -- specifies if user should have an administrator role or not
Returns
mongo _id of user that was created
Return type
str
delete_job(project_uid, job_uid, relayout=True, nofail=False)
Deletes a Job after killing it (if running), clearing it, setting the "deleted" property, and recalculating the tree layout
Parameters
project_uid (str) -- the uid of the project that contains the job to delete
job_uid (str) -- the uid of the job to delete
relayout (bool) -- specifies if the tree layout should be recalculated or not
delete_project(project_uid, request_user_id, all_jobs_in_project, all_workspaces_in_project)
Iterates through each job and workspace associated with the project to be deleted and deletes both, and then disables the project itself
Parameters
project_uid (str) -- the uid of the project to be deleted
request_user_id (str) -- the uid of the user that is requesting the project to be deleted
all_jobs_in_project (list) -- list of all jobs to be fully deleted
all_workspaces_in_project (list) -- list of all workspaces that will be deleted by association
Returns
string confirmation that the workspace has been deleted
Return type
str
Raises
AssertionError -- if for some reason a workspace isn't deleted, this will be thrown
delete_project_user_access(project_uid, requester_user_id, delete_user_id)
Removes a user's access from a project.
Parameters
project_uid (str) -- the uid of the project to revoke access to
requester_user_id (str) -- the _id of the user requesting a user to be removed from the project
delete_user_id (str) -- the uid of the user to remove from the project
Raises
AssertionError
delete_user(email, requesting_user_email, requesting_user_password)
Remove a user from the cryoSPARC. Only administrators may do this
Parameters
email (str) -- user's email
requesting_user_email (str) -- your cryoSPARC login email
requesting_user_password (str) -- your cryoSPARC password
Returns
confirmation message
Return type
str
delete_workspace(project_uid, workspace_uid, request_user_id, jobs_inside_one_workspace, jobs_inside_multiple_workspaces)
Iterates through each job associated with the workspace to be deleted and either deletes or updates the job, and then disables the workspace itself
Parameters
project_uid (str) -- the uid of the project containing the workspace to be deleted
workspace_uid (str) -- the uid of the workspace to be deleted
request_user_id (str) -- the uid of the user that is requesting the workspace to be deleted
jobs_inside_one_workspace (list) -- list of all jobs to be fully deleted
jobs_inside_multiple_workspaces (list) -- list of all jobs to be updated so that the workspace uid is removed from its list of "workspace_uids"
Returns
string confirmation that the workspace has been deleted
Return type
str
enqueue_job(project_uid, job_uid, lane=False, hostname=False, gpus=False, no_check_inputs_ready=False)
Add the job in the given project to the queue for the given worker lane (default lane if not specified)
Parameters
project_uid (str) -- uid of the project containing the job to queue
job_uid (str) -- job uid to queue
lane (str, optional) -- name of the worker lane onto which to queue., defaults to False
hostname (str, optional) -- the hostname of the target worker node
gpus (list, optional) -- list of GPU indexes on which to queue the given job., defaults to False
no_check_inputs_ready (bool, optional) -- if True, forgoes checking whether inputs are ready (not recommended), defaults to False
Returns
job status (e.g., 'launched' if success)
Return type
str
Example
1
enqueue_job('P3', 'J42', lane='cryoem1')
Copied!
get_job(project_uid, job_uid, *args, **kwargs)
Get a job object with optionally only a few fields specified in \*args
Parameters
project_uid (str) -- uid of project
job_uid (str) -- uid of job
Returns
mongo result set
Return type
dict
1
get_job('P1','J1','status','abc','def')
Copied!
get_project(project_uid)
Returns the information about a single project
Parameters
project_uid (str) -- the id of the project
Returns
the information related to the project thats stored in the database
Return type
str
get_scheduler_lanes()
Returns a list of lanes that are registered with the master scheduler
Returns
list of dicts -- information about each lane
get_scheduler_target_cluster_info(name)
Returns cluster information based on the passed cluster name
Parameters
name (str) -- name of cluster/lane (lane must be of type 'cluster')
Returns
json string containing cluster information
Return type
str
Raises
AssertionError
get_scheduler_target_cluster_script(name)
Returns cluster job submission template for specified cluster
Parameters
name (str) -- name of cluster/lane (lane must be of type 'cluster')
Returns
job scheduler script template
Return type
str
Raises
AssertionError
get_scheduler_targets()
Returns a list of worker nodes that are registered with the master scheduler
Returns
list of dicts -- information about each worker node
get_system_info()
Returns system-related information related to the cryosparc app
Returns
dictionary listing information about cryosparc environment
Return type
dict
get_worker_nodes()
Returns a list of worker nodes registered in the master scheduler
Returns
list of dicts -- information about each worker node
import_project(owner_user_id, abs_path_export_project_dir)
Allows a user to import any project that was exported by cryoSPARC
Parameters
owner_user_id (str) -- the mongo object id ("_id") of the user requesting to import the project
abs_path_export_project_dir (str) -- the absolute project directory containing the jobs to import
abs_path_database_export_dir (str) -- the absolute database export directory (created by cryoSPARC's export_project()) to import
job_add_to_workspace(project_uid, job_uid, workspace_uid)
Adds a job to the specified workspace
Parameters
project_uid (str) -- the id of the project
job_uid (str) -- the id of the job to add
workspace_uid (str) -- the id of the workspace
Returns
the number of modified documents
Return type
int
job_remove_from_workspace(project_uid, job_uid, workspace_uid)
Removes a job from the specified workspace
Parameters
project_uid (str) -- the id of the project
job_uid (str) -- the id of the job to remove
workspace_uid (str) -- the id of the workspace
Returns
the number of modified documents
Return type
int
kill_job(project_uid, job_uid, killed_by_user_id=None)
Kill the given running job
Parameters
project_uid (str) -- uid of the project that contain the job to kill
job_uid (str) -- job uid to kill
killed_by_user_id (str) -- ID of user that killed the job, optional
Raises
AssertionError
list_projects()
Returns a list of all projects available
Returns
all projects available in the database
Return type
list
list_users()
Show a list of all cryosparc users
list_workspaces(project_uid=None)
List all workspaces inside a given project
Parameters
project_uid (str, optional) -- the uid of the project (e.g. P1), defaults to None
Returns
list of workpaces in project or all projects if not specified
Return type
list
make_job(job_type, project_uid, workspace_uid, user_id, created_by_job_uid=None, title=None, params={}, input_group_connects={}, enable_bench=False, priority=None)
Create a new job with the given type in the given project/workspace
To see all available job types, see cryosparc_compute/jobs/register.py (look for the value of the contains keys).
To see what parameters are available for a job and what values are available, reference the build.py file in cryosparc_compute/jobs that pertains to the desired job type.
Parameters
job_type (str) -- Type of job
project_uid (str) -- project ID, e.g., P3
workspace_uid (str) -- workspace ID, e.g., W1
user_id (str) -- ID of user
created_by_job_uid (str, optional) -- ID of the parent job that created this, defaults to None
title (str, optional) -- Descriptive title for what this job is for, defaults to None
params (dict, optional) -- Parameter settings for the job, defaults to {}
input_group_connects (dict, optional) -- Connected input groups, which each key is name and each value has format JXX.output_group_name, defaults to {}
Returns
ID of new job
Return type
str
Example
The following makes a 3-class ab-initio job with particles connected from a Select 2D Classes job:
1
juid = cli.make_job(
2
    job_type='homo_abinit',
3
    project_uid='P3',
4
    workspace_uid='W1',
5
    user_id=cli.GetUser('[email protected]')['_id'],
6
    params={'abinit_K': 3},
7
    input_group_connects={'particles': 'J41.particles_selected'})
8
​
9
print(juid)  # "J42"
Copied!
refresh_job_types()
Reload the available parameter types on jobs created prior to a cryoSPARC update. Run this following a cryoSPARC update to ensure jobs run with the correct parameters.
Returns
list of dicts -- the results of the mongodb query
remove_scheduler_lane(name)
Removes the specified lane and any targets assigned under the lane in the master scheduler
NOTE: This will remove any worker node associated with the specified lane.
Parameters
name (str) -- the name of the lane to remove
remove_scheduler_target_cluster(name)
Removes the specified cluster/lane and any targets assigned under the lane in the master scheduler
NOTE: This will remove any worker node associated with the specified cluster/lane.
Parameters
name (str) -- the name of the cluster/lane to remove
Returns
"True" if successful
Return type
bool
remove_scheduler_target_node(hostname)
Removes a target worker node from the master scheduler
Parameters
hostname (str) -- the hostname of the target worker node to remove
request_delete_project(project_uid, request_user_id)
Allows the user to confirm what jobs and workspaces they will potentially be deleting when requesting to delete a project.
Parameters
project_uid (str) -- the uid of the project to be deleted
request_user_id (str) -- the uid of the user that is requesting the project to be deleted
Returns
If the project does not have any jobs or workspaces related to it, this function will go ahead and "disable" the project and return a string confirmation.
Return type
str
Returns
If the project has jobs or workspaces associated with it, it will return 2 lists. First list are all non-deleted jobs that exist within the project, and second list are all workspaces within the project to be deleted.
Return type
lists
request_delete_workspace(project_uid, workspace_uid, request_user_id)
Allows the user to confirm what jobs they will potentially be deleting when requesting to delete a workspace.
Parameters
project_uid (str) -- the uid of the project containing the workspace to be deleted
workspace_uid (str) -- the uid of the workspace to be deleted
Returns
If the workspace does not have any jobs related to it, this function will go ahead and "disable" the workspace and return a string confirmation.
Return type
str
Returns
If the workspace has jobs associated with it, it will return 2 lists. First list are jobs that can be deleted outright, and second list are jobs that are a part of other workspaces.
Return type
lists
reset_password(email, newpass)
Reset a cryosparc user's password
Parameters
email (str) -- the user's email address
password (str) -- the user's new password
Returns
the number of modified documents
Return type
str
set_project_owner(project_uid, user_id)
Updates a project's owner
Parameters
project_uid (str) -- the id of the project to delete
user_id (str) -- the owner's user id
set_scheduler_target_node_cache(hostname, cache_reserve=None, cache_quota=None)
Sets the cache reserve and cache quota for a target worker node
Parameters
hostname (str) -- the hostname of the target worker node
cache_reserve (int) -- the size (in MB) to reserve on the SSD for the cryosparc cache
cache_quota (int) -- the max size (in MB) to use on the SSD for the cryosparc cache
Raises
AssertionError
set_scheduler_target_node_lane(hostname, lane)
Sets the lane of a target worker node
Parameters
hostname (str) -- the hostname of the target worker node
lane (str) -- the name of the lane to assign to the target worker node:
Returns
target worker node's updated configurations
Return type
list
Raises
AssertionError
set_scheduler_target_property(hostname, key, value)
Sets a property for the target worker node
Parameters
hostname (str) -- the hostname of the target worker node
key (str) -- the key of the property whose value is being modified
value (str) -- the actual value to set for the property
Returns
list of dicts -- information about all targets
Raises
AssertionError
set_user_allowed_prefix_dir(user_id, allowed_prefix)
Sets directories that users are allowed to query from the file browser
Parameters
user_id (str) -- the mongo id of the user to update
allowed_prefix (str) -- the path of the directory the user can query inside (must start with "/", and must be an absolute path)
Returns
True if successful
Return type
bool
Raises
AssertionError
update_job(project_uid, job_uid, attrs, operation='$set')
Updates a specific job's document with the provided attributes
Parameters
project_uid (str) -- uid of project
job_uid (str) -- uid of job
attrs (dict) -- the attribute to modify in the job document
operation (str) -- the operation to perform on the document (default: '$set')
update_project(project_uid, attrs, operation='$set', export=True)
Updates a project's attributes
Parameters
project_uid (str) -- the id of the project to update
attrs (dict) -- the key to update and its value
operation (str) -- the type of mongodb operation to perform
Example
1
update_project(project_uid, {'project_dir' : new_abs_path})
Copied!
update_project_directory(project_uid, new_project_dir)
Safely updates the project directory of a project given a directory. Checks if the directory exists, is readable, and writeable.
Parameters
project_uid (str) -- the uid of the project to update
new_project_dir (str) -- the absolute path of the new directory
update_project_root_dir(project_uid, new_project_dir_container)
Updates the root directory of a project (creates a new project (PXXX) folder within the new root directory and updates the project document)
NOTE: the root project directory passed cannot contain a folder with the same project (PXXX) folder name
Parameters
project_uid (str) -- the uid of the project to update
new_project_dir_container (str) -- the new "root" path where the project (PXXX) folder is to be created
update_project_size(project_uid)
Calculates the size of the project. Similar to running du -sL inside the project dir
update_user(email, password, username=None, first_name=None, last_name=None, admin=None)
Updates a cryosparc user's details. Email and password are required, other params will only be set if they are not empty.
Parameters
email (str) -- the user's email address
password (str) -- the user's password
username (str) -- new username of the user, optional
first_name (str) -- new given name of the user, optional
last_name (str) -- new surname of the user, optional
admin (bool) -- whether the user should be admin or not, optional
Returns
confirmation message if successful or not
Return type
str
verify_cluster(name)
Used to ensure cluster has been properly configured by executing a generic 'info' command
Parameters
name (str) -- name of cluster/lane (lane must be of type 'cluster')
Returns
the result of the command execution
Return type
str
Raises
AssertionError