CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • Worker Management with cryosparcw
  • cryosparcw env
  • cryosparcw call <command>
  • cryosparcw connect <options>
  • cryosparcw gpulist
  • cryosparcw ipython
  • cryosparcw patch
  • cryosparcw update
  • cryosparcw newcuda <path> (CryoSPARC v4.3 and older)
  1. Setup, Configuration and Management
  2. Management and Monitoring

cryosparcw reference

How to use the cryosparcw utility for managing cryoSPARC workers

Previouscryosparcm cli referenceNextSoftware System Guides

Last updated 1 year ago

Worker Management with cryosparcw

Run all commands in this section while logged into the workstation or worker nodes where the cryosparc_worker package is installed.

Verify that cryosparcw is in the worker's PATH with this command:

which cryosparcw

and ensure the output is not empty.

Alternatively navigate to the worker installation directory and in bin/cryosparcw. Example:

cd /path/to/cryosparc_worker
bin/cryosparcw gpulist

cryosparcw env

Equivalent to , but for worker nodes.

cryosparcw call <command>

Execute a shell command in a transient CryoSPARC worker shell environment. For example,

cryosparcw call which python

prints the path of the CryoSPARC worker environment’s Python executable.

cryosparcw connect <options>

Run this command from a worker node or workstation to register a worker with the master, or to edit the configuration on an existing worker.

Enter cryosparcw connect --help to see full usage details.

$ cryosparcw connect --help
usage: connect.py [-h] [--worker WORKER] [--master MASTER] [--port PORT]
                  [--sshstr SSHSTR] [--update] [--nogpu] [--gpus GPUS]
                  [--nossd] [--ssdpath SSDPATH] [--ssdquota SSDQUOTA]
                  [--ssdreserve SSDRESERVE] [--lane LANE] [--newlane]
                  [--rams RAMS] [--cpus CPUS] [--monitor_port MONITOR_PORT]

Connect to cryoSPARC master

optional arguments:
  -h, --help            show this help message and exit
  --worker WORKER
  --master MASTER
  --port PORT
  --sshstr SSHSTR
  --update
  --nogpu
  --gpus GPUS
  --nossd
  --ssdpath SSDPATH
  --ssdquota SSDQUOTA
  --ssdreserve SSDRESERVE
  --lane LANE
  --newlane
  --rams RAMS
  --cpus CPUS
  --monitor_port MONITOR_PORT

Example command to connect a worker in a single-machine workstation on a new resources lane.

cryosparcw connect \
    --worker localhost \
    --master localhost \
    --port 39000 \
    --ssdpath /scratch/cryosparc_cache \
    --lane default \
    --newlane

Overview of available options.

  • --worker <WORKER>: (Required) Hostname of the worker that the master can use for access the worker. If the master and worker are running on the same machine (e.g., a workstation), specify --worker localhost If the master and worker are on the same local network and the master has SSH access to the worker, enter --worker $(hostname)

  • --master <MASTER>: (Required) Hostname of the local network machine

  • --port <PORT>: Port on which the master node is running (default 39000)

  • --update: Update an existing worker configuration instead of registering a new one

  • --sshstr <SSHSTR>: SSH-login string for the master to use to send commands to the workers (default $(whoami)@<WORKER>)

  • --nogpu: Registers a worker without any GPUs installed or with GPU access disabled

  • --gpus <GPUS>: Comma-separated list of GPU slots indexes that this worker has access to, starting with 0. e.g., to only enable the last two GPUs on a 4-GPU machine, enter --gpus 2,3

  • --nossd: If specified, this worker does not have access to a Solid-State Drive (SSD) to use for caching particle data

  • --ssdpath <SSDPATH>: Path to the location of the mounted SSD drive on the file system to use for caching. e.g., --ssdpath /scratch/cryosparc_cache

  • --ssdquota <SSDQUOTA>: The maximum amount of space on the SSD that cryoSPARC uses on this worker, in megabytes (MB). Workers automatically remove older cache files to remain below this quota

  • --ssdreserve <SSDRESERVE>: The amount of space to initially reserve on the SSD for this worker

  • --lane <LANE>: Which of cryoSPARC's scheduler lanes to add this worker to. Use with --newlane to automatically create a new lane

  • --newlane: Create a new scheduler lane to use for this worker instead of using an existing one

  • --rams <RAMS>: Comma-separated list of 8GB Random Access Memory (RAM) slots indexes to use for cryoSPARC jobs

  • --cpus <CPUS>: Comma-separated list of CPU core indexes to use for cryoSPARC jobs

  • --monitor_port <MONITOR_PORT>: Not used

cryosparcw gpulist

Lists which GPUs the CryoSPARC worker processes have access to.

$ cryosparcw gpulist
  Detected 4 CUDA devices.

   id           pci-bus  name
   ---------------------------------------------------------------
       0      0000:02:00.0  Quadro GP100
       1      0000:84:00.0  Quadro GP100
       2      0000:83:00.0  GeForce GTX 1080 Ti
       3      0000:03:00.0  GeForce GTX TITAN X
   ---------------------------------------------------------------

Use this to verify that the worker is installed correctly

cryosparcw ipython

cryosparcw patch

Install a patch previously downloaded on the master node with cryosparcm patch --download that is copied to the worker installation directory.

cryosparcw update

cryosparcw newcuda <path> (CryoSPARC v4.3 and older)

CryoSPARC versions v4.4.0 and newer include a bundled CUDA Toolkit and no longer have this command.

Specifies a new path to the CUDA installation to use. Example usage:

cryosparcw newcuda /usr/local/cuda-11.0

Starts an shell in CryoSPARC's worker environment.

Used for manual worker updates. See for details

ipython
Software Updates and Patches
cryosparcm env
Software Updates
See cryosparcm patch documentation.