CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • cryosparc_master/config.sh
  • cryosparc_worker/config.sh
  1. Setup, Configuration and Management
  2. Management and Monitoring

Environment variables

(Advanced) Specify additional environment variables in the configuration files to augment CryoSPARC's low-level behaviour.

To set or change one of these environment settings, add a new line to one the config.sh files with the following format (substitute VARIABLE and VALUE as indicated in the next sections):

export VARIABLE="VALUE"

Or set the value based on a different environment variable provided by the system:

export VARIABLE="${OTHER_VARIABLE}"

Which config.sh file you use depends on which variable you need to change. The variables available for each file are described below.

cryosparc_master/config.sh

Note: Restart CryoSPARC with cryosparcm restart after changing this file.

Variable
Description
Default Value

CRYOSPARC_CLIENT_TIMEOUT

How many seconds to wait for network requests between CryoSPARC master/master processes before timing out with an error

300

CRYOSPARC_CLUSTER_JOB_MONITOR_INTERVAL

The amount of time to wait (in seconds) in between status updates for cluster jobs.

10

CRYOSPARC_CLUSTER_JOB_MONITOR_MAX_RETRIES

The maximum amount of retries for cluster job status updates.

1000000

CRYOSPARC_DB_CONNECTION_TIMEOUT_MS

The amount of time to wait (in milliseconds) for the database to respond when starting it.

20000

CRYOSPARC_DISABLE_IMPORT_ON_MASTER

In master/worker or cluster modes, Import Jobs always run on the default master node. Set this to true to allow running them on any node

false

CRYOSPARC_FORCE_HOSTNAME

In master/worker or cluster modes, cryosparcm commands always run on the machine where cryosparc_master was initially installed. Set this to true to allow running on any machine

false

CRYOSPARC_FORCE_USER

cryosparcm commands must be run by the same UNIX user account that owns the cryosparc_master installation. Set this to true to allow running cryosparcm from any user account

false

CRYOSPARC_HOSTNAME_CHECK

Override the installed master hostname when checking that cryosparcm is running on the correct host. See also CRYOSPARC_FORCE_HOSTNAME

-

CRYOSPARC_IGNORE_HIDDEN_FILES

When checking that job directory is empty before running a job, set to true to ignore any filenames that being with .. Use this if your file system or OS automatically creates hidden files

false

CRYOSPARC_IO_URING

Set to false to force-disable io_uring for fast disk read operations during jobs. May be required for some older operating systems or systems with an incorrect io_uring implementation.

true

CRYOSPARC_LICENSE_SERVER_ADDR

Override CryoSPARC license server address. Use for systems that require access through a proxy.

https://get.cryosparc.com

CRYOSPARC_LIVE_DATA_MANAGEMENT_SCRIPT_ENABLE

false

CRYOSPARC_LIVE_DATA_MANAGEMENT_SCRIPT_PATH

-

CRYOSPARC_MONGO_CACHE_GB

How much RAM to allocate for MongoDB database query cache, in GB

4

CRYOSPARC_MOTION_CORRECTION_CPUS_PER_GPU

How many CPU cores to allocate for each GPU used in Patch, Full-frame and Local Motion Correction jobs

6

CRYOSPARC_MOTION_CORRECTION_RAM_MB_PER_GPU

How much RAM (in MB) to allocate for each GPU used in Patch, Full-frame and Local Motion Correction jobs

15000

CRYOSPARC_PROJECT_DIR_PREFIX

The prefix to add to a project directory name on the filesystem when it is created.

CS-

CRYOSPARC_SLACK_WEBHOOK_URL

If set, CryoSPARC makes an HTTP request to this URL each time a job's status changes with some job metadata encoded in JSON

-

CRYOSPARC_SSD_CACHE_LIFETIME_DAYS

Whenever a job requires SSD cache, it automatically checks for and removes files that haven't been accessed in more than the number of days specified by this variable. Note: Files may remain on the SSD for longer than this amount since they only get cleaned up when a job runs

30

CRYOSPARC_DISABLE_EXTERNAL_REQUESTS

Set to true to prevent the application from requesting external HTTPS resources used to display information modules on the homepage.

-

REQUESTS_CA_BUNDLE

May be required for connection to the CryoSPARC license verification server on systems with outdated Certificate-Authority files or that filter HTTPS requests through a proxy. Specify a path to a file or folder that contains the certificates.

-

CRYOSPARC_HEARTBEAT_SECONDS

CryoSPARC jobs running on worker nodes regularly report their status to the master command server to indicate that they are still running and active. If a job fails to report for more than this number of seconds (e.g., due to stalling, a slow network or a silent error), CryoSPARC marks the job as failed. Increase for very busy/low-resource worker nodes or slow/unreliable connections between the master and worker nodes. Increasing may reduce heartbeat-related job failures.

180

Note: This file includes the following environment variables that are specified at installation time:

  • CRYOSPARC_LICENSE_ID

  • CRYOSPARC_MASTER_HOSTNAME

  • CRYOSPARC_DB_PATH

  • CRYOSPARC_BASE_PORT

cryosparc_worker/config.sh

No restart is required after changing this file.

Variable
Description
Default Value

CRYOSPARC_CACHE_NUM_THREADS

2

CRYOSPARC_CACHE_LOCK_STRATEGY

(v4.5+) Distributed locking strategy to use when multiple running jobs access the SSD cache simultaneously. Set to file to use a file-system POSIX lock. Set to master to use the CryoSPARC master as a broker. Use the master strategy for caches on distributed file systems such as GPFS and BeeGFS where POSIX locks are disabled or unavailable.

file

CRYOSPARC_CLIENT_TIMEOUT

How many seconds to wait for network requests between CryoSPARC worker/master processes before timing out with an error

300

CRYOSPARC_IMPROVED_SSD_CACHE

Use the new, more reliable SSD cache system (v4.4+)

true

CRYOSPARC_IO_URING

Set to false to force-disable io_uring for fast disk read operations during jobs. May be required for some older operating systems or systems with an incorrect io_uring implementation.

true

CRYOSPARC_NO_PAGELOCK

By default, CryoSPARC uses the CUDA driver's pagelocked_empty function to allocate GPU memory. This causes CUDA-driver errors on some systems. Set this variable to true to use numpy.empty

false

CRYOSPARC_SSD_PATH

Set this variable to override the SSD cache path provided when you installed the worker. Useful if the SSD cache path is generated by your cluster as an environment variable when the job is scheduled

-

CRYOSPARC_TIFF_IO_SHM

(available in CryoSPARC versions 3.2 through 4.5.3) When reading TIFF or EER movie files, CryoSPARC reads the entire file into memory before decompressing. This significantly improves performance on some networked file systems but uses more memory. Set to false to perform decompression directly from disk

true

Note: This file includes the following environment variables that are specified at installation time:

  • CRYOSPARC_LICENSE_ID

  • CRYOSPARC_CUDA_PATH

  • CRYOSPARC_USE_GPU

PreviousManagement and MonitoringNext(Optional) Hosting CryoSPARC Through a Reverse Proxy

Last updated 5 months ago

Set to true to run the script specified in CRYOSPARC_LIVE_DATA_MANAGEMENT_SCRIPT_PATH every time a CryoSPARC Live session's data management status changes. See

The path to a bash script that runs every time a session's data management status changes: See . Use this to perform archive or deletion processes specific to your system (e.g., move archive data to an S3 bucket). The script is given arguments for the project UID, session UID, the datatype that changed (micrographs, raw, particles, metadata, or thumbnails), and the status of that datatype (active, archiving, archived, deleted, deleting or missing)

(v4.3+) Number of threads to use during caching when copying particle .mrc files from the project directory to the cache directory. Set to 1 to disable threading and copy files sequentially.

Live Session Data Management Tutorial
Live Session Data Management Tutorial
Leveraging Multiple Threads to Copy Particles