CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • From microscope to structure, in minutes
  • Who is CryoSPARC Live built for?
  • Use cases for CryoSPARC Live
  • Live processing at microscope, during data collection
  • Live processing offsite, during data collection
  • Seamless first cut processing of previously-collected data
  • What does CryoSPARC Live enable?
  • Continuous import and expedited preprocessing
  • Adjustable parameters and saveable configurations
  • Streamlined exposure curation (threshold-based and manual)
  • Ability to test and refine picking strategies while collection is ongoing
  • Make go/no go decisions about a sample using Streaming 2D classification
  • 3D reconstruction and Streaming 3D refinement during data collection
  • Add or free up compute resources during a Session
  • Export of results and integration with cryoSPARC
  • Programmatic control of CryoSPARC Live
  • What users say
  • History and development
  • Embedded CryoSPARC Live
  • Get Started
  1. Real-time processing in cryoSPARC Live

About CryoSPARC Live

From microscope to structure, in minutes. The fastest, streamlined path to high-resolution structures, during data collection or afterwards.

PreviousWebinar RecordingsNextPrerequisites and Compute Resources Setup

Last updated 2 years ago

From microscope to structure, in minutes

CryoSPARC Live is a software platform that enables:

  • Real-time cryo-EM data quality assessment

  • Decision making based on 2D and 3D results during live data collection

  • An expedited, streamlined workflow for processing previously collected data

  • Direct seamless interoperation with CryoSPARC for advanced processing

CryoSPARC Live is built to enable experimentation with parameters on the fly, while the software efficiently manages the reprocessing that is necessary in order to test or effect parameter changes, and while maintaining the overall progress of the Live session.

Who is CryoSPARC Live built for?

CryoSPARC Live is built for:

  • Data collection facilities, cryo-EM cores and and microscope operators, who want to make the most of microscope time with real-time data quality assessment and collection;

  • Facilities and service providers, who want to provide their users with expedited information about sample quality and/or delivery of 3D maps; and

  • Individual users, who wish to gain insights about data quality by performing 3D reconstructions in real-time with data collection, or on previously collected data in a seamless and "first cut" manner.

Use cases for CryoSPARC Live

Live processing at microscope, during data collection

CryoSPARC Live can be used by microscope operators during a collection session. We recommend the exposures are written to fast disks as I/O can become a bottleneck.

Live processing offsite, during data collection

CryoSPARC Live can be engaged to read files that are being uploaded to the user, e.g., via AWS or other similar service. Live will read in new exposures as they are found.

Seamless first cut processing of previously-collected data

Finally, we recommend using the Live workflow for expedited preprocessing (see below) and a first-cut look at data quality in 2D and 3D, for all datasets including those already collected.

What does CryoSPARC Live enable?

Continuous import and expedited preprocessing

CryoSPARC Live watches specified directories for new files and processes them as they become available. CryoSPARC Live preprocessing includes four steps: motion correction, CTF estimation, particle picking and extraction. CryoSPARC Live can sustain a throughput of 450 or more exposures per hour, per GPU, for K3 data. On a 4-GPU machine, that can scale to 1800+ exposures per hour! For K2 or Falcon data, performance can be even higher, upwards of 650 exposures per hour per GPU.

Particles from preprocessing are seamlessly transitioned into 2D classification and 3D reconstruction, and preprocessed exposures can also be exported for further processing in CryoSPARC or other software.

For extensive benchmarks and throughput statistics, please see:

Adjustable parameters and saveable configurations

  • Exposure groups: It is possible to add, remove, and ignore exposures from one or more Exposure Groups (collections of exposures with the same optical parameters). This makes it possible to process subsets of a larger dataset and make comparisons.

Learn more about Configuration Profiles in the UI Overview:

Streamlined exposure curation (threshold-based and manual)

Users can review several useful statistics (e.g., CTF Fit, Defocus, Total Motion, etc) across a dataset and apply thresholds to automatically reject incoming and already-processed exposures that do not fall within the desired range for one or more parameters. Exposures can also be rejected manually. Rejected exposures are excluded from further processing unless thresholds are changed, which can be done at any time during a Live session. On export, exposures are split into different output groups (e.g., rejected and accepted) for further advanced processing outside of the Live workflow.

Ability to test and refine picking strategies while collection is ongoing

It is possible to have finalized a picking strategy within the first few hundred exposures, which can then be left to run on the remaining exposures as they come in. Blob-based picking is active by default at the start of any Live session and manual picking can be engaged at any time. Blob/manual picks can be curated and used to generate templates for template-based picking, or available templates can be loaded directly into the Session. Pick scores (NCC and Power Threshold) can be used to include or exclude picks. These and particle extraction parameters can be tested on a single or few exposures before being applied to the entire dataset, and can be updated as many times as necessary. Updated particle locations will be fed automatically into later stages of the pipeline (e.g., Streaming 2D Classification).

Make go/no go decisions about a sample using Streaming 2D classification

Real time streaming 2D classification enables assessing sample quality, preferred orientation issues and presence/absence of the expected target and/or ligands, as well as large conformational variability.

2D Classification in CryoSPARC Live picks up newly extracted particles from incoming exposures and automatically updates 2D classes every few minutes using a new streaming method. This means that after starting a Streaming 2D Classification job, class averages will update automatically as new particles become available from upstream. Class averages can be selected as soon as the initial classification is complete, and these selections will be retained, enabling Streaming 3D Refinement to take in new particles from the selected classes and update the reconstruction in real time.

Early feedback in 2D (and 3D, see below) can confirm whether a collection should continue, or whether upstream steps such as sample preparation may require improvement. Within a few hours of starting a Live session, it is possible to make a "go/no go" decision about the sample and assess issues that may result in a poor 3D reconstruction. Thus, it is possible to save on microscope time, or, to at least inform the user ahead of time about the quality of result they may be able to expect.

Learn more about CryoSPARC Live Jobs and Session-Level Functions:

3D reconstruction and Streaming 3D refinement during data collection

The ability to generate a refinement during data collection is important as a diagnostic during collection, as a first-cut structure from which to continue further processing, and in many cases is comparable to the highest resolution structure(s) that can be generated after extensive advanced processing in CryoSPARC. Streaming 3D Refinement in CryoSPARC Live also will pick up newly available particles from Streaming 2D Classification so that over the course of a collection, the 3D structure updates every few minutes.

For a detailed walkthrough of setting up your own Live session, please see:

Add or free up compute resources during a Session

Over the course of a CryoSPARC Live Session, it is possible to adjust the compute resources dedicated to processing. The Number of Preprocessing GPU Workers can be increased or decreased during a session in order to free up compute resources or add more parallelization capacity for preprocessing. Additionally, the compute lanes being used can be adjusted.

Learn more about GPU requirements for Live:

Export of results and integration with cryoSPARC

CryoSPARC Live is tightly integrated with CryoSPARC. Each CryoSPARC Live Session is housed within a CryoSPARC Project, so the results of live processing can always be used seamlessly for further advanced processing in CryoSPARC as well as for export.

It is common to take the final map(s), exposures and particle stack(s) from CryoSPARC Live and hand them off directly to e.g., users of a microscope facility or lab, or to use the motion-corrected, CTF-estimated exposures directly for advanced processing without redoing preprocessing steps.

Programmatic control of CryoSPARC Live

CryoSPARC Live Sessions and jobs can also be controlled via the command line:

What users say

"CryoSPARC Live enables the Pacific Northwest Cryo-EM Center to keep up with data coming from all 5 of our microscopes, a data volume that has at times exceeded 15TB (30,000+ images) per day. We can monitor not only standard image pre-processing metrics, but also directly observe improvements to 3D maps concurrent with data collection! In many cases we’ve stopped data collection with a 2.2-3.5Å map already in hand. We can also quickly identify pathologies such as poor orientational distributions, or tune imaging conditions to improve alignment of difficult particles."

- Craig Yoshioka, Center Co-Director, Pacific Northwest Center for Cryo-EM (PNCC)

In our facility, CryoSPARC Live has completely revolutionised the way we collect and process cryoEM data. Initially, we commissioned CryoSPARC Live with the primary goal of feedback on micrograph quality through on-the-fly motion correction and CTF estimation. But it has turned out to be so much more than we hoped for. We now use it for all initial processing of data coming through our facility. Our latest GPU workstation was specifically specced out for rapid CryoSPARC processing as it has become our primary in-house processing tool. Some of the aspects we love about CryoSPARC Live include: the GUI is intuitive, easy to use, and easy to learn for new users; processing is extremely fast; parameter changes are propagated throughout the workflow in a streaming fashion; easy linkages with ‘regular’ CryoSPARC make continued processing simple; picking tools are easy to use and work for almost all targets, and micrograph curation is powerful and easy to apply. CryoSPARC has become such an essential part of our data collection and analysis workflow; we could not imagine working without it. Highly valuable microscope time is now used much more efficiently and our microscope users are very satisfied.

- Simon HJ Brown, PhD, Customer Solutions Expert, Cryo Electron Microscopy - Molecular Horizons, University of Wollongong

"CryoSPARC live may literally make your jaw drop. The speed of processing combined with the capability to make adjustments in real time are remarkable. It allows you to make go/no go decisions for each experiment using 2D and 3D results generated almost as fast as the images can be collected. Perhaps most impressive is the ability to go back and change almost any parameter on the fly, and it will immediately reprocess the images from that step, and key metrics such as motion and CTF fit can be examined in interactive plots where ranges can be set and immediately applied for image curation. Together these features take a large step toward the future in reducing both the time and resources that are needed to achieve a successful outcome for each data collection."

- Jeff Speir, Director of Operations, NanoImaging Services Inc. (NIS)

"My group and I have been using CryoSPARC Live beta for a little over a year with great successes. We currently only run CryoSPARC Live for our in house projects and it has been of great help. It is easy to set up, very easy to use and very fast. There is no need to create scripts or any slightly complicated task. On most projects with very limited human interaction and no knowledge of the target of interest we can reach high resolution overnight before the run on the microscope is finished. The CryoSPARC-live features allow us to monitor the quality of the acquisition and make modifications in real time if necessary, whether it is a sample problem (e.g. ice thickness) or an issue with the alignment of the microscope. We are looking forward to extend the use to the core facility users allowing them to get real time feedback on their runs."

- Eric Hanssen, Head, Advanced Microscopy Facility and Associate Professor, University of Melbourne

"CryoSPARC Live is a wonderful tool that not only gets researchers excited about their cryoEM experiments but lets microscope operators know they are acquiring high-quality processable data. With on-the-fly feedback users and staff are able to engage with each other to identify bottlenecks and modify data collection strategies in real-time to conduct optimized experiments for a sample. 3D feedback is critical because it ensures we are able to collect a full dataset of their macromolecule of interest, thereby allowing our users to accelerate their biomedical research."

- Edward Eng, Manager, New York Structural Biology Center (NYSBC)

"Having used CryoSPARC Live at both New York Structural Biology Center (USA) and The Hospital for Sick Children (Canada), it has become an invaluable tool in real time assessment of samples for a frequent user like me. By having the movie frames aligned, CTF estimated, particles picked, 2D classification and ab initio done on the fly allows me to quickly judge which grid/sample is worth collecting on. As all the statistics can also be neatly presented as an overview, it is straightforward too to pick out trends and exceptions in the data: For instance I have noticed cases where a bunch of micrographs had poor estimated resolution, only to pinpoint the problem to a single grid square, allowing me to avoid similar grid squares subsequently. The final big advantage of CryoSPARC Live is that the resulting data can easily be passed onto the conventional CryoSPARC pipeline for further processing – saving both time and energy!"

- Yong Zi Tan, Postdoctoral Fellow, The Hospital for Sick Children

History and development

CryoSPARC Live was first released as a private beta in May 2019. Based on extensive beta testing at dozens of facilities and labs globally, we have incorporated feedback and worked to improve the workflow, with several iterations already released. CryoSPARC Live will continue to evolve with advancements in data collection, user feedback and automation of the cryo-EM workflow.

Embedded CryoSPARC Live

Get Started

Microscope/camera and job parameters: Any of these can be adjusted over the course of a session, and combinations of parameters can be saved as "" for use in future sessions. For example, microscope parameters such as the pixel size, spherical aberration and accelerating voltage are likely to be consistent for a given instrument and can be applied quickly using Configuration Profiles at the start of a new Session. If working on multiple samples of the same protein or complex, it may be useful to save picking and extraction parameters to save time.

In 2022, we collaborated with Thermo Fisher Scientific Inc. to make available Embedded CryoSPARC Live, a version of CryoSPARC Live that is designed to seamlessly integrated with Thermo ScientificTM cryo-transmission electron microscope systems. To learn more about the collaboration, please visit:

Performance Metrics
UI Overview
Live Jobs and Session-Level Functions
New Live Session: Start to Finish Guide
Prerequisites and Compute Resources Setup
Managing a CryoSPARC Live Session from the CLI
https://cryosparc.com/embedded-cryosparc-live
Prerequisites and Compute Resources Setup
How to Access cryoSPARC Live
New Live Session: Start to Finish Guide
FAQs and Troubleshooting
Processing EMPIAR-10288 in cryoSPARC Live.
Configuration Profiles