CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • At a Glance
  • Description
  • Inputs
  • Movies
  • Commonly Adjusted Parameters
  • Only process this many movies
  • Low-memory mode
  • Save results in 16-bit floating point
  • Output denoiser training data
  • Num. movies for denoiser training data
  • Output F-crop factor:
  • Start frame and End frame
  • Override knots X
  • Outputs
  • Micrographs
  • Micrographs incomplete
  • Common Next Steps
  • Method Details
  • Example Workflow
  • References
  1. Processing Data in cryoSPARC
  2. All Job Types in CryoSPARC
  3. Motion Correction

Job: Patch Motion Correction

PreviousMotion CorrectionNextJob: Full-Frame Motion Correction

Last updated 4 months ago

At a Glance

Model full-frame and anisotropic motion in movies, and apply dose weighting, to produce motion-corrected micrographs.

Description

During movie collection, complex 3D deformations of the ice occur. When projected into a flat image, these 3D deformations appear as anisotropic motions of the ice, in which different regions of the image move in different directions. See e.g., (Thorne, 2020) for more details about the mechanism of anisotropic beam-induced motion.

Patch Motion Correction models this anisotropy by tiling a movie into patches and estimating the displacement of each patch in each frame under a motion model that is smooth over both space and time. It then corrects for the estimated motion by shifting each pixel based on its modeled displacement in each frame. Similar approaches are used in software such as MotionCor2 (Zheng et al. 2017) and Warp (Tegunov et al. 2018). Patch motion correction also applies .

Inputs

Movies

This job requires raw movies in .mrc , .tif , .eer, or .mrc.bz2 format, typically from an Import Movies job.

Commonly Adjusted Parameters

Only process this many movies

This parameter selects n movies, randomly to process. It is most useful when working with a subset of movies to assess data quality before committing to a full processing pipeline. If this is left blank, all movies are processed.

To select the same movies every time, set the Random seedparameter to a constant value.

Low-memory mode

Reading movie files from disk is slow. To speed up processing, motion correction jobs process one movie and load the next simultaneously. However, this can lead to GPUs with low memory (i.e., < 16 GB VRAM) to run out of memory. Turning this option on causes the job to wait to load the next movie until it has finished processing the current movie. This slows down the job, but can prevent out-of-memory errors.

Save results in 16-bit floating point

Output denoiser training data

Num. movies for denoiser training data

Output F-crop factor:

Reduce the size of the output, aligned micrographs by this factor. If the F-crop (Fourier-crop) factor is 1.0, no cropping is performed. A crop factor of 0.5 downsamples output micrographs by half, discarding the highest-frequency half of the Fourier components, etc. This significantly reduces the file size of the final micrographs, but limits the maximum resolution possible for resulting reconstructions.

Start frame and End frame

These parameters allow discarding frames from the start and end of the input movies, before motion correction is applied. This can be helpful in case there are artefacts or issues with early or late frames. For example, some cameras can have issues where they output one or more blank frames at the end of a movie, and these can disturb motion estimation.

Override knots X

Importantly, the knot parameters do not change the number of patches or size of patches used by Patch Motion Correction. Patches are 500 Å wide and cannot be resized by the user.

Outputs

Micrographs

Micrographs incomplete

Micrographs which were not successfully corrected are output in this group. Typically this is due to some error specific to the movie — the Event Log should contain more information on why a particular movie failed.

Common Next Steps

Method Details

Patch Motion Correction progresses through three main steps for each movie: rigid motion estimation, anisotropic motion estimation, and motion correction.

Next, the micrograph is split into overlapping 500 Å patches. In order to model anisotropic motion (i.e. deformation of the sample), the motion trajectories of all patches are jointly optimized to produce the highest degree of correlation across all frames of the movie. In this optimization setup, for each patch, each frame (denoted by a Z dimension representing time) is allowed to shift in both X and Y directions.

The optimization of patch trajectories is constrained using a spline function. The number of knots (i.e. degrees of freedom) in the spline function is automatically selected by CryoSPARC based on the magnification and dose rate of the input movies, and the spline serves to restrict the possible patch trajectories that the optimization can model to ensure that they are smooth over both space and time. Furthermore, after optimization, the spline function allows interpolation of displacement for each pixel, rather than only at the patch level.

As mentioned, the number of spline knots control the number of degrees of freedom used by the motion model in each of the three dimensions (X, Y, and time). A small number of knots causes the model to be smooth, which protects from overfitting but may also prevent the spline functions from fully capturing the dynamics of patch movement. A large number can have the opposite effect. CryoSPARC automatically selects the number of knots.

The full motion model, optimized using data across all frames, provides an estimate of the anisotropic motion during the entire movie:

which in turn can be used to model each pixel’s trajectory through time and space:

After motion estimation, the final spline function provides the optimal prediction of the shift of any given pixel for each frame. This prediction is used to shift each pixel in each frame by the modeled displacement to produce a single, motion corrected averaged micrograph from which particles can be picked and extracted. Note that this is distinct from other motion correction jobs, like Local Motion Correction, that model and correct the motion of individual particles and produce extracted particles rather than a complete micrograph.

Example Workflow

References

  1. Zheng, S. Q. et al. MotionCor2: anisotropic correction of beam-induced motion for improved cryo-electron microscopy. Nature Methods 14, 331–332 (2017).

Save the output micrographs in 16-bit floating point. We recommend that this option is turned on to save space at minimal loss of accuracy. See the for more information.

When this parameter is turned on, Patch Motion Correction will produce the even and odd half-micrographs necessary to train the . Note that these half-micrographs do take up a small amount of disk space — the training data pair takes up as much space as the motion-corrected micrograph. For a 4k frame saved in 16-bit format, the default 200 movie training set takes up approximately 12 GB of disk space.

This parameter sets the number of movies for which denoiser training data is produced and defaults to 200. Generally, 100 micrographs are sufficient to train the denoiser. By default, more training data is produced in case some micrographs with training data are excluded in, e.g., a job. If space is a significant concern, or if no micrographs will be excluded before training the denoiser, this parameter can be reduced to the number of micrographs you plan to use when training the denoiser.

Particle images can also be Fourier-cropped (downsampled) at the stage. It is often common to initially process downsampled particle images (to speed up particle curation and early reconstructions) and then return to full resolution particle images downstream in final refinements. This type of workflow is made easier if the motion corrected micrographs are not downsampled, and the downsampling is instead only applied at the extraction stage. As such, for a general workflow and non-super-resolution movies, we recommend using the default Output F-crop factor of 1.0 during motion correction. When working with super-resolution movies, we recommend downsampling to the physical detector pixel size (i.e., setting Output F-crop factor to 0.5 for 2x super-resolution movies). Of course, there are some datasets that benefit from the full, super-resolution pixel size, and some which benefit from more aggressive downsampling.

Spline knots control the smoothness of motion trajectories that will be optimized by Patch motion correction (see below). CryoSPARC automatically determines the number of knots in each spatial and temporal direction based on characteristics of the input movies (e.g., magnification, dose rate, etc) and it is not generally necessary to change these parameters. These parameters can be used to enforce additional smoothness on motion trajectories (setting knots to a low value) or allow patch motion correction to model additional anisotropy (setting knots to a high value) beyond its automatic settings, if needed.

Aligned micrographs for further downstream preprocessing. These micrographs also have rigid motion estimates, which are required for jobs like and .

Typically, micrographs must have their CTF estimated, via e.g., , before any further processing can occur.

Rigid motion is estimated in much the same way as . More detail is available in that article, but in short, a full-frame trajectory is iteratively optimized to produce the highest degree of correlation between all frames.

Robert Thorne, “Hypothesis for a Mechanism of Beam-Induced Motion in Cryo-Electron Microscopy,” IUCrJ, 2020, .

Tegunov, D. & Cramer, P. Real-time cryo-EM data pre-processing with Warp. bioRxiv (2018) doi:.

16-bit floating point article
Micrograph Denoiser
Curate Exposures
Extract from Micrographs
Reference-Based Motion Correction
Local Motion Correction
Patch CTF Estimation
Full-Frame Motion Correction
Tutorial: Patch Motion and Patch CTF
https://doi.org/10.1107/S2052252520002560
10.1038/s41592-019-0580-y
Method Details
Dose Weighting
An example micrograph from EMPIAR 10674.
An example of a single frame’s patch displacement. An arrow represents the displacement of a patch in a single frame. Note that the length of the arrows are scaled for visibility.
X and Y displacement models for a single frame are shown with pink and blue surfaces, respectively. Using this function, each pixel’s displacement can be modeled.
Each pixel’s displacement varies as it moves through time, and so do the spline functions.