CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • At A Glance
  • Description
  • Inputs
  • Source movies (optional)
  • Commonly Adjusted Parameters
  • Micrographs data path
  • Length of prefix to cut
  • Microscope parameters
  • Negative stain data
  • Skip header check
  • Outputs
  • Imported micrographs
  • Failed micrographs
  • Common Next Steps
  • Linking Micrographs to Input Movies
  1. Processing Data in cryoSPARC
  2. All Job Types in CryoSPARC
  3. Import

Job: Import Micrographs

PreviousJob: Import MoviesNextJob: Import Particle Stack

Last updated 1 year ago

At A Glance

Import one or more micrographs into CryoSPARC for processing.

Description

The Import Micrographs job imports single-frame micrographs into CryoSPARC for downstream processing. Typically, these micrographs will have been pre-processed through motion correction tools outside of CryoSPARC, or come from a camera that does was imaging single frames rather than movies.

Importantly, the micrograph files that are imported by this job are not copied — they are merely linked (via a symbolic link) into the CryoSPARC project directory where this job is running. It is thus critical that the micrograph files are not deleted or moved while they are being processed in CryoSPARC.

The Import Micrographs job accepts an optional input of movies (e.g., from an Import Movies job). This links the micrographs (and therefore, particles extracted from the micrographs) to their source movies, which allows for workflows that require movies such as Reference Based Motion Correction. Thus, although this input is optional, we recommend connecting the movies if they are available.

Inputs

Source movies (optional)

Imported movies from an Import Movies job. If jobs which require movies (such as ) will be used later, this input should be connected so that resulting particles are properly linked to their movies. See the for more information.

Commonly Adjusted Parameters

Micrographs data path

The absolute path to the micrographs to import (in MRC format). Wildcards can be used to select multiple micrographs.

Length of prefix to cut

Microscope parameters

Raw pixel size and Total exposure dose are typically selected during data collection, while Accelerating voltage and Spherical aberration are properties of the microscope. It is best to specify these properties if they are known. Note that the pixel size of the micrographs may not be the physical detector pixel size if the original movies were downsampled during motion correction or if they were collected in a super-resolution mode.

Negative stain data

If Negative Stain Data is on, this indicates that there are light particles on dark background (-1). If it's off, this indicates the micrographs have dark particles on light background (cryo-em data, +1).

Skip header check

Turned on by default in v4.2+, off by default in prior versions.

When Skip header check is turned off, each micrograph file's header is read by the job to ensure that all micrographs are of the same size in pixels and physical extent. The header check helps to detect corrupt files which otherwise may cause errors in downstream jobs, but also can take a long time due to the number of file system operations needed to read the headers.

When this parameter is turned off (i.e., when the header check is used), set the Number of CPUs to parallelize during header check parameter to parallelize reading of exposure headers.

Outputs

Imported micrographs

The imported micrographs are ready for further processing.

Failed micrographs

If Skip Header Check is off, micrographs with an inconsistent size are output here and are typically not used in downstream processing.

Common Next Steps

Linking Micrographs to Input Movies

In the scenario where you want to link your micrographs to a movie dataset that is already in CryoSPARC, you can connect the movies to the inputs of this job.

Since CryoSPARC does not know how the micrograph names correspond to movie names, you will have to help the job connect the correct movie and micrograph. Most motion correction packages add characters to the beginning and/or end of a filename, but leave some identifying region in the middle of the filename unchanged. The Import Micrographs job thus provides four parameters, each of which accepts a number telling the job how many characters to trim from the beginning and end of the micrograph and movie filenames to be left with this identifying region.

Consider the following case:

Importing movies from /bulk8/data/rposert-dev/CS-rposert-guide-work/J4/motioncorrected/*.mrc
Importing 24 files
Attempting to find corresponding filenames in imported micrographs and connected input movies..
  Example source (input) movie filename:
 003573716055357609725_CB1__00006_Feb18_23.39.55.tif
  Example query (imported) micrograph filename:
 001334752721923690937_CB1__01978_Feb20_11.29.13_background.mrc

In these lines from the Import Micrographs job log, we see that the movies have a unique ID (twenty-one numbers) added to the front, and they end in .tif. The micrographs also have a (different) twenty-one character ID, but end in _background.mrc. We therefore need to cut twenty-one characters off the front of both the movies and the micrographs, four characters off the end of the movies, and fifteen characters off the end of the movies.

The Import Micrographs job displays the first filename in alphabetical order in the job log. Since many programs add a unique ID to the beginning of the filename during import and motion correction, the filenames displayed in the log will likely not have any part of their filename in common. You are looking for the part of the filename which would match for a movie derived from an imported micrograph.

The parameters for the job should therefore be:

Length of movie path prefix to cut : 21
Length of movie path suffix to cut : 4
Length of mic. path prefix to cut  : 21
Length of mic. path suffix to cut  : 15

Now the imported micrographs will be linked to the movies in CryoSPARC, allowing you to come back to preprocessing steps easily.

These parameters are used to properly link micrographs to their input movies. See the section for help setting these parameters.

Imported micrographs are typically taken into a job to estimate their contrast transfer function parameters.

Patch CTF
Reference Based Motion Correction
relevant section
Linking Micrographs to Input Movies