CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • Step 1 - Preprocess Data
  • Step 2 - Create Denoising Job
  • Step 3 - Create Denoising Job for Training
  • Step 4 - Create Denoising Job for Newly Trained Model
  • Next Steps
  1. Processing Data in cryoSPARC
  2. All Job Types in CryoSPARC
  3. Deep Picking
  4. Topaz (Bepler, et al)

T20S Proteasome: Topaz Micrograph Denoising Tutorial

Topaz denoising tutorial via the Topaz wrapper available in CryoSPARC.

PreviousT20S Proteasome: Topaz Particle Picking TutorialNextJob: Topaz Train and Job: Topaz Cross Validation

Last updated 2 years ago

Step 1 - Preprocess Data

  • Preprocess the T20S subset by completing steps 1-12 in the tutorial found in the Cryo-EM Data Processing in cryoSPARC: Introductory Tutorial section found here:

  • Ensure that the Import Movies, and CTF estimation jobs from the linked tutorial are completed as they will be required for the Topaz Denoise job. The ouputs from these two jobs will be used as inputs for the denoising job.

Step 2 - Create Denoising Job

  • Select Topaz Denoise (BETA) from the Job Builder. Drag and drop the exposures_success output from the completed CTF estimation job into the micrographs input.

  • Use the file browser (activated by clicking the folder icon) to locate the Topaz executable path found earlier for the Path to Topaz Executable field. Instructions on how to find the Topaz executable path can be found above.

  • Queue the job.

  • The job is using the provided pretrained Topaz denoising model on the subset of 20 micrographs from the T20S tutorial.

  • Once the job is completed, observe the micrograph images outputted in the log. Depending on the value of the "Number of plots to show" parameter, the job will show side-by-side micrograph comparisons where the left side features the original micrograph and the right side features the denoised micrograph. This helps determine if the denoised micrographs will be in picking or other related tasks.

Step 3 - Create Denoising Job for Training

  • Select Topaz Denoise (BETA) from the Job Builder. Drag and drop the exposures_success output from the completed CTF estimation job and the imported_movies output from the completed Import Movies job into the micrographs and training_micrographs inputs respectively.

  • Use the file browser to locate the same Topaz executable path used in step 2. See Step 2.2 for more details.

  • Queue the job.

  • The job is training a new Topaz denoising model on the subset of 20 micrographs from the T20S tutorial. Once the training is complete, it will use the model to denoise the input micrographs. When the denoising is completed, the job will output both the denoised micrographs and the newly trained model.

  • As done in step 2.5, observe the shown micrograph comparisons between the original and denoised micrographs.

  • When training a new model, the job will also output plots of the training and validation loss. The plots for both losses should be descending overtime. If the plot for the training loss is decreasing while the plot for the validation loss is increasing, this indicates that the model has overfit and training parameters must be tuned. The simplest approach to resolving overfitting is to reduce the learning rate.

  • The job will output denoised micrographs that barely look denoised. That is because the training data is the exact subset of data that is being denoised. When a greater variety of training data is used to train a model, the denoising will be much more noticeable.

Step 4 - Create Denoising Job for Newly Trained Model

  • Select Topaz Denoise (BETA) from the Job Builder. Drag and drop the exposures_success output from the completed CTF estimation job and the topaz_denoise_model output from the completed Topaz Denoise (BETA) job from step 3 into the micrographs and denoise_model inputs respectively.

  • Use the file browser to locate the same Topaz executable path used in step 2. See Step 2.2 for more details.

  • Queue the job.

  • The job is used the previously trained Topaz denoising model on the same subset of 20 micrographs. This step serves to demonstrate how to use trained Topaz denoising models. When using trained models outside of this tutorial, the input micrographs should be different from those used to trained the model.

  • As done in step 2.5, observe the shown micrograph comparisons between the original and denoised micrographs. The output denoised micrographs should be nearly identical to those from step 3 as the model is denoising the same micrographs using the same model as the job from step 3.

Next Steps

Denoised micrographs mainly serve to improve particle picking using Manual Picker and deep learning pickers such as the Topaz particle picker. Denoising micrographs have no impact on preprocessing methods such as Motion Correction and CTF Estimation nor do they affect the performance of other pickers such as Blob Picker and Template Picker. However, the denoised micrographs can serve to help visualize particles in the Inspect Particle Picks job after using any of the pickers including the aforementioned Blob Picker and Template Picker.

To observe this functionality, continue the T20S tutorial until step 9 until step 9. At step 9 of the linked tutorial, pass the denoised micrographs from step 2 of this tutorial instead of the micrographs output from the Template Picker. The T20S tutorial can be found here:

Get Started with CryoSPARC: Introductory Tutorial (v3)
Passing inputs to the Topaz Denoise job
Side-by-Side Micrograph Denoising Comparison
Passing training inputs to the Topaz Denoise job
Topaz Denoising Loss Plot
Passing pretrained inputs to the Topaz Denoise job