CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • Introduction
  • Topaz License
  • Installing Topaz
  • Create an conda Python environment
  • Find the Topaz Executable Path
  • (Optional) Create a topaz.sh wrapper script
  • (Optional) Set Topaz executable path as project-level default parameter
  • Verify Topaz Installation
  • Job Types
  • Topaz Deep Picking Tutorials
  1. Processing Data in cryoSPARC
  2. All Job Types in CryoSPARC
  3. Deep Picking

Topaz (Bepler, et al)

Overview of the Topaz wrapper available through CryoSPARC.

PreviousJob: Deep Picker Train and Job: Deep Picker InferenceNextT20S Proteasome: Topaz Particle Picking Tutorial

Last updated 4 months ago

Introduction

The wrapper in CryoSPARC incorporates deep learning models used in Topaz to automatically pick particles with a set of previously-picked particles or to denoise micrographs. The wrapper consists of four jobs:

  • Topaz Train

  • Topaz Cross Validation

  • Topaz Extract

  • Topaz Denoise

Use the first three jobs for particle picking. Use final job for micrograph denoising.

Topaz is a particle detection tool created by Tristan Bepler, Alex J. Noble and team:

Bepler, T., Morin, A., Rapp, M. et al. Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. Nat Methods 16, 1153–1160 (2019) doi:10.1038/s41592-019-0575-8

Bepler, T., Noble, A.J., Berger, B. Topaz-Denoise: general deep denoising models for cryoEM. bioRxiv 838920 (2019) doi: <https://doi.org/10.1101/838920>

Topaz License

Structura Biotechnology Inc. and CryoSPARC do not license Topaz nor distribute Topaz binaries. Please ensure you have your own copy of Topaz licensed and installed under the terms of its .

Installing Topaz

CryoSPARC requires a Topaz installation in a dedicated conda environment. The steps below are based on and were modified in December 2024 to force installation of Topaz v0.2.5 and compatible libraries. Newer versions of Topaz are available, but may not be compatible with the Topaz wrapper in CryoSPARC.

Create an conda Python environment

Do not use the conda Python installed with CryoSPARC. This installation is destroyed during CryoSPARC updates.

Important considerations for Master/Worker or Cluster installations:

  • The path to the Anaconda installation on the machine hosting cryosparc_master must exactly match the path on machines hosting cryosparc_worker

  • The Anaconda installation directory must be accessible by the CryoSPARC Linux user account with the required permissions for executing the topaz binary

Create and activate aconda environment named topaz environment with the following commands:

conda create -n topaz python=3.6
conda activate topaz  # changes to the topaz conda environment

For Topaz 0.2.5, thecondaenvironment should be configured with Python 3.6, regardless of the versions of other Python installations on the system.

Once the conda environment has been created and activated, run the following command to install Topaz:

conda install topaz=0.2.5 mkl=2024.0.0 -c tbepler -c pytorch -c conda-forge

After running the preceding command, with the condaenvironment still active, the command

topaz --version

should print TOPAZ 0.2.5a to the terminal.

Find the Topaz Executable Path

Once Topaz is installed and the conda environment is active in your current shell, enter the following command to determine the full path to the topaz binary:

which topaz

The output should look similar to this:

/home/cryosparcuser/anaconda3/envs/topaz/bin/topaz

In the CryoSPARC interface, specify this as the value for the "Path to Topaz executable" parameter:

If using this path results in errors when running a Topaz job (often with a message such as "topaz did not produce valid output"), try aliasing Topaz with a shell script that also activates the correct conda environment. This process is described in the next section.

(Optional) Create a topaz.sh wrapper script

CryoSPARC has its own conda installation and environment, which may conflict with the Topaz environment. To prevent this, create a shell script that deactivates the CryoSPARC environment and activates the topaz one.

Create a topaz.sh file in a well known location such as the home directory (e.g., ~/topaz.sh). Add the following contents, making the noted substitutions:

#!/usr/bin/env bash
if command -v conda > /dev/null 2>&1; then
    conda deactivate > /dev/null 2>&1 || true  # ignore any errors
    conda deactivate > /dev/null 2>&1 || true  # ignore any errors
fi
unset _CE_CONDA
unset CONDA_DEFAULT_ENV
unset CONDA_EXE
unset CONDA_PREFIX
unset CONDA_PROMPT_MODIFIER
unset CONDA_PYTHON_EXE
unset CONDA_SHLVL
unset PYTHONPATH
unset LD_PRELOAD
unset LD_LIBRARY_PATH

source $HOME/anaconda3/etc/profile.d/conda.sh
conda activate topaz
exec topaz $@
  • Substitute $HOME/anaconda3 on line 17 with the Anaconda or Miniforge installation directory.

Make this file executable by the CryoSPARC user from the command line

chmod +x topaz.sh

In the CryoSPARC interface, specify the full path to topaz.sh as the "Path to Topaz executable" parameter:

(Optional) Set Topaz executable path as project-level default parameter

To avoid having to locate and set the Topaz executable path when building every Topaz job, in v4.0.2 onwards you can set a project-level default that will apply to all newly created Topaz jobs.

Navigate to the projects view, select a project and choose the Topaz executable path under the 'Project Level Parameters' module within the sidebar details panel:

Verify Topaz Installation

wget http://bergerlab-downloads.csail.mit.edu/topaz/topaz-tutorial-data.tar.gz
tar -xzvf topaz-tutorial-data.tar.gz
mkdir -p data/EMPIAR-10025/processed
mkdir -p data/EMPIAR-10025/processed/micrographs
/path/to/topaz preprocess -v -s 8 -o data/EMPIAR-10025/processed/micrographs/ data/EMPIAR-10025/rawdata/micrographs/*.mrc
/path/to/topaz convert -s 8 -o data/EMPIAR-10025/processed/particles.txt data/EMPIAR-10025/rawdata/particles.txt
  • Substitute /path/to/topaz with the full path to be used for CryoSPARC jobs

  • For the preprocess command, specify one of

    • --device n where n is a specific GPU number to run the test on

    • --num-workers n where n is the number of processes to use

Job Types

Topaz Deep Picking Tutorials

Use a previously-installed Anaconda Python (3.6+), or . Alternatively, provides the required conda functionality.

Log out of the current command shell and log in again to ensure no conda environment is active. Run the following commands to verify that the Topaz Installation is working correctly. These are adapted from the . Note the substitutions below.

Topaz
GNU General Public License v3.0
Topaz developers' instructions
install new one
Miniforge3
Topaz Quick start guide
Job: Topaz Train and Job: Topaz Cross Validation
Job: Topaz Extract
Job: Topaz Denoise
T20S Proteasome: Topaz Particle Picking Tutorial
T20S Proteasome: Topaz Micrograph Denoising Tutorial
You can use the input field or file browser to enter a path to the Topaz executable.