# Topaz (Bepler, et al) ## Introduction The [**Topaz**](https://github.com/tbepler/topaz) wrapper in CryoSPARC incorporates deep learning models used in Topaz to automatically pick particles with a set of previously-picked particles or to denoise micrographs. The wrapper consists of four jobs: * Topaz Train * Topaz Cross Validation * Topaz Extract * Topaz Denoise Use the first three jobs for particle picking. Use final job for micrograph denoising. Topaz is a particle detection tool created by Tristan Bepler, Alex J. Noble and team: ``` Bepler, T., Morin, A., Rapp, M. et al. Positive-unlabeled convolutional neural networks for particle picking in cryo-electron micrographs. Nat Methods 16, 1153–1160 (2019) doi:10.1038/s41592-019-0575-8 Bepler, T., Noble, A.J., Berger, B. Topaz-Denoise: general deep denoising models for cryoEM. bioRxiv 838920 (2019) doi: ``` ## Topaz License Structura Biotechnology Inc. and CryoSPARC do not license Topaz nor distribute Topaz binaries. Please ensure you have your own copy of Topaz licensed and installed under the terms of its [GNU General Public License v3.0](https://github.com/tbepler/topaz/blob/master/LICENSE). ## Installing Topaz CryoSPARC requires a Topaz installation in a dedicated `conda` environment. The steps below are based on [Topaz developers' instructions](https://github.com/tbepler/topaz/tree/v0.2.5?tab=readme-ov-file#installation) and were modified in December 2024 to force installation of Topaz v0.2.5 and compatible libraries. Newer versions of Topaz are available, but may not be compatible with the Topaz wrapper in CryoSPARC. ### Create an `conda` Python environment {% hint style="warning" %} Do not use the `conda` Python installed with CryoSPARC. This installation is destroyed during CryoSPARC updates. {% endhint %} {% hint style="info" %} Important considerations for Master/Worker or Cluster installations: * The path to the Anaconda installation on the machine hosting `cryosparc_master` must exactly match the path on machines hosting `cryosparc_worker` * The Anaconda installation directory must be accessible by the CryoSPARC Linux user account with the required permissions for executing the `topaz` binary {% endhint %} Use a previously-installed Anaconda Python (3.6+), or [install new one](https://www.anaconda.com/). Alternatively, [Miniforge3](https://github.com/conda-forge/miniforge?tab=readme-ov-file#miniforge3) provides the required `conda` functionality. Create and activate a`conda` environment named `topaz` environment with the following commands: > ```bash > conda create -n topaz python=3.6 > conda activate topaz # changes to the topaz conda environment > ``` For Topaz 0.2.5, the`conda`environment should be configured with Python 3.6, regardless of the versions of other Python installations on the system. Once the `conda` environment has been created and activated, run the following command to install Topaz: ```bash conda install topaz=0.2.5 mkl=2024.0.0 -c tbepler -c pytorch -c conda-forge ``` After running the preceding command, with the `conda`environment still active, the command ``` topaz --version ``` should print `TOPAZ 0.2.5a` to the terminal. ### Find the Topaz Executable Path Once Topaz is installed and the `conda` environment is active in your current shell, enter the following command to determine the full path to the `topaz` binary: ``` which topaz ``` The output should look similar to this: ``` /home/cryosparcuser/anaconda3/envs/topaz/bin/topaz ``` In the CryoSPARC interface, specify this as the value for the "Path to Topaz executable" parameter: ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FFZzCErXEqQuAJTPavEoI%2Ftopaz-executable-path.png?alt=media\&token=c1faeec1-f4a1-44eb-93cf-1a7063fd5f06) If using this path results in errors when running a Topaz job (often with a message such as "topaz did not produce valid output"), try aliasing Topaz with a shell script that also activates the correct `conda` environment. This process is described in the next section. ### (Optional) Create a `topaz.sh` wrapper script CryoSPARC has its own `conda` installation and environment, which may conflict with the Topaz environment. To prevent this, create a shell script that deactivates the CryoSPARC environment and activates the topaz one. Create a `topaz.sh` file in a well known location such as the home directory (e.g., `~/topaz.sh`). Add the following contents, making the noted substitutions: ```bash #!/usr/bin/env bash if command -v conda > /dev/null 2>&1; then conda deactivate > /dev/null 2>&1 || true # ignore any errors conda deactivate > /dev/null 2>&1 || true # ignore any errors fi unset _CE_CONDA unset CONDA_DEFAULT_ENV unset CONDA_EXE unset CONDA_PREFIX unset CONDA_PROMPT_MODIFIER unset CONDA_PYTHON_EXE unset CONDA_SHLVL unset PYTHONPATH unset LD_PRELOAD unset LD_LIBRARY_PATH source $HOME/anaconda3/etc/profile.d/conda.sh conda activate topaz exec topaz $@ ``` * Substitute `$HOME/anaconda3` on line 17 with the Anaconda or Miniforge installation directory. Make this file executable by the CryoSPARC user from the command line ``` chmod +x topaz.sh ``` In the CryoSPARC interface, specify the full path to `topaz.sh` as the "Path to Topaz executable" parameter: ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FtPrF94QQMqQDWHEJgcxa%2Ftopaz-sh-path.png?alt=media\&token=15da4adf-95e2-4519-b552-6b2c6d9208e0) ### (Optional) Set Topaz executable path as project-level default parameter To avoid having to locate and set the Topaz executable path when building every Topaz job, in v4.0.2 onwards you can set a project-level default that will apply to all newly created Topaz jobs. Navigate to the projects view, select a project and choose the Topaz executable path under the 'Project Level Parameters' module within the sidebar details panel:

You can use the input field or file browser to enter a path to the Topaz executable.

## Verify Topaz Installation Log out of the current command shell and log in again to ensure no conda environment is active. Run the following commands to verify that the Topaz Installation is working correctly. These are adapted from the [Topaz Quick start guide](https://github.com/tbepler/topaz/blob/dd35a9e391ff5535374c2983d3d93e6b406c35f5/tutorial/01_quick_start_guide.ipynb). Note the substitutions below. ```bash wget http://bergerlab-downloads.csail.mit.edu/topaz/topaz-tutorial-data.tar.gz tar -xzvf topaz-tutorial-data.tar.gz mkdir -p data/EMPIAR-10025/processed mkdir -p data/EMPIAR-10025/processed/micrographs /path/to/topaz preprocess -v -s 8 -o data/EMPIAR-10025/processed/micrographs/ data/EMPIAR-10025/rawdata/micrographs/*.mrc /path/to/topaz convert -s 8 -o data/EMPIAR-10025/processed/particles.txt data/EMPIAR-10025/rawdata/particles.txt ``` * Substitute `/path/to/topaz` with the full path to be used for CryoSPARC jobs * For the `preprocess` command, specify one of * `--device n` where `n` is a specific GPU number to run the test on * `--num-workers n` where `n` is the number of processes to use ## Job Types {% content-ref url="topaz/job-topaz-train-beta" %} [job-topaz-train-beta](https://guide.cryosparc.com/processing-data/all-job-types-in-cryosparc/deep-picking/topaz/job-topaz-train-beta) {% endcontent-ref %} {% content-ref url="topaz/job-topaz-extract-beta" %} [job-topaz-extract-beta](https://guide.cryosparc.com/processing-data/all-job-types-in-cryosparc/deep-picking/topaz/job-topaz-extract-beta) {% endcontent-ref %} {% content-ref url="topaz/job-topaz-denoise-beta" %} [job-topaz-denoise-beta](https://guide.cryosparc.com/processing-data/all-job-types-in-cryosparc/deep-picking/topaz/job-topaz-denoise-beta) {% endcontent-ref %} ## Topaz Deep Picking Tutorials {% content-ref url="topaz/topaz-tutorial" %} [topaz-tutorial](https://guide.cryosparc.com/processing-data/all-job-types-in-cryosparc/deep-picking/topaz/topaz-tutorial) {% endcontent-ref %} {% content-ref url="topaz/t20s-proteasome-deep-micrograph-denoising-tutorial" %} [t20s-proteasome-deep-micrograph-denoising-tutorial](https://guide.cryosparc.com/processing-data/all-job-types-in-cryosparc/deep-picking/topaz/t20s-proteasome-deep-micrograph-denoising-tutorial) {% endcontent-ref %}