# Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+) {% hint style="info" %} The "Extensive Workflow" job has been renamed to "Extensive Validation" in CryoSPARC v4.3.0+. For the version of this guide applicable to CryoSPARC versions ≤v4.2, please see: [Extensive Workflow](https://guide.cryosparc.com/setup-configuration-and-management/software-system-guides/tutorial-verify-cryosparc-installation-with-the-extensive-workflow-sysadmin-guide-1) {% endhint %} {% hint style="success" %} In CryoSPARC v5.0+, many job types have been added to the Extensive Validation Job's test set, and in advanced mode (see below), nearly all job types that exist are launched, providing for a comprehensive test of CryoSPARC. {% endhint %} ## Introduction CryoSPARC's "Extensive Validation" job orchestrates a full 3D target reconstruction for two datasets: * T20S Proteasome ([EMPIAR-10025](http://pdbe.org/EMPIAR-10025)) from a small subset of movies (\~8GB) * Tobacco Mosaic Virus ([EMPIAR-10305](https://www.ebi.ac.uk/empiar/EMPIAR-10305/)) CryoSPARC's engineering team uses this job to automatically test and benchmark CryoSPARC between releases. System Administrators may use the Extensive Validation job to verify that CryoSPARC is correctly configured following a fresh installation or an update. ![Table view of a workspace with an Extensive Validation run](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FzyS0nhuVXmFUcmVMLzim%2Fv4.3.0-extensive-validation-table-view.png?alt=media\&token=f8ca6d1d-4f55-43d5-9ce0-0b02f6588705) ### Benchmarking vs. Testing The Extensive Validation job has two modes: Benchmark mode and Testing mode.

Switch the Extensive Workflow job from "Testing" mode to "Benchmark" mode using the drop-down menu.

In Benchmark mode, jobs with pre-defined parameters run sequentially on the worker node in order. Each job accesses available system resources independently of other jobs to collect accurate runtime statistics. **Benchmark mode is useful for evaluating the overall performance of a worker node.** In Testing mode, jobs run in parallel when possible. Multiple parameter combinations of each job are dispatched. **Testing mode is useful for ensuring that a CryoSPARC instance is installed correctly.** Both Benchmark and Testing modes verify the following system requirements: * CryoSPARC system and license installation * Worker/Cluster configuration * GPU and CUDA driver installation * SSD caching ### Datasets Available {% hint style="info" %} CryoSPARC downloads the selected dataset into the project directory when the Extensive Validation job first runs in the current project. {% endhint %} #### [EMPIAR-10025](https://www.ebi.ac.uk/empiar/EMPIAR-10025/) * **Number of movies:** 20 * **Frames per movie:** 38 * **Movie size:** 7420 × 7676 (K2 Super Resolution) * **Pixel size:** 0.86 Å * **Particles processed:** 10,000 * **Particle box size (pixels):** 448
Jobs run in Benchmark Mode (up to CryoSPARC v4.7) 1. Import Movies 2. Patch Motion Correction 3. Patch CTF Estimation 4. Manually Curate Exposures 5. Blob Picker 6. Inspect Particle Picks 7. Extract From Micrographs (CPU) 8. 2D Classification 9. Select 2D 10. Template Picker 11. Inspect Particle Picks 12. Extract From Micrographs (CPU) 13. 2D Classification (50 Class) 14. 2D Classification (100 Class) (All Job Types Enabled) 15. 2D Classification (200 Class) (All Job Types Enabled) 16. Select 2D 17. Particle Sets Tools 18. Ab-Initio Reconstruction 19. Ab-Initio Reconstruction (3 Class) (All Job Types Enabled) 20. Homogeneous Refinement 21. Non-Uniform Refinement 22. Homogeneous Refinement Legacy (All Job Types Enabled) 23. Non-Uniform Refinement Legacy (All Job Types Enabled) 24. 3D Classification 25. 3D Variability (3 Mode) 26. 3D Variability (6 Mode) (All Job Types Enabled)
Jobs run in Testing Mode (up to CryoSPARC v4.7) 1. Import Movies 2. Patch Motion Correction 3. Full-Frame Motion Correction 4. Patch CTF Estimation 5. CTFFIND4 6. Curate Exposures (Stream A) 7. Curate Exposures (Stream B) 8. Blob Picker (Stream A) 9. Blob Picker (Stream B) 10. Inspect Picks (Stream A) 11. Inspect Picks (Stream B) 12. Extract From Micrographs (Stream A) 13. Local Motion Correction (Stream B) 14. 2D Classification (Stream A) # all jobs past this point are in Stream A 15. Select 2D 16. Template Picker 17. Inspect Picks 18. Extract From Micrographs 19. 2D Classification (50 Class) 20. 2D Classification (100 Class) 21. 2D Classification (200 Class) 22. Select 2D 23. Particle Sets Tools 24. Ab-Initio Reconstruction 25. Ab-Initio Reconstruction (3 Class) 26. Homogeneous Refinement 27. Non-Uniform Refinement 28. Homogeneous Refinement (Legacy) 29. Non-Uniform Refinement (Legacy) 30. Heterogeneous Refinement (3 Class) 31. Heterogeneous Refinement (6 Class) 32. 3D Classification (Simple mode) 33. 3D Classification (PCA mode) 34. 3D Variability (3 mode) 35. 3D Variability (6 mode) 36. Sharpening Tools 37. Validation (FSC) 38. Global CTF Refinement 39. Local CTF Refinement 40. 3D Variability Display
Jobs run in Benchmark Mode (CryoSPARC v5.0+) 1. Import Movies 2. Patch Motion Correction 3. Patch CTF Estimation 4. Manually Curate Exposures 5. Blob Picker 6. Inspect Particle Picks 7. Extract From Micrographs (CPU) 8. Micrograph Junk Detector (v5.0+) 9. 2D Classification 10. Downsample Particles (v5.0+) 11. Select 2D 12. Template Picker 13. Inspect Particle Picks 14. Extract From Micrographs (CPU) 15. 2D Classification (50 Class) 16. 2D Classification (100 Class) (All Job Types Enabled) 17. 2D Classification (200 Class) (All Job Types Enabled) 18. Select 2D 19. Particle Sets Tools 20. Ab-Initio Reconstruction 21. Ab-Initio Reconstruction (3 Class) (All Job Types Enabled) 22. Heterogeneous Refinement (v5.0+) 23. Homogeneous Refinement 24. Non-Uniform Refinement 25. Subset Particles by Statistic 26. Volume Tools 27. Orientation Diagnostics 28. 3D Variability (3 Mode) 29. 3D Variability (6 Mode) (All Job Types Enabled) 30. Local Refinement
Jobs run in Testing Mode (CryoSPARC v5.0+) 1. Import Movies 2. Import Micrographs 3. Import 3D Volumes 4. Import Particle Stack 5. Import Templates 6. Patch Motion Correction 7. Full-Frame Motion Correction 8. Import Beam Shift 9. Patch CTF Estimation 10. CTFFIND4 11. Check For Corrupt Micrographs 12. Generate Micrograph Thumbnails 13. Curate Exposures (Stream A) 14. Curate Exposures (Stream B) 15. Exposure Tools 16. Exposure Sets Tool (Split) 17. Exposure Sets Tool (Intersect) 18. Blob Picker (Stream A) 19. Blob Picker (Stream B) 20. Inspect Picks (Stream A) 21. Remove Duplicate Particles 22. Blob Picker Tuner 23. Inspect Picks (Stream B) 24. Reassign Particles to Micrographs 25. Patch CTF Extraction 26. Local Motion Correction (Stream B) 27. Local Motion Correction (Multi) (Stream B) 28. Extract From Micrographs (CPU) (Stream A) 29. Extract From Micrographs (GPU) 30. Micrograph Junk Detector 31. Restack Particles 32. 2D Classification (Stream A) # all jobs past this point are in Stream A 33. Cache Particles on SSD 34. Check For Corrupt Particles 35. Class Probability Filter 36. Downsample Particles 37. Select 2D 38. Average Power Spectra 39. Template Picker 40. Inspect Picks 41. Extract From Micrographs 42. Exposure Group Utilities (Combine Particles) 43. Exposure Group Utilities (Split Particles) 44. Exposure Group Utilities (Combine Exposures) 45. Exposure Group Utilities (Split Exposures) 46. 2D Classification (50 Class) 47. 2D Classification (100 Class) 48. 2D Classification (200 Class) 49. Rebalance 2D Classes 50. Select 2D 51. Particle Sets Tools 52. Reconstruct 2D Classes 53. Ab-Initio Reconstruction 54. Ab-Initio Reconstruction (3 Class) 55. Split Volumes Group 56. Simulate Data 57. Reference Based Auto Select 3D 58. Volume Tools 59. Create Templates (Non-Helical) 60. Create Templates (Helical) 61. Heterogeneous Refinement 62. Homogeneous Refinement 63. Non-Uniform Refinement 64. Subset Particles by Statistic 65. Volume Alignment Tools 66. Volume Tools (Mask) 67. Reference Based Auto Select 2D (Sobel) 68. Reference Based Auto Select 2D (Cluster) 69. Reference Based Auto Select 2D (Thresholds) 70. Rebalance Orientations 71. Orientation Diagnostics 72. Heterogenous Reconstruction Only 73. Local Resolution Estimation 74. Symmetry Search Utility 75. Apply Helical Symmetry 76. Reference Based Motion Correction 77. 3D Flex Data Prep 78. Patch Motion to Local Motion 79. 3D Classification 80. Sharpening Tools 81. Validation (FSC) 82. Global CTF Refinement 83. 3D Variability (3 mode) 84. 3D Variability (6 mode) 85. Local Refinement 86. Recenter Trajectories 87. Apply Trajectories 88. Local Filtering 89. 3D Flex Mesh Prep 90. Align 3D Maps 91. Regroup 3D Classes 92. Particle Subtraction 93. 3D Flex Training 94. Local CTF Refinement 95. ResLog Analysis 96. 3D Variability Display 97. 3D Flex Generator 98. 3D Flex Reconstruction
#### [EMPIAR-10305](https://www.ebi.ac.uk/empiar/EMPIAR-10305/) * **Number of movies:** 62 * **Frames per movie:** 20 * **Movie size:** 7420 × 7676 (K2 Super Resolution) * **Pixel size:** 0.32 Å * **Particles processed:** \~30,000 * **Particle box size (pixels):** 512
Jobs run in Benchmark Mode (up to CryoSPARC v4.7) 1. Import Movies 2. Patch Motion Correction 3. Patch CTF Estimation 4. Curate Exposures 5. Blob Picker 6. Inspect Picks 7. Extract From Micrographs 8. 2D Classification 9. Select 2D 10. Template Picker 11. Inspect Picks 12. Extract From Micrographs 13. 2D Classification 14. Select 2D 15. Particle Sets Tools 16. Ab-Initio Reconstruction 17. Homogeneous Refinement 18. Non-Uniform Refinement 19. 3D Classification 20. 3D Variability
Jobs run in Testing Mode (up to CryoSPARC v4.7) 1. Import Movies 2. Patch Motion Correction 3. Patch CTF Estimation 4. Filament Tracer 5. Inspect Picks 6. Extract From Micrographs 7. 2D Classification 8. Select 2D 9. Helical Refinement 10. Local CTF Refinement 11. Global CTF Refinement 12. Symmetry Expansion 13. Homogeneous Reconstruct Only
Jobs run in Benchmark Mode (CryoSPARC v5.0+) 1. Import Movies 2. Patch Motion Correction 3. Patch CTF Estimation 4. Filament Tracer 5. Inspect Particle Picks 6. Extract From Micrographs (CPU) 7. 2D Classification 8. Select 2D Classes 9. Helical Refinement 10. Local CTF Refinement 11. Global CTF Refinement 12. Symmetry Expansion 13. Homogeneous Reconstruction Only
Jobs run in Testing Mode (CryoSPARC v5.0+) 1. Import Movies 2. Patch Motion Correction 3. Patch CTF Estimation 4. Filament Tracer 5. Inspect Particle Picks 6. Extract From Micrographs (CPU) 7. 2D Classification 8. Select 2D Classes 9. Helical Refinement 10. Local CTF Refinement 11. Global CTF Refinement 12. Symmetry Expansion 13. Homogeneous Reconstruction Only
## Prerequisites {% content-ref url="../how-to-download-install-and-configure" %} [how-to-download-install-and-configure](https://guide.cryosparc.com/setup-configuration-and-management/how-to-download-install-and-configure) {% endcontent-ref %} ## Creating and Running the Extensive Validation Job 1. Open the CryoSPARC web interface 2. In the dashboard, create a new Project from the navigation bar and create an initial workspace. ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2Fzsh9gXX6Za1Fal9xMdMg%2Fnew-project-selection-v4.3.png?alt=media\&token=425272f1-fce8-404c-88fe-4a397182d165) Specify a descriptive title such as "Extensive Validation Testing" and directory for the project to store its data.
**Best practices:** Create a new workspace and run the Extensive Validation in that workspace each time CryoSPARC updates and restarts. Name each workspace with the latest installed version of CryoSPARC that the job runs on. For example, when testing CryoSPARC v4.3.0, name the workspace "v4.3.0 Benchmark and Validation" 4\. Select the Job Builder from the sidebar and select the **"Extensive Validation"** job (under the Validation category). ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FQTyeLNgKPh6Pd2dwmJBn%2Fv4.3-extensive-validation-builder.png?alt=media\&token=cb50e7d1-86c3-414d-9001-7dbfbabef0e0) *5. (Optional)* If desired, change the job parameters. 6\. Select the node or cluster that the Extensive Validation jobs should run on. ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FrnaByM6wI9XNuBPUuS8i%2Fextensive-validation-resource-param-sec.png?alt=media\&token=93986ff1-12b2-401e-8788-8677890fd9c8) ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FDxHc5GicFN5qc1zPEUO5%2Fv4.3-resource-selection.png?alt=media\&token=7ef51e43-6af1-4357-b83b-755d68119e00) Queue the job from the Job Builder sidebar. Open the job's Event log (either click/tap the Job card header or select the Job card and press the `Space` key) to monitor its progress. The Validation job logs each spawned job as it is queued and logs how long it takes to complete. ![](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FAnWpBc6IeaIQ8L4dMa7v%2Fextensive-validation-job-overview-tab.png?alt=media\&token=435bed08-7e5b-44c4-9815-1ea75a3049eb) Close the Job modal with the `×` button. This shows the workspace overview with CryoSPARC jobs spawned by the Extensive Validation job Once all jobs successfully complete, the Extensive Validation job status changes to "Completed". This means the installation was successful. Users may now be notified to start or resume processing! ## Troubleshooting Failed Jobs Extensive Validation will fails if any spawned job fails. Scroll through the workspace to find jobs with the "Failed" status. Open the failed job's Event Log. Scroll to the bottom to see why the job failed. Common failure reasons include * [Cannot verify license](https://discuss.cryosparc.com/t/new-install-error-connecting-to-cryosparc-license-server/3825) or license key entered incorrectly * [Incorrect filesystem permissions](https://discuss.cryosparc.com/t/file-r-w-permissions/3544) * [CUDA not set up correctly](https://discuss.cryosparc.com/t/3d-variability-analysis-errors-v2-9-0/3110) * [SSD cache full or not set up correctly](https://discuss.cryosparc.com/t/changing-ssd-no-run-directory/2916) * [Worker registration issue](https://discuss.cryosparc.com/t/exit-status-255-error/2177) * [Not enough GPU memory available](https://discuss.cryosparc.com/t/patchmotion-failure-2-13-2/3971) Once the configuration issue is resolved, restart the Extensive Validation job: Either create a new workspace and job as already noted, or clear the existing Extensive Workflow job and re-queue. ## Additional Extensive Testing For an even *more* extensive system test, the Extensive Validation job provides the parameter "Run Advanced Jobs" ![Turn on the "Run Advanced Jobs" radio button to run the full workflow.](https://1916621962-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2F-M7DGv3GkRvGGpbVPCgg%2Fuploads%2FRVY8O5MzKzkKoOkPMo8v%2Fv4.3-ext-wflo-advanced-toggle.png?alt=media\&token=2f6ce388-586a-4d7e-9943-5145b03e545c) With "Run Advanced Jobs" enabled, additional validation jobs run in parallel. Use this to verify multi-GPU performance on a single node. Advanced jobs available for each dataset are listed in the "Datasets Available" section above. ## Expected Results To compare the results of Extensive Validation runs, use "Benchmark" mode. This locks in the parameters and runs each job serially to ensure all system resources are available independently. The benchmark results may be viewed in the "Manage" panel, under the "Benchmarks" tab.
There are several reference benchmarks available for comparison with your CryoSPARC installation, including benchmarks completed on AWS EC2 instances. Select one or more benchmark rows and click "Compare" to compare benchmarks.