Comment on page
Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
The "Extensive Workflow" job has been renamed to "Extensive Validation" in CryoSPARC v4.3.0+. For the version of this guide applicable to CryoSPARC versions ≤v4.2, please see: Extensive Workflow
CryoSPARC's "Extensive Validation" job orchestrates a full 3D target reconstruction for two datasets:
CryoSPARC's engineering team uses this job to automatically test and benchmark CryoSPARC between releases.
System Administrators may use the Extensive Validation job to verify that CryoSPARC is correctly configured following a fresh installation or an update.
Table view of a workspace with an Extensive Validation run
The Extensive Validation job has two modes: Benchmark mode and Testing mode.
Switch the Extensive Workflow job from "Testing" mode to "Benchmark" mode using the drop-down menu.
In Benchmark mode, jobs with pre-defined parameters run sequentially on the worker node in order. Each job accesses available system resources independently of other jobs to collect accurate runtime statistics. Benchmark mode is useful for evaluating the overall performance of a worker node.
In Testing mode, jobs run in parallel when possible. Multiple parameter combinations of each job are dispatched. Testing mode is useful for ensuring that a CryoSPARC instance is installed correctly.
Both Benchmark and Testing modes verify the following system requirements:
- CryoSPARC system and license installation
- Worker/Cluster configuration
- GPU and CUDA driver installation
- SSD caching
CryoSPARC downloads the selected dataset into the project directory when the Extensive Validation job first runs in the current project.
- Number of movies: 20
- Frames per movie: 38
- Movie size: 7420 × 7676 (K2 Super Resolution)
- Pixel size: 0.86 Å
- Particles processed: 10,000
- Particle box size (pixels): 448
- 1.Import Movies
- 2.Patch Motion Correction
- 3.Patch CTF Estimation
- 4.Curate Exposures
- 5.Blob Picker
- 6.Inspect Picks
- 7.Extract From Micrographs
- 8.2D Classification
- 9.Select 2D
- 10.Template Picker
- 11.Inspect Picks
- 12.Extract From Micrographs
- 13.2D Classification (50 Class)
- 14.2D Classification (100 Class) (All Job Types Enabled)
- 15.2D Classification (200 Class) (All Job Types Enabled)
- 16.Select 2D
- 17.Particle Sets Tools
- 18.Ab-Initio Reconstruction
- 19.Ab-Initio Reconstruction (3 Class) (All Job Types Enabled)
- 20.Homogeneous Refinement
- 21.Non-Uniform Refinement
- 22.Homogeneous Refinement Legacy (All Job Types Enabled)
- 23.Non-Uniform Refinement Legacy (All Job Types Enabled)
- 24.3D Classification
- 25.3D Variability (3 Mode)
- 26.3D Variability (6 Mode) (All Job Types Enabled)
- 1.Import Movies
- 2.Patch Motion Correction
- 3.Full-Frame Motion Correction
- 4.Patch CTF Estimation
- 5.CTFFIND4
- 6.Curate Exposures (Stream A)
- 7.Curate Exposures (Stream B)
- 8.Blob Picker (Stream A)
- 9.Blob Picker (Stream B)
- 10.Inspect Picks (Stream A)
- 11.Inspect Picks (Stream B)
- 12.Extract From Micrographs (Stream A)
- 13.Local Motion Correction (Stream B)
- 14.2D Classification (Stream A) # all jobs past this point are in Stream A
- 15.Select 2D
- 16.Template Picker
- 17.Inspect Picks
- 18.Extract From Micrographs
- 19.2D Classification (50 Class)
- 20.2D Classification (100 Class)
- 21.2D Classification (200 Class)
- 22.Select 2D
- 23.Particle Sets Tools
- 24.Ab-Initio Reconstruction
- 25.Ab-Initio Reconstruction (3 Class)
- 26.Homogeneous Refinement
- 27.Non-Uniform Refinement
- 28.Homogeneous Refinement (Legacy)
- 29.Non-Uniform Refinement (Legacy)
- 30.Heterogeneous Refinement (3 Class)
- 31.Heterogeneous Refinement (6 Class)
- 32.3D Classification (Simple mode)
- 33.3D Classification (PCA mode)
- 34.3D Variability (3 mode)
- 35.3D Variability (6 mode)
- 36.Sharpening Tools
- 37.Validation (FSC)
- 38.Global CTF Refinement
- 39.Local CTF Refinement
- 40.3D Variability Display
- Number of movies: 62
- Frames per movie: 20
- Movie size: 7420 × 7676 (K2 Super Resolution)
- Pixel size: 0.32 Å
- Particles processed: ~30,000
- Particle box size (pixels): 512
- 1.Import Movies
- 2.Patch Motion Correction
- 3.Patch CTF Estimation
- 4.Curate Exposures
- 5.Blob Picker
- 6.Inspect Picks
- 7.Extract From Micrographs
- 8.2D Classification
- 9.Select 2D
- 10.Template Picker
- 11.Inspect Picks
- 12.Extract From Micrographs
- 13.2D Classification
- 14.Select 2D
- 15.Particle Sets Tools
- 16.Ab-Initio Reconstruction
- 17.Homogeneous Refinement
- 18.Non-Uniform Refinement
- 19.3D Classification
- 20.3D Variability
- 1.Import Movies
- 2.Patch Motion Correction
- 3.Patch CTF Estimation
- 4.Filament Tracer
- 5.Inspect Picks
- 6.Extract From Micrographs
- 7.2D Classification
- 8.Select 2D
- 9.Helical Refinement
- 10.Local CTF Refinement
- 11.Global CTF Refinement
- 12.Symmetry Expansion
- 13.Homogeneous Reconstruct Only
- 1.Open the CryoSPARC web interface
- 2.In the dashboard, create a new Project from the navigation bar and create an initial workspace.
Specify a descriptive title such as "Extensive Validation Testing" and directory for the project to store its data.
Best practices: Create a new workspace and run the Extensive Validation in that workspace each time CryoSPARC updates and restarts. Name each workspace with the latest installed version of CryoSPARC that the job runs on. For example, when testing CryoSPARC v4.3.0, name the workspace "v4.3.0 Benchmark and Validation"
4. Select the Job Builder from the sidebar and select the "Extensive Validation" job (under the Validation category).
5. (Optional) If desired, change the job parameters.
6. Select the node or cluster that the Extensive Validation jobs should run on.
Queue the job from the Job Builder sidebar. Open the job's Event log (either click/tap the Job card header or select the Job card and press the
Space key) to monitor its progress. The Validation job logs each spawned job as it is queued and logs how long it takes to complete.
Close the Job modal with the
× button. This shows the workspace overview with CryoSPARC jobs spawned by the Extensive Validation job Once all jobs successfully complete, the Extensive Validation job status changes to "Completed". This means the installation was successful. Users may now be notified to start or resume processing!
Extensive Validation will fails if any spawned job fails.
Scroll through the workspace to find jobs with the "Failed" status. Open the failed job's Event Log.
Scroll to the bottom to see why the job failed.
Example Import Movies failure because the Gain Reference was not found
Common failure reasons include
Once the configuration issue is resolved, restart the Extensive Validation job: Either create a new workspace and job as already noted, or clear the existing Extensive Workflow job and re-queue.
For an even more extensive system test, the Extensive Validation job provides the parameter "Run Advanced Jobs"
Turn on the "Run Advanced Jobs" radio button to run the full workflow.
With "Run Advanced Jobs" enabled, additional validation jobs run in parallel. Use this to verify multi-GPU performance on a single node. Advanced jobs available for each dataset are listed in the "Datasets Available" section above.
To compare the results of Extensive Validation runs, use "Benchmark" mode. This locks in the parameters and runs each job serially to ensure all system resources are available independently. The benchmark results may be viewed in the "Manage" panel, under the "Benchmarks" tab.
There are several reference benchmarks available for comparison with your CryoSPARC installation, including benchmarks completed on AWS EC2 instances. Select one or more benchmark rows and click "Compare" to compare benchmarks.
