Processing times from automated workflow examples
This page contains the amount of compute time for various stages of data processing using each workflow along with the compute node configuration.
Workflow v1 applied to GPCR datasets
Processing times in hours for each dataset processed on a node featuring an AMD Threadripper Pro 5975WX CPU, 2× NVIDIA RTX 4090 GPUs, and 256 GB DDR4 ECC RAM (3200 MT/s). Two 4 TB WD Black SN850X NVMe SSDs in RAID 0 were utilized for CryoSPARC’s internal particle caching system, while raw movies and motion corrected micrographs were stored on a large ZFS HDD array accessed via 10 Gb Ethernet. Times are shown for each stage of processing, as well as the total run time. For five datasets, times are also shown for preprocessing and RBMC stages using an 8-GPU server (grey), showing that 2-3x speedup in these stages is possible and an overall speedup of up to 1.7x by using additional hardware.
Timings in this table are as of CryoSPARC v4.7
10346
3.7
0.6
3.3
1.0
2.7
11.3
2,059
4092 x 5760
10359
6.2
1.0
5.3
3.2
5.8
21.5
3,167
4092 x 5760
10361
1.8
0.6
3.6
1.2
2.2
9.4
1,013
4092 x 5760
(8-GPU)
0.7
1.5
7.6
10673
9.8
0.9
6.3
2.6
6.3
25.9
5,734
4092 x 5760
11352
4.2
0.5
3.6
1.1
1.9
11.3
2,864
4092 x 5760
10288
2.5
0.7
3.7
1.7
2.3
10.9
2,753
3710 x 3838
10853
17.9
2.5
13.0
9.7
16.5
59.6
12,299
4092 x 5760
(8-GPU)
6.7
9
40.9
10854
8.0
1.4
6.0
2.6
4.6
22.6
5,359
4092 x 5760
10855
14.3
2.0
11.1
7.5
14.3
49.2
9,765
4092 x 5760
(8-GPU)
5.4
7.7
33.7
10856
8.3
1.7
8.1
3.6
8.3
30.0
5,759
4092 x 5760
10936
17.0
1.5
6.3
2.3
11.1
38.2
7,518
8184 x 11520
(8-GPU)
6.4
8.6
25.1
11119
16.1
1.7
5.7
2.3
7.3
33.1
11,241
4092 x 5760
11350
5.5
0.9
4.9
1.3
2.5
15.1
4,140
4092 x 5760
11435
14.8
1.1
5.0
1.6
7.8
30.3
6,816
8184 x 11520
11448
9.1
1.1
5.8
2.0
18.3
36.3
3,916
8184 x 11520
11474
19.7
1.2
6.6
5.7
30.6
63.8
8,330
8184 x 11520
11432
21.8
1.0
6.2
2.5
10.8
42.3
8,874
8184 x 11520
11433
15.4
1.6
9.5
4.3
33.0
63.8
6,772
8184 x 11520
11134
15.6
1.4
10.6
2.6
9.0
39.2
6,846
8184 x 11520
11135
35.9
2.2
13.3
1.2
13.1
65.7
14,634
8184 x 11520
(8-GPU)
13.9
9.3
39.9
11136
14.7
1.3
9.4
1.6
9.6
36.6
5,818
8184 x 11520
Workflow v2 applied to CAK datasets
Processing times in hours for each dataset processed on a node featuring an AMD EPYC 75F3 CPU, 8× NVIDIA RTX A100 (40GB) GPUs, and 512 GB DDR4 ECC RAM (3200 MT/s). Two 8 TB WD Black SN840 NVMe SSDs in RAID 0 were utilized for CryoSPARC’s internal particle caching system, while raw movies and motion corrected micrographs were stored on a large ZFS HDD array accessed via 10 Gb Ethernet. We utilized parallelization across resources where possible; Patch Motion Correction (8 GPUs), Patch CTF Estimation (8 GPUs), Particle Extraction (2 GPUs) and Reference Based Motion Correction (6 GPUs). For EMPIAR-11823e and 11823f, particle stacks were initially processed separately (requiring 16.1 and 16.6 hours, respectively) before being merged. We anticipate that total processing time for these combined datasets would align with 11793 or 11807 given the similar movie and particle counts. Average processing times across all datasets were approximately 21 hours.
Timings in this table are as of CryoSPARC v5.0
11800
3.1
1.7
7.0
6.0
17.8
5,805
4096 x 4096
11793
4.5
2.5
9.6
6.7
23.3
9,118
4096 x 4096
11799
2.7
1.7
7.5
5.2
17.1
5,780
4096 x 4096
11807
5.6
2.7
9.6
7.1
25.0
10,017
4096 x 4096
11823a
3.4
1.6
7.8
5.9
18.7
5,729
4096 x 4096
11823b
2.8
1.6
8.6
4.8
17.8
5,583
4096 x 4096
11823c
3.0
1.4
7.3
5.3
17.0
5,210
4096 x 4096
11823d
3.0
1.5
7.4
5.9
17.8
5,203
4096 x 4096
11823ef
5.2
3.0
14.5
10.0
32.7
10,597
4096 x 4096
Last updated