FAQs and Troubleshooting

FAQs

General

Who should use CryoSPARC Live?

CryoSPARC Live is built for anyone with access to a microscope or cryo-EM single particle data:

• Data collection facilities, cryo-EM cores and microscope operators who want to make the most of microscope time with real-time data quality assessment

• Individual users who want to gain insights about data quality by performing 3D reconstructions in real-time with data collection

• Users who are looking for the simplest, fastest way to process previously collected data in a seamless and "first cut" manner

CryoSPARC Live is available free of charge for academic use.

How can I access CryoSPARC Live?

To access the Live web application, please see: How to Access CryoSPARC Live

CryoSPARC Live jobs are housed within CryoSPARC projects, so there is full integration between the two applications and you can continue processing in CryoSPARC using the outputs of Live.

What are the hardware requirements for Live?

Please see Prerequisites and Compute Resources Setup

What kind of data can I process in CryoSPARC Live?

CryoSPARC Live supports reading in movie files (.tif, .mrc, .mrc.bz2 and .eer). For more information about EER file support in CryoSPARC, please see: EER File Support

How fast is processing in CryoSPARC Live?

CryoSPARC Live can use multiple GPUs concurrently for pre-processing and reconstruction jobs to maximize achievable throughput. With 4 GPUs assigned to preprocessing (and sufficiently fast storage), cryoSPARC Live can sustain a rate of up to 1 exposure per 1.4 seconds, or over 60,000 exposures per 24-hours. For more performance metrics, please see: Performance Metrics

How do I view keyboard shortcuts?

Click on the Main Menu in any CryoSPARC Live Session and select "Keyboard Shortcuts" for a list of all keyboard shortcuts available.

View keyboard shortcuts via the main menu.

Processing Data in Live

I am processing a brand new dataset and I am not sure which values to enter for minimum and maximum particle diameter.

In this case, start with crude estimates of your particle diameter based on the molecular weight of the molecule. These diameters are only used for blob picking, and will not affect downstream results other than the quality of picks. Once the Live session is started, use the picking tab and the exposure viewer to display your picks as circles (open the dropdown beside the pick counts at the top of the exposure viewer to change the display mode from dots to circles). These circles will be drawn at the same size as the particle diameter that you had entered.

Based on this, choose new values for the minimum and maximum particle diameter, and use Test mode (click the 'Test' button on the picking tab) to reprocess the currently selected exposure using the new parameters that you provided. Once this is complete (you must wait for the selected micrograph to be picked up and processed by a Live worker) you can visualize the new picks, and adjust thresholds to determine the quality of the picks using these new parameters. You can repeat test mode as many times as needed to achieve good picking results.The same strategy can also be used with the template picker.

Once you have picks that are reasonable for your data, be sure to also adjust the extraction box size on the configuration tab based on your new knowledge of the particle diameter, and "Apply to All' so that all exposures are re-extracted at the new box size.

I Enabled an exposure group but later realized it was wrongly configured. How can I change this?

Click New to add a new Exposure Group. Configure this group as desired and click Enable. Then, Remove the incorrect group. CryoSPARC Live requires a minimum of one exposure group to be enabled at all times; you cannot remove an exposure group if it is the only one that exists. Note: exposure groups can only be removed if they do not have any exposures already found or processed within them. You can ignore all the exposures from a group that has already been established, or you can clear the session.

Press the "Remove" button to delete the exposure group. This button is only enabled if there are more than one exposure groups available.

Where can I see job status / progress ?

Jobs will update their status and relevant outputs (e.g., plots) in the CryoSPARC Live interface on their respective tabs. For more details, click into the job icon for any job to view its streamlog directly within the Live interface. Finally, you can always interact with Live jobs through your regular CryoSPARC interface: navigate to the Project where the Live Session is housed and click on the job number or press SPACEBAR to open the streamlog.

Click on the "+" icon while in the Single Session page to view a job's streamlog directly.

Click on the job icon to view its streamlog directly.

Where can I see and interact with job outputs?

You can interact with Live jobs through your regular CryoSPARC interface and download/use their outputs for further processing. Navigate to the Project/workspace where the Live Session is housed to view all Live jobs. 3D maps can be download directly from the Volume Viewer in the CryoSPARC Live interface.

The sidebar shows that that a few of my exposures have failed. How can I find which ones have failed?

You can scroll through the exposure feed and look for the "rejected" icon on any thumbnails that have failed. Alternatively, navigate to the Browse Tab and filter by 'Failed` exposures to view a list of all failed exposure UIDs. Click on any row to view that exposure.

I forgot to flip the gain reference file and the session is already running. How can I fix this?

Navigate to the Configuration Tab and ensure the "Show advanced" checkbox is selected. Under Microscope/Camera Parameter, adjust the toggles for flipping the gain reference in X or Y as required. Once you change the parameter, click Apply to All or Apply to Future to save the change and cause the CryoSPARC Live GPU Workers to re-process the exposures with the new parameter.

Use the search bar to quickly find the parameter you're looking for.

How do I switch from Blob Picking to Template Picking or vice versa? How do I re-extract particles based on my new picking settings?

The Blob Picker is the default active picker in CryoSPARC Live and is engaged as soon as exposures start to be processed. To engage the Template Picker at any time during the session, follow these instructions: New Live Session: Start to Finish Guide

I want to change the extraction box size.

During a running Session, you can navigate to the Configuration Tab at any time to change the extraction box size for any of the pickers. Once you change the parameter, click Apply to All to save the change and cause the CryoSPARC Live GPU Workers to re-extract all particles with the new box size. Note that you should not use Apply for Future in this case, as only the new box size particles will be used in streaming processing.

I want to specify custom parameters for a Live job.

You can specify custom parameters for Streaming 2D Classification, Ab-Initio Reconstruction and Streaming Refinement jobs. To do so, navigate to the respective tab from the sidebar and click on the Gear icon. Click on the build icon (hammer) to Build with custom parameters. This will create a new job in building status.

Click on the build icon to create a new job that you can modify parameters for.

Navigate to the CryoSPARC interface (to the Project where this Live Session is housed). Edit the parameters of the building job and then return to the Live interface to queue/start the job.

Click "Queue" to schedule your job with custom parameters.

I manually rejected an exposure. How do I un-reject?

Select the manually rejected exposure and navigate to the Individual Exposure Tab. To un-reject the exposure individually, click on the dropdown menu above the Exposure Viewer and click Un-reject Exposure, OR use the keyboard shortcut "R". To reset all manually rejected exposures, click on the dropdown menu in the header and click Reset manual rejected exposures.

What does Test Adjustments do?

Test Adjustments is useful for fine-tuning picking parameters and thresholds. If you are not sure how your parameter changes might affect processing or if you would like to experiment, you can use 'Test Adjustments' which will cause only the currently selected exposure to be re-picked and re-extracted with the new picking parameter changes. Once this process is complete, the new pick locations will appear on the active micrograph. Exposures to which a Test parameter has been applied are indicated with a purple "T" on their respective thumbnails. You can apply 'Test Adjustments' on as many individual exposures as you like.

An example Test exposure

Once satisfied with the new picker settings, click 'Activate for All' or 'Activate for Future' as desired. This will trigger re-picking and re-extraction. Unless one of the Activate buttons is clicked following Test Adjustments, the exposure on which Test Adjustments was run will simply be excluded from any further processing (i.e., from Streaming 2D Classification and Streaming 3D Refinement). To undo Test Adjustments mode on a particular exposure, i.e., to reset it so that it can be included in further processing, click on the dropdown above the Exposure Viewer and click Reprocess exposure.

When and how often will my Streaming 2D classes update?

2D classes update in two ways. First, when new particles are available, those particles are classified into the existing classes, and the number of particles in each class is updated. This allows the new particles to be filtered based on class selection, and selected particles flow on to 3D refinement. This can be ver fast, taking only seconds. Second, once the number of newly classified particles accumulates enough, the templates themselves are updated with the signal from the new particles, and all existing particles are re-classified based on the new templates. This can take a few minutes depending on how many particles are extracted.

When and how often will my Streaming 3D Refinement update?

3D refinement updates with a new 3D structure when enough new particles have arrived from 2D classification. At this point, the 3D reference structure currently resolved is backtracked to a lower resolution, and all existing particles (including new particles) are used to refine the structure to high resolution until convergence. This can take several minutes depending on the number of particles, box size, symmetry, and convergence rate for the given dataset.

How can I download 3D volume(s) generated in Live?

You can download 3D volumes directly from the Volume Viewer in the Live interface. Alternatively, you can download them from within the CryoSPARC Project where the Live Session is housed.

Download map directly from the Volume Viewer.

Compute Configuration

I want to increase or decrease the number of GPUs being used for preprocessing in my Live Session.

Pause the Session. This will cause any running CryoSPARC Live GPU Workers to complete their current task and spin down. Any auxiliary and reconstruction jobs (Generate Templates, Ab-Initio Reconstruction, Streaming 2D Classification and Streaming Refinement) will be killed and marked as completed, so their outputs can be used to potentially resume processing.

Make any adjustments to the compute configuration page and then Start the session again. This will spin up new CryoSPARC Live GPU Worker jobs using the values set. Any Auxiliary or Reconstruction jobs will need to be manually restarted via their respective tabs.

If I Pause a session now and come back to it later, can I pick up where I left off?

If you Pause a session, any active CryoSPARC Live GPU Workers will be spun down and any auxiliary and reconstruction jobs (Generate Templates, Ab-Initio Reconstruction, Streaming 2D Classification and Streaming Refinement) will be killed and marked as completed, so their outputs can be used to potentially resume processing. When you restart the session, remaining exposures will start to process where they left off, regardless of the stage they finished at. You will need to manually go to the 2D class and 3D refinement tabs to start/resume those processes.

Data Management

How can I safely delete CryoSPARC Live jobs or outputs?

Like other processing jobs, CryoSPARC Live jobs generate intermediate results and metadata. To free up space, users may wish to archive or permanently delete some of these outputs. CryoSPARC v3.0.0+ includes a new toolset for managing, archiving and deleting data from cryoSPARC Live sessions. Please see the detailed tutorial here: CryoSPARC Live Session Data Management

I am finished with the Live workflow and I want to continue processing in CryoSPARC or elsewhere.

Please see View/interact with Outputs and Perform Further Processing: New Session: Start to Finish Guide

Troubleshooting

Please also see our CryoSPARC Troubleshooting page for general CryoSPARC issues: Troubleshooting

My Live Session is running but one or more of my jobs failed. How can I troubleshoot and/or resume processing?

Open the job streamlog by clicking on the job number within the CryoSPARC Live interface or navigate to the CryoSPARC interface to do the same.

• CryoSPARC Live GPU Workers: If these fail, you can Pause the Session and then Start it again. Any exposure that haven't been preprocessed (i.e., are found but haven't had all of Motion Correction, CTF Estimation, Particle Picking or Extraction performed on them yet, or are new incoming exposures) will be picked up normally by the new CryoSPARC Live GPU Workers that are spun up, and will be processed accordingly.

• CryoSPARC Live Exposure Export or CryoSPARC Live Particle Export: If these jobs fail, you can simply try to run them again without Pausing the session.

• Auxiliary Jobs

  • Generate Templates: If this job fails, you will need to start it again by clicking "Generate Templates" once again

  • Ab-Initio Reconstruction: If this job fails, you will need to start it again by clicking "Restart"

• Streaming Jobs (Streaming 2D Classification and Streaming Refinement): If these jobs fail, you will need to "Restart" or "Attempt Resume"

One or more of my exposures failed to process. How do I reset?

Select the failed exposure and navigate to the Individual Exposure Tab. You can view the traceback here. To reset the exposure individually, click on the dropdown menu above the Exposure Viewer and click "Reprocess Exposure". To reset all failed exposures, click on the dropdown menu in the header and click Reset Failed Exposures.

My Live jobs are queued with 'GPU not available'.

This is most likely because there are insufficient GPU resources allocated to each lane in the CryoSPARC Live compute configuration. Pause the session and navigate to the configuration tab to adjust compute settings, then click Start to continue processing.

We recommend 4 GPUs be allocated for Live processing for the ability to perform preprocessing and real-time 2D/3D processing during a data collection session: Prerequisites and Compute Resources Setup

How can I view queued and running jobs?

Click on the footer in CryoSPARC Live for a summary of all running and queued Live jobs. CryoSPARC Live jobs are also shown in the CryoSPARC Resource Manager.

Where can I get more help for CryoSPARC Live?

Please post on the CryoSPARC Discussion Forum under the CryoSPARC Live category.