Low Level Results Interface
Last updated
Last updated
When a CryoSPARC job completes, it produces a number of output groups. These are familiar to all CryoSPARC users: output groups have names like Particles, Volumes, Exposures, etc. These outputs contain all of the information produced by the job for each kind of output item. Within output groups, there is an additional organizational level, called output results. Each output result has a name (e.g., blob, ctf, locations, mscope_params, etc.) and contains information of only a specific type. Together, a set of output results comprise an output group.
It is easier to understand groups and results with a concrete example. Consider a Particles output group. At the bare minimum, a Particles output must have all of the information about the particle images themselves: where to find them on the disk, their pixel size, etc. However, there is often more information than just the images. Perhaps the particles have CTF estimates or 2D or 3D pose estimates. Thus, the Particles output group contains each of these types of information as individual output results.
When creating a new job, you connect the parent job’s output group to the descendent job (from the point of view of the descendent job, it’s an input group). Taking a concrete example, you might connect the Particles and Volume outputs from a Homogeneous Refinement to the relevant Local Refinement inputs. The Local Refinement now has access to all of the output results produced by the Homogeneous Refinement. As the Local Refinement proceeds, it will update the particles’ poses (in the alignments3D output result) and then produce its own output group with the updated alignments3D result.
In some cases, you may want to replace only a single result in an output group with the result from another job. For example, perhaps the Homogeneous Refinement was performed with a curated set of downsampled particles, but you want to perform a Local Refinement with the full size particle images. You could do this using standard CryoSPARC jobs:
Use Particle Sets Tools to find the overlap between the curated, downsampled particles and the uncurated, full-size particles.
Perform a Homogeneous Refinement of the full-size particles against the downsampled map to assign poses to the full-size particles.
Perform the Local Refinement.
This process would work, but consider step 2. We already have pose estimates for these particles, but they were found using the downsampled particles, so we cannot use them with the Extract from Micrographs output group using the normal job building interface. We are therefore forced to perform an alignment which is otherwise unnecessary and potentially slow.
If we could use all of the information from the downsampled particles except for the blob output result (which contains the particle image data at a specific box size), we could use the existing poses (the alignments3D result from the downsampled Homogeneous Refinement’s particles output group) with the full-size images (the blob result from the Extract from Micrographs job’s Particles output group). This output result swapping is performed with the low level results interface.
In this section, we demonstrate the use of the low level results interface by working through the example above. Keep in mind that any result can be replaced using the low level interface — there is nothing unique about the blob output result which we replace here.
We begin by setting up an Extract From Micrographs job to extract particles picked with Blob Picker. In addition to extracting a full-size image, we use the Second (small) F-crop box size (pix) parameter to extract a much smaller image of each particle. These images will be used during particle curation, for which resolution is much less important than speed.
Extraction box size (pix)
452
Second (small) F-crop box size (pix)
96
Save results in 16-bit floating point
On
We proceed through a standard particle curation pipeline using the 96 pixel images until we have a clean particle stack. Because these particle images were downsampled to a pixel size of approximately 5.25 Å, a Homogeneous Refinement easily reaches the Nyquist resolution of 10.5 Å.
We’d like to perform a Local Refinement of the full-size particle images using the poses found with this Homogeneous Refinement using the low level interface. To do so, we will first create a Local Refinement job and connect the Particles and Volumes output groups to the Local Refinement as usual.
We can reveal an input or output’s results by clicking the small arrow on the left-hand side of the group’s box:
These boxes display which specific output results are connected to the input group of the currently building job. As mentioned above, they are automatically connected with the appropriate output results from the output group. However, we want to replace the blob result from the Homogeneous Refinement with the blob result from the full size extracted images. To do so, we open the Extract From Micrographs job and navigate to the Outputs tab. This tab contains a larger version of the output groups seen in the sidebar. Within these larger output groups, we see small boxes corresponding to the individual output results.
To replace the Homogeneous Refinement’s blob result with that of Extract from Micrographs, we drag the blob output result from the Particles Extracted output group to the blob result of the Particle stacks input group.
Note that while dragging an output result, any result slot which can accept that result turns green. After dropping the result into the slot, the job number displayed in the input result updates to reflect the change. In this case, the result initially reads J688.particles.blob.F. This indicates that the result uses the blob result of the particles output group from the final iteration (this is the meaning of the F) of job 688.
Once the result is replaced using the low level results interface, it reads J683.particles.blob.F, meaning it will use the blob result from J683's particles group. The other slots are unchanged, so the ctf and poses from J688 will still be used.
If we run this Local Refinement, we see that the initial poses from the Homogeneous Refinement are used, but the full-size images produce a much higher resolution map.
Some jobs save several versions of output groups. By default, the final version produced by the job is used. However, in some cases it may be desirable to use the output group from an earlier iteration. For example, early iterations of Ab-Initio Reconstruction are often good sources of junk volumes for particle curation. The Low Level Results Interface can also be used to select a specific version of an output result.
You can access earlier versions by clicking the Versions dropdown at the bottom of the output group box. This opens a panel listing each of the iterations. Click an iteration to select it. The name of the output result will update to reflect the change — for example, going from J683.particles.alignments3D to J683.particles.alignments3D.3 if Iteration 3 is selected. Connecting the output result to an input’s result slot will now use the selected version.
Data in this section comes from , originally collected by Kim and colleagues.
In the figure above, you may note that the particles’ location and pick stats are passthroughs. When jobs do not modify the information in an output group, they are treated as passthroughs. Under the hood, this means that CryoSPARC does not save a new copy of the passed-through information for the new job. Instead, the job’s output contains the location of the original information. More information about passthroughs is available .
CryoSPARC and CryoSPARC Tools both automatically fetch passthrough information, so in most cases you can treat passthroughs as if they were a typical output group slot. External programs which directly read .cs files, however, will not have access to passthroughs without special attention. To create a .cs file which contains all of the metadata (including passthroughs), the job first.
