CryoSPARC Guide
  • About CryoSPARC
  • Current Version
  • Licensing
    • Non-commercial license agreement
  • Setup, Configuration and Management
    • CryoSPARC Architecture and System Requirements
    • CryoSPARC Installation Prerequisites
    • How to Download, Install and Configure
      • Obtaining A License ID
      • Downloading and Installing CryoSPARC
      • CryoSPARC Cluster Integration Script Examples
      • Accessing the CryoSPARC User Interface
    • Deploying CryoSPARC on AWS
      • Performance Benchmarks
    • Using CryoSPARC with Cluster Management Software
    • Software Updates and Patches
    • Management and Monitoring
      • Environment variables
      • (Optional) Hosting CryoSPARC Through a Reverse Proxy
      • cryosparcm reference
      • cryosparcm cli reference
      • cryosparcw reference
    • Software System Guides
      • Guide: Updating to CryoSPARC v4
      • Guide: Installation Testing with cryosparcm test
      • Guide: Verify CryoSPARC Installation with the Extensive Validation Job (v4.3+)
      • Guide: Verify CryoSPARC Installation with the Extensive Workflow (≤v4.2)
      • Guide: Performance Benchmarking (v4.3+)
      • Guide: Download Error Reports
      • Guide: Maintenance Mode and Configurable User Facing Messages
      • Guide: User Management
      • Guide: Multi-user Unix Permissions and Data Access Control
      • Guide: Lane Assignments and Restrictions
      • Guide: Queuing Directly to a GPU
      • Guide: Priority Job Queuing
      • Guide: Configuring Custom Variables for Cluster Job Submission Scripts
      • Guide: SSD Particle Caching in CryoSPARC
      • Guide: Data Management in CryoSPARC (v4.0+)
      • Guide: Data Cleanup (v4.3+)
      • Guide: Reduce Database Size (v4.3+)
      • Guide: Data Management in CryoSPARC (≤v3.3)
      • Guide: CryoSPARC Live Session Data Management
      • Guide: Manipulating .cs Files Created By CryoSPARC
      • Guide: Migrating your CryoSPARC Instance
      • Guide: EMDB-friendly XML file for FSC plots
    • Troubleshooting
  • Application Guide (v4.0+)
    • A Tour of the CryoSPARC Interface
    • Browsing the CryoSPARC Instance
    • Projects, Workspaces and Live Sessions
    • Jobs
    • Job Views: Cards, Tree, and Table
    • Creating and Running Jobs
    • Low Level Results Interface
    • Filters and Sorting
    • View Options
    • Tags
    • Flat vs Hierarchical Navigation
    • File Browser
    • Blueprints
    • Workflows
    • Inspecting Data
    • Managing Jobs
    • Interactive Jobs
    • Upload Local Files
    • Managing Data
    • Downloading and Exporting Data
    • Instance Management
    • Admin Panel
  • Cryo-EM Foundations
    • Image Formation
      • Contrast in Cryo-EM
      • Waves as Vectors
      • Aliasing
  • Expectation Maximization in Cryo-EM
  • Processing Data in cryoSPARC
    • Get Started with CryoSPARC: Introductory Tutorial (v4.0+)
    • Tutorial Videos
    • All Job Types in CryoSPARC
      • Import
        • Job: Import Movies
        • Job: Import Micrographs
        • Job: Import Particle Stack
        • Job: Import 3D Volumes
        • Job: Import Templates
        • Job: Import Result Group
        • Job: Import Beam Shift
      • Motion Correction
        • Job: Patch Motion Correction
        • Job: Full-Frame Motion Correction
        • Job: Local Motion Correction
        • Job: MotionCor2 (Wrapper) (BETA)
        • Job: Reference Based Motion Correction (BETA)
      • CTF Estimation
        • Job: Patch CTF Estimation
        • Job: Patch CTF Extraction
        • Job: CTFFIND4 (Wrapper)
        • Job: Gctf (Wrapper) (Legacy)
      • Exposure Curation
        • Job: Micrograph Denoiser (BETA)
        • Job: Micrograph Junk Detector (BETA)
        • Interactive Job: Manually Curate Exposures
      • Particle Picking
        • Interactive Job: Manual Picker
        • Job: Blob Picker
        • Job: Template Picker
        • Job: Filament Tracer
        • Job: Blob Picker Tuner
        • Interactive Job: Inspect Particle Picks
        • Job: Create Templates
      • Extraction
        • Job: Extract from Micrographs
        • Job: Downsample Particles
        • Job: Restack Particles
      • Deep Picking
        • Guideline for Supervised Particle Picking using Deep Learning Models
        • Deep Network Particle Picker
          • T20S Proteasome: Deep Particle Picking Tutorial
          • Job: Deep Picker Train and Job: Deep Picker Inference
        • Topaz (Bepler, et al)
          • T20S Proteasome: Topaz Particle Picking Tutorial
          • T20S Proteasome: Topaz Micrograph Denoising Tutorial
          • Job: Topaz Train and Job: Topaz Cross Validation
          • Job: Topaz Extract
          • Job: Topaz Denoise
      • Particle Curation
        • Job: 2D Classification
        • Interactive Job: Select 2D Classes
        • Job: Reference Based Auto Select 2D (BETA)
        • Job: Reconstruct 2D Classes
        • Job: Rebalance 2D Classes
        • Job: Class Probability Filter (Legacy)
        • Job: Rebalance Orientations
        • Job: Subset Particles by Statistic
      • 3D Reconstruction
        • Job: Ab-Initio Reconstruction
      • 3D Refinement
        • Job: Homogeneous Refinement
        • Job: Heterogeneous Refinement
        • Job: Non-Uniform Refinement
        • Job: Homogeneous Reconstruction Only
        • Job: Heterogeneous Reconstruction Only
        • Job: Homogeneous Refinement (Legacy)
        • Job: Non-uniform Refinement (Legacy)
      • CTF Refinement
        • Job: Global CTF Refinement
        • Job: Local CTF Refinement
        • Job: Exposure Group Utilities
      • Conformational Variability
        • Job: 3D Variability
        • Job: 3D Variability Display
        • Job: 3D Classification
        • Job: Regroup 3D Classes
        • Job: Reference Based Auto Select 3D (BETA)
        • Job: 3D Flexible Refinement (3DFlex) (BETA)
      • Postprocessing
        • Job: Sharpening Tools
        • Job: DeepEMhancer (Wrapper)
        • Job: Validation (FSC)
        • Job: Local Resolution Estimation
        • Job: Local Filtering
        • Job: ResLog Analysis
        • Job: ThreeDFSC (Wrapper) (Legacy)
      • Local Refinement
        • Job: Local Refinement
        • Job: Particle Subtraction
        • Job: Local Refinement (Legacy)
      • Helical Reconstruction
        • Helical symmetry in CryoSPARC
        • Job: Helical Refinement
        • Job: Symmetry search utility
        • Job: Average Power Spectra
      • Utilities
        • Job: Exposure Sets Tool
        • Job: Exposure Tools
        • Job: Generate Micrograph Thumbnails
        • Job: Cache Particles on SSD
        • Job: Check for Corrupt Particles
        • Job: Particle Sets Tool
        • Job: Reassign Particles to Micrographs
        • Job: Remove Duplicate Particles
        • Job: Symmetry Expansion
        • Job: Volume Tools
        • Job: Volume Alignment Tools
        • Job: Align 3D maps
        • Job: Split Volumes Group
        • Job: Orientation Diagnostics
      • Simulations
        • Job: Simulate Data (GPU)
        • Job: Simulate Data (Legacy)
    • CryoSPARC Tools
    • Data Processing Tutorials
      • Case study: End-to-end processing of a ligand-bound GPCR (EMPIAR-10853)
      • Case Study: DkTx-bound TRPV1 (EMPIAR-10059)
      • Case Study: Pseudosymmetry in TRPV5 and Calmodulin (EMPIAR-10256)
      • Case Study: End-to-end processing of an inactive GPCR (EMPIAR-10668)
      • Case Study: End-to-end processing of encapsulated ferritin (EMPIAR-10716)
      • Case Study: Exploratory data processing by Oliver Clarke
      • Tutorial: Tips for Membrane Protein Structures
      • Tutorial: Common CryoSPARC Plots
      • Tutorial: Negative Stain Data
      • Tutorial: Phase Plate Data
      • Tutorial: EER File Support
      • Tutorial: EPU AFIS Beam Shift Import
      • Tutorial: Patch Motion and Patch CTF
      • Tutorial: Float16 Support
      • Tutorial: Particle Picking Calibration
      • Tutorial: Blob Picker Tuner
      • Tutorial: Helical Processing using EMPIAR-10031 (MAVS)
      • Tutorial: Maximum Box Sizes for Refinement
      • Tutorial: CTF Refinement
      • Tutorial: Ewald Sphere Correction
      • Tutorial: Symmetry Relaxation
      • Tutorial: Orientation Diagnostics
      • Tutorial: BILD files in CryoSPARC v4.4+
      • Tutorial: Mask Creation
      • Case Study: Yeast U4/U6.U5 tri-snRNP
      • Tutorial: 3D Classification
      • Tutorial: 3D Variability Analysis (Part One)
      • Tutorial: 3D Variability Analysis (Part Two)
      • Tutorial: 3D Flexible Refinement
        • Installing 3DFlex Dependencies (v4.1–v4.3)
      • Tutorial: 3D Flex Mesh Preparation
    • Webinar Recordings
  • Real-time processing in cryoSPARC Live
    • About CryoSPARC Live
    • Prerequisites and Compute Resources Setup
    • How to Access cryoSPARC Live
    • UI Overview
    • New Live Session: Start to Finish Guide
    • CryoSPARC Live Tutorial Videos
    • Live Jobs and Session-Level Functions
    • Performance Metrics
    • Managing a CryoSPARC Live Session from the CLI
    • FAQs and Troubleshooting
  • Guides for v3
    • v3 User Interface Guide
      • Dashboard
      • Project and Workspace Management
      • Create and Build Jobs
      • Queue Job, Inspect Job and Other Job Actions
      • View and Download Results
      • Job Relationships
      • Resource Manager
      • User Management
    • Tutorial: Job Builder
    • Get Started with CryoSPARC: Introductory Tutorial (v3)
    • Tutorial: Manually Curate Exposures (v3)
  • Resources
    • Questions and Support
Powered by GitBook
On this page
  • Job Management
  • Current Jobs Dialog
  • Lanes
  • Main App Footer
  • Job Actions
  • Core Actions
  • Multi Actions
  1. Application Guide (v4.0+)

Managing Jobs

PreviousInspecting DataNextInteractive Jobs

Last updated 2 years ago

Job Management

Current Jobs Dialog

This is the hub for managing jobs and viewing instance usage. The Current Jobs panel shows all jobs in progress in a grid of their containing lanes.

The control bar at the top of the view contains a view toggle for filtering between all jobs, queued jobs, or active jobs. On the right side of the bar, instance token usage is visible (if applicable). Token usage is broken down by each active instance and summarized in total.

Lanes

The main area is composed of lanes and their attached jobs. This view has been condensed from earlier versions of CryoSPARC by moving all inactive lanes into a single tracker pinned to the left side of the viewing area. Only lanes that are currently active will be displayed separately. Each lane is composed of a three parts.

Header

The header shows the lane title, type, and the number of jobs currently attached to the lane. The job number will change if viewing jobs filtered by active or queued, and will show what number out of the total are currently being displayed.

The header also houses additional information visible when clicked to expand. This will reveal the title, name, and description of that lane.

Jobs

The jobs section houses unique job cards with features specific to managing the instance.

The card header shows the status colour indicator followed by the job’s project ID and individual job ID. On the right side is the number of GPU tokens being used by the job, its running priority, and its current status. The job type is displayed below.

The job body includes information about who created it and how long it has been attached to the lane in its current status. Below is an action button to kill the job, and a trigger button for the job’s quick access menu.

The job card can also be selected and inspected in the sidebar, and opened from the “View Job” button at the bottom of the sidebar (to be noted: this will close the manage dialog as only one dialog can be open at a time, but it is possible to go back to the manage dialog by simply pressing the native browser back button).

Footer

The lane footer contains a “View Details” button that will navigate to the flattened jobs browse section set to the table view, and will set filters for the relevant lane and job statuses of “Queued, Launched, Started, Waiting, and Running”. This allows for more detailed and dynamic inspection of jobs running on the lane.

Main App Footer

The app footer is designed to give contextual information about the jobs currently active in the instance at a glance. It is highly dynamic and will size up or down depending on the space available in the window. The footer is made up of two sections, the active job section and the active target section.

  • Active Jobs: The active job section is composed of a details button which is followed by job pill buttons. The details button shows a count of how many jobs are currently queued and/or active. This button will open up the “Current Jobs” dialog when clicked. The job pill buttons show the job’s project ID, job ID, and a status indicator. Hovering a job button will show the full job type in a tooltip, and clicking on a job will open that job’s inspection dialog (this will not navigate you away from your current page).

  • Active Targets: The active targets section has its own details button which similarly opens the “Current Jobs” dialog. The target buttons show the applicable lane’s name and can be clicked to open a target dialog with a small grid of job cards representing the jobs running on that target. Each card has nearly identical functionality to those in the “Current Jobs” dialog.

Job Actions

All jobs have variety of actions available for both processing data as well as doing general management and maintenance. These actions can be accessed from two distinct locations, the quick actions menu, and the sidebar actions panel.

Sidebar Actions Panel

The sidebar actions panel is analogous to the actions section in the jobs sidebar of previous versions of CryoSPARC. This panel can be expanded by clicking the “Actions” button on the footer of the job sidebar. The actions panel overlays the main sidebar area, and allows you to continue to see and scroll through all of the information available in the sidebar. This panel is focused on managing the current job’s data and only includes core job actions (these are detailed below). This panel will remain open until you decide to close it, which allows for ease of iteration while processing data within a job whether the job dialog is open or closed.

Quick Actions Menu

Core Actions

Queue Job

Link Job

To avoid redundant processing, it's possible to link jobs across multiple workspaces to continue processing in a new or existing workspace. Any changes you make to the job in one workspace will be reflected in the other, as both workspaces will be displaying the exact same job. If a linked job is deleted from any workspace, all other instances of it in other workspaces will also be deleted. You can unlink a job from a workspace if needed.

Move Job

When re-organizing your workspaces, you may find it necessary to move a job from one workspace to another. This action will cause the job to disappear from its original workspace and move to the new one.

Kill Job

You can kill a job while it is running. A pop-up message will allow you to confirm the action. Killed jobs will display as orange and will remain as part of the workspace until you delete them. Killed jobs will not have their intermediate results expunged until they are cleared.

Restart Job

Any job that is in a killable or clearable state can be restarted. This action will run through a process of automatically killing the job, clearing the job, and then queueing the job to the same lane with the same parameters it was initially run with.

Clear Job

When a job has been killed, has failed, or has been completed, it can be cleared. This means that its results and outputs are all erased from the file system, but its connections to other jobs and parameters are retained. This makes it easy to correct a mistakenly set parameter, or to re-run a failed job. Clearing also removes queued jobs from the queue and resets jobs to Building status.

Clear Intermediate Results

Unused intermediate results can be cleared from iterative jobs to save space. This action can be executed at a job level (by clicking "Clear Intermediate Results" in the quick actions menu or actions panel) or at a project level for every job (by clicking "Clear Intermediate Results" in the project level quick actions menu or actions panel). This function will remove all unused outputs created by iterative jobs that save raw data at every iteration. Final results for every result slot will be retained, whether they have been used elsewhere or not.

Export Job

Clone Job

Cloning is particularly useful when you wish to process subsequent jobs in a new workspace for better organization of your experiment, or if you wish to quickly replicate a job (to try another setting of parameters) without dragging and dropping inputs into the Job Builder. Select the job you wish to clone, and from the quick action menu or sidebar actions panel, click 'Clone job'. By default, the job will be cloned in the current workspace. Click 'Queue' to launch the job. Note that you can always edit the parameters of a cloned job before Queuing. You can also change inputs by removing connected inputs and dragging in new inputs.

Mark Job as Complete

Delete Job

While selected, you can delete a job from the quick actions menu or sidebar actions panel. A pop-up message will ask you to confirm the deletion. Once deleted, the job will disappear from the workspace (except in the tree view, if it has children that need to be displayed). Note that job IDs within a workspace are unique, so a new job in the same workspace will not be assigned the same ID as a previously run or previously deleted job. Deleted jobs are cleared before deletion, meaning that their intermediate and final results are erased from disk.

Multi Actions

Multi actions are actions that only appear when multiple jobs have been selected. This can be done by holding the command key and clicking on job cards or rows to select multiple, or by holding the command key and pressing or holding either the left or right arrow keys with one or more jobs already selected.

Once multiple jobs have been selected a new multi-actions menu will become available in place of the default actions menu when right clicking on job cards or rows. A multi-actions palette is also available and can be activated by clicking the “Actions” button at the bottom of the multi-select sidebar.

Jobs can be removed from the selection by either clicking the “clear” button in the top right corner of the job cards in the multi-select sidebar, or by holding the command key and clicking selected job cards or rows.

All selected jobs can be cleared by either clicking the “clear” button on the righthand side of the multi-select sidebar header, clicking the background of the cards view, or pressing the escape key.

Queue Jobs

This action allows you to select multiple applicable jobs and choose a lane to queue them too. All selected jobs will be queued to the chosen lane, with the exception of interactive jobs and master direct import jobs, which will automatically be queued to the default lane even if a different lane was chosen.

Kill Jobs

This action will kill all applicable selected jobs, and works identically to the singular “Kill Job” action.

Clear Jobs

This action will clear all applicable selected jobs, and works identically to the singular “Clear Job” action.

Restart Jobs

This action allows you to restart any selection of jobs that are in a killable or clearable state. It will automatically kill any running jobs, clear all stopped jobs, and then queue each job to the same lane with the same parameters it was initially run with.

Only a single restart process can be performed at a time in order to prevent jobs states from becoming inconsistent during the operation.

Mark Jobs as Complete

This action will mark all applicable selected jobs as complete, and works identically to the singular “Mark Jobs as Complete” action.

Link Jobs

This action allows you to link all selected jobs to another workspace. When selecting this option a confirmation alert will be shown with a list of all selected jobs for review as well as options to choose the workspace where the selected jobs will be linked, or create a new workspace to link the jobs to. When the link is confirmed the jobs will be linked to the chosen or new workspace and you will be navigated into the workspace with the linked jobs selected.

Move Jobs

This action functions similarly to the “Link Jobs” action and allows you to move all selected jobs into another workspace. When selecting this option a confirmation alert will be shown with a list of all selected jobs for review as well as options to choose the workspace where the selected jobs will be moved, or create a new workspace to move the jobs into. When the move is confirmed the jobs will be moved into the chosen or new workspace and you will be navigated into the workspace with the moved jobs selected.

Clone Jobs

This action allows you to clone a variable number of jobs based on how many you have selected. When this option is chosen an alert will open showing you the jobs to be cloned with an option to clone them into either the current workspace, another workspace, or a new workspace. When cloning is confirmed the jobs will be cloned into the chosen or new workspace, if the chosen workspace is different from the one you are currently in you will be navigated into it. The cloned jobs will be selected automatically after the action is complete.

When selecting “Clone Jobs” the alert will open with the “Clone” confirmation button automatically focused. If you wish to clone the selected jobs into the current workspace, you can simply press the enter key to do this in an expedient manner.

Clone Job Chain

This action allows you to clone a chain of jobs between two selections. This can only be done with a valid chain of jobs, if unrelated jobs are selected as the start and end points you will be alerted that the chain is not valid. When this option is selected a dialog with the same options as the “Clone Jobs” action will open. Here you can see all of the jobs that would be created in the chain (this list is all of the jobs between the two selected points), and options to select a workspace to clone into or create a new workspace to clone into. When cloning is confirmed the job chain will be cloned into the chosen or new workspace, if the chosen workspace is different from the one you are currently in you will be navigated into it.

Clear Intermediate Results

This action will clear the intermediate results of all applicable selected jobs, and works identically to the singular “Clear Intermediate Results” action.

Export Jobs

This action will export all applicable selected jobs, and works identically to the singular “Export Job” action.

Unlink Jobs

This action will unlink all selected linked jobs from the current workspace, and works identically to the singular “Unlink Job” action.

Delete Jobs

This action allows you to multi-select any number of jobs and delete all of them. A confirmation alert will open once this option is selected with a list of all of the jobs to be deleted available for review. Once confirmed, this action will permanently delete all selected jobs.

The quick actions menu is a context menu that can be opened on job cards or rows and gives a wide range general actions and job specific actions. In order to access this menu you must either right click on a job card or row, or click the triple dot menu button on the job card header. The quick actions menu has five sections: the core actions section, multi actions section, quick actions section, general actions section, and navigation section. . The navigation section has options to enter the job’s containing project and view all workspaces inside, or enter the job’s workspace and see all jobs inside. The core actions and multi actions sections are covered below.

Queuing jobs will connect them to a selected lane and set their status to queued until the necessary resources become available to run them. This is covered further in the “” guide section.

Any individual job can be exported, for sharing, manipulation, or archiving. Jobs must be exported manually in order to create a "consolidated" exported-job directory which is then importable. For more details, please see: .

This option allows you to mark failed or killed jobs in CryoSPARC as completed in order to allow their latest outputs to be used for further processing. For more details, please see: .

The quick actions section is populated by common next steps in a processing chain, and allows you to quickly build subsequent jobs with the inputs already connected.
The general actions section includes the tags menu where you can add or remove tags
Creating and Running Jobs
Data Management in CryoSPARC
Exporting a job