Jobs
Last updated
The fundamental unit of data processing within CryoSPARC is a job. Jobs are given inputs and parameters, do work, and through that work, create outputs. The outputs of one job can be connected as the inputs of another job, and through this process, allow you to create branching pipelines through which you can explore and refine your data.
The most common representation of a job in CryoSPARC is a “Job Card”. This card is composed of a number of different sections used to organize job information in a consistent way with flexibility to accommodate the specific differences between job types.
The header shows the job’s ID and a status indicator with the colour of the job’s current status (e.g. green for completed, red for failed, etc). The whole lefthand side of the header is a button that will open the job dialog when clicked.
The righthand side of the header shows all of the available job actions, including buttons to toggle the outputs view, show the quick actions menu, and a button to star the job for future reference.
Underneath the header, you can give your job a unique title if you wish.
The description is an optional text field that can be added and edited in the sidebar (pencil icon). This field supports markdown and will turn into a scrollable sub-section on the card if a non-trivial amount of information is added.
The corners of the job card are populated with at-a-glance information - info tags - about the outputs, such as the number of particles or classes. This view allows you to get a quick overview of your job without needing to open it.
The outputs view can be accessed either by clicking the outputs view toggle (cube with arrow icon) in the card header, or by pressing the O key to switch to the view on all cards at once. This view shows a list of all of the outputs created by the job. These outputs can be used in two ways:
Outputs can be dragged and dropped into the inputs of a building job, visible in the sidebar when the job is selected. Further information is available in the “Tutorial: Job Builder” guide page.
Outputs can alternatively be clicked, which will add them to the “Job Cart”, a system that allows you to filter sequential jobs if their inputs match the selected outputs. This system can be further explored in the “Creating and Running Jobs” section of the guide.
The job’s footer contains a variety of information to gain a better idea of the processing that has occurred within a job at a glance. Each icon can be hovered to reveal a tooltip with additional relevant information.
The timer shows how long your job has been running or how long it ran for. It can be hovered to show who created the job and when they created it, along with the time it concluded processing if it is completed, killed, or failed.
The lane icon shows the name of the lane to which the job has been assigned. Hovering will show the target name and type.
The GPU icon shows the number of GPUs used by the job, and if running, will show the names of the GPUs in use. Hovering will show the number of GPUs used and available on the target.
The gear icon shows the number of custom parameters set on the job, hovering shows those parameters and their values.
Tag buttons shown beside the tag icon represent the tags that have been applied to the job. Clicking on these tag buttons will set the given tag as a filter in the current view.
There are a variety of actions available for interacting with jobs. These range from directly assigning jobs the computational resources that they need to run, to organizing jobs between different workspaces, and even building templates from pre-existing jobs to expedite your work.
Job actions can be accessed in two places: the “Quick Actions Menu”, and the sidebar “Actions Menu”.
You can access the quick actions menu by clicking the trigger button in a card header (the trigger button is represented by an icon with three horizontal dots) or by right clicking on the card.
This menu can be accessed by clicking the “Actions” button at the bottom of the job sidebar with the job you wish to perform actions on selected. This menu contains all job core and compound actions, but does not include quick actions, selection actions, or navigation actions.
Queue Job: Queuing a job will assign the necessary computational resources to it so that is can begin processing your data.
Link Job: Linking a job allows you to connect a job in one workspace to another workspace within the same project. The job will exist in both workspaces as though it were created in them.
Move Job: Moving a job will connect it to another workspace, and remove it from the current one.
Clone Job: Cloning a job will create an identical copy of it, this includes all of its custom parameters and inputs.
Kill Job: Killing a job will interrupt its processing and completely stop it from running. The job will release its computational resources and tokens.
Clear Job: Clearing a job resets it to building status so that its parameters and inputs can be edited again. This is necessary to return the job to an operable state after it has failed or has been killed.
Clear Intermediate Results: This will clear all unused outputs created by the job, in order to save storage space used by the raw data.
Export Job: This will export all essential files (images, streamlog events, etc) into a folder inside of its parent project’s directory. This folder can be used to import the job into another project or even instance.
Mark Job as Complete: Marking a job as complete will change its status to complete and allow you to access its outputs as though the job had completed normally. This can be useful if a job fails or is killed but the generation of its outputs had reached a point sufficient for further data processing.
Unlink Job from Workspace: Unlinking a job will remove it from the current workspace. It will still be fully accessible in all other workspaces it had been linked to or from.
Delete Job: Deleting a job removes it permanently from all views, and renders it inoperable for further data processing. Deleted jobs can still be accessed through a filter for accounting purposes, and continue exist in the database in a limited form.
These actions can only be performed with multiple jobs selected.
Clone {x} Jobs
Clone Job Chain (only available for valid chains of jobs): This action allows you to clone all jobs between, and including, two initial job selections. One job represents the start of the chain, and the other represents the end of the chain.
Delete {x} Jobs
Create Workflow: Workflows are a multi-job template that allow you to quickly build pre-populated sets of jobs to create automated pipelines. They can be explored in more detail in the “Workflows” section of the guide.
Quick create actions are touched on in more detail in the “Creating and running jobs” section of the documentation. These actions allow for the creation of predefined template jobs that are automatically connected to the outputs of the current job.
Create Blueprint: Blueprints are a per job template that can be used to quickly create a job and populate its parameters, or populate/overwrite the parameters of a pre-existing job. They are covered in more depth in the “Blueprints” section of the guide.
Edit tags: Allows you to add or remove pre-configured tags from the job. Tags and tagging are expanded upon in the “Tags” section in the guide.
Select Job Chain: Selects all jobs between two initial job selections, one represents the start of the chain, and the other represents the end of the chain.
Select Ancestor Jobs: Selects the chain of jobs that lead up to and connect to the current job.
Select Descendent Jobs: Selects the chain of jobs that connect to and branch out from the current job.
Select Job Chain: Selects all jobs between two initial job selections, one represents the start of the chain, and the other represents the end of the chain.
Select Ancestor Jobs: Selects the chain of jobs that lead up to and connect to the current job.
Select Descendent Jobs: Selects the chain of jobs that connect to and branch out from the current job.
View workspaces in {project}: This action will navigate you to the workspaces page corresponding to the project that the job is inside.
View jobs in {project} {workspace}: This action will navigate you to the jobs page within the workspace that the job is assigned to.
