# CryoSPARC Cluster Integration Script Examples

CryoSPARC can integrate with cluster scheduler systems. This page contains examples of integration setups.

Due to the many variations of cluster scheduler systems and their configurations, the examples here will need to be modified for your own specific use case.

For information on what each variable means, see [Downloading and Installing CryoSPARC](/setup-configuration-and-management/how-to-download-install-and-configure/downloading-and-installing-cryosparc.md#connect-a-cluster-to-cryosparc).

## GPU Resource Management

When CryoSPARC launches a job to the cluster, the number of GPUs requested by the user is used in the submission script, but CryoSPARC does not know which specific devices it will be allocated and therefore each job simply tries to use GPUs with device numbers starting at zero. For example, a 2-GPU job will try to use GPUs \[0, 1] when submitted to a cluster. It is the responsibility of the cluster system to correctly allocate requested GPU resources to CryoSPARC jobs while insulating those allocated resources from interference by other jobs. For this purpose, the SLURM scheduler, for example, can combine [Generic Resource (GRES)](https://slurm.schedmd.com/gres.html) management with (Linux) [cgroup](https://slurm.schedmd.com/cgroups.html)[ controls](https://slurm.schedmd.com/cgroups.html).

## A SLURM Example

{% code title="cluster\_info.json" %}

```json
{
    "name": "slurm-lane1",
    "worker_bin_path": "/path/to/cryosparc_worker/bin/cryosparcw",
    "send_cmd_tpl": "{{ command }}",
    "qsub_cmd_tpl": "/opt/slurm/bin/sbatch {{ script_path_abs }}",
    "qstat_cmd_tpl": "/opt/slurm/bin/squeue -j {{ cluster_job_id }}",
    "qdel_cmd_tpl": "/opt/slurm/bin/scancel {{ cluster_job_id }}",
    "qinfo_cmd_tpl": "/opt/slurm/bin/sinfo"
}
```

{% endcode %}

{% code title="cluster\_script.sh" %}

```django
#!/usr/bin/env bash

#SBATCH --job-name cryosparc_{{ project_uid }}_{{ job_uid }}
#SBATCH --cpus-per-task={{ num_cpu }}
#SBATCH --gres=gpu:{{ num_gpu }}
#SBATCH --mem={{ ram_gb|int }}G
#SBATCH --comment="created by {{ cryosparc_username }}"
#SBATCH --output={{ job_dir_abs }}/{{ project_uid }}_{{ job_uid }}_slurm.out
#SBATCH --error={{ job_dir_abs }}/{{ project_uid }}_{{ job_uid }}_slurm.err

{{ run_cmd }}
```

{% endcode %}

## A PBS Example

{% code title="cluster\_info.json" %}

```json
{
    "name" : "pbscluster",
    "worker_bin_path" : "/path/to/cryosparc_worker/bin/cryosparcw",
    "cache_path" : "/path/to/local/SSD/on/cluster/nodes",
    "send_cmd_tpl" : "ssh loginnode {{ command }}",
    "qsub_cmd_tpl" : "qsub {{ script_path_abs }}",
    "qstat_cmd_tpl" : "qstat -as {{ cluster_job_id }}",
    "qdel_cmd_tpl" : "qdel {{ cluster_job_id }}",
    "qinfo_cmd_tpl" : "qstat -q"
}
```

{% endcode %}

{% code title="cluster\_script.sh" %}

```django
#!/bin/bash

#PBS -N cryosparc_{{ project_uid }}_{{ job_uid }}
#PBS -l select=1:ncpus={{ num_cpu }}:ngpus={{ num_gpu }}:mem={{ (ram_gb*1000)|int }}mb:gputype=P100
#PBS -o {{ job_dir_abs }}/cluster.out
#PBS -e {{ job_dir_abs }}/cluster.err

{{ run_cmd }}
```

{% endcode %}

## A Gridengine Example

{% code title="cluster\_info.json" %}

```json
{
    "name" : "ugecluster",
    "worker_bin_path" : "/u/cryosparcuser/cryosparc/cryosparc_worker/bin/cryosparcw",
    "cache_path" : "/scratch/cryosparc_cache",
    "send_cmd_tpl" : "{{ command }}",
    "qsub_cmd_tpl" : "qsub {{ script_path_abs }}",
    "qstat_cmd_tpl" : "qstat -j {{ cluster_job_id }}",
    "qdel_cmd_tpl" : "qdel {{ cluster_job_id }}",
    "qinfo_cmd_tpl" : "qstat -q default.q"
}
```

{% endcode %}

{% code title="cluster\_script.sh" %}

```django
#!/bin/bash

## What follows is a simple UGE script:
## Job Name
#$ -N cryosparc_{{ project_uid }}_{{ job_uid }}

## Number of CPUs (select 1 CPU always, and oversubscribe as GPU is per core value)
##$ -pe smp {{ num_cpu }}
#$ -pe smp 1

## Memory per CPU core
#$ -l m_mem_free={{ (ram_gb)|int }}G

## Number of GPUs 
#$ -l gpu_card={{ num_gpu }}

## Time limit 4 days
#$ -l h_rt=345600

## STDOUT/STDERR
#$ -o {{ job_dir_abs }}/cluster.out
#$ -e {{ job_dir_abs }}/cluster.err
#$ -j y

## Number of threads
export OMP_NUM_THREADS={{ num_cpu }}

echo "HOSTNAME: $HOSTNAME"

{{ run_cmd }}
```

{% endcode %}


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://guide.cryosparc.com/setup-configuration-and-management/how-to-download-install-and-configure/cryosparc-cluster-integration-script-examples.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.