HTC Computing Guides

Get started

System overview Roadmap to getting started Log into CHTC Quick reference: Basic shell commands Quick reference: HTCondor commands

Submit jobs

Submit jobs using HTCondor

Job submission basics Submit multiple jobs Submit an interactive job Monitor your jobs

Arguments and shell scripting

Basic scripting and job submission with arguments Practice: Pass arguments from the submit file to the executable script Practice: Pass multiple arguments to multiple jobs with one submit file

Manage data

Use and transfer data in jobs on the HTC system Manage Large Data in HTC Jobs Check disk quota and usage Directly transfer files between ResearchDrive and your jobs Transfer files to/from CHTC

Local computer ResearchDrive (via smbclient) Globus Private GitHub repo

Software

Overview Containers

What are containers? Using software in Apptainer containers

Use Apptainer containers More about Apptainer containers

Practice: Build a container

Convert Docker images to Apptainer/Singularity images

Using software in Docker containers

Use Docker containers More about Docker containers Reference: Explore and test Docker containers

Recipes repository

Other Modes of Software Use

Compile software in an interactive job Licensed software and modules Use software available in modules

Workflows

Checkpoint jobs DAGMan workflow manager

Submit workflows with DAGman Practice: Run a simple DAGMan workflow Practice: Run an diamond DAGMan workflow

Special use cases

Use GPUs Interactive GPU Jobs Scale beyond local HTC capacity Submit high memory jobs Machine learning jobs Custom Linux versions

Troubleshooting

Status page Known issues Request variable memory Windows/Linux incompatibility Get help

External Resources

HTCondor manual Software Carpentry UW-Madison software training for students

Guide Sections

Get started Submit jobs Manage data Software Workflows Special use cases Troubleshooting External Resources