Big Computing
Big Computing at Warwick and Beyond gives an overview of the dedicated computing facilities at Warwick and more widely, which allow you to do "more" than you can on a laptop. Sometimes this means running single large programs, such as a large hydrodynamic simulation for flow in a pipe (High Performance Computing, HPC); sometimes this means running very many small programs (High Throughput Computing, HTC).
Any computing which can't be done on a laptop in a few hours counts!
This two-day workshop aims to cover all the background knowledge to allow you to confidently use these systems, from requesting resources, to making effective use of them. We focus on Warwick's Sulis cluster, but we also discuss the other options there are available right now.
Throughout the workshop we are available to give hands-on support for any issues relating to "big computing" including code help, getting access to machines, and choosing how to run your jobs.
Rough Topics List
- Signing up for services
- Understanding your resource needs
- Choosing suitable machines
- How to actually run a job - from submission to completion
- Running multiple jobs together in a batch
- Databases for IO improvement
- Skills for parallel computing - how machines differ and how to use this effectively
Slides for 1 Day short Workshop
See links below: Introduction, Signup hints, Cloud computing and the first two parts of the Challenges section, Getting started and Reproducibility
Slides for Full 2 Day Workshop
Day 1 AM IntroductionLink opens in a new window
Day 1 AM Terms and Technologies
Day 1 PM GNU Parallel for running multiple programs
Day 1 PM Signup Hints and Info
Day 2 AM Big Computing Challenges Part 1 - Getting Started
Day 2 AM Big Computing Challenges Part 2 - Reproducibility
Day 2 PM Big Computing Challenges Part 3 - Files and IO
Handy Links
Signup instructions for Warwick SCRTP desktop (prereq. for cluster access) - https://warwick.ac.uk/research/rtp/sc/desktop/myaccount
Signup instructions for Warwick's Avon cluster - https://warwick.ac.uk/research/rtp/sc/hpc/register/
Warwick's Sulis cluster info - https://warwick.ac.uk/research/rtp/sc/sulis/
New to Computing? - Research Software Engineering FAQ - https://warwick.ac.uk/research/rtp/sc/rse/faq/
New to Working on a Remote Computer? Our video guide - https://warwick.ac.uk/research/rtp/sc/rse/training/remoteAccess
Livestream
December 2022 stream links:
Day 1 Live stream: https://youtu.be/bVADcievAY4
Day 2 Live stream link: https://youtu.be/vNbi-BkeDvk
December 2023 stream link:
Live stream or review link: https://www.youtube.com/watch?v=jomefMNSEg0