IS&T RCS Tutorial - Multi-GPU PyTorch

3:30 pm on Wednesday, June 2, 2021
5:30 pm on Wednesday, June 2, 2021
Zoom - Registered attendees will be emailed the Zoom link approximately two days before the tutorial
In this advanced tutorial, we will demonstrate how you can request and utilize multiple GPUs on the SCC to process Deep Learning workloads under the PyTorch Framework. We will explore different parallelization paradigms. The tutorial will not be hands-on due to scarcity of available GPUs. However, you can follow our examples using the code we will provide during the tutorial. In addition, we will illustrate how to measure your code’s performance, and consequently evaluate the suitability for parallelization over multiple GPUs. The aim is to equip you with the tools and knowledge that could help you make the best use of the available resources. Prerequisite: some prior knowledge of Machine Learning, Deep Learning, PyTorch, Python, and Software development is necessary.