<aside> ❗ Headings and text with “▶” can be toggled to expand contained info. You can also see the Table of Contents as a collapsible sidebar on the right →

</aside>

Table of Contents

Introduction

In this module, students will be introduced to the steps of designing ML accelerators with GPUs. It will provide an opportunity for understanding the relationship between ML accelerators and GPUs, uncovering how GPUs play a vital role in accelerating machine learning tasks.

Objectives

Be able to understand the basic kernels of ML accelerators for GPUs.

Readings

Required

Optional

Notes / Things that stand out

GPU and ML

Why are GPUs Good for ML?

• ML workloads have a high number of floating point operations.
• They are also high data parallel operations.
• GPUs provide dense floating point operations.
• GPUs have high memory bandwidth (e.g. GDDR, HBM), and can make better use of them.
• GPUs can easily employ flexible data format standards (e.g. TF, BF, INT4 etc).
• ML workloads are statistical computing based computations, which works well with GPU’s underlying execution models.

DNN Operation Categories

• Element-wise operations.
  • e.g. Activation operations.