OCCAM: Optimal Data Reuse for Convolutional Neural Networks Article Swipe

MAKE IMPACT

Ashish Gondimalla , Jianqiao Liu , T. N. Vijaykumar , Mithuna Thottethodi · YOU? · · 2021 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2106.14138

Convolutional neural networks (CNNs) are emerging as powerful tools for image processing in important commercial applications. We focus on the important problem of improving the latency of image recognition. CNNs' large data at each layer's input, filters, and output poses a memory bandwidth problem. While previous work captures only some of the enormous data reuse, full reuse implies that the initial input image and filters are read once from off chip and the final output is written once off chip without spilling the intermediate layers' data to off-chip. We propose Occam to capture full reuse via four contributions. (1) We identify the necessary condition for full reuse. (2) We identify the dependence closure as the sufficient condition to capture full reuse using the least on-chip memory. (3) Because the dependence closure is often too large to fit in on-chip memory, we propose a dynamic programming algorithm that optimally partitions a given CNN to guarantee the least off-chip traffic at the partition boundaries for a given on-chip capacity. Occam's partitions reside on different chips forming a pipeline so that a partition's filters and dependence closure remain on-chip as different images pass through (i.e., each partition incurs off-chip traffic only for its inputs and outputs). (4) because the optimal partitions may result in an unbalanced pipeline, we propose staggered asynchronous pipelines (STAP) which replicates the bottleneck stages to improve throughput by staggering the mini-batches across the replicas. Importantly, STAP achieves balanced pipelines without changing Occam's optimal partitioning. Our simulations show that Occam cuts off-chip transfers by 21x and achieves 2.06x and 1.36x better performance, and 33\% and 24\% better energy than the base case and Layer Fusion, respectively. On an FPGA implementation, Occam performs 5.1x better than the base case.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2106.14138

PDF

https://arxiv.org/pdf/2106.14138

OA Status

green

Related Works

10

OpenAlex ID

https://openalex.org/W4311701604

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4311701604

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2106.14138

Digital Object Identifier

Title

OCCAM: Optimal Data Reuse for Convolutional Neural Networks

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2021

Year of publication

Publication date

2021-06-27

Full publication date if available

Authors

Ashish Gondimalla, Jianqiao Liu, T. N. Vijaykumar, Mithuna Thottethodi

List of authors in order

Landing page

https://arxiv.org/abs/2106.14138

Publisher landing page

PDF URL

https://arxiv.org/pdf/2106.14138

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2106.14138

Direct OA link when available

Concepts

Computer science, occam, Parallel computing, Convolutional neural network, Pipeline (software), Reuse, Chip, Computer engineering, Artificial intelligence, Programming language, Engineering, Waste management, Telecommunications

Top concepts (fields/topics) attached by OpenAlex

Cited by

0

Total citation count in OpenAlex

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload