Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs Article Swipe

MAKE IMPACT

Dingyi Dai , Yichi Zhang , Jiahao Zhang , Zhanqiu Hu , Yaohui Cai , Qi Sun , Zhiru Zhang · YOU? · · 2024 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2401.17544

Quantization is a crucial technique for deploying deep learning models on resource-constrained devices, such as embedded FPGAs. Prior efforts mostly focus on quantizing matrix multiplications, leaving other layers like BatchNorm or shortcuts in floating-point form, even though fixed-point arithmetic is more efficient on FPGAs. A common practice is to fine-tune a pre-trained model to fixed-point for FPGA deployment, but potentially degrading accuracy. This work presents QFX, a novel trainable fixed-point quantization approach that automatically learns the binary-point position during model training. Additionally, we introduce a multiplier-free quantization strategy within QFX to minimize DSP usage. QFX is implemented as a PyTorch-based library that efficiently emulates fixed-point arithmetic, supported by FPGA HLS, in a differentiable manner during backpropagation. With minimal effort, models trained with QFX can readily be deployed through HLS, producing the same numerical results as their software counterparts. Our evaluation shows that compared to post-training quantization, QFX can quantize models trained with element-wise layers quantized to fewer bits and achieve higher accuracy on both CIFAR-10 and ImageNet datasets. We further demonstrate the efficacy of multiplier-free quantization using a state-of-the-art binarized neural network accelerator designed for an embedded FPGA (AMD Xilinx Ultra96 v2). We plan to release QFX in open-source format.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2401.17544

PDF

https://arxiv.org/pdf/2401.17544

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4391462652

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4391462652

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2401.17544

Digital Object Identifier

Title

Trainable Fixed-Point Quantization for Deep Learning Acceleration on FPGAs

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2024

Year of publication

Publication date

2024-01-31

Full publication date if available

Authors

Dingyi Dai, Yichi Zhang, Jiahao Zhang, Zhanqiu Hu, Yaohui Cai, Qi Sun, Zhiru Zhang

List of authors in order

Landing page

https://arxiv.org/abs/2401.17544

Publisher landing page

PDF URL

https://arxiv.org/pdf/2401.17544

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2401.17544

Direct OA link when available

Concepts

Quantization (signal processing), Field-programmable gate array, Acceleration, Computer science, Point (geometry), Deep learning, Artificial intelligence, Algorithm, Mathematics, Physics, Embedded system, Geometry, Classical mechanics

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload