Differentiable programming in machine learning Article Swipe

PDF

Related Concepts

Python (programming language) Computer science Automatic differentiation Strengths and weaknesses Artificial intelligence Differentiable function Deep learning Machine learning Programming language Mathematics Philosophy Mathematical analysis Epistemology Computation

Marija Kostić , Dražen Drašković ·

YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.5937/tehnika2306699k · OA: W4391014952

This paper explains automatic differentiation, discussing two primary modes - forward and backward - and their respective implementation methods. In the context of issues encountered in machine learning and deep learning, the forward mode is deemed more suitable as it efficiently differentiates functions with numerous inputs compared to outputs. Given Python's pivotal role in the ML landscape, the paper elaborates on two widely used deep learning libraries-PyTorch and TensorFlow. While both these libraries support automatic differentiation, they adopt distinct approaches, each carrying its unique strengths and weaknesses.