User-transparent Distributed TensorFlow Article Swipe

MAKE IMPACT

Abhinav Vishnu , Joseph Manzano , Charles Siegel , Jeff Daily · YOU? · · 2017 · Open Access · · DOI: https://doi.org/10.48550/arxiv.1704.04560

Deep Learning (DL) algorithms have become the {\em de facto} choice for data analysis. Several DL implementations -- primarily limited to a single compute node -- such as Caffe, TensorFlow, Theano and Torch have become readily available. Distributed DL implementations capable of execution on large scale systems are becoming important to address the computational needs of large data produced by scientific simulations and experiments. Yet, the adoption of distributed DL implementations faces significant impediments: 1) most implementations require DL analysts to modify their code significantly -- which is a show-stopper, 2) several distributed DL implementations are geared towards cloud computing systems -- which is inadequate for execution on massively parallel systems such as supercomputers. This work addresses each of these problems. We provide a distributed memory DL implementation by incorporating required changes in the TensorFlow runtime itself. This dramatically reduces the entry barrier for using a distributed TensorFlow implementation. We use Message Passing Interface (MPI) -- which provides performance portability, especially since MPI specific changes are abstracted from users. Lastly -- and arguably most importantly -- we make our implementation available for broader use, under the umbrella of Machine Learning Toolkit for Extreme Scale (MaTEx) at {\texttt http://hpc.pnl.gov/matex}. We refer to our implementation as MaTEx-TensorFlow.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/1704.04560

PDF

https://arxiv.org/pdf/1704.04560

OA Status

green

Cited By

3

References

6

Related Works

10

OpenAlex ID

https://openalex.org/W2612037482

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W2612037482

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.1704.04560

Digital Object Identifier

Title

User-transparent Distributed TensorFlow

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2017

Year of publication

Publication date

2017-04-15

Full publication date if available

Authors

Abhinav Vishnu, Joseph Manzano, Charles Siegel, Jeff Daily

List of authors in order

Landing page

https://arxiv.org/abs/1704.04560

Publisher landing page

PDF URL

https://arxiv.org/pdf/1704.04560

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/1704.04560

Direct OA link when available

Concepts

Implementation, Computer science, Software portability, Distributed computing, Cloud computing, Interface (matter), Message Passing Interface, Deep learning, Parallel computing, Node (physics), Massively parallel, Artificial intelligence, Message passing, Operating system, Programming language, Bubble, Structural engineering, Maximum bubble pressure method, Engineering

Top concepts (fields/topics) attached by OpenAlex

Cited by

3

Total citation count in OpenAlex

Citations by year (recent)

2020: 2, 2019: 1

Per-year citation counts (last 5 years)

References (count)

6

Number of works referenced by this work

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload