Towards Scalable Neural Representation for Diverse Videos Article Swipe

MAKE IMPACT

Bo He , Xitong Yang , Hanyu Wang , Zuxuan Wu , Hao Chen , Shuaiyi Huang , Yixuan Ren , Ser-Nam Lim , Abhinav Shrivastava · YOU? · · 2023 · Open Access · · DOI: https://doi.org/10.48550/arxiv.2303.14124

Implicit neural representations (INR) have gained increasing attention in representing 3D scenes and images, and have been recently applied to encode videos (e.g., NeRV, E-NeRV). While achieving promising results, existing INR-based methods are limited to encoding a handful of short videos (e.g., seven 5-second videos in the UVG dataset) with redundant visual content, leading to a model design that fits individual video frames independently and is not efficiently scalable to a large number of diverse videos. This paper focuses on developing neural representations for a more practical setup -- encoding long and/or a large number of videos with diverse visual content. We first show that instead of dividing videos into small subsets and encoding them with separate models, encoding long and diverse videos jointly with a unified model achieves better compression results. Based on this observation, we propose D-NeRV, a novel neural representation framework designed to encode diverse videos by (i) decoupling clip-specific visual content from motion information, (ii) introducing temporal reasoning into the implicit neural network, and (iii) employing the task-oriented flow as intermediate output to reduce spatial redundancies. Our new model largely surpasses NeRV and traditional video compression techniques on UCF101 and UVG datasets on the video compression task. Moreover, when used as an efficient data-loader, D-NeRV achieves 3%-10% higher accuracy than NeRV on action recognition tasks on the UCF101 dataset under the same compression ratios.

Type

preprint

Language

en

Landing Page

http://arxiv.org/abs/2303.14124

PDF

https://arxiv.org/pdf/2303.14124

OA Status

green

Cited By

1

Related Works

10

OpenAlex ID

https://openalex.org/W4361021222

Raw OpenAlex JSON

OpenAlex ID

https://openalex.org/W4361021222

Canonical identifier for this work in OpenAlex

DOI

https://doi.org/10.48550/arxiv.2303.14124

Digital Object Identifier

Title

Towards Scalable Neural Representation for Diverse Videos

Work title

Type

preprint

OpenAlex work type

Language

en

Primary language

Publication year

2023

Year of publication

Publication date

2023-03-24

Full publication date if available

Authors

Bo He, Xitong Yang, Hanyu Wang, Zuxuan Wu, Hao Chen, Shuaiyi Huang, Yixuan Ren, Ser-Nam Lim, Abhinav Shrivastava

List of authors in order

Landing page

https://arxiv.org/abs/2303.14124

Publisher landing page

PDF URL

https://arxiv.org/pdf/2303.14124

Direct link to full text PDF

Open access

Yes

Whether a free full text is available

OA status

green

Open access status per OpenAlex

OA URL

https://arxiv.org/pdf/2303.14124

Direct OA link when available

Concepts

Computer science, ENCODE, Encoding (memory), Artificial intelligence, Task (project management), Scalability, Representation (politics), Data compression, Pattern recognition (psychology), Computer vision, Gene, Economics, Biochemistry, Law, Database, Politics, Management, Political science, Chemistry

Top concepts (fields/topics) attached by OpenAlex

Cited by

1

Total citation count in OpenAlex

Citations by year (recent)

2025: 1

Per-year citation counts (last 5 years)

Related works (count)

10

Other works algorithmically related by OpenAlex

Full payload