NeRV: Neural Representations for Videos
We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB im...
Saved in:
Published in | arXiv.org |
---|---|
Main Authors | , , , , , |
Format | Paper |
Language | English |
Published |
Ithaca
Cornell University Library, arXiv.org
26.10.2021
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB image. Video encoding in NeRV is simply fitting a neural network to video frames and decoding process is a simple feedforward operation. As an image-wise implicit representation, NeRV output the whole image and shows great efficiency compared to pixel-wise implicit representation, improving the encoding speed by 25x to 70x, the decoding speed by 38x to 132x, while achieving better video quality. With such a representation, we can treat videos as neural networks, simplifying several video-related tasks. For example, conventional video compression methods are restricted by a long and complex pipeline, specifically designed for the task. In contrast, with NeRV, we can use any neural network compression method as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches (H.264, HEVC \etc). Besides compression, we demonstrate the generalization of NeRV for video denoising. The source code and pre-trained model can be found at https://github.com/haochen-rye/NeRV.git. |
---|---|
AbstractList | We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as frame sequences, we represent videos as neural networks taking frame index as input. Given a frame index, NeRV outputs the corresponding RGB image. Video encoding in NeRV is simply fitting a neural network to video frames and decoding process is a simple feedforward operation. As an image-wise implicit representation, NeRV output the whole image and shows great efficiency compared to pixel-wise implicit representation, improving the encoding speed by 25x to 70x, the decoding speed by 38x to 132x, while achieving better video quality. With such a representation, we can treat videos as neural networks, simplifying several video-related tasks. For example, conventional video compression methods are restricted by a long and complex pipeline, specifically designed for the task. In contrast, with NeRV, we can use any neural network compression method as a proxy for video compression, and achieve comparable performance to traditional frame-based video compression approaches (H.264, HEVC \etc). Besides compression, we demonstrate the generalization of NeRV for video denoising. The source code and pre-trained model can be found at https://github.com/haochen-rye/NeRV.git. |
Author | Wang, Hanyu Ser-Nam Lim Ren, Yixuan Chen, Hao Shrivastava, Abhinav He, Bo |
Author_xml | – sequence: 1 givenname: Hao surname: Chen fullname: Chen, Hao – sequence: 2 givenname: Bo surname: He fullname: He, Bo – sequence: 3 givenname: Hanyu surname: Wang fullname: Wang, Hanyu – sequence: 4 givenname: Yixuan surname: Ren fullname: Ren, Yixuan – sequence: 5 fullname: Ser-Nam Lim – sequence: 6 givenname: Abhinav surname: Shrivastava fullname: Shrivastava, Abhinav |
BookMark | eNrjYmDJy89LZWLgNDI2NtS1MDEy4mDgLS7OMjAwMDIzNzI1NeZkUPdLDQqzUvBLLS1KzFEISi0oSi1OzStJLMnMzytWSMsvUgjLTEnNL-ZhYE1LzClO5YXS3AzKbq4hzh66BUX5haWpxSXxWfmlRXlAqXgjUwszM3NjAxNTY-JUAQBmEDE5 |
ContentType | Paper |
Copyright | 2021. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
Copyright_xml | – notice: 2021. This work is published under http://creativecommons.org/licenses/by-nc-nd/4.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License. |
DBID | 8FE 8FG ABJCF ABUWG AFKRA AZQEC BENPR BGLVJ CCPQU DWQXO HCIFZ L6V M7S PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
DatabaseName | ProQuest SciTech Collection ProQuest Technology Collection Materials Science & Engineering Collection ProQuest Central (Alumni) ProQuest Central ProQuest Central Essentials ProQuest Central Technology Collection ProQuest One Community College ProQuest Central Korea SciTech Premium Collection (Proquest) (PQ_SDU_P3) ProQuest Engineering Collection Engineering Database Publicly Available Content Database ProQuest One Academic Eastern Edition (DO NOT USE) ProQuest One Academic ProQuest One Academic UKI Edition ProQuest Central China Engineering Collection |
DatabaseTitle | Publicly Available Content Database Engineering Database Technology Collection ProQuest Central Essentials ProQuest One Academic Eastern Edition ProQuest Central (Alumni Edition) SciTech Premium Collection ProQuest One Community College ProQuest Technology Collection ProQuest SciTech Collection ProQuest Central China ProQuest Central ProQuest Engineering Collection ProQuest One Academic UKI Edition ProQuest Central Korea Materials Science & Engineering Collection ProQuest One Academic Engineering Collection |
DatabaseTitleList | Publicly Available Content Database |
Database_xml | – sequence: 1 dbid: 8FG name: ProQuest Technology Collection url: https://search.proquest.com/technologycollection1 sourceTypes: Aggregation Database |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Physics |
EISSN | 2331-8422 |
Genre | Working Paper/Pre-Print |
GroupedDBID | 8FE 8FG ABJCF ABUWG AFKRA ALMA_UNASSIGNED_HOLDINGS AZQEC BENPR BGLVJ CCPQU DWQXO FRJ HCIFZ L6V M7S M~E PIMPY PQEST PQQKQ PQUKI PRINS PTHSS |
ID | FETCH-proquest_journals_25866730453 |
IEDL.DBID | BENPR |
IngestDate | Thu Oct 10 15:38:34 EDT 2024 |
IsOpenAccess | true |
IsPeerReviewed | false |
IsScholarly | false |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_journals_25866730453 |
OpenAccessLink | https://www.proquest.com/docview/2586673045?pq-origsite=%requestingapplication% |
PQID | 2586673045 |
PQPubID | 2050157 |
ParticipantIDs | proquest_journals_2586673045 |
PublicationCentury | 2000 |
PublicationDate | 20211026 |
PublicationDateYYYYMMDD | 2021-10-26 |
PublicationDate_xml | – month: 10 year: 2021 text: 20211026 day: 26 |
PublicationDecade | 2020 |
PublicationPlace | Ithaca |
PublicationPlace_xml | – name: Ithaca |
PublicationTitle | arXiv.org |
PublicationYear | 2021 |
Publisher | Cornell University Library, arXiv.org |
Publisher_xml | – name: Cornell University Library, arXiv.org |
SSID | ssj0002672553 |
Score | 3.3675826 |
SecondaryResourceType | preprint |
Snippet | We propose a novel neural representation for videos (NeRV) which encodes videos in neural networks. Unlike conventional representations that treat videos as... |
SourceID | proquest |
SourceType | Aggregation Database |
SubjectTerms | Decoding Neural networks Pipeline design Representations Rye Source code Video compression |
Title | NeRV: Neural Representations for Videos |
URI | https://www.proquest.com/docview/2586673045 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1JSwMxFH7YDoI3V1xqCSh4Cs5kls54EZQZi9ChDFp6K9kGBOmWevW3-xJSPQg9hsAjPJK3fnkfwK0QGecybGkiNSYoMo0oj0VLNQ8VV5YO3k1iGtXZ8D15naZTX3AzHla5tYnOUKuFtDXye5bmlqESI5DH5Ypa1ijbXfUUGh0IWJTYNm3wVNbj5rfKwrIBxszxP0PrvEd1CMGYL_X6CPb0_Bj2HehSmhO4q3UzeSB2Pgb_JI3DpPqvQHNDMJokkw-lF-YUbqry7XlIt9Jn_gaY2d954zPoYiqvz4GwIsQHFElVtJiXKsEHQulU50oWzKrlAnq7JF3u3r6CA2YRF2hZWdaD7mb9pa_RZW5EHzp59dL32sHV6Lv8AXv3djA |
link.rule.ids | 786,790,12792,21416,33406,33777,43633,43838 |
linkProvider | ProQuest |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwfV1bS8MwFD5oi-ibV7xMDSj4FOzStWt9EZSNqlsZZY69lVxOQZBtLvP_m4RMH4Q9Bw7hkHznki_nA7gVIuVcRg3tSDQFikzalMeiocgjxZWVg3eTmIZlWrx3XqfJ1DfctKdVrjHRAbWaS9sjv2dJZhUqTQbyuPiiVjXKvq56CY1tCO3IzSyA8KlXjqrfLgtLuyZnjv8BrYse_X0IR3yBywPYwtkh7DjSpdRHcFdiNXkgdj4G_ySV46T6r0AzTUw2SSYfCuf6GG76vfFzQdfWa38CdP233_gEAlPK4ykQlkfmArWlyhtTlyrBu0JhgpmSObNuOYPWJkvnm5evYbcYDwf14KV8u4A9ZtkXBmVZ2oJgtfzGSxM-V-LK--gHoXJ3Dw |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=NeRV%3A+Neural+Representations+for+Videos&rft.jtitle=arXiv.org&rft.au=Chen%2C+Hao&rft.au=He%2C+Bo&rft.au=Wang%2C+Hanyu&rft.au=Ren%2C+Yixuan&rft.date=2021-10-26&rft.pub=Cornell+University+Library%2C+arXiv.org&rft.eissn=2331-8422 |