Efficient data processing pipeline for event-based vision datasets: techniques and insights

Event-based vision datasets have emerged as a critical asset in advancing the capabilities of real-time perception systems, particularly in fields such as robotics, autonomous vehicles, and human-computer interaction. These datasets enable low-latency processing by capturing asynchronous, pixel-leve...

Full description

Saved in:
Bibliographic Details
Published inEngineering Research Express Vol. 6; no. 4; pp. 45238 - 45251
Main Authors Aitsam, Muhammad, Jimenez Rodriguez, Alejandro, Di Nuovo, Alessandro
Format Journal Article
LanguageEnglish
Published IOP Publishing 01.12.2024
Subjects
Online AccessGet full text

Cover

Loading…
Abstract Event-based vision datasets have emerged as a critical asset in advancing the capabilities of real-time perception systems, particularly in fields such as robotics, autonomous vehicles, and human-computer interaction. These datasets enable low-latency processing by capturing asynchronous, pixel-level changes in the scene, providing a distinct advantage over traditional frame-based systems. However, the diverse formats and characteristics of event-based datasets pose significant challenges for efficient processing and analysis, hindering their broader adoption and integration. In this paper, we present a versatile and comprehensive data processing pipeline designed to address these challenges by supporting multiple event-data formats, including newer formats such as EVT2 and EVT3. This pipeline not only converts data into widely supported formats like AEDAT and NPZ, but also ensures that the unique characteristics of event-based data-such as temporal precision and sparse event representation-are preserved throughout the conversion process. By applying this pipeline to several open-source datasets, we establish a standardized, efficient methodology for dataset manipulation that enhances compatibility and reproducibility in event-based vision research. Additionally, we introduce a novel high-resolution event-based action dataset, converted into various formats using our pipeline, which opens new avenues for exploring event-based techniques in action recognition. This dataset and our pipeline serve as valuable resources for the research community, enabling advancements in real-time vision applications and fostering greater collaboration and standardization across studies.
AbstractList Event-based vision datasets have emerged as a critical asset in advancing the capabilities of real-time perception systems, particularly in fields such as robotics, autonomous vehicles, and human-computer interaction. These datasets enable low-latency processing by capturing asynchronous, pixel-level changes in the scene, providing a distinct advantage over traditional frame-based systems. However, the diverse formats and characteristics of event-based datasets pose significant challenges for efficient processing and analysis, hindering their broader adoption and integration. In this paper, we present a versatile and comprehensive data processing pipeline designed to address these challenges by supporting multiple event-data formats, including newer formats such as EVT2 and EVT3. This pipeline not only converts data into widely supported formats like AEDAT and NPZ, but also ensures that the unique characteristics of event-based data-such as temporal precision and sparse event representation-are preserved throughout the conversion process. By applying this pipeline to several open-source datasets, we establish a standardized, efficient methodology for dataset manipulation that enhances compatibility and reproducibility in event-based vision research. Additionally, we introduce a novel high-resolution event-based action dataset, converted into various formats using our pipeline, which opens new avenues for exploring event-based techniques in action recognition. This dataset and our pipeline serve as valuable resources for the research community, enabling advancements in real-time vision applications and fostering greater collaboration and standardization across studies.
Author Jimenez Rodriguez, Alejandro
Aitsam, Muhammad
Di Nuovo, Alessandro
Author_xml – sequence: 1
  givenname: Muhammad
  orcidid: 0000-0002-4157-7282
  surname: Aitsam
  fullname: Aitsam, Muhammad
  organization: Sheffield Hallam University , Howard St, Sheffield City Centre, Sheffield, S1 1WB, United Kingdom
– sequence: 2
  givenname: Alejandro
  surname: Jimenez Rodriguez
  fullname: Jimenez Rodriguez, Alejandro
  organization: Sheffield Hallam University , Howard St, Sheffield City Centre, Sheffield, S1 1WB, United Kingdom
– sequence: 3
  givenname: Alessandro
  surname: Di Nuovo
  fullname: Di Nuovo, Alessandro
  organization: Sheffield Hallam University , Howard St, Sheffield City Centre, Sheffield, S1 1WB, United Kingdom
BookMark eNp9kE1LAzEQhoNUsGrvHnPy5Npkk51NvUmpH1DwoicPIU0mbaRm12Rb9N-7tSIeROYww_C8w_Ack0FsIhJyxtklZ0qNSxC8UDCpxsZNSlEdkOHPavBrPiKjnMOCSQAONa-H5HnmfbABY0ed6QxtU2OxZ-KStqHFdYhIfZMobnukWJiMjm5DDk384jN2-Yp2aFcxvG0wUxMdDTGH5arLp-TQm3XG0Xc_IU83s8fpXTF_uL2fXs8LyyeyK6wApgAVKgM1GmGdl16VoMCyvoQAlExwh4obW0pVofXG9IEKQJi6FCeE7e_a1OSc0Os2hVeTPjRneudH7wTonQC999NHLvaR0LT6pdmk2D_4H37-B47pXYOWmsmqFEq3zotPhVp2zw
CODEN ERENBL
Cites_doi 10.1155/2021/8973482
10.1109/ACCESS.2020.2996661
10.1109/JSSC.2012.2230553
10.1109/ICCV48922.2021.00266
10.3389/fnins.2015.00437
10.1145/263109.263162
10.1109/IJCNN60899.2024.10650870
10.1126/sciadv.adi1480
10.1109/ACCESS.2022.3219440
10.3389/fnins.2017.00309
ContentType Journal Article
Copyright 2024 The Author(s). Published by IOP Publishing Ltd
Copyright_xml – notice: 2024 The Author(s). Published by IOP Publishing Ltd
DBID O3W
TSCCA
AAYXX
CITATION
DOI 10.1088/2631-8695/ad9235
DatabaseName Open Access: IOP Publishing Free Content
IOPscience (Open Access)
CrossRef
DatabaseTitle CrossRef
DatabaseTitleList CrossRef
Database_xml – sequence: 1
  dbid: O3W
  name: IOP Publishing
  url: http://iopscience.iop.org/
  sourceTypes:
    Enrichment Source
    Publisher
DeliveryMethod fulltext_linktorsrc
EISSN 2631-8695
ExternalDocumentID 10_1088_2631_8695_ad9235
erxad9235
GrantInformation_xml – fundername: HORIZON EUROPE Marie Sklodowska-Curie Actions
  grantid: 955778
  funderid: https://doi.org/10.13039/100018694
GroupedDBID O3W
TSCCA
AAYXX
CITATION
ID FETCH-LOGICAL-c194t-c36086e8e8a67ea3cdf4f82686c0c0c336e4031de81ac2485ecfaa6e85663a723
IEDL.DBID IOP
ISSN 2631-8695
IngestDate Wed Nov 27 13:00:15 EST 2024
Tue Nov 26 22:22:23 EST 2024
IsDoiOpenAccess true
IsOpenAccess true
IsPeerReviewed true
IsScholarly true
Issue 4
Language English
License Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI.
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-c194t-c36086e8e8a67ea3cdf4f82686c0c0c336e4031de81ac2485ecfaa6e85663a723
Notes ERX-105596.R1
ORCID 0000-0002-4157-7282
OpenAccessLink https://proxy.k.utb.cz/login?url=https://iopscience.iop.org/article/10.1088/2631-8695/ad9235
PageCount 14
ParticipantIDs iop_journals_10_1088_2631_8695_ad9235
crossref_primary_10_1088_2631_8695_ad9235
PublicationCentury 2000
PublicationDate 2024-12-01
PublicationDateYYYYMMDD 2024-12-01
PublicationDate_xml – month: 12
  year: 2024
  text: 2024-12-01
  day: 01
PublicationDecade 2020
PublicationTitle Engineering Research Express
PublicationTitleAbbrev ERX
PublicationTitleAlternate Eng. Res. Express
PublicationYear 2024
Publisher IOP Publishing
Publisher_xml – name: IOP Publishing
References Fang (erxad9235bib23) 2021
Lv (erxad9235bib7) 2021; 2021
erxad9235bib24
Kamarudin (erxad9235bib19) 2015; 10
Orchard (erxad9235bib18) 2015; 9
Aitsam (erxad9235bib6) 2024
Dong (erxad9235bib20) 2023; 6
Gallego (erxad9235bib1) 2019
Baby (erxad9235bib8) 2018; 3
erxad9235bib10
Fang (erxad9235bib5) 2023; 9
erxad9235bib16
Gehrig (erxad9235bib14) 2022; 3
erxad9235bib15
Lin (erxad9235bib9) 2022; 13667
erxad9235bib13
Li (erxad9235bib22) 2017; 11
erxad9235bib2
erxad9235bib3
Serrano-Gotarredona (erxad9235bib21) 2013; 48
Mogul (erxad9235bib17) 1997; 27
Lenz (erxad9235bib4) 2021
Khan (erxad9235bib12) 2020; 8
Aitsam (erxad9235bib11) 2022; 10
Rebecq (erxad9235bib25) 2018
References_xml – volume: 2021
  start-page: 1
  year: 2021
  ident: erxad9235bib7
  article-title: Dynamic vision sensor tracking method based on event correlation index
  publication-title: Complexity
  doi: 10.1155/2021/8973482
  contributor:
    fullname: Lv
– year: 2018
  ident: erxad9235bib25
  article-title: Esim: an open event camera simulator
  contributor:
    fullname: Rebecq
– volume: 8
  start-page: 103149
  year: 2020
  ident: erxad9235bib12
  article-title: Lossless compression of data from static and mobile dynamic vision sensors-performance and trade-offs
  publication-title: IEEE Access
  doi: 10.1109/ACCESS.2020.2996661
  contributor:
    fullname: Khan
– volume: 48
  start-page: 827
  year: 2013
  ident: erxad9235bib21
  article-title: A 128,128 1.5 contrast sensitivity 0.9 fpn 3 micro sec latency 4 mw asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers
  publication-title: IEEE J. Solid-State Circuits
  doi: 10.1109/JSSC.2012.2230553
  contributor:
    fullname: Serrano-Gotarredona
– volume: 13667
  start-page: 578
  year: 2022
  ident: erxad9235bib9
  article-title: Dvs-voltmeter: stochastic process-based event simulator for dynamic vision sensors
  publication-title: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
  contributor:
    fullname: Lin
– year: 2021
  ident: erxad9235bib23
  article-title: Incorporating learnable membrane time constant to enhance learning of spiking neural networks
  doi: 10.1109/ICCV48922.2021.00266
  contributor:
    fullname: Fang
– year: 2021
  ident: erxad9235bib4
  article-title: Tonic: event-based datasets and transformations
  contributor:
    fullname: Lenz
– volume: 3
  year: 2018
  ident: erxad9235bib8
  article-title: Dynamic vision sensors for human activity recognition
  contributor:
    fullname: Baby
– volume: 9
  start-page: 1
  year: 2015
  ident: erxad9235bib18
  article-title: Converting static image datasets to spiking neuromorphic datasets using saccades
  publication-title: Frontiers in Neuroscience
  doi: 10.3389/fnins.2015.00437
  contributor:
    fullname: Orchard
– volume: 27
  start-page: 181
  year: 1997
  ident: erxad9235bib17
  article-title: Potential benefits of delta encoding and data compression for http
  publication-title: ACM SIGCOMM Computer Communication Review
  doi: 10.1145/263109.263162
  contributor:
    fullname: Mogul
– start-page: 1–8
  year: 2024
  ident: erxad9235bib6
  doi: 10.1109/IJCNN60899.2024.10650870
  contributor:
    fullname: Aitsam
– ident: erxad9235bib24
– ident: erxad9235bib16
– volume: 6
  year: 2023
  ident: erxad9235bib20
  article-title: Bullying10k: a neuromorphic dataset towards privacy-preserving bullying recognition
  contributor:
    fullname: Dong
– ident: erxad9235bib3
– ident: erxad9235bib10
– volume: 10
  year: 2015
  ident: erxad9235bib19
  article-title: Comparison of image classification techniques using caltech 101 dataset texture-based image retrieval
  publication-title: Article in Journal of Theoretical and Applied Information Technology
  contributor:
    fullname: Kamarudin
– volume: 3
  year: 2022
  ident: erxad9235bib14
  article-title: Are high-resolution event cameras really needed?
  contributor:
    fullname: Gehrig
– volume: 9
  year: 2023
  ident: erxad9235bib5
  article-title: Spikingjelly: an open-source machine learning infrastructure platform for spike-based intelligence
  publication-title: Science Advances
  doi: 10.1126/sciadv.adi1480
  contributor:
    fullname: Fang
– volume: 10
  start-page: 122261
  year: 2022
  ident: erxad9235bib11
  article-title: Neuromorphic computing for interactive robotics: a systematic review
  publication-title: IEEE Access
  doi: 10.1109/ACCESS.2022.3219440
  contributor:
    fullname: Aitsam
– year: 2019
  ident: erxad9235bib1
  article-title: Event-based vision: a survey
  contributor:
    fullname: Gallego
– ident: erxad9235bib15
– volume: 11
  start-page: 5
  year: 2017
  ident: erxad9235bib22
  article-title: Cifar10-dvs: an event-stream dataset for object classification
  publication-title: Frontiers in Neuroscience
  doi: 10.3389/fnins.2017.00309
  contributor:
    fullname: Li
– ident: erxad9235bib13
– ident: erxad9235bib2
SSID ssib046616717
ssib037096498
Score 2.3269281
Snippet Event-based vision datasets have emerged as a critical asset in advancing the capabilities of real-time perception systems, particularly in fields such as...
SourceID crossref
iop
SourceType Aggregation Database
Enrichment Source
Publisher
StartPage 45238
SubjectTerms event based vision dataset
event data processing
neuromophic computing application
Title Efficient data processing pipeline for event-based vision datasets: techniques and insights
URI https://iopscience.iop.org/article/10.1088/2631-8695/ad9235
Volume 6
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1NS8QwEA27evHiByqunznowUN2bZOmqZ5EXNSDenBxQSGk-QARarEVxF_vpOmqKyIihdLDNG2HtO8NefOK0K5OMiWY08QZ5wjLHSN5ajOS5KlmaeZSG1S-l_xsxC7GybiDjj56YZ7K9tPfh8NgFBxS2ArixCDmNCKCZ8lAGaAnSRfNUiG41_OdX11PJhNNgZyzz1qCARBxqF3apcqfBpqCpi5c_gvSDBfQ_eQeg8Dksf9S53399s2-8Z8PsYjmWwaKj0PoEurYYhndnTZWEoBA2GtGcRn6BwDXcPlQ-p51i4He4sbviXjoMzi0pTfxla2rQ_zhB1thVRj8UFS-8q9W0Gh4enNyRtr_LhAdZawmmnIodKywQvHUKqqNYw7KEMH1AWyUcsvgW2CsiJT2lmhWO6XgBKCGVKUxXUUzxVNh1xAWsXEZjZzxNoSxYrk1USYUkAygFSJKe2h_knJZBnsN2SyLCyF9iqRPkQwp6qE9yKZs37Hql7idqTj7_Cq5ZNKbx1MhS-PW_zjSBpqLgcME9commqmfX-wWcJA6327mGuyv6O07tyrV7g
link.rule.ids 314,780,784,27924,27925,38865,38890,53841,53867
linkProvider IOP Publishing
linkToPdf http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV3NS8MwFA9ugngRRcXv5aAHD3G2SdPUm-jG_GB6cDjwENJ8wC5dWSf45_vSdOpARHrp4TWEl7z-fo-89wtCpzrJlGBOE2ecIyx3jOSpzUiSp5qlmUttqPId8sGI3Y-TcXPPad0LMy2bX_8FvAah4ODCpiBOdGNOIyJ4lnSVAXqSdEvjWmgVIpf77OuJvi42FE2BoLPvfIIBGHHIX5rjyt8GW4KnFkzhB9r0N9FGQxPxdZjUFlqxxTZ669V6DwAT2Bd24jIU-QP44HJS-sZyi4GD4lqUiXh8Mjj0jtf2lZ1XV_hLtLXCqjB4UlQ-Pa920Kjfe7kZkOZyBKKjjM2JphyyESusUDy1imrjmINcQXB9CQ-l3DIIWGNFpLTXLbPaKQUfAH-jKo3pLmoX08LuISxi4zIaOeO1AmPFcmuiTChgAoD9Ikr30fnCJ7IMGhiyPrsWQnr_Se8_Gfy3j87AabIJhOoPu86SnZ19SC6Z9ArvVEhY0oN_jtRBa8-3ffl4N3w4ROsxcI5QbXKE2vPZuz0GzjDPT-p98Qm63bnd
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Efficient+data+processing+pipeline+for+event-based+vision+datasets%3A+techniques+and+insights&rft.jtitle=Engineering+Research+Express&rft.au=Aitsam%2C+Muhammad&rft.au=Jimenez+Rodriguez%2C+Alejandro&rft.au=Di+Nuovo%2C+Alessandro&rft.date=2024-12-01&rft.pub=IOP+Publishing&rft.eissn=2631-8695&rft.volume=6&rft.issue=4&rft_id=info:doi/10.1088%2F2631-8695%2Fad9235&rft.externalDocID=erxad9235
thumbnail_l http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2631-8695&client=summon
thumbnail_m http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2631-8695&client=summon
thumbnail_s http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2631-8695&client=summon