On the Agreement between Manual and Automated Methods for Single-Trial Detection and Estimation of Features from Event-Related Potentials: e0134127
The agreement between humans and algorithms on whether an event-related potential (ERP) is present or not and the level of variation in the estimated values of its relevant features are largely unknown. Thus, the aim of this study was to determine the categorical and quantitative agreement between m...
Saved in:
Published in | PloS one Vol. 10; no. 8 |
---|---|
Main Authors | , , , , |
Format | Journal Article |
Language | English |
Published |
01.08.2015
|
Online Access | Get full text |
Cover
Loading…
Abstract | The agreement between humans and algorithms on whether an event-related potential (ERP) is present or not and the level of variation in the estimated values of its relevant features are largely unknown. Thus, the aim of this study was to determine the categorical and quantitative agreement between manual and automated methods for single-trial detection and estimation of ERP features. To this end, ERPs were elicited in sixteen healthy volunteers using electrical stimulation at graded intensities below and above the nociceptive withdrawal reflex threshold. Presence/absence of an ERP peak (categorical outcome) and its amplitude and latency (quantitative outcome) in each single-trial were evaluated independently by two human observers and two automated algorithms taken from existing literature. Categorical agreement was assessed using percentage positive and negative agreement and Cohen's Kappa , whereas quantitative agreement was evaluated using Bland-Altman analysis and the coefficient of variation. Typical values for the categorical agreement between manual and automated methods were derived, as well as reference values for the average and maximum differences that can be expected if one method is used instead of the others. Results showed that the human observers presented the highest categorical and quantitative agreement, and there were significantly large differences between detection and estimation of quantitative features among methods. In conclusion, substantial care should be taken in the selection of the detection/estimation approach, since factors like stimulation intensity and expected number of trials with/without response can play a significant role in the outcome of a study. |
---|---|
AbstractList | The agreement between humans and algorithms on whether an event-related potential (ERP) is present or not and the level of variation in the estimated values of its relevant features are largely unknown. Thus, the aim of this study was to determine the categorical and quantitative agreement between manual and automated methods for single-trial detection and estimation of ERP features. To this end, ERPs were elicited in sixteen healthy volunteers using electrical stimulation at graded intensities below and above the nociceptive withdrawal reflex threshold. Presence/absence of an ERP peak (categorical outcome) and its amplitude and latency (quantitative outcome) in each single-trial were evaluated independently by two human observers and two automated algorithms taken from existing literature. Categorical agreement was assessed using percentage positive and negative agreement and Cohen's Kappa , whereas quantitative agreement was evaluated using Bland-Altman analysis and the coefficient of variation. Typical values for the categorical agreement between manual and automated methods were derived, as well as reference values for the average and maximum differences that can be expected if one method is used instead of the others. Results showed that the human observers presented the highest categorical and quantitative agreement, and there were significantly large differences between detection and estimation of quantitative features among methods. In conclusion, substantial care should be taken in the selection of the detection/estimation approach, since factors like stimulation intensity and expected number of trials with/without response can play a significant role in the outcome of a study. |
Author | Andersen, Ole K Redondo, David EMedina Arguissain, Federico G Moerch, Carsten D Manresa, Jose ABiurrun |
Author_xml | – sequence: 1 givenname: Jose surname: Manresa middlename: ABiurrun fullname: Manresa, Jose ABiurrun – sequence: 2 givenname: Federico surname: Arguissain middlename: G fullname: Arguissain, Federico G – sequence: 3 givenname: David surname: Redondo middlename: EMedina fullname: Redondo, David EMedina – sequence: 4 givenname: Carsten surname: Moerch middlename: D fullname: Moerch, Carsten D – sequence: 5 givenname: Ole surname: Andersen middlename: K fullname: Andersen, Ole K |
BookMark | eNqVj0FOw0AMRUcIJFrgBiy8ZJOQyZSksKsgFZsKBN1XQ-O0qSZ2mfHAQbgwQ9ULsLL-13_f9lidEhMqda2LXJta3-44erIu3yc7L7SZ6LI-USN9b8qsKgtzrsYh7IrizkyraqR-XghkizDbeMQBSeAD5RuRYGEpWgeWWphF4cEKtrBA2XIboGMP7z1tHGZL36fYEwqupWc6AE2QPgF_kjuYo5XoMVGeB2i-0pbsDd2h8JUlydQQHgCP516qsy4ZeHWcF-pm3iwfn7O958-IQVZDH9bonCXkGFZ6amqT3plMzT-ivzSyYzU |
ContentType | Journal Article |
DBID | 7TK |
DOI | 10.1371/journal.pone.0134127 |
DatabaseName | Neurosciences Abstracts |
DatabaseTitle | Neurosciences Abstracts |
DatabaseTitleList | Neurosciences Abstracts |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Sciences (General) |
EISSN | 1932-6203 |
GroupedDBID | --- 123 29O 2WC 3V. 53G 5VS 7RV 7TK 7X2 7X7 7XC 88E 8AO 8C1 8CJ 8FE 8FG 8FH 8FI 8FJ A8Z AAFWJ ABDBF ABIVO ABJCF ABUWG ACGFO ACIHN ACIWK ACPRK ADBBV ADRAZ AEAQA AENEX AFKRA AFRAH AHMBA ALIPV ALMA_UNASSIGNED_HOLDINGS AOIJS APEBS ARAPS ATCPS BAWUL BBNVY BBORY BCNDV BENPR BGLVJ BHPHI BKEYQ BPHCQ BVXVI BWKFM CCPQU CS3 D1I D1J D1K DIK DU5 E3Z EAP EAS EBD EMOBN ESTFP ESX EX3 F5P FPL FYUFA GROUPED_DOAJ GX1 HCIFZ HH5 HMCUK HYE IAO IEA IHR IHW INH INR IOV IPY ISE ISR ITC K6- KB. KQ8 L6V LK5 LK8 M0K M1P M48 M7P M7R M7S M~E NAPCQ O5R O5S OK1 P2P P62 PATMY PDBOC PIMPY PQQKQ PROAC PSQYO PTHSS PV9 PYCSY RNS RPM RZL SV3 TR2 UKHRP WOQ WOW ~02 ~KM |
ID | FETCH-proquest_miscellaneous_18373386483 |
IEDL.DBID | M48 |
IngestDate | Sat Oct 26 01:34:34 EDT 2024 |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 8 |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-proquest_miscellaneous_18373386483 |
Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 content type line 23 ObjectType-Feature-2 |
PQID | 1837338648 |
PQPubID | 23462 |
ParticipantIDs | proquest_miscellaneous_1837338648 |
PublicationCentury | 2000 |
PublicationDate | 20150801 |
PublicationDateYYYYMMDD | 2015-08-01 |
PublicationDate_xml | – month: 08 year: 2015 text: 20150801 day: 01 |
PublicationDecade | 2010 |
PublicationTitle | PloS one |
PublicationYear | 2015 |
SSID | ssj0053866 |
Score | 3.9462042 |
Snippet | The agreement between humans and algorithms on whether an event-related potential (ERP) is present or not and the level of variation in the estimated values of... |
SourceID | proquest |
SourceType | Aggregation Database |
Title | On the Agreement between Manual and Automated Methods for Single-Trial Detection and Estimation of Features from Event-Related Potentials: e0134127 |
URI | https://search.proquest.com/docview/1837338648 |
Volume | 10 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwjV1LS8NAEB76uHgR6wOtGkbwUA8pTfPYRBCpNbUIqUVb6K0kze6pJNokoBf_hH_Y2W3qRUUvc9pd2NmZnW925wFwHi5IUpghdGHIpxsRhbrXiQ2di0jELKJreSHznYORM5xa9zN7VoFNz9aSgdmPrp3sJzVdLduvL2_XpPBXqmsDMzaT2s9pwtsdWaGsy6pQ71qmKeU8sL7-FUi7HadMoPtt5rdLWVmawQ5slxARe-szbUCFJ7vQKJUww1ZZKfpiDz4eEiQAhz1ymtUzH5ZhVxiEstQohkmMvSJPCZbyGAPVLTpDwqn4RCZryfWJlD-85bmKyErUBJ-0fp3QiKlAiREL8slRJqKgL8MjdRVBRwuO01wGGxEnL5GXG9uH1sCf9If6ZntzkiT5PRAmPC2yOSk3I4fVsVzzAGoJseQQUHA7JhU2POF5spabZ7sEeLod0yU7xiLzCM7-XK75jzHHsEVQxF6H1p1ALV8V_JTMfR5pUGUzRtTtG5IO7jSo3_ij8aOmHGhNnbCk7_4nmny5ig |
link.rule.ids | 315,786,790,870,24339,27946,27947,31744,33291,33398,33769 |
linkProvider | Scholars Portal |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=On+the+Agreement+between+Manual+and+Automated+Methods+for+Single-Trial+Detection+and+Estimation+of+Features+from+Event-Related+Potentials%3A+e0134127&rft.jtitle=PloS+one&rft.au=Manresa%2C+Jose+ABiurrun&rft.au=Arguissain%2C+Federico+G&rft.au=Redondo%2C+David+EMedina&rft.au=Moerch%2C+Carsten+D&rft.date=2015-08-01&rft.eissn=1932-6203&rft.volume=10&rft.issue=8&rft_id=info:doi/10.1371%2Fjournal.pone.0134127&rft.externalDBID=NO_FULL_TEXT |