AVOCADO: Visualization of Workflow-Derived Data Provenance for Reproducible Biomedical Research
A major challenge in data‐driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are reproducibile. In order to communicate and reproduce multi‐step analysis workflows executed on datasets that contain data for dozens or hundreds o...
Saved in:
| Published in | Computer graphics forum Vol. 35; no. 3; pp. 481 - 490 |
|---|---|
| Main Authors | , , , |
| Format | Journal Article |
| Language | English |
| Published |
England
Blackwell Publishing Ltd
01.06.2016
|
| Subjects | |
| Online Access | Get full text |
| ISSN | 0167-7055 1467-8659 |
| DOI | 10.1111/cgf.12924 |
Cover
Loading…
| Abstract | A major challenge in data‐driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are reproducibile. In order to communicate and reproduce multi‐step analysis workflows executed on datasets that contain data for dozens or hundreds of samples, it is crucial to be able to visualize the provenance graph at different levels of aggregation. Most existing approaches are based on node‐link diagrams, which do not scale to the complexity of typical data provenance graphs. In our proposed approach, we reduce the complexity of the graph using hierarchical and motif‐based aggregation. Based on user action and graph attributes, a modular degree‐of‐interest (DoI) function is applied to expand parts of the graph that are relevant to the user. This interest‐driven adaptive approach to provenance visualization allows users to review and communicate complex multi‐step analyses, which can be based on hundreds of files that are processed by numerous workflows. We have integrated our approach into an analysis platform that captures extensive data provenance information, and demonstrate its effectiveness by means of a biomedical usage scenario. |
|---|---|
| AbstractList | A major challenge in data-driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are reproducibile. In order to communicate and reproduce multi-step analysis workflows executed on datasets that contain data for dozens or hundreds of samples, it is crucial to be able to visualize the provenance graph at different levels of aggregation. Most existing approaches are based on node-link diagrams, which do not scale to the complexity of typical data provenance graphs. In our proposed approach, we reduce the complexity of the graph using hierarchical and motif-based aggregation. Based on user action and graph attributes, a modular degree-of-interest (DoI) function is applied to expand parts of the graph that are relevant to the user. This interest-driven adaptive approach to provenance visualization allows users to review and communicate complex multi-step analyses, which can be based on hundreds of files that are processed by numerous workflows. We have integrated our approach into an analysis platform that captures extensive data provenance information, and demonstrate its effectiveness by means of a biomedical usage scenario.A major challenge in data-driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are reproducibile. In order to communicate and reproduce multi-step analysis workflows executed on datasets that contain data for dozens or hundreds of samples, it is crucial to be able to visualize the provenance graph at different levels of aggregation. Most existing approaches are based on node-link diagrams, which do not scale to the complexity of typical data provenance graphs. In our proposed approach, we reduce the complexity of the graph using hierarchical and motif-based aggregation. Based on user action and graph attributes, a modular degree-of-interest (DoI) function is applied to expand parts of the graph that are relevant to the user. This interest-driven adaptive approach to provenance visualization allows users to review and communicate complex multi-step analyses, which can be based on hundreds of files that are processed by numerous workflows. We have integrated our approach into an analysis platform that captures extensive data provenance information, and demonstrate its effectiveness by means of a biomedical usage scenario. A major challenge in data-driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are reproducibile. In order to communicate and reproduce multi-step analysis workflows executed on datasets that contain data for dozens or hundreds of samples, it is crucial to be able to visualize the provenance graph at different levels of aggregation. Most existing approaches are based on node-link diagrams, which do not scale to the complexity of typical data provenance graphs. In our proposed approach, we reduce the complexity of the graph using hierarchical and motif-based aggregation. Based on user action and graph attributes, a modular degree-of-interest (DoI) function is applied to expand parts of the graph that are relevant to the user. This interest-driven adaptive approach to provenance visualization allows users to review and communicate complex multi-step analyses, which can be based on hundreds of files that are processed by numerous workflows. We have integrated our approach into an analysis platform that captures extensive data provenance information, and demonstrate its effectiveness by means of a biomedical usage scenario. |
| Author | Luger, S. Gehlenborg, N. Stitz, H. Streit, M. |
| AuthorAffiliation | 2 Harvard Medical School, United States of America 1 Johannes Kepler University Linz, Austria |
| AuthorAffiliation_xml | – name: 2 Harvard Medical School, United States of America – name: 1 Johannes Kepler University Linz, Austria |
| Author_xml | – sequence: 1 givenname: H. surname: Stitz fullname: Stitz, H. organization: Johannes Kepler University Linz, Austria – sequence: 2 givenname: S. surname: Luger fullname: Luger, S. organization: Johannes Kepler University Linz, Austria – sequence: 3 givenname: M. surname: Streit fullname: Streit, M. organization: Johannes Kepler University Linz, Austria – sequence: 4 givenname: N. surname: Gehlenborg fullname: Gehlenborg, N. organization: Harvard Medical School, United States of America |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/29973745$$D View this record in MEDLINE/PubMed |
| BookMark | eNqNkl9vFCEUxYmpsX_0wS9gJvFFH6YFBobBhybrrl2N1TVGu4-EYe60tOywwszW-unF7m6jjSbCAwR-5-TAvftop_MdIPSU4EOSxpE5bw8JlZQ9QHuElSKvSi530B4maS8w57toP8ZLjDETJX-EdqmUohCM7yE1OpuNR5PZq-zMxkE7-0P31neZb7O5D1et89f5BIJdQZNNdK-zT8GvoNOdgaz1IfsMy-CbwdjaQfba-gU01miXziPoYC4eo4etdhGebNYD9PXkzZfx2_x0Nn03Hp3mhhPG8lboQpcVUFEzYtJsTCtN09aEsEbUhgNj0DBDqZa0oq0kUPGygAYXXBY1KQ7Q8dp3OdQpg4GuD9qpZbALHW6U11b9edPZC3XuV6rEVAjOksGLjUHw3waIvVrYaMA53YEfoqK4ZAmU_4GSinLOmKA0oc_voZd-CF36iURhwoTEtEjUs9_D36XelikBR2vABB9jgFYZ298WKr3FOkWw-tUIKjWCum2EpHh5T7E1_Ru7cb-2Dm7-Darx9GSryNcKG3v4fqfQ4UqVKTFX849T9X4uMP1QTdWk-Am5sdCQ |
| CitedBy_id | crossref_primary_10_1109_TVCG_2020_2990336 crossref_primary_10_1109_TVCG_2018_2865024 crossref_primary_10_1093_bib_bby019 crossref_primary_10_1177_1473871616667767 crossref_primary_10_1016_j_visinf_2018_12_002 crossref_primary_10_1111_cgf_14035 crossref_primary_10_1109_TVCG_2019_2934629 crossref_primary_10_1016_j_procs_2017_05_216 crossref_primary_10_1007_s10115_021_01645_6 crossref_primary_10_1016_j_cag_2023_02_002 crossref_primary_10_1177_14738716211032653 crossref_primary_10_1523_ENEURO_0476_23_2024 crossref_primary_10_1016_j_ijmedinf_2024_105690 crossref_primary_10_2196_42289 crossref_primary_10_1109_TVCG_2017_2744805 crossref_primary_10_1007_s00778_017_0486_1 crossref_primary_10_1111_cgf_15272 crossref_primary_10_1109_TVCG_2019_2934658 crossref_primary_10_2196_51297 crossref_primary_10_1093_llc_fqad020 crossref_primary_10_1109_TVCG_2021_3135697 crossref_primary_10_1109_ACCESS_2022_3194072 crossref_primary_10_1109_TVCG_2017_2745298 crossref_primary_10_1109_TVCG_2021_3104026 crossref_primary_10_1109_TVCG_2018_2864769 crossref_primary_10_1016_j_envsoft_2019_04_009 |
| Cites_doi | 10.1109/IV.2006.130 10.1161/CIRCRESAHA.114.303819 10.1371/journal.pone.0127612 10.1109/TVCG.2013.225 10.1145/166117.166125 10.1109/TVCG.2015.2467251 10.1126/science.aac8041 10.1126/science.348.6242.1411 10.1186/gb-2010-11-8-r86 10.1109/TVCG.2015.2467551 10.1186/s12859-015-0550-z 10.1145/22627.22342 10.1109/TVCG.2009.108 10.1145/1029208.1029225 10.1145/989863.989941 10.1109/TVCG.2015.2513389 10.7155/jgaa.00001 10.1093/nar/gkt328 10.1109/TVCG.2011.185 10.1093/bib/bbs078 10.1007/978-0-85729-079-3 10.1038/ng.2761 10.1093/bioinformatics/btq415 10.1007/11890850_14 10.1093/bib/bbl022 10.1109/TVCG.2009.84 10.1109/PACIFICVIS.2015.7156375 10.1038/nrg2641 10.1109/TVCG.2013.109 10.1038/483531a 10.1109/IV.2006.54 10.1109/TVCG.2015.2468078 10.1126/science.298.5594.824 |
| ContentType | Journal Article |
| Copyright | 2016 The Author(s), Computer Graphics Forum published by John Wiley & Sons Ltd. and The Eurographics Association. 2016 The Eurographics Association and John Wiley & Sons Ltd. |
| Copyright_xml | – notice: 2016 The Author(s), Computer Graphics Forum published by John Wiley & Sons Ltd. and The Eurographics Association. – notice: 2016 The Eurographics Association and John Wiley & Sons Ltd. |
| DBID | BSCLL 24P AAYXX CITATION NPM 7SC 8FD JQ2 L7M L~C L~D F28 FR3 7X8 5PM |
| DOI | 10.1111/cgf.12924 |
| DatabaseName | Istex Wiley Online Library Open Access CrossRef PubMed Computer and Information Systems Abstracts Technology Research Database ProQuest Computer Science Collection Advanced Technologies Database with Aerospace Computer and Information Systems Abstracts Academic Computer and Information Systems Abstracts Professional ANTE: Abstracts in New Technology & Engineering Engineering Research Database MEDLINE - Academic PubMed Central (Full Participant titles) |
| DatabaseTitle | CrossRef PubMed Computer and Information Systems Abstracts Technology Research Database Computer and Information Systems Abstracts – Academic Advanced Technologies Database with Aerospace ProQuest Computer Science Collection Computer and Information Systems Abstracts Professional Engineering Research Database ANTE: Abstracts in New Technology & Engineering MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic Technology Research Database CrossRef PubMed Computer and Information Systems Abstracts |
| Database_xml | – sequence: 1 dbid: 24P name: Wiley Online Library Open Access url: https://authorservices.wiley.com/open-science/open-access/browse-journals.html sourceTypes: Publisher – sequence: 2 dbid: NPM name: PubMed url: https://proxy.k.utb.cz/login?url=http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| EISSN | 1467-8659 |
| EndPage | 490 |
| ExternalDocumentID | PMC6027754 4107220521 29973745 10_1111_cgf_12924 CGF12924 ark_67375_WNG_KW702M8G_D |
| Genre | article Journal Article Feature |
| GrantInformation_xml | – fundername: NHGRI NIH HHS grantid: R00 HG007583 – fundername: Austrian Science Fund FWF grantid: P 27975 |
| GroupedDBID | .3N .4S .DC .GA .Y3 05W 0R~ 10A 15B 1OB 1OC 29F 31~ 33P 3SF 4.4 50Y 50Z 51W 51X 52M 52N 52O 52P 52S 52T 52U 52W 52X 5GY 5HH 5LA 5VS 66C 6J9 702 7PT 8-0 8-1 8-3 8-4 8-5 8UM 8VB 930 A03 AAESR AAEVG AAHHS AANLZ AAONW AASGY AAXRX AAZKR ABCQN ABCUV ABDBF ABDPE ABEML ABPVW ACAHQ ACBWZ ACCFJ ACCZN ACFBH ACGFS ACPOU ACSCC ACXBN ACXQS ADBBV ADEOM ADIZJ ADKYN ADMGS ADOZA ADXAS ADZMN ADZOD AEEZP AEGXH AEIGN AEIMD AEMOZ AENEX AEQDE AEUQT AEUYR AFBPY AFEBI AFFNX AFFPM AFGKR AFPWT AFZJQ AHBTC AHEFC AITYG AIURR AIWBW AJBDE AJXKR AKVCP ALAGY ALMA_UNASSIGNED_HOLDINGS ALUQN AMBMR AMYDB ARCSS ASPBG ATUGU AUFTA AVWKF AZBYB AZFZN AZVAB BAFTC BDRZF BFHJK BHBCM BMNLL BMXJE BNHUX BROTX BRXPI BSCLL BY8 CAG COF CS3 CWDTD D-E D-F DCZOG DPXWK DR2 DRFUL DRSTM DU5 EAD EAP EBA EBO EBR EBS EBU EDO EJD EMK EST ESX F00 F01 F04 F5P FEDTE FZ0 G-S G.N GODZA H.T H.X HF~ HGLYW HVGLF HZI HZ~ I-F IHE IX1 J0M K1G K48 LATKE LC2 LC3 LEEKS LH4 LITHE LOXES LP6 LP7 LUTES LW6 LYRES MEWTI MK4 MRFUL MRSTM MSFUL MSSTM MXFUL MXSTM N04 N05 N9A NF~ O66 O9- OIG P2W P2X P4D PALCI PQQKQ Q.N Q11 QB0 QWB R.K RDJ RIWAO RJQFR ROL RX1 SAMSI SUPJJ TH9 TN5 TUS UB1 V8K W8V W99 WBKPD WIH WIK WOHZO WQJ WRC WXSBR WYISQ WZISG XG1 ZL0 ZZTAW ~IA ~IF ~WT 24P AAHQN AAMNL AANHP AAYCA ACRPL ACUHS ACYXJ ADNMO AFWVQ AHQJS ALVPJ AAYXX ADMLS AEYWJ AGHNM AGQPQ AGYGG CITATION AAMMB AEFGJ AGXDD AIDQK AIDYY NPM 7SC 8FD JQ2 L7M L~C L~D F28 FR3 7X8 5PM |
| ID | FETCH-LOGICAL-c5144-f7a3a68e27b41c1c1dcf9cdfb114d7bc5e44ed4c22a9282f91e8563ed03593b13 |
| IEDL.DBID | DR2 |
| ISSN | 0167-7055 |
| IngestDate | Thu Aug 21 18:26:42 EDT 2025 Fri Jul 11 09:18:50 EDT 2025 Thu Jul 10 18:05:47 EDT 2025 Fri Jul 25 23:46:21 EDT 2025 Mon Jul 21 06:03:21 EDT 2025 Tue Jul 01 02:23:07 EDT 2025 Thu Apr 24 22:58:54 EDT 2025 Wed Jan 22 16:22:48 EST 2025 Wed Oct 30 09:50:01 EDT 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 3 |
| Language | English |
| License | Attribution This is an open access article under the terms of the Creative Commons Attribution License, which permits use, distribution and reproduction in any medium, provided the original work is properly cited. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c5144-f7a3a68e27b41c1c1dcf9cdfb114d7bc5e44ed4c22a9282f91e8563ed03593b13 |
| Notes | istex:59A92BF1996CA7F10DF1F6E01BDC850574840C1D ArticleID:CGF12924 Supporting InformationSupporting Information ark:/67375/WNG-KW702M8G-D SourceType-Scholarly Journals-1 ObjectType-Feature-1 content type line 14 ObjectType-Article-1 ObjectType-Feature-2 content type line 23 Equal contribution |
| OpenAccessLink | https://proxy.k.utb.cz/login?url=https://onlinelibrary.wiley.com/doi/abs/10.1111%2Fcgf.12924 |
| PMID | 29973745 |
| PQID | 1801479023 |
| PQPubID | 30877 |
| PageCount | 10 |
| ParticipantIDs | pubmedcentral_primary_oai_pubmedcentral_nih_gov_6027754 proquest_miscellaneous_2064775954 proquest_miscellaneous_1825544722 proquest_journals_1801479023 pubmed_primary_29973745 crossref_citationtrail_10_1111_cgf_12924 crossref_primary_10_1111_cgf_12924 wiley_primary_10_1111_cgf_12924_CGF12924 istex_primary_ark_67375_WNG_KW702M8G_D |
| ProviderPackageCode | CITATION AAYXX |
| PublicationCentury | 2000 |
| PublicationDate | 2016-06 June 2016 2016-06-00 2016-Jun 20160601 |
| PublicationDateYYYYMMDD | 2016-06-01 |
| PublicationDate_xml | – month: 06 year: 2016 text: 2016-06 |
| PublicationDecade | 2010 |
| PublicationPlace | England |
| PublicationPlace_xml | – name: England – name: Oxford |
| PublicationTitle | Computer graphics forum |
| PublicationTitleAlternate | Computer Graphics Forum |
| PublicationYear | 2016 |
| Publisher | Blackwell Publishing Ltd |
| Publisher_xml | – name: Blackwell Publishing Ltd |
| References | Kaiser J.: The Cancer Test. Science 348, 6242 (2015), 1411-1413. 2 Aittokallio T., Schwikowski B.: Graph-based Methods for Analysing Networks in Cell Biology. Briefings in Bioinformatics 7, 3 (2006), 243-255. 4 Maguire E., Rocca-Serra P., Sansone S.-A., Davies J., Chen M.: Visual Compression of Workflow Visualizations with Automated Detection of Macro Motifs. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2576-2585. 4 Wolstencroft K., Haines R., et al.: The Taverna Workflow Suite: Designing and Executing Workflows of Web Services on the Desktop, Web or in the Cloud. Nucleic Acids Research 41, W1 (2013), W557-W561. 2 van den Elzen S., Holten D., Blaas J., van Wijk J.J.: Reducing Snapshots to Points: A Visual Analytics Approach to Dynamic Network Exploration. IEEE Transactions on Visualization and Computer Graphics (InfoVis '15) 22, 1 (2015), 1-10. 4 Refinery Platform: AVOCADO Usage Scenario. https://github.com/refinery-platform/sample-data/tree/develop/avocado_usage-scenario, 2015. Accessed: 2016-03-07. 7 Omberg L., Ellrott K., Yuan Y., Kandoth C., Wong C., Kellen M.R., Friend S.h., Stuart J., Liang h., Margolin A.A.: Enabling Transparent and Collaborative Computational Analysis of 12 Tumor Types within The Cancer Genome Atlas. Nature Genetics 45, 10 (2013), 1121-1126. 4 van Ham F., Perer A.: "Search, Show Context, Expand on Demand": Supporting Large Graph Exploration with Degree-of-Interest. IEEE Transactions on Visualization and Computer Graphics (InfoVis '09) 15, 6 (2009), 953-960. 4 Park P.J.: Chip-Seq: Advantages and Challenges of a Maturing Technology. Nature Reviews Genetics 10, 10 (2009), 669-680. 7 Begley C.G., Ioannidis J.P.: Reproducibility in Science Improving the Standard for Basic and Preclinical Research. Circulation research 116, 1 (2015), 116-126. 2 Aigner W., Miksch S., Schumann H., Tominski C.: Visualization of Time-Oriented Data. Springer, 2011. 4 Milo R., Shen-Orr S., Itzkovitz S., Kashtan N., Chklovskii D., Alon U.: Network Motifs: Simple Building Blocks of Complex Networks. Science 298, 5594 (2002), 824-827. 4 Bostock M., Ogievetsky V., Heer J.: D3 Data-Driven Documents. IEEE Transactions on Visualization and Computer Graphics (InfoVis '11) 17, 12 (2011), 2301-2309. 6 Abello J., Hadlak S., Schumann H., Schulz H.-J.: A Modular Degree-of-Interest Specification for the Visual Analysis of Large Dynamic Networks. IEEE Transactions on Visualization and Computer Graphics (InfoVis '13) 20, 3 (2014), 337-350. 4 Jünger M., Mutzel P.: 2-Layer Straightline Crossing Minimization: Performance of Exact and Heuristic Algorithms. Journal of Graph Algorithms and Application 1 (1997), 1-25. 6 Vehlow C., Kao D.P., Bristow M.R., Hunter L.E., Weiskopf D., Görg C.: Visual Analysis of Biological Data-Knowledge Networks. BMC Bioinformatics 16, 1 (2015). 4 Begley C.G., Ellis L.M.: Drug development: Raise standards for preclinical cancer research. Nature 483, 7391 (2012), 531-533. 2 González-Beltrán A., Li P., Zhao J., Avila-Garcia M.S., Roos M., Thompson M., van der Horst E., Kaliyaperumal R., Luo R., Lee T.-L., Lam T.-w., Edmunds S.C., Sansone S.-A., Rocca-Serra P.: From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics. PLOS ONE 10, 7 (2015), e0127612. 2 Elmqvist N., Fekete J.-D.: Hierarchical Aggregation for Information Visualization: Overview, Techniques, and Design Guidelines. IEEE Transactions on Visualization and Computer Graphics 16, 3 (2010), 439-454. 4, 5 Ragan E., Endert A., Sanyal J., Chen J.: Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes. IEEE Transactions on Visualization and Computer Graphics (InfoVis '15) 22, 1 (2015), 31-40. 2 Rocca-Serra P., Brandizi M., Maguire E., Sklyar N., Taylor C., Begley K., Field D., Harris S., Hide W., Hofmann O., Neumann S., Sterk P., Tong W., Sansone S.-A.: ISA Software Suite: Supporting Standards-compliant Experimental Annotation and Enabling Curation at the Community Level. Bioinformatics 26, 18 (2010), 2354-2356. 2 Buck S.: Solving reproducibility. Science 348, 6242 (2015), 1403-1403. 2 Huang Y., Gottardo R.: Comparability and reproducibility of biomedical data. Briefings in Bioinformatics 14, 4 (2013), 391-401. 2 Yoghourdjian V., Dwyer T., Gange G., Kieffer S., Klein K., Marriott K.: High-Quality Ultra-Compact Grid Layout of Grouped Networks. IEEE Transactions on Visualization and Computer Graphics (InfoVis '15) 22, 1 (2015), 339-348. 9 Goecks J., Nekrutenko A., Taylor J.: Galaxy: A Comprehensive Approach for Supporting Accessible, Reproducible, and Transparent Computational Research in the Life Sciences. Genome Biology 11, 8 (2010), R86. 2 2010; 11 2012; 483 2010; 16 2015; 16 2011 2013; 45 2002; 298 2015; 10 2013; 41 2009 2006; 7 2006 2005 1997; 1 2004 1993 2002 2015; 348 2011; 17 2014; 20 2013; 19 2013; 14 2010; 26 2009; 10 2015; 116 2015; 22 1986 2016 2015 2014 2006; 4145 2009; 15 Brandes U. (e_1_2_11_11_2) 2002 e_1_2_11_31_2 e_1_2_11_30_2 e_1_2_11_13_2 Lee B. (e_1_2_11_25_2) 2006 e_1_2_11_35_2 e_1_2_11_12_2 e_1_2_11_34_2 e_1_2_11_33_2 e_1_2_11_10_2 e_1_2_11_32_2 e_1_2_11_6_2 e_1_2_11_28_2 e_1_2_11_27_2 e_1_2_11_4_2 e_1_2_11_26_2 e_1_2_11_3_2 e_1_2_11_2_2 e_1_2_11_47_2 e_1_2_11_48_2 Noel S. (e_1_2_11_29_2) 2004 e_1_2_11_20_2 e_1_2_11_43_2 Hadlak S. (e_1_2_11_21_2) 2015 e_1_2_11_44_2 e_1_2_11_45_2 e_1_2_11_46_2 e_1_2_11_24_2 e_1_2_11_9_2 e_1_2_11_23_2 e_1_2_11_40_2 Furnas G.W. (e_1_2_11_15_2) 1986 e_1_2_11_22_2 e_1_2_11_41_2 e_1_2_11_7_2 e_1_2_11_42_2 e_1_2_11_17_2 e_1_2_11_16_2 e_1_2_11_14_2 e_1_2_11_36_2 e_1_2_11_37_2 e_1_2_11_19_2 e_1_2_11_38_2 e_1_2_11_18_2 e_1_2_11_39_2 Bavoil L. (e_1_2_11_8_2) 2005 Archambault D. (e_1_2_11_5_2) 2009 |
| References_xml | – reference: Begley C.G., Ioannidis J.P.: Reproducibility in Science Improving the Standard for Basic and Preclinical Research. Circulation research 116, 1 (2015), 116-126. 2 – reference: Omberg L., Ellrott K., Yuan Y., Kandoth C., Wong C., Kellen M.R., Friend S.h., Stuart J., Liang h., Margolin A.A.: Enabling Transparent and Collaborative Computational Analysis of 12 Tumor Types within The Cancer Genome Atlas. Nature Genetics 45, 10 (2013), 1121-1126. 4 – reference: van Ham F., Perer A.: "Search, Show Context, Expand on Demand": Supporting Large Graph Exploration with Degree-of-Interest. IEEE Transactions on Visualization and Computer Graphics (InfoVis '09) 15, 6 (2009), 953-960. 4 – reference: Aittokallio T., Schwikowski B.: Graph-based Methods for Analysing Networks in Cell Biology. Briefings in Bioinformatics 7, 3 (2006), 243-255. 4 – reference: Aigner W., Miksch S., Schumann H., Tominski C.: Visualization of Time-Oriented Data. Springer, 2011. 4 – reference: Refinery Platform: AVOCADO Usage Scenario. https://github.com/refinery-platform/sample-data/tree/develop/avocado_usage-scenario, 2015. Accessed: 2016-03-07. 7 – reference: van den Elzen S., Holten D., Blaas J., van Wijk J.J.: Reducing Snapshots to Points: A Visual Analytics Approach to Dynamic Network Exploration. IEEE Transactions on Visualization and Computer Graphics (InfoVis '15) 22, 1 (2015), 1-10. 4 – reference: Huang Y., Gottardo R.: Comparability and reproducibility of biomedical data. Briefings in Bioinformatics 14, 4 (2013), 391-401. 2 – reference: Wolstencroft K., Haines R., et al.: The Taverna Workflow Suite: Designing and Executing Workflows of Web Services on the Desktop, Web or in the Cloud. Nucleic Acids Research 41, W1 (2013), W557-W561. 2 – reference: Yoghourdjian V., Dwyer T., Gange G., Kieffer S., Klein K., Marriott K.: High-Quality Ultra-Compact Grid Layout of Grouped Networks. IEEE Transactions on Visualization and Computer Graphics (InfoVis '15) 22, 1 (2015), 339-348. 9 – reference: Kaiser J.: The Cancer Test. Science 348, 6242 (2015), 1411-1413. 2 – reference: Vehlow C., Kao D.P., Bristow M.R., Hunter L.E., Weiskopf D., Görg C.: Visual Analysis of Biological Data-Knowledge Networks. BMC Bioinformatics 16, 1 (2015). 4 – reference: Abello J., Hadlak S., Schumann H., Schulz H.-J.: A Modular Degree-of-Interest Specification for the Visual Analysis of Large Dynamic Networks. IEEE Transactions on Visualization and Computer Graphics (InfoVis '13) 20, 3 (2014), 337-350. 4 – reference: Bostock M., Ogievetsky V., Heer J.: D3 Data-Driven Documents. IEEE Transactions on Visualization and Computer Graphics (InfoVis '11) 17, 12 (2011), 2301-2309. 6 – reference: González-Beltrán A., Li P., Zhao J., Avila-Garcia M.S., Roos M., Thompson M., van der Horst E., Kaliyaperumal R., Luo R., Lee T.-L., Lam T.-w., Edmunds S.C., Sansone S.-A., Rocca-Serra P.: From Peer-Reviewed to Peer-Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics. PLOS ONE 10, 7 (2015), e0127612. 2 – reference: Begley C.G., Ellis L.M.: Drug development: Raise standards for preclinical cancer research. Nature 483, 7391 (2012), 531-533. 2 – reference: Park P.J.: Chip-Seq: Advantages and Challenges of a Maturing Technology. Nature Reviews Genetics 10, 10 (2009), 669-680. 7 – reference: Jünger M., Mutzel P.: 2-Layer Straightline Crossing Minimization: Performance of Exact and Heuristic Algorithms. Journal of Graph Algorithms and Application 1 (1997), 1-25. 6 – reference: Maguire E., Rocca-Serra P., Sansone S.-A., Davies J., Chen M.: Visual Compression of Workflow Visualizations with Automated Detection of Macro Motifs. IEEE Transactions on Visualization and Computer Graphics 19, 12 (2013), 2576-2585. 4 – reference: Ragan E., Endert A., Sanyal J., Chen J.: Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes. IEEE Transactions on Visualization and Computer Graphics (InfoVis '15) 22, 1 (2015), 31-40. 2 – reference: Goecks J., Nekrutenko A., Taylor J.: Galaxy: A Comprehensive Approach for Supporting Accessible, Reproducible, and Transparent Computational Research in the Life Sciences. Genome Biology 11, 8 (2010), R86. 2 – reference: Elmqvist N., Fekete J.-D.: Hierarchical Aggregation for Information Visualization: Overview, Techniques, and Design Guidelines. IEEE Transactions on Visualization and Computer Graphics 16, 3 (2010), 439-454. 4, 5 – reference: Buck S.: Solving reproducibility. Science 348, 6242 (2015), 1403-1403. 2 – reference: Rocca-Serra P., Brandizi M., Maguire E., Sklyar N., Taylor C., Begley K., Field D., Harris S., Hide W., Hofmann O., Neumann S., Sterk P., Tong W., Sansone S.-A.: ISA Software Suite: Supporting Standards-compliant Experimental Annotation and Enabling Curation at the Community Level. Bioinformatics 26, 18 (2010), 2354-2356. 2 – reference: Milo R., Shen-Orr S., Itzkovitz S., Kashtan N., Chklovskii D., Alon U.: Network Motifs: Simple Building Blocks of Complex Networks. Science 298, 5594 (2002), 824-827. 4 – year: 2011 – volume: 298 start-page: 824 issue: 5594 year: 2002 end-page: 827 article-title: Network Motifs: Simple Building Blocks of Complex Networks publication-title: Science – volume: 22 start-page: 339 issue: 1 year: 2015 end-page: 348 article-title: High‐Quality Ultra‐Compact Grid Layout of Grouped Networks publication-title: IEEE Transactions on Visualization and Computer Graphics (InfoVis ‘15) – volume: 45 start-page: 1121 issue: 10 year: 2013 end-page: 1126 article-title: Enabling Transparent and Collaborative Computational Analysis of 12 Tumor Types within The Cancer Genome Atlas publication-title: Nature Genetics – volume: 348 start-page: 1403 issue: 6242 year: 2015 end-page: 1403 article-title: Solving reproducibility publication-title: Science – volume: 15 start-page: 953 issue: 6 year: 2009 end-page: 960 article-title: “Search, Show Context, Expand on Demand”: Supporting Large Graph Exploration with Degree‐of‐Interest publication-title: IEEE Transactions on Visualization and Computer Graphics (InfoVis ‘09) – volume: 10 start-page: e0127612 issue: 7 year: 2015 article-title: From Peer‐Reviewed to Peer‐Reproduced in Scholarly Publishing: The Complementary Roles of Data Models and Workflows in Bioinformatics publication-title: PLOS ONE – start-page: 87 year: 2009 end-page: 94 – start-page: 31 year: 2002 end-page: 44 – start-page: 166 year: 2006 end-page: 173 – volume: 1 start-page: 1 year: 1997 end-page: 25 article-title: 2‐Layer Straightline Crossing Minimization: Performance of Exact and Heuristic Algorithms publication-title: Journal of Graph Algorithms and Application – volume: 4145 start-page: 118 year: 2006 end-page: 132 – volume: 20 start-page: 337 issue: 3 year: 2014 end-page: 350 article-title: A Modular Degree‐of‐Interest Specification for the Visual Analysis of Large Dynamic Networks publication-title: IEEE Transactions on Visualization and Computer Graphics (InfoVis ’13) – volume: 116 start-page: 116 issue: 1 year: 2015 end-page: 126 article-title: Reproducibility in Science Improving the Standard for Basic and Preclinical Research publication-title: Circulation research – volume: 17 start-page: 2301 issue: 12 year: 2011 end-page: 2309 article-title: D3 Data‐Driven Documents publication-title: IEEE Transactions on Visualization and Computer Graphics (InfoVis ’11) – year: 2016 – start-page: 17 year: 2006 end-page: 24 – year: 2014 – start-page: 109 year: 2004 end-page: 118 – start-page: 421 year: 2004 end-page: 424 – start-page: 57 year: 1993 end-page: 64 – start-page: 175 year: 2015 end-page: 182 – volume: 19 start-page: 2576 issue: 12 year: 2013 end-page: 2585 article-title: Visual Compression of Workflow Visualizations with Automated Detection of Macro Motifs publication-title: IEEE Transactions on Visualization and Computer Graphics – volume: 22 start-page: 31 issue: 1 year: 2015 end-page: 40 article-title: Characterizing Provenance in Visualization and Data Analysis: An Organizational Framework of Provenance Types and Purposes publication-title: IEEE Transactions on Visualization and Computer Graphics (InfoVis ’15) – volume: 483 start-page: 531 issue: 7391 year: 2012 end-page: 533 article-title: Drug development: Raise standards for preclinical cancer research publication-title: Nature – volume: 41 start-page: W557 issue: W1 year: 2013 end-page: W561 article-title: The Taverna Workflow Suite: Designing and Executing Workflows of Web Services on the Desktop, Web or in the Cloud publication-title: Nucleic Acids Research – volume: 348 start-page: 1411 issue: 6242 year: 2015 end-page: 1413 article-title: The Cancer Test publication-title: Science – volume: 7 start-page: 243 issue: 3 year: 2006 end-page: 255 article-title: Graph‐based Methods for Analysing Networks in Cell Biology publication-title: Briefings in Bioinformatics – volume: 16 start-page: 1 year: 2015 article-title: Visual Analysis of Biological Data‐Knowledge Networks publication-title: BMC Bioinformatics – start-page: 1 year: 2006 end-page: 5 – volume: 26 start-page: 2354 issue: 18 year: 2010 end-page: 2356 article-title: ISA Software Suite: Supporting Standards‐compliant Experimental Annotation and Enabling Curation at the Community Level publication-title: Bioinformatics – volume: 14 start-page: 391 issue: 4 year: 2013 end-page: 401 article-title: Comparability and reproducibility of biomedical data publication-title: Briefings in Bioinformatics – volume: 10 start-page: 669 issue: 10 year: 2009 end-page: 680 article-title: Chip‐Seq: Advantages and Challenges of a Maturing Technology publication-title: Nature Reviews Genetics – volume: 16 start-page: 439 issue: 3 year: 2010 end-page: 454 article-title: Hierarchical Aggregation for Information Visualization: Overview, Techniques, and Design Guidelines publication-title: IEEE Transactions on Visualization and Computer Graphics – volume: 11 start-page: R86 issue: 8 year: 2010 article-title: Galaxy: A Comprehensive Approach for Supporting Accessible, Reproducible, and Transparent Computational Research in the Life Sciences publication-title: Genome Biology – volume: 22 start-page: 1 issue: 1 year: 2015 end-page: 10 article-title: Reducing Snapshots to Points: A Visual Analytics Approach to Dynamic Network Exploration publication-title: IEEE Transactions on Visualization and Computer Graphics (InfoVis ’15) – start-page: 16 year: 1986 end-page: 23 – start-page: 135 year: 2005 end-page: 142 – year: 2015 – ident: e_1_2_11_41_2 doi: 10.1109/IV.2006.130 – ident: e_1_2_11_10_2 doi: 10.1161/CIRCRESAHA.114.303819 – ident: e_1_2_11_17_2 doi: 10.1371/journal.pone.0127612 – ident: e_1_2_11_34_2 – ident: e_1_2_11_26_2 doi: 10.1109/TVCG.2013.225 – ident: e_1_2_11_33_2 doi: 10.1145/166117.166125 – ident: e_1_2_11_48_2 doi: 10.1109/TVCG.2015.2467251 – ident: e_1_2_11_13_2 doi: 10.1126/science.aac8041 – ident: e_1_2_11_23_2 doi: 10.1126/science.348.6242.1411 – ident: e_1_2_11_18_2 doi: 10.1186/gb-2010-11-8-r86 – ident: e_1_2_11_37_2 doi: 10.1109/TVCG.2015.2467551 – ident: e_1_2_11_46_2 doi: 10.1186/s12859-015-0550-z – start-page: 31 volume-title: Lecture Notes in Computer Science year: 2002 ident: e_1_2_11_11_2 – ident: e_1_2_11_7_2 – start-page: 16 volume-title: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’86) year: 1986 ident: e_1_2_11_15_2 doi: 10.1145/22627.22342 – volume-title: Proceedings of the Eurographics Conference on Visualization (EuroVis ’15) – State of The Art Reports year: 2015 ident: e_1_2_11_21_2 – ident: e_1_2_11_45_2 doi: 10.1109/TVCG.2009.108 – start-page: 109 volume-title: Proceedings of the ACM Workshop on Visualization and Data Mining for Computer Security (VizSEC/DMSEC ’04) year: 2004 ident: e_1_2_11_29_2 doi: 10.1145/1029208.1029225 – ident: e_1_2_11_19_2 doi: 10.1145/989863.989941 – ident: e_1_2_11_36_2 – ident: e_1_2_11_39_2 doi: 10.1109/TVCG.2015.2513389 – ident: e_1_2_11_22_2 doi: 10.7155/jgaa.00001 – start-page: 1 volume-title: Proceedings of the AVI Workshop on BEyond time and errors: novel evaluation methods for information visualization (BELIV'06) year: 2006 ident: e_1_2_11_25_2 – start-page: 135 volume-title: Proceedings of the IEEE Conference on Visualization (VIS ’05) year: 2005 ident: e_1_2_11_8_2 – ident: e_1_2_11_47_2 doi: 10.1093/nar/gkt328 – ident: e_1_2_11_24_2 – volume-title: Refinery Platform ident: e_1_2_11_35_2 – ident: e_1_2_11_12_2 doi: 10.1109/TVCG.2011.185 – ident: e_1_2_11_16_2 – ident: e_1_2_11_20_2 doi: 10.1093/bib/bbs078 – ident: e_1_2_11_4_2 doi: 10.1007/978-0-85729-079-3 – ident: e_1_2_11_30_2 doi: 10.1038/ng.2761 – ident: e_1_2_11_38_2 doi: 10.1093/bioinformatics/btq415 – ident: e_1_2_11_2_2 doi: 10.1007/11890850_14 – ident: e_1_2_11_6_2 doi: 10.1093/bib/bbl022 – start-page: 87 volume-title: Proceedings of the IEEE Conference on Graphics Interface (GI ’09) year: 2009 ident: e_1_2_11_5_2 – ident: e_1_2_11_43_2 – ident: e_1_2_11_14_2 doi: 10.1109/TVCG.2009.84 – ident: e_1_2_11_32_2 – ident: e_1_2_11_40_2 doi: 10.1109/PACIFICVIS.2015.7156375 – ident: e_1_2_11_31_2 doi: 10.1038/nrg2641 – ident: e_1_2_11_3_2 doi: 10.1109/TVCG.2013.109 – ident: e_1_2_11_9_2 doi: 10.1038/483531a – ident: e_1_2_11_42_2 doi: 10.1109/IV.2006.54 – ident: e_1_2_11_44_2 doi: 10.1109/TVCG.2015.2468078 – ident: e_1_2_11_27_2 – ident: e_1_2_11_28_2 doi: 10.1126/science.298.5594.824 |
| SSID | ssj0004765 |
| Score | 2.3258326 |
| Snippet | A major challenge in data‐driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are... A major challenge in data-driven biomedical research lies in the collection and representation of data provenance information to ensure that findings are... |
| SourceID | pubmedcentral proquest pubmed crossref wiley istex |
| SourceType | Open Access Repository Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 481 |
| SubjectTerms | Agglomeration Analysis Biomedical research Categories and Subject Descriptors (according to ACM CCS) Complexity Computer graphics Graph theory Graphs H.5.2 [Information Systems]: Information Interfaces and Presentation-User Interfaces-Graphical user interfaces (GUI) J.3 [Life and Medical Science] Biology and genetics Mathematical analysis Platforms Representations Studies Visualization Workflow |
| Title | AVOCADO: Visualization of Workflow-Derived Data Provenance for Reproducible Biomedical Research |
| URI | https://api.istex.fr/ark:/67375/WNG-KW702M8G-D/fulltext.pdf https://onlinelibrary.wiley.com/doi/abs/10.1111%2Fcgf.12924 https://www.ncbi.nlm.nih.gov/pubmed/29973745 https://www.proquest.com/docview/1801479023 https://www.proquest.com/docview/1825544722 https://www.proquest.com/docview/2064775954 https://pubmed.ncbi.nlm.nih.gov/PMC6027754 |
| Volume | 35 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnZ3JjtQwEIatYeYCB_YlMLQMQmguaRHHjjtwaqbpHgGzCJjlgBTFG7SmlaBeAHHiHXhDnoQqZ6EbZiSEcomUSuSUq-LfSfkLIY-cSKxTkQoTyUTIjeFhyns2jHWkUhlFzlXVFnvJziF_eSJO1sizZi1MxYdoX7hhZvjnNSZ4rmZLSa4_uC4MVgxZoFirhYLozW90FJeJaLjeSIypqUJYxdOeuTIWbaBbv54lNP-ul1zWsX4gGl4h75tbqOpPTruLuerqb3_QHf_zHq-Sy7VApf0qoq6RNVtcJ5eWsIU3SN4_2oce3H9Kj8YzXJRZLeWkpaP47t1Nyi8_v_8YgPVna-ggn-f0YFrCUxUjjIJKpiD7PWl2rCaWPvcEAAwW2tQB3iSHwxfvtnfC-lcNoQbFxUMn8zhPepZJxSMNm9Eu1QaiIOJGKi0s59ZwzViewiTPpZHtiSS2BgmCsYriW2S9KAt7h1BlUmMNY5ETjEtuFDjASJBRieDKSBuQrabTMl1zzPF3GpOsmc-A1zLvtYA8bE0_VfCOs4we-55vLfLpKVa7SZEd742yV8fyCdvtjbJBQDab0MjqRJ9lEdJ3ZArKJyAP2sOQovjdJS9suUAbmLdxpHKeb8Nw0a8UqYD23K6irW0QKAZoDhcBkStx2BogInz1SDH-6FHhCX6ix2tu-TA73wvZ9mjod-7-u-k9chHkY1IVzm2S9fl0Ye-DRJurDrnA-EGHbPQHu6_fdnxm_gJ5BDra |
| linkProvider | Wiley-Blackwell |
| linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV1LbxMxELZKe4AeeLcEChiEUC8b4V17nUVcSkMSaJsi1NcFWeu1DVGjXZQmgDjxH_iH_BJmvA8SaCWE9rKSZ1f2eMb-xh5_JuSJE7F1mukglqEIuDE8SHjHBlHGdCIZc67MthjGg0P-5kScLJEX9VmYkh-iWXBDz_DjNTo4LkjPeXn2wbVhtgr5JbKCN3r7gOrdb_IoLmNRM3sjZ0zFK4R5PM2nC7PRCir263lQ8--MyXkk66ei3jXyvm5EmYFy2p5NdTv79ge_4_-28jq5WmFUulUa1Q2yZPObZHWOufAWSbeO9qET95_To9EZnsssT3PSwlFcfnfj4svP7z-6IP3ZGtpNpyl9OylgYEUjowCUKSB_TzY70mNLX3oSALQXWqcC3iaHvVcH24Oguq0hyAB08cDJNErjjg2l5iyDx2QuyQwYAuNG6kxYzq3hWRimCcR5LmG2I-LIGiQRjDSL1shyXuT2DqHaJMaaMGROhFxyo0EBRgKSigXXRtoW2ax7TWUVlTneqDFWdUgDWlNeay3yuBH9VPJ3nCf01Hd9I5FOTjHhTQp1POyrnWP5LNzr9FW3RTZq21CVr58phgQ8MgHw0yKPmmLwUtx6SXNbzFAGQjeOxJwXy4R47leKREB91ktzayoEoAGqw0WLyAVDbASQJXyxJB999GzhMe7S4z83vZ1drAW13e_5l7v_LvqQXB4c7O2q3dfDnXvkCqDJuMyj2yDL08nM3gfENtUPvGP-Aq4OPIo |
| linkToPdf | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV3bbtQwELVKKyF44F4IFDAIob5k1SS-JPBUGnYLhW2F6OUByYpjG1ZdJdV2FxBP_AN_yJcwkxu70EoI5SVSJpE9PhMfOzMnhDxxXFinA-0LGXKfGcP8hMXWj_JAJzIInKuzLYZie5-9PuJHS-R5WwtT60N0G24YGdX7GgP8xLi5IM8_uh5MViG7QFaY2IgR0um739pRTAreCnujZEwjK4RpPN2tC5PRCvr161lM8--EyXkiW81E_avkQ9uHOgHluDeb6l7-7Q95x__s5DVypWGodLOG1HWyZIsb5PKcbuFNkm0e7MIQ7j6jB6NTrMqsazlp6Shuvrtx-eXn9x8pWH-2hqbZNKN7kxJeqwgxCjSZAu-vpGZHemzpi0oCANFC20TAW2S___L91rbf_KvBz4FyMd_JLMpEbEOpWZDDYXKX5AZgEDAjdc4tY9awPAyzBFZ5LglszEVkDUoIRjqIVslyURb2DqHaJMaaMAwcD5lkRoMDjAQeJTjTRlqPrLeDpvJGyBz_pzFW7YIGvKYqr3nkcWd6Uqt3nGX0tBr5ziKbHGO6m-TqcDhQO4dyI3wbD1TqkbUWGqqJ9FMVoPyOTID6eORRdxliFD-8ZIUtZ2gDCzeGspzn24RY9St5wqE9t2u0dQ0CygDNYdwjcgGHnQFqhC9eKUafKq1wgd_o8ZnrFczO94LaGvSrk7v_bvqQXNxL--rNq-HOPXIJqKSok-jWyPJ0MrP3ga5N9YMqLH8B8WI7Qg |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=AVOCADO%3A+Visualization+of+Workflow%E2%80%93Derived+Data+Provenance+for+Reproducible+Biomedical+Research&rft.jtitle=Computer+graphics+forum&rft.au=Stitz%2C+H.&rft.au=Luger%2C+S.&rft.au=Streit%2C+M.&rft.au=Gehlenborg%2C+N.&rft.date=2016-06-01&rft.issn=0167-7055&rft.eissn=1467-8659&rft.volume=35&rft.issue=3&rft.spage=481&rft.epage=490&rft_id=info:doi/10.1111%2Fcgf.12924&rft.externalDBID=n%2Fa&rft.externalDocID=10_1111_cgf_12924 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0167-7055&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0167-7055&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0167-7055&client=summon |