Meta Matters - Enriching & Exploiting Your Metadata

• Published in: International journal of population data science Vol. 3; no. 4
• Main Author: Hacker, Alex
• Format: Journal Article
• Language: English
• Published: Swansea University 11.09.2018
• ISSN: 2399-4908; 2399-4908
• DOI: 10.23889/ijpds.v3i4.1016

IntroductionData is nothing without context: if you don't know how, when or why a variable was gathered, it's nigh impossible to draw conclusions from it. This presentation discusses different sorts of metadata and how they can be gathered, stored, and used to enrich data; drawing examples from our biobank. Objectives and ApproachEach data item has two types of metadata: variable-level and value-level. For example, consider a questionnaire. The variable-level metadata covers each question: exact wording, validation rules for the answers, etc. The value-level metadata covers each individual answer: details of the questioner, date and time of response, and so on. We also have database-level metadata: datasets which list every dataset or every field in the database. While some of this information needs to be gathered alongside the data itself, much can be extracted or imputed from results or documentation. We present some generalizable examples. ResultsLike any other data, metadata is only worth having if you’re using it. We will present principles and examples of applications that we have developed for it: Data management – Deriving useful variables and tables, and helping to make your data easier to parse, extract, and validate. Presentation – Making your data more human-readable by labelling variables and decoding values. Documentation – Metadata tables make ideal repositories for granular institutional knowledge about your data: known issues, potential pitfalls, or explanations for missing values. Analysis – Identifying which metadata variables are most valuable for analysts, and how best to provide them. Automation – Using the metadata to generate code that can automatically produce summary statistics, tables, graphs… and more metadata! Conclusion/ImplicationsEvery dataset comes with some metadata. When examined and built upon, it can deepen understanding of the data within, as well as becoming a powerful resource in its own right.