A Systematic Review of Technical Evaluation in Telemedicine Systems

• Published in: 2006 International Conference of the IEEE Engineering in Medicine and Biology Society Vol. 2006; pp. 6320 - 6323
• Main Authors: Thiyagarajan, C.A., Clarke, M.
• Format: Conference Proceeding Journal Article
• Language: English
• Published: United States IEEE 2006
• Subjects: Biomedical Technology; Broadcasting; Databases, Bibliographic; Delay effects; Equipment Design; evaluation; Health Services Research; Humans; Information Systems; Measurement standards; Motion Pictures; Protocols; Quality Assurance, Health Care; Technology Assessment, Biomedical - methods; Teleconferencing; teleconsultation; Telemedicine; Telemedicine - instrumentation; Telemedicine - methods; Tellurium; Testing; Time measurement; User-Computer Interface; Vision defects
• DOI: 10.1109/IEMBS.2006.260377
• ISBN: 9781424400324; 1424400325
• ISSN: 1557-170X

We conducted a systematic review of the literature to critically analyse the evaluation and assessment frameworks that have been applied to telemedicine systems. Subjective methods were predominantly used for technical evaluation (59 %), e.g. Likert scale. Those including objective measurements (41%) were restricted to simple metrics such as network time delays. Only three papers included a rigorous standards based objective approach. Our investigation has been unable to determine a definitive standards-based telemedicine evaluation framework that exists in the literature that may be applied systematically to assess and compare telemedicine systems. We conclude that work needs to be done to address this deficiency. We have therefore developed a framework that has been used to evaluate videoconferencing systems telemedicine applications. Our method seeks to be simple to allow relatively inexperienced users to make measurements, is objective and repeatable, is standards based, is inexpensive and requires little specialist equipment. We use the EIA 1956 broadcast test card to assess resolution, grey scale and for astigmatism. Colour discrimination is assessed with the TE 106 and Ishihara 24 colour scale chart. Network protocol analysis software is used to assess network performance (throughput, delay, jitter, packet loss)