Gestures that people can understand and use
Recent advances in computing devices push researchers to envision new interaction modalities that go beyond traditional mouse and keyboard input. Typical examples are large displays for which researchers hope to create more “natural” means of interaction by using human gestures and body movements as...
Saved in:
Published in | Journal of visual languages and computing Vol. 25; no. 5; pp. 572 - 576 |
---|---|
Main Authors | , , |
Format | Journal Article |
Language | English |
Published |
Elsevier Ltd
01.10.2014
|
Subjects | |
Online Access | Get full text |
Cover
Loading…
Abstract | Recent advances in computing devices push researchers to envision new interaction modalities that go beyond traditional mouse and keyboard input. Typical examples are large displays for which researchers hope to create more “natural” means of interaction by using human gestures and body movements as input. In this article, we reflect about this goal of designing gestures that people can easily understand and use and how designers of gestural interaction can capitalize on the experience of 30 years of research on visual languages to achieve it. Concretely, we argue that gestures can be regarded as “visual expressions to convey meaning” and thus are a visual language. Based on what we have learned from visual language research in the past, we then explain why the design of a generic gesture set or language that spans many applications and devices is likely to fail. We also discuss why we recommend using gestural manipulations that enable users to directly manipulate on-screen objects instead of issuing commands with symbolic gestures whose meaning varies among different users, contexts, and cultures.
•Point of view on the design of gestures for human–computer interaction.•Gestural languages as visual languages: capitalizing on experience of VL research.•Stimulating publications in JVLC of articles on gestural interaction. |
---|---|
AbstractList | Recent advances in computing devices push researchers to envision new interaction modalities that go beyond traditional mouse and keyboard input. Typical examples are large displays for which researchers hope to create more “natural” means of interaction by using human gestures and body movements as input. In this article, we reflect about this goal of designing gestures that people can easily understand and use and how designers of gestural interaction can capitalize on the experience of 30 years of research on visual languages to achieve it. Concretely, we argue that gestures can be regarded as “visual expressions to convey meaning” and thus are a visual language. Based on what we have learned from visual language research in the past, we then explain why the design of a generic gesture set or language that spans many applications and devices is likely to fail. We also discuss why we recommend using gestural manipulations that enable users to directly manipulate on-screen objects instead of issuing commands with symbolic gestures whose meaning varies among different users, contexts, and cultures.
•Point of view on the design of gestures for human–computer interaction.•Gestural languages as visual languages: capitalizing on experience of VL research.•Stimulating publications in JVLC of articles on gestural interaction. |
Author | Jetter, Hans-Christian Ardito, Carmelo Costabile, Maria Francesca |
Author_xml | – sequence: 1 givenname: Carmelo surname: Ardito fullname: Ardito, Carmelo email: carmelo.ardito@uniba.it organization: Dipartimento di Informatica, Università di Bari Aldo Moro, via Orabona 4, 70125 Bari, Italy – sequence: 2 givenname: Maria Francesca surname: Costabile fullname: Costabile, Maria Francesca email: maria.costabile@uniba.it organization: Dipartimento di Informatica, Università di Bari Aldo Moro, via Orabona 4, 70125 Bari, Italy – sequence: 3 givenname: Hans-Christian surname: Jetter fullname: Jetter, Hans-Christian email: h.jetter@ucl.ac.uk organization: Intel ICRI Cities, University College London, Gower Street, London, UK |
BookMark | eNp9j01LxDAURYOM4MzoH3DVvbS-JE0_wI0MOgoDbmbhLiQvL9hS2yFpB_z3toxrF5d3N-dxz4at-qEnxu45ZBx48dhm7bnDTADPMygzAHHF1hxqlVZKytXSc5XWovi8YZsYWwAoK1ms2cOe4jgFisn4ZcbkRMOpowRNn0y9oxBH07tkyRTpll1700W6-7tbdnx9Oe7e0sPH_n33fEhRAowpt55sjmjqqi64KipRWVmWCr2w8yTpZM6FRQ_oc8sdYWVIgfd1bpEbIbdMXN5iGGIM5PUpNN8m_GgOerHVrV5s9WKrodSz7Qw9XSCah50bCjpiQz2SawLhqN3Q_If_AmR3X-M |
CitedBy_id | crossref_primary_10_1080_0144929X_2024_2310097 crossref_primary_10_1016_j_apergo_2022_103833 crossref_primary_10_1016_j_promfg_2015_07_172 crossref_primary_10_1145_2682623 crossref_primary_10_1016_j_jvlc_2015_03_001 crossref_primary_10_1080_0144929X_2021_1970227 crossref_primary_10_1177_1071181320641088 crossref_primary_10_1177_1541931213601660 crossref_primary_10_1080_10447318_2021_1898825 crossref_primary_10_1177_0018720818780544 |
Cites_doi | 10.1145/142750.143052 10.1016/S1045-926X(05)80032-1 10.1006/jvlc.1994.1011 10.1145/1936652.1936676 10.1145/2559206.2559220 10.1109/MS.1987.229792 10.1145/1744161.1744163 10.1016/S1045-926X(05)80017-5 10.1145/1357054.1357089 10.1109/VL.1995.520784 10.1145/1226969.1226996 10.1006/jvlc.1997.0037 |
ContentType | Journal Article |
Copyright | 2014 Elsevier Ltd |
Copyright_xml | – notice: 2014 Elsevier Ltd |
DBID | AAYXX CITATION |
DOI | 10.1016/j.jvlc.2014.07.002 |
DatabaseName | CrossRef |
DatabaseTitle | CrossRef |
DatabaseTitleList | |
DeliveryMethod | fulltext_linktorsrc |
Discipline | Applied Sciences Languages & Literatures Computer Science |
EISSN | 1095-8533 |
EndPage | 576 |
ExternalDocumentID | 10_1016_j_jvlc_2014_07_002 S1045926X14000639 |
GroupedDBID | --K --M .DC .~1 0R~ 1B1 1RT 1~. 1~5 29L 4.4 457 4G. 5GY 5VS 7-5 71M 8P~ 9JN AACTN AAEDT AAEDW AAIKJ AAKOC AALRI AAOAW AAQFI AAQXK AAXUO AAYFN ABBOA ABJNI ABMAC ABXDB ABYKQ ACDAQ ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADFGL ADJOM ADMUD AEBSH AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHHHB AHZHX AIALX AIEXJ AIKHN AITUG AJBFU AJOXV ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD ASPBG AVWKF AXJTR AZFZN BKOJK BLXMC CAG COF CS3 DM4 EBS EFBJH EJD EO8 EO9 EP2 EP3 FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q G8K GBLVA GBOLZ HLZ HMY HVGLF HZ~ IHE J1W KOM LG5 LX9 M3X MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 R2- RIG ROL RPZ SBC SDF SDG SDP SES SEW SPC SSS SSV SSZ T5K UHS WUQ XPP ZMT ZU3 ~G- AAXKI AAYXX AKRWK CITATION |
ID | FETCH-LOGICAL-c300t-1bfeb4cca9896156828b3775cf2b1093d3412bcf0cf4b1dec8ae50ff94bc1a23 |
IEDL.DBID | .~1 |
ISSN | 1045-926X |
IngestDate | Thu Sep 12 16:39:42 EDT 2024 Fri Feb 23 02:32:37 EST 2024 |
IsPeerReviewed | true |
IsScholarly | true |
Issue | 5 |
Keywords | Symbolic gestures Manipulations Visual expressions Gestural languages |
Language | English |
LinkModel | DirectLink |
MergedId | FETCHMERGED-LOGICAL-c300t-1bfeb4cca9896156828b3775cf2b1093d3412bcf0cf4b1dec8ae50ff94bc1a23 |
PageCount | 5 |
ParticipantIDs | crossref_primary_10_1016_j_jvlc_2014_07_002 elsevier_sciencedirect_doi_10_1016_j_jvlc_2014_07_002 |
PublicationCentury | 2000 |
PublicationDate | October 2014 2014-10-00 |
PublicationDateYYYYMMDD | 2014-10-01 |
PublicationDate_xml | – month: 10 year: 2014 text: October 2014 |
PublicationDecade | 2010 |
PublicationTitle | Journal of visual languages and computing |
PublicationYear | 2014 |
Publisher | Elsevier Ltd |
Publisher_xml | – name: Elsevier Ltd |
References | Bordegoni (bib4) 1994; 5 M.L. Maher, T. Clausner, B. Tversky, D. Kirsh, J. Kay, A. Danielescu, K. Grace, Gesture-Based Interaction Design: Communication and Cognition, Workshop at CHI'14, 2014. Chang, Levialdi (bib5) 1990; 1 J. Hurtienne, J.H. Israel, Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction, in: Proceedings of the International Conference on Tangible and Embedded Interaction, ACM, Baton Rouge, Louisiana, 2007, pp. 127–134. R.J.K. Jacob, A. Girouard, L.M. Hirshfield, M.S. Horn, O. Shaer, E.T. Solovey, J. Zigelbaum, Reality-based interaction: a framework for post-wimp interfaces, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Florence, Italy, 2008, pp. 201–210. Wigdor, Morrison (bib9) 2010 T. Ballendat, N. Marquardt, S. Greenberg, Proxemic interaction: designing for a proximity and orientation-aware environment, in: Proceedings of the ACM International Conference on Interactive Tabletops and Surfaces, ACM, Saarbrücken, Germany, 2010, pp. 121–130. P. Bottoni, M.F. Costabile, S. Levialdi, P. Mussio, Formalising visual languages, in: Proceedings of the IEEE International Symposium on Visual Languages, 1995, pp. 45–52. Mauney, Howarth, Wirtanen, Capra (bib18) 2010 Jetter, Reiterer, Geyer (bib21) 2013; 18 H.-C. Jetter, J. Gerken, H. Reiterer, Natural user interfaces: why we need better model-worlds, not better gestures, in: Presented at CHI׳10 Workshop on Natural User Interfaces, 2010. Fowler (bib12) 2010 D. Saffer, A Call to Arms for Interaction Designers, 2007. Norman (bib17) 2010; 17 S. Elrod, R. Bruce, R. Gold, D. Goldberg, F. Halasz, W. Janssen, D. Lee, K. McCall, E. Pedersen, K. Pier, J. Tang, B. Welch, Liveboard: a large interactive display supporting group meetings, presentations, and remote collaboration, in: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, ACM, Monterey, California, USA, 1992, pp. 599–607. Kurtenbach, Hulteen (bib3) 1990 (accessed 31.03.14). Saffer (bib6) 2008 Glinert, Kopache, McIntyre (bib10) 1990; 1 Shi-Kuo (bib15) 1987; 4 Catarci, Costabile, Levialdi, Batini (bib11) 1997; 8 R. George, Ocgm (Pronounced Occam[‘S Razor]) is the Replacement for Wimp, 2009. Bordegoni (10.1016/j.jvlc.2014.07.002_bib4) 1994; 5 Fowler (10.1016/j.jvlc.2014.07.002_bib12) 2010 10.1016/j.jvlc.2014.07.002_bib13 10.1016/j.jvlc.2014.07.002_bib14 Jetter (10.1016/j.jvlc.2014.07.002_bib21) 2013; 18 Wigdor (10.1016/j.jvlc.2014.07.002_bib9) 2010 10.1016/j.jvlc.2014.07.002_bib16 Norman (10.1016/j.jvlc.2014.07.002_bib17) 2010; 17 Catarci (10.1016/j.jvlc.2014.07.002_bib11) 1997; 8 10.1016/j.jvlc.2014.07.002_bib20 Mauney (10.1016/j.jvlc.2014.07.002_bib18) 2010 Glinert (10.1016/j.jvlc.2014.07.002_bib10) 1990; 1 Kurtenbach (10.1016/j.jvlc.2014.07.002_bib3) 1990 Shi-Kuo (10.1016/j.jvlc.2014.07.002_bib15) 1987; 4 10.1016/j.jvlc.2014.07.002_bib19 10.1016/j.jvlc.2014.07.002_bib2 10.1016/j.jvlc.2014.07.002_bib1 Chang (10.1016/j.jvlc.2014.07.002_bib5) 1990; 1 Saffer (10.1016/j.jvlc.2014.07.002_bib6) 2008 10.1016/j.jvlc.2014.07.002_bib8 10.1016/j.jvlc.2014.07.002_bib7 |
References_xml | – volume: 18 start-page: 1 year: 2013 end-page: 20 ident: bib21 article-title: Blended interaction: understanding natural human–computer interaction in post-wimp interactive spaces publication-title: Pers. Ubiquit. Comput. contributor: fullname: Geyer – volume: 5 start-page: 205 year: 1994 end-page: 225 ident: bib4 article-title: An environment for the specification and recognition of dynamic gestures publication-title: J. Vis. Lang. Comput. contributor: fullname: Bordegoni – year: 2008 ident: bib6 article-title: Designing Gestural Interfaces: Touchscreens and Interactive Devices contributor: fullname: Saffer – volume: 1 start-page: 311 year: 1990 ident: bib5 article-title: Foreword publication-title: J. Vis. Lang. Comput. contributor: fullname: Levialdi – volume: 17 start-page: 6 year: 2010 end-page: 10 ident: bib17 article-title: Natural user interfaces are not natural publication-title: Interactions contributor: fullname: Norman – year: 1990 ident: bib3 article-title: Gestures in Human–Computer Communication contributor: fullname: Hulteen – start-page: 3193 year: 2010 end-page: 3196 ident: bib9 article-title: Special interest group: designing user interfaces for multi-touch and surface-gesture devices publication-title: CHI׳10 Extended Abstracts on Human Factors in Computing Systems contributor: fullname: Morrison – volume: 4 start-page: 29 year: 1987 end-page: 39 ident: bib15 article-title: Visual languages: a tutorial and survey publication-title: IEEE Softw. contributor: fullname: Shi-Kuo – volume: 1 start-page: 3 year: 1990 end-page: 39 ident: bib10 article-title: Exploring the general-purpose visual alternative publication-title: J. Vis. Lang. Comput. contributor: fullname: McIntyre – volume: 8 start-page: 215 year: 1997 end-page: 260 ident: bib11 article-title: Visual query systems for databases: a survey publication-title: J. Vis. Lang. Comput. contributor: fullname: Batini – year: 2010 ident: bib12 article-title: Domain Specific Languages contributor: fullname: Fowler – start-page: 4015 year: 2010 end-page: 4020 ident: bib18 article-title: Cultural similarities and differences in user-defined gestures for touchscreen user interfaces publication-title: CHI׳10 Extended Abstracts on Human Factors in Computing Systems contributor: fullname: Capra – start-page: 3193 year: 2010 ident: 10.1016/j.jvlc.2014.07.002_bib9 article-title: Special interest group: designing user interfaces for multi-touch and surface-gesture devices contributor: fullname: Wigdor – start-page: 4015 year: 2010 ident: 10.1016/j.jvlc.2014.07.002_bib18 article-title: Cultural similarities and differences in user-defined gestures for touchscreen user interfaces contributor: fullname: Mauney – ident: 10.1016/j.jvlc.2014.07.002_bib1 doi: 10.1145/142750.143052 – volume: 1 start-page: 3 year: 1990 ident: 10.1016/j.jvlc.2014.07.002_bib10 article-title: Exploring the general-purpose visual alternative publication-title: J. Vis. Lang. Comput. doi: 10.1016/S1045-926X(05)80032-1 contributor: fullname: Glinert – volume: 5 start-page: 205 year: 1994 ident: 10.1016/j.jvlc.2014.07.002_bib4 article-title: An environment for the specification and recognition of dynamic gestures publication-title: J. Vis. Lang. Comput. doi: 10.1006/jvlc.1994.1011 contributor: fullname: Bordegoni – ident: 10.1016/j.jvlc.2014.07.002_bib2 doi: 10.1145/1936652.1936676 – ident: 10.1016/j.jvlc.2014.07.002_bib8 doi: 10.1145/2559206.2559220 – year: 2010 ident: 10.1016/j.jvlc.2014.07.002_bib12 contributor: fullname: Fowler – volume: 18 start-page: 1 issue: 5 year: 2013 ident: 10.1016/j.jvlc.2014.07.002_bib21 article-title: Blended interaction: understanding natural human–computer interaction in post-wimp interactive spaces publication-title: Pers. Ubiquit. Comput. contributor: fullname: Jetter – volume: 4 start-page: 29 year: 1987 ident: 10.1016/j.jvlc.2014.07.002_bib15 article-title: Visual languages: a tutorial and survey publication-title: IEEE Softw. doi: 10.1109/MS.1987.229792 contributor: fullname: Shi-Kuo – volume: 17 start-page: 6 year: 2010 ident: 10.1016/j.jvlc.2014.07.002_bib17 article-title: Natural user interfaces are not natural publication-title: Interactions doi: 10.1145/1744161.1744163 contributor: fullname: Norman – volume: 1 start-page: 311 year: 1990 ident: 10.1016/j.jvlc.2014.07.002_bib5 article-title: Foreword publication-title: J. Vis. Lang. Comput. doi: 10.1016/S1045-926X(05)80017-5 contributor: fullname: Chang – ident: 10.1016/j.jvlc.2014.07.002_bib20 doi: 10.1145/1357054.1357089 – ident: 10.1016/j.jvlc.2014.07.002_bib14 – ident: 10.1016/j.jvlc.2014.07.002_bib7 – ident: 10.1016/j.jvlc.2014.07.002_bib13 – year: 1990 ident: 10.1016/j.jvlc.2014.07.002_bib3 contributor: fullname: Kurtenbach – year: 2008 ident: 10.1016/j.jvlc.2014.07.002_bib6 contributor: fullname: Saffer – ident: 10.1016/j.jvlc.2014.07.002_bib16 doi: 10.1109/VL.1995.520784 – ident: 10.1016/j.jvlc.2014.07.002_bib19 doi: 10.1145/1226969.1226996 – volume: 8 start-page: 215 year: 1997 ident: 10.1016/j.jvlc.2014.07.002_bib11 article-title: Visual query systems for databases: a survey publication-title: J. Vis. Lang. Comput. doi: 10.1006/jvlc.1997.0037 contributor: fullname: Catarci |
SSID | ssj0007836 |
Score | 2.1331728 |
Snippet | Recent advances in computing devices push researchers to envision new interaction modalities that go beyond traditional mouse and keyboard input. Typical... |
SourceID | crossref elsevier |
SourceType | Aggregation Database Publisher |
StartPage | 572 |
SubjectTerms | Gestural languages Manipulations Symbolic gestures Visual expressions |
Title | Gestures that people can understand and use |
URI | https://dx.doi.org/10.1016/j.jvlc.2014.07.002 |
Volume | 25 |
hasFullText | 1 |
inHoldings | 1 |
isFullTextHit | |
isPrint | |
link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV07T8MwED5VZWHhUV7lUXlALJVpHnZaj1VFKVC6UKRuUezYohUqFUkZ-e2cE4eHkBgYIiWnOLLO9n3fOXdngPMIUcIEqaB-ivSNISTQhBtFfWTPCU8UcgabKHw_iUaP7HbGZzUYVLkwNqzS2f7SphfW2kk6Tpud1XzeeUBHgosgmqGLUACtzWBHMMI5ffn-FeZhsxTKigSc2rdd4kwZ47V4e7ZlDH1WFPB0Wyu_wOkb4Ax3YMsxRdIvO7MLNb1swLZjjcStyQxF1cEMlawBh2O3C5mRCzL-LJyc7UH7GlHA3pL8KclJGT9OUL1k_ZnlQuy1zvQ-TIdX08GIuvMSqAo9L6e-NFoyHBLRE0hUInSmZNjtcmUCaatGpYhYgVTGU4ZJP9Wql2juGSOYVH4ShAdQX74s9REQ4-lIpYnw8HNMp6HoMSns4pcKWwrWhHalp3hVVsWIq3CxRWy1Glutxp79uR00gVeqjH-MbYxm-492x_9sdwKb9qkMuTuFev661mdIHXLZKuZGCzb6N3ejyQd79MJo |
link.rule.ids | 315,786,790,4521,24144,27955,27956,45618,45712 |
linkProvider | Elsevier |
linkToHtml | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwnV07T8MwED6VMsDCo7zK0wNiqULzcNJ6RBVQIO1CkbpZsWOLVqhUJGXkt3NOnAJCYmCIFDlxFJ3j-75zvjsDnEeIEtpPmeOlSN8oQoKThFo6HrLnJEwkcgaTKDwYRv0nej8OxzXoVbkwRlZpfX_p0wtvbVva1prt-WTSfsRAImR-NMYQoQDaFVg1bMDoui4_vnQeJk2hLEkQOuZ2mzlTirym7y-mjqFHiwqedm3lFzp9Q5ybLdiwVJFclW-zDTU1a8CmpY3ETsoMm6qdGaq2BuzHdhkyIxckXlZOznagdYswYE5J_pzkpBSQE7QvWSzTXIg5FpnahdHN9ajXd-yGCY4MXDd3PKGVoDgmrMuQqUQYTYmg0wml9oUpG5UiZPlCaldqKrxUyW6iQldrRoX0Ej_Yg_rsdaYOgGhXRTJNmIuPoyoNWJcKZma_kNiT0Sa0KjvxeVkWg1d6sSk3VuXGqtw1f7f9JoSVKfmPweXot__od_jPfmew1h8NYh7fDR-OYN1cKfV3x1DP3xbqBHlELk6L7-QTRHPEAQ |
openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Gestures+that+people+can+understand+and+use&rft.jtitle=Journal+of+visual+languages+and+computing&rft.au=Ardito%2C+Carmelo&rft.au=Costabile%2C+Maria+Francesca&rft.au=Jetter%2C+Hans-Christian&rft.date=2014-10-01&rft.pub=Elsevier+Ltd&rft.issn=1045-926X&rft.eissn=1095-8533&rft.volume=25&rft.issue=5&rft.spage=572&rft.epage=576&rft_id=info:doi/10.1016%2Fj.jvlc.2014.07.002&rft.externalDocID=S1045926X14000639 |
thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1045-926X&client=summon |
thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1045-926X&client=summon |
thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1045-926X&client=summon |