Development of gaze feature extraction and skill level identification techniques for visual evaluation of fluid simulation results
In the manufacturing industry, knowledge transfer from experts to beginners is getting to be important issue because of decreasing number of experts. To support knowledge transfer, various support systems have been developed. However, it is difficult to present knowledge that matches the user's...
Saved in:
| Published in | Kikai Gakkai ronbunshū = Transactions of the Japan Society of Mechanical Engineers Vol. 91; no. 944; p. 24-00261 |
|---|---|
| Main Authors | , |
| Format | Journal Article |
| Language | Japanese |
| Published |
The Japan Society of Mechanical Engineers
2025
一般社団法人 日本機械学会 |
| Subjects | |
| Online Access | Get full text |
Cover
Loading…
| Abstract | In the manufacturing industry, knowledge transfer from experts to beginners is getting to be important issue because of decreasing number of experts. To support knowledge transfer, various support systems have been developed. However, it is difficult to present knowledge that matches the user's skill level. On the other hand, techniques for judging user's skill level using biological information such as EEG (Electroencephalography), HRV (Heart rate variability), and gaze have been developed. In this study, we focused on the gaze, which is easier to measure than EEG and HRV. It has been reported that the content of work and skill level were estimated using machine learning from the features of gaze. However, the features of the gaze used differ depending on the subject. In this study, we focused on gaze and pupil size to evaluate the differences between expert, intermediate, and beginner in a task of searching for vortices in fluid simulation images. As a result of comparing the gaze and pupil size of 8 experts, 8 intermediates and 8 beginners, it was found significant differences in the fixation duration, the number of fixations, and the number of gaze movements. Although there was no significant difference in pupil size, which indicates cognitive load, between experts and intermediates, both groups had larger pupil sizes compared to beginners. Experts explore vortices with high cognitive load over long periods and many gaze movement, intermediates explore vortices with high cognitive load over short periods and few gaze movement, and beginners explore vortices with low cognitive load over short periods and few gaze movement. We used Random Forest to learn the fixation duration, number of fixations, number of gaze movements, and pupil size, and classified the skill level with the accuracy of 83.1 ± 11.6% using the features with high importance. These results imply the gaze shows the difference in the users' skill levels and show the prospect of presenting appropriate knowledge to the user of the design support system by using the gaze-measurement result. |
|---|---|
| AbstractList | In the manufacturing industry, knowledge transfer from experts to beginners is getting to be important issue because of decreasing number of experts. To support knowledge transfer, various support systems have been developed. However, it is difficult to present knowledge that matches the user's skill level. On the other hand, techniques for judging user's skill level using biological information such as EEG (Electroencephalography), HRV (Heart rate variability), and gaze have been developed. In this study, we focused on the gaze, which is easier to measure than EEG and HRV. It has been reported that the content of work and skill level were estimated using machine learning from the features of gaze. However, the features of the gaze used differ depending on the subject. In this study, we focused on gaze and pupil size to evaluate the differences between expert, intermediate, and beginner in a task of searching for vortices in fluid simulation images. As a result of comparing the gaze and pupil size of 8 experts, 8 intermediates and 8 beginners, it was found significant differences in the fixation duration, the number of fixations, and the number of gaze movements. Although there was no significant difference in pupil size, which indicates cognitive load, between experts and intermediates, both groups had larger pupil sizes compared to beginners. Experts explore vortices with high cognitive load over long periods and many gaze movement, intermediates explore vortices with high cognitive load over short periods and few gaze movement, and beginners explore vortices with low cognitive load over short periods and few gaze movement. We used Random Forest to learn the fixation duration, number of fixations, number of gaze movements, and pupil size, and classified the skill level with the accuracy of 83.1 ± 11.6% using the features with high importance. These results imply the gaze shows the difference in the users' skill levels and show the prospect of presenting appropriate knowledge to the user of the design support system by using the gaze-measurement result. |
| Author | WATANUKI, Keiichi MAEDA, Taichi |
| Author_FL | WATANUKI Keiichi MAEDA Taichi |
| Author_FL_xml | – sequence: 1 fullname: MAEDA Taichi – sequence: 2 fullname: WATANUKI Keiichi |
| Author_xml | – sequence: 1 fullname: MAEDA, Taichi organization: Production Engineering and MONOZUKURI Innovation Center, Center for Sustainability, Research & Development Group, Hitachi, Ltd – sequence: 2 fullname: WATANUKI, Keiichi organization: Graduate School of Science and Engineering, Saitama University |
| BackLink | https://cir.nii.ac.jp/crid/1390866514695030656$$DView record in CiNii |
| BookMark | eNo9kTtv3DAMgIWiBZKm-QOZNHR1qpdlaSzSV4oAXZJZoCXqoqtOTi370GTsL6_v3N5CAuTHjyD4lrwuQ0FCrji75sLaD9MIpW7rDq-FahgTmr8i54KbrrGd5mfkstbUM6lFa2TXnpM_n3CPeXjaYZnoEOkGXpBGhGkekeLvReenNBQKJdD6M-VM82GAprAMpJg8HNsT-seSfs1YaRxGuk91hkxxD3legUUd85wWSdrNea2NWOc81XfkTYRc8fJfviAPXz7f33xr7n58vb35eNcEroVuRO-ZiRxCL0Lste0woIgxBNvzFlTU0mAwXHm5QMp7sdyIXBnDW2NMlPKC3K7eMMDWPY1pB-OzGyC5Y2EYNw7GKfmMTkjTW7R9H6VRUVmIHHnXSohWicAOrverq6TkfDpELi0zWrdcadsyyXSrF-z7im3rBBs8Lf2_5_QtZ7mzSh2SUO74txPkH2F0WORfSnOadg |
| ContentType | Journal Article |
| Copyright | 2025 The Japan Society of Mechanical Engineers |
| Copyright_xml | – notice: 2025 The Japan Society of Mechanical Engineers |
| DBID | RYH DOA |
| DOI | 10.1299/transjsme.24-00261 |
| DatabaseName | CiNii Complete DOAJ - Directory of Open Access Journals |
| DatabaseTitleList | |
| Database_xml | – sequence: 1 dbid: DOA name: DOAJ Directory of Open Access Journals url: https://www.doaj.org/ sourceTypes: Open Website |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| DocumentTitleAlternate | 流体解析結果の目視評価における視線特徴の抽出と熟練度識別技術の開発 |
| DocumentTitle_FL | 流体解析結果の目視評価における視線特徴の抽出と熟練度識別技術の開発 |
| EISSN | 2187-9761 |
| EndPage | 24-00261 |
| ExternalDocumentID | oai_doaj_org_article_238b9e9bbf384f49af1e1753af942d03 article_transjsme_91_944_91_24_00261_article_char_en |
| GroupedDBID | ALMA_UNASSIGNED_HOLDINGS GROUPED_DOAJ JSF KQ8 RJT RYH |
| ID | FETCH-LOGICAL-d1626-2bc08f1adb2dfb697ede2ffdd9b15a4f638ed814c3f1a4cc2625e148815888f33 |
| IEDL.DBID | DOA |
| IngestDate | Wed Aug 27 01:29:04 EDT 2025 Thu Jun 26 21:14:24 EDT 2025 Thu May 08 13:50:23 EDT 2025 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 944 |
| Language | Japanese |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-d1626-2bc08f1adb2dfb697ede2ffdd9b15a4f638ed814c3f1a4cc2625e148815888f33 |
| OpenAccessLink | https://doaj.org/article/238b9e9bbf384f49af1e1753af942d03 |
| ParticipantIDs | doaj_primary_oai_doaj_org_article_238b9e9bbf384f49af1e1753af942d03 nii_cinii_1390866514695030656 jstage_primary_article_transjsme_91_944_91_24_00261_article_char_en |
| PublicationCentury | 2000 |
| PublicationDate | 2025 |
| PublicationDateYYYYMMDD | 2025-01-01 |
| PublicationDate_xml | – year: 2025 text: 2025 |
| PublicationDecade | 2020 |
| PublicationTitle | Kikai Gakkai ronbunshū = Transactions of the Japan Society of Mechanical Engineers |
| PublicationTitleAlternate | 日本機械学会論文集 |
| PublicationTitle_FL | 日本機械学会論文集 |
| PublicationYear | 2025 |
| Publisher | The Japan Society of Mechanical Engineers 一般社団法人 日本機械学会 |
| Publisher_xml | – name: The Japan Society of Mechanical Engineers – name: 一般社団法人 日本機械学会 |
| References | Kamahara, J., Nagamatsu, T., Hiroe, M., Ito, K., Aoyagi, S. and Takada, K., Comparison of gaze analysis method for determining proficiency level by measuring the gaze of facial image, DEIM Forum2020 (2020), C1-5 (in Japanese). Couceiro, R., Barbosa, R., Durães, J., Duarte, G., Castelhano, J., Duarte, C., Teixeira, C., Laranjeiro, N., Medeiros, J., Carvalho, D. P., Branco, C. M. and Madeira, H., Spotting problematic code lines using nonintrusive programmers’ biofeedback, IEEE 30th International Symposium on software Reliability Engineering (ISSRE) (2019), pp.93-103. Horiguchi, N., Ohtsubo, K. and Yoneyama, S., A study on analytical method for restoring failure on mold manufacturing to know-how (A method of comparative analysis using matrix), Transactions of the JSME (in Japanese), Vol.82, No.842 (2016), DOI: 10.1299/transjsme.16-00020. Hershman, R., Henik, A. and Cohen, N., A novel blink detection method based on pupillometry noise, Behavior Research Methods (2018), pp.107–114, Springer. Moriya, T. and Nakajima, A., Systematization of manufacturing knowledge by case-based approach, Journal of the Society of Instrument and Control Engineers, Vol.46, No.7 (2007), pp.564–570 (in Japanese). Japan Vocational Ability Development Association, Skill Testing & Certification, available from <https://www.javada.or.jp/jigyou/gino/giken.html>, (accessed on 7th January, 2025). Maeda, T. and Watanuki, K., Evaluation of gazes of experts and beginners in vortex search of fluid-simulation, International Conference on Design and Concurrent Engineering 2023 & Manufacturing Systems Conference 2023 (2023b), Regular Paper: No.20. Chew, Y. J., Ohtomi, K. and Suzuki, H., Skill metrics for mobile crane operators based on gaze fixation pattern, Advances in Human Aspects of Transportation (2017), pp.1139–1149, Springer. Ghia, U., Ghia, N. K. and Shin, T. C., High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method, Journal of computational physics 48 (1982), pp.387-411. Sakata, Y., Hara, S., Yamashita, M. and Aoyama, K., Knowledge management for design development and maintenance through text mining of product information (Failure case classification by text mining about product failures), 32nd Design Engineering and System Division Conference of JSME, No.22- 20 (2022), 1202 (in Japanese). Borisov, V., Kasneci, E. and Kasneci, G., Robust cognitive load detection from wrist-band sensors, Computers in Human Behavior Reports 4 (2021), 100116. Hanafusa, R., Yamagishi, S., Matsumoto, S. and Kashima, T., Automatic classification of eye tracking patterns in reading program based on machine learning, The 29th Annual Conference of the Japanese Society for Artificial Intelligence (2015), 3N3-2 (in Japanese). Mathôt, S., Fabius, J., Heusden, E. V. and Stigchel, S. V., Safe and sensible preprocessing and baseline correction of pupil-size data, Behavior Research Methods (2018), pp.94–106, Springer. Saito, K., Analysis of review quality by using gaze data during document review, Unisys Technology Review, Vol.139, (2019), pp.31-42 (in Japanese). Asaka, Y. and Watanuki, K., The relationship between brain activity and accuracy of replicating actions in the process of embodied knowledge acquisition, Transactions of the JSME (in Japanese), Vol.82, No.842 (2016), DOI: 10.1299/transjsme.16-00150. Maeda, T. and Watanuki, K., Evaluation of the difference in gaze of expert and beginner in the phenomenon evaluation of fluid simulation, Transactions of the JSME (in Japanese), Vol.89, No.919 (2023a), DOI: 10.1299/transjsme.22-00169. Baig, Z. M. and Kavakli, M., Classification of user competency level using EEG and convolutional neural network in 3D modeling application, Expert Systems with Applications Vol.146 (2020), 113202, ELSEVIER. Hiekata, K., Yamato, H. and Oishi, W., Development of knowledge transfer support framework based on design data – A Case Study on Marine Propeller Design -, Journal of the Japan Society of Naval Architects and Ocean Engineers, Vol.6 (2007), pp.151–158 (in Japanese). Takahashi, R., Watanuki, K., Kaede, K. and Muramatsu, K., Development of a new index for lie detection using changes in pupil diameter, 29th Design Engineering and System Division Conference of JSME, No.19-35 (2019), 2404 (in Japanese). The Japan Society of Mechanical Engineers Certification Program for Computational Mechanics Engineers, Levels of certification, available from <https://www.jsme.or.jp/cee/about>, (accessed on 7th January, 2025). Anne, R., Thomas, N. and Sven, M., Differences in analysis and interpretation of technical systems by expert and novice engineering designers, International Conference on Engineering Design, ICED15 (2015), pp.339–348. |
| References_xml | – reference: Asaka, Y. and Watanuki, K., The relationship between brain activity and accuracy of replicating actions in the process of embodied knowledge acquisition, Transactions of the JSME (in Japanese), Vol.82, No.842 (2016), DOI: 10.1299/transjsme.16-00150. – reference: Couceiro, R., Barbosa, R., Durães, J., Duarte, G., Castelhano, J., Duarte, C., Teixeira, C., Laranjeiro, N., Medeiros, J., Carvalho, D. P., Branco, C. M. and Madeira, H., Spotting problematic code lines using nonintrusive programmers’ biofeedback, IEEE 30th International Symposium on software Reliability Engineering (ISSRE) (2019), pp.93-103. – reference: Hiekata, K., Yamato, H. and Oishi, W., Development of knowledge transfer support framework based on design data – A Case Study on Marine Propeller Design -, Journal of the Japan Society of Naval Architects and Ocean Engineers, Vol.6 (2007), pp.151–158 (in Japanese). – reference: Baig, Z. M. and Kavakli, M., Classification of user competency level using EEG and convolutional neural network in 3D modeling application, Expert Systems with Applications Vol.146 (2020), 113202, ELSEVIER. – reference: Maeda, T. and Watanuki, K., Evaluation of gazes of experts and beginners in vortex search of fluid-simulation, International Conference on Design and Concurrent Engineering 2023 & Manufacturing Systems Conference 2023 (2023b), Regular Paper: No.20. – reference: Moriya, T. and Nakajima, A., Systematization of manufacturing knowledge by case-based approach, Journal of the Society of Instrument and Control Engineers, Vol.46, No.7 (2007), pp.564–570 (in Japanese). – reference: Horiguchi, N., Ohtsubo, K. and Yoneyama, S., A study on analytical method for restoring failure on mold manufacturing to know-how (A method of comparative analysis using matrix), Transactions of the JSME (in Japanese), Vol.82, No.842 (2016), DOI: 10.1299/transjsme.16-00020. – reference: Maeda, T. and Watanuki, K., Evaluation of the difference in gaze of expert and beginner in the phenomenon evaluation of fluid simulation, Transactions of the JSME (in Japanese), Vol.89, No.919 (2023a), DOI: 10.1299/transjsme.22-00169. – reference: Hanafusa, R., Yamagishi, S., Matsumoto, S. and Kashima, T., Automatic classification of eye tracking patterns in reading program based on machine learning, The 29th Annual Conference of the Japanese Society for Artificial Intelligence (2015), 3N3-2 (in Japanese). – reference: Takahashi, R., Watanuki, K., Kaede, K. and Muramatsu, K., Development of a new index for lie detection using changes in pupil diameter, 29th Design Engineering and System Division Conference of JSME, No.19-35 (2019), 2404 (in Japanese). – reference: Borisov, V., Kasneci, E. and Kasneci, G., Robust cognitive load detection from wrist-band sensors, Computers in Human Behavior Reports 4 (2021), 100116. – reference: The Japan Society of Mechanical Engineers Certification Program for Computational Mechanics Engineers, Levels of certification, available from <https://www.jsme.or.jp/cee/about>, (accessed on 7th January, 2025). – reference: Sakata, Y., Hara, S., Yamashita, M. and Aoyama, K., Knowledge management for design development and maintenance through text mining of product information (Failure case classification by text mining about product failures), 32nd Design Engineering and System Division Conference of JSME, No.22- 20 (2022), 1202 (in Japanese). – reference: Mathôt, S., Fabius, J., Heusden, E. V. and Stigchel, S. V., Safe and sensible preprocessing and baseline correction of pupil-size data, Behavior Research Methods (2018), pp.94–106, Springer. – reference: Japan Vocational Ability Development Association, Skill Testing & Certification, available from <https://www.javada.or.jp/jigyou/gino/giken.html>, (accessed on 7th January, 2025). – reference: Saito, K., Analysis of review quality by using gaze data during document review, Unisys Technology Review, Vol.139, (2019), pp.31-42 (in Japanese). – reference: Hershman, R., Henik, A. and Cohen, N., A novel blink detection method based on pupillometry noise, Behavior Research Methods (2018), pp.107–114, Springer. – reference: Chew, Y. J., Ohtomi, K. and Suzuki, H., Skill metrics for mobile crane operators based on gaze fixation pattern, Advances in Human Aspects of Transportation (2017), pp.1139–1149, Springer. – reference: Ghia, U., Ghia, N. K. and Shin, T. C., High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method, Journal of computational physics 48 (1982), pp.387-411. – reference: Kamahara, J., Nagamatsu, T., Hiroe, M., Ito, K., Aoyagi, S. and Takada, K., Comparison of gaze analysis method for determining proficiency level by measuring the gaze of facial image, DEIM Forum2020 (2020), C1-5 (in Japanese). – reference: Anne, R., Thomas, N. and Sven, M., Differences in analysis and interpretation of technical systems by expert and novice engineering designers, International Conference on Engineering Design, ICED15 (2015), pp.339–348. |
| SSID | ssib036258375 ssib016970096 ssib051641555 ssj0002911760 |
| Score | 2.3783607 |
| Snippet | In the manufacturing industry, knowledge transfer from experts to beginners is getting to be important issue because of decreasing number of experts. To... |
| SourceID | doaj nii jstage |
| SourceType | Open Website Publisher |
| StartPage | 24-00261 |
| SubjectTerms | Fluid simulation Gaze Knowledge transfer Machine learning Skill level Vortex |
| Title | Development of gaze feature extraction and skill level identification techniques for visual evaluation of fluid simulation results |
| URI | https://www.jstage.jst.go.jp/article/transjsme/91/944/91_24-00261/_article/-char/en https://cir.nii.ac.jp/crid/1390866514695030656 https://doaj.org/article/238b9e9bbf384f49af1e1753af942d03 |
| Volume | 91 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| ispartofPNX | Transactions of the JSME (in Japanese), 2025, Vol.91(944), pp.24-00261-24-00261 |
| link | http://utb.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwrV09T90wFLUKUxlQaYt4fFQeWFNix07isUUgVIlOILFFdmxXeYRHRd5j6NCBX95zk5Bm69LFUSLbiXzse--JfI8ZOwXIdbS1TEzqy0RFFROnMpXAEtqiIMGtnBKFr7_nV7fq252-mx31RXvCBnngYeDO4FKcCca5mJXoytgoAqlL2miU9IPOJ3zejExhJoncFBScv97DSmswsWnmapAEOFI9_Y2RWPNFno5ZNTDRZ2tyFcvuIXyWKumZyqjqD1e1ROBGGflbq6aZeaPLd2x3DCP5l-Hz99ibpX3Pdmbigh_Yy2w_EH-M_If9FXgMvZAnh0l-GlIauF153t03bctbasAbP24g6jHjk8hrxxHf8uem2-DFf0XCqevYbhp00jyMZ4FxcPhNu-4-stvLi5vzq2Q8cSHxAswmka5Oyyisd9JHh0EMPsgYvTdOaKsiFmvwpVB1hkqqriXGNYBQlUKDSccs22fbq8dVOGDcFplRtCELJkURhl6JqJ3RRto8mHTBvtJoVj8HUY2KZK77BwC_GsGv_gX-gp0PWEzdvLac4KuMqIxSdJGq6oGcKlFiG6zDgp0AyKpuqEREnJIGIDyI0USodH74Pz71iL2VdHRw__fmmG2vnzbhBPHM2n3qpy7K698XfwAKF_H6 |
| linkProvider | Directory of Open Access Journals |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Development+of+gaze+feature+extraction+and+skill+level+identification+techniques+for+visual+evaluation+of+fluid+simulation+results&rft.jtitle=Transactions+of+the+JSME+%28in+Japanese%29&rft.au=MAEDA%2C+Taichi&rft.au=WATANUKI%2C+Keiichi&rft.date=2025&rft.pub=The+Japan+Society+of+Mechanical+Engineers&rft.eissn=2187-9761&rft.volume=91&rft.issue=944&rft.spage=24-00261&rft.epage=24-00261&rft_id=info:doi/10.1299%2Ftransjsme.24-00261&rft.externalDocID=article_transjsme_91_944_91_24_00261_article_char_en |