Teaching Plan Generation and Evaluation With GPT-4: Unleashing the Potential of LLM in Instructional Design

• Published in: IEEE transactions on learning technologies Vol. 17; pp. 1445 - 1459
• Main Authors: Hu, Bihao, Zheng, Longwei, Zhu, Jiayi, Ding, Lishan, Wang, Yilei, Gu, Xiaoqing
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2024; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Automated teaching plan generation; Computer Assisted Instruction; Education; Instructional design; instructional design capabilities; Knowledge based systems; Large language models; large language models (LLMs); Mathematical analysis; Mathematical models; mathematical problem chains; Mathematics; Mathematics Instruction; Modules; Pedagogical Content Knowledge; pedagogical content knowledge (PCK); Professional development; Secondary School Mathematics; Task analysis; Teacher Improvement; Teacher Qualifications; Teachers; Teaching; Technological innovation
• ISSN: 1939-1382; 2372-0050
• DOI: 10.1109/TLT.2024.3384765

This study explores and analyzes the specific performance of large language models (LLMs) in instructional design, aiming to unveil their potential strengths and possible weaknesses. Recently, the influence of LLMs has gradually increased in multiple fields, yet exploratory research on their application in education remains relatively scarce. In response to this situation, our research, grounded in pedagogical content knowledge theory, initially formulated an instructional design framework based on mathematical problem chains and corresponding prompt instructions. Subsequently, a comprehensive tool for assessing LLM's instructional design capabilities was developed. Utilizing Generative Pretrained Transformer 4, a high school mathematics teaching plan dataset was generated. Finally, the performance of LLMs in instructional design was evaluated. The evaluation results revealed that the teaching plans generated by LLMs excel in setting instructional objectives, identifying teaching priorities, organizing problem chains and teaching activities, articulating subject content, and selecting methods and strategies. Particularly commendable performance was noted in the modules of statistics and functions. However, there is room for improvement in aspects related to mathematical culture and interdisciplinary assessment, as well as in the geometry and algebra modules. Lastly, this study proposes initiatives, such as LLM prompt-based teacher training and the integration of mathematics-focused LLMs. These suggestions aim to advance personalized instructional design and professional development of teachers, offering educators new insights into the in-depth application of LLMs.