Implementation of Handwritten Digit Recognition Using CNN in an Augmented Reality-Based Geometry Learning Application

Abstract

Instruction on 3D shapes in elementary schools is still dominated by the use of two-dimensional media, making it difficult for students to visualise three-dimensional objects. Furthermore, most Augmented Reality (AR)-based learning applications focus primarily on presenting learning materials without providing an assessment mechanism that supports handwritten student answers. This study aims to implement Handwritten Digit Recognition (HDR) based on a Convolutional Neural Network (CNN) in an AR-based 3D-shape learning application. The proposed approach integrates CNN-based handwritten digit recognition as an evaluation mechanism within the AR-based learning application, allowing students' handwritten answers to be automatically recognised and evaluated. The CNN model utilises the LeNet-5 architecture and was trained using a combined dataset consisting of 70,000 MNIST images and 10,000 handwritten images independently collected by the researcher. The testing results show that the model achieved an accuracy of 98.32%, while HDR testing within the application using 250 samples of student handwriting achieved an accuracy of 96.80%. The results indicate that the implementation of CNN-based HDR in an AR-based 3D shape learning application can recognise handwritten digits with high accuracy and provide an automated learning evaluation mechanism based on handwritten answers.