Design of an Intelligent Vehicle Manifest Recording System at the Bengkalis-Sungai Pakning Ro-Ro Ferry Crossing Based on Deep Learning and Optical Character Recognition

Abstract

Vehicle manifest recording in Ro-Ro ferry services is still predominantly conducted manually, which may lead to operational inefficiencies and data inconsistencies. This study presents an automated vehicle manifest recording system for the Bengkalis–Sungai Pakning Ro-Ro ferry crossing by leveraging deep learning and Optical Character Recognition (OCR) technologies. The proposed system utilizes CCTV or IP cameras to capture vehicle images, performs frame extraction from video streams, and applies YOLOv11 for real-time vehicle and license plate detection. The detected license plate regions are subsequently processed using an OCR module to extract textual vehicle identification information. The detection model was trained using a publicly available vehicle and license plate dataset. Experimental evaluation on the vehicle and license plate dataset shows that the YOLOv11 model achieves a precision of 85.9%, recall of 84.0%, and mAP@0.5 of 87.8% for vehicle and plate detection. OCR evaluation conducted on real operational test images indicates a recognition success rate of 57.5%, with an average confidence score of 0.63 for successfully recognized plates. Further analysis reveals that illumination level and plate scale (distance proxy) are the dominant factors affecting OCR performance, while tilt angle exhibits moderate influence. These results indicate that the proposed framework provides reliable visual detection performance and identifies critical environmental constraints that must be addressed for robust automated manifest deployment in Ro-Ro ferry environments.