Design and implementation of a weapon storage access control system based on hand gesture recognition and face recognition on Raspberry Pi 5
DOI:
https://doi.org/10.35335/cit.Vol18.2026.1633.pp161-170Keywords:
Hand Gesture Recognition, LSTM, Multimodal Biometric, Raspberry Pi 5, Sequential FusionAbstract
This study presents a multimodal biometric access control system for weapon storage facilities, integrating hand gesture recognition and face recognition through a sequential fusion architecture on Raspberry Pi 5. The sequential design activates face verification only after correct gesture authentication, optimizing computational efficiency on edge hardware while establishing a dual-layer security barrier. The gesture module combines MediaPipe Hands landmark extraction with LSTM-based temporal classification, achieving near-perfect accuracy across four gesture classes. The face module employs dlib's ResNet-34 for 128-dimensional embedding comparison, with an empirically recalibrated Euclidean distance threshold of 0.34 to eliminate false acceptance risks identified during intrusion testing. Evaluation under controlled conditions yielded 0% False Reject Rate and 0% False Accept Rate across 60 trials, with reliable GPIO-controlled solenoid actuation. Results demonstrate that sequential fusion of behavioral and physiological biometrics on a single edge device provides a viable security solution for high-risk access control applications.
Downloads
References
A. K. Jain, A. Ross, and S. Prabhakar, “An Introduction to Biometric Recognition,” IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 1, pp. 4–20, Jan. 2004, doi: 10.1109/TCSVT.2003.818349.
A. Saldamli, G.; Dalli, A.; Taha, K.; Al-Fuqaha, “Security and privacy issues in physical and cyber biometric systems,” pp. 282–289, 2020, doi: 10.1109/ICIoT48696.2020.9089455.
J. Galbally, S. Marcel, and J. Fierrez, “Biometric Antispoofing Methods: A Survey in Face Recognition,” IEEE Access, vol. 2, pp. 1530–1552, 2014, doi: 10.1109/ACCESS.2014.2381273.
R. Tolosana, R. Vera-Rodriguez, J. Fierrez, A. Morales, and J. Ortega-Garcia, “Deepfakes and beyond: A Survey of face manipulation and fake detection,” Inf. Fusion, vol. 64, pp. 131–148, Dec. 2020, doi: 10.1016/j.inffus.2020.06.014.
A. R. and A. Jain, “Multimodal biometrics: An overview,” in 12th EUSIPCO, 2004, pp. 1221–1224. doi: 0908.1417.
M. S. H. and G. Muhammad, “An audio-visual emotion recognition system using deep learning fusion for a cognitive wireless framework,” IEEE Wirel. Commun., vol. 26, no. 3, pp. 62–68, 2021.
F. Zhang et al., “MediaPipe Hands: On-device Real-time Hand Tracking,” Jun. 2020, [Online]. Available: http://arxiv.org/abs/2006.10214
C. Lugaresi et al., “MediaPipe: A Framework for Building Perception Pipelines,” Jun. 2019, [Online]. Available: http://arxiv.org/abs/1906.08172
A. A. Ilham, I. Nurtanio, Ridwang, and Syafaruddin, “Applying LSTM and GRU Methods to Recognize and Interpret Hand Gestures, Poses, and Face-Based Sign Language in Real Time,” J. Adv. Comput. Intell. Intell. Informatics, vol. 28, no. 2, pp. 265–272, Mar. 2024, doi: 10.20965/jaciii.2024.p0265.
B. Sundar and T. Bagyammal, “American Sign Language Recognition for Alphabets Using MediaPipe and LSTM,” Procedia Comput. Sci., vol. 215, pp. 642–651, 2022, doi: 10.1016/j.procs.2022.12.066.
H.-J. Kim and S.-W. Baek, “Application of Wearable Gloves for Assisted Learning of Sign Language Using Artificial Neural Networks,” Processes, vol. 11, no. 4, p. 1065, Apr. 2023, doi: 10.3390/pr11041065.
D. E. King, “High quality face recognition with deep metric learning.” Accessed: Mar. 15, 2026. [Online]. Available: http://blog.dlib.net/2017/02/high-quality-face-recognition-with-deep.html
K. He, X. Zhang, S. Ren, and J. Sun, “Deep Residual Learning for Image Recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Jun. 2016, pp. 770–778. doi: 10.1109/CVPR.2016.90.
H. Li, K. Ota, and M. Dong, “Learning IoT in Edge: Deep Learning for the Internet of Things with Edge Computing,” IEEE Netw., vol. 32, no. 1, pp. 96–101, Jan. 2018, doi: 10.1109/MNET.2018.1700202.
W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge Computing: Vision and Challenges,” IEEE Internet Things J., vol. 3, no. 5, pp. 637–646, Oct. 2016, doi: 10.1109/JIOT.2016.2579198.
J. A. Mensah, J. K. Appati, E. K. . Boateng, E. Ocran, and L. Asiedu, “FaceNet recognition algorithm subject to multiple constraints: Assessment of the performance,” Sci. African, vol. 23, p. e02007, Mar. 2024, doi: 10.1016/j.sciaf.2023.e02007.
A. Jha, Ishita, P. G. Shenwai, A. Batra, S. Kotian, and P. Modi, “GesSure: A Robust Face-Authentic Enabled Dynamic Gesture Recognition GUI Application,” Int. J. Cybern. Informatics, vol. 11, no. 4, pp. 19–30, Aug. 2022, doi: 10.5121/ijci.2022.110402.
and A. S. A. Indriani, M. Harris, “Applying hand gesture recognition for user guide application using MediaPipe,” in 2nd ISSAT, 2021, pp. 101–108.
M. Abadi et al., “TensorFlow: A system for large-scale machine learning,” May 2016, [Online]. Available: http://arxiv.org/abs/1605.08695
C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” J. Big Data, vol. 6, no. 1, p. 60, Dec. 2019, doi: 10.1186/s40537-019-0197-0.
M. Sokolova and G. Lapalme, “A systematic analysis of performance measures for classification tasks,” Inf. Process. Manag., vol. 45, no. 4, pp. 427–437, Jul. 2009, doi: 10.1016/j.ipm.2009.03.002.
F. Schroff, D. Kalenichenko, and J. Philbin, “FaceNet: A unified embedding for face recognition and clustering,” in 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE, Jun. 2015, pp. 815–823. doi: 10.1109/CVPR.2015.7298682.
J. Diez-Tomillo, J. M. Alcaraz-Calero, and Q. Wang, “Dynamic-Distance-Based Thresholding for UAV-Based Face Verification Algorithms,” Sensors, vol. 23, no. 24, p. 9909, Dec. 2023, doi: 10.3390/s23249909.
ISO/IEC, “ISO/IEC 19795-1:2021 Information technology — Biometric performance testing and reporting,” 2021. [Online]. Available: https://www.iso.org/standard/73515.html
M. S. U. and W. Slany, “Visual programming for human detection using FaceNet in Pocket Code,” Int. J. Interact. Mob. Technol., vol. 18, no. 13, pp. 382–396, 2024, doi: 10.3991/ijim.v18i13.49277.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Daffa Rahman, Sunarta Sunarta, Bagus Hendra Saputra
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
