Penerapan Transfer Learning dan Explainable AI untuk Pengenalan Tokoh Wayang Berbasis Web
DOI:
https://doi.org/10.46880/jmika.Vol10No2.pp983-990Keywords:
Transfer Learning, EfficientNetV2, Explainable AI, Grad-CAM, WayangAbstract
Recognizing wayang characters remains difficult because many classes share similar visual traits, capture conditions vary, and labeled data is scarce. This study develops a web-based wayang character recognition system that combines transfer learning with visual explanation for classroom use. Two lightweight backbones, EfficientNetV2-S and DeiT-Tiny, were fine-tuned on ten characters using horizontal flipping, RandAugment, and TrivialAugment, then evaluated through five-fold stratified cross-validation. Performance was measured using accuracy, macro-F1, and the Matthews Correlation Coefficient. EfficientNetV2-S reached a mean accuracy of 86.8%, a macro-F1 of 0.87, and an MCC of 0.85, slightly above DeiT-Tiny at 86.0%, 0.86, and 0.83. Confusion analysis identified recurring errors between visually similar pairs such as Nakula-Sadewa and Gareng-Petruk. Grad-CAM overlays were integrated into a Streamlit interface so that every prediction carries visual evidence that non-experts can inspect. Median inference latency on a mid-range CPU was 28 ms for EfficientNetV2-S and 32 ms for DeiT-Tiny, fast enough for interactive use. The system shows that accuracy, speed, and interpretability can be balanced within the constraints of cultural heritage datasets.
References
Ardiyasa, I. P. (2020). Wayang (puppet) character transformation model in augmented reality. Lekesan: Interdisciplinary Journal of Asia Pacific Arts, 3(2), 54–63. https://doi.org/10.31091/lekesan.v3i2.1141
Chicco, D., & Jurman, G. (2020). The advantages of the Matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation. BMC Genomics, 21(1), 1–13. https://doi.org/10.1186/s12864-019-6413-7
Chicco, D., & Jurman, G. (2021). An ensemble learning approach for enhanced classification of patients with hepatitis and cirrhosis. IEEE Access, 9, 24485–24498. https://doi.org/10.1109/ACCESS.2021.3057.196
Cubuk, E. D., Zoph, B., Shlens, J., & Le, Q. V. (2020). RandAugment: Practical automated data augmentation with a reduced search space. Advances in Neural Information Processing Systems, 33, 18613–18624.
Han, K., Wang, Y., Guo, J., & Wu, E. (2024).
ParameterNet: Parameters are all you need for large-scale visual pretraining of mobile networks. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 15751–15761. https://doi.org/10.1109/CVPR52733.2024.01491
Irawan, C., Rachmawanto, E. H., & Hadi, H. P. (2025). Ensemble learning layer for wayang recognition using CNN-based ResNet-50 and LSTM. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 10(1). https://doi.org/10.22219/kinetik.v10i1.2053
Khosravi, H., Shum, S. B., Chen, G., Conati, C.,
Tsai, Y.-S., Kay, J., Knight, S., Martinez-Maldonado, R., Sadiq, S., & Gašević, D. (2022). Explainable Artificial Intelligence in education. Computers and Education: Artificial Intelligence, 3, 100074. https://doi.org/https://doi.org/10.1016/j.caea i.2022.100074
Kritandani, W., Aryani, R., & Rakasiwi, T. (2024). A report review: Artificial intelligence and the future of teaching and learning. International Research Education Journal, 6(2), 245–253.
https://doi.org/10.17977/um043v6i2p245-253
Kusrini, K., Yudianto, M. R. A., & Al Fatta, H.
(2022). The effect of Gaussian filter and data preprocessing on the classification of Punakawan puppet images with the convolutional neural network algorithm. International Journal of Electrical and Computer Engineering, 12(4), 3752–3761. https://doi.org/10.11591/ijece.v12i4.pp3752-3761
Lee, J., Mukhanov, L., Molahosseini, A. S., Minhas, U., Hua, Y., del Rincon, J., Dichev, K., Hong, C.-H., & Vandierendonck, H. (2023). Resource-Efficient Convolutional Networks: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques. ACM Comput. Surv., 55(13s). https://doi.org/10.1145/3587095
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., & Guo, B. (2021). Swin
Transformer: Hierarchical vision transformer using shifted windows. Proceedings of the IEEE/CVF International Conference on Computer Vision, 9992–10002.
Loshchilov, I., & Hutter, F. (2019). Decoupled weight decay regularization. Proceedings of the International Conference on Learning Representations (ICLR).
Mendoza, D., Romero, F., & Trippel, C. (2024). Model selection for latency-critical inference serving. Proceedings of the Nineteenth European Conference on Computer Systems, 1016–1038.
Müller, S. G., & Hutter, F. (2021). TrivialAugment: Tuning-free yet state-of-the-art data augmentation. Proceedings of the IEEE/CVF International Conference on Computer Vision, 774–782.
Nigade, V., Bauszat, P., Bal, H., & Wang, L. (2024). Inference serving with end-to-end latency SLOs over dynamic edge networks. Real-Time Systems, 60(2). https://doi.org/10.1007/s11241-024-09418-4
Papa, L., Russo, P., Amerini, I., & Zhou, L. (2024). A survey on efficient vision transformers: Algorithms, techniques, and performance benchmarking. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 7682–7700. https://doi.org/10.1109/TPAMI.2024.3392941
Sudiatmika, I. B. K., Artana, M., Utami, N. W.,
Putra, M. A. P., & Dewi, E. G. A. (2021).
Mask R-CNN for Indonesian shadow puppet recognition and classification. Journal of Physics: Conference Series, 1783(1), 12032.
https://doi.org/10.1088/1742-6596/1783/1/012032
Supriyanti, W., & Anggoro, D. A. (2021). Classification of Pandavas Figure in Shadow Puppet Images using Convolutional Neural Networks. Khazanah Informatika: Jurnal Ilmu Komputer dan Informatika, 7(1), 18–24.
https://doi.org/10.23917/khif.v7i1.12484
Takahashi, K., Yamamoto, K., Kuchiba, A., &
Koyama, T. (2022). Confidence interval for micro-averaged F1 and macro-averaged F1 scores. Applied Intelligence, 52(5), 4961–4972. https://doi.org/10.1007/s10489-021-02635-5
Tan, M., & Le, Q. V. (2021). EfficientNetV2: Smaller models and faster training. Proceedings of Machine Learning Research, 139, 10096–10106.
Tena, S., Hartanto, R., & Ardiyanto, I. (2023). Content-Based Image Retrieval for Traditional Indonesian Woven Fabric Images Using a Modified Convolutional Neural Network Method. Journal of Imaging, 9(8).
https://doi.org/10.3390/jimaging9080165
Touvron, H., Cord, M., Douze, M., Massa, F.,
Sablayrolles, A., & Jégou, H. (2021). Training data-efficient image transformers and distillation through attention. Proceedings of Machine Learning Research, 139, 10347–10357.
Wang, H., Wang, Z., Du, M., Yang, F., Zhang, Z.,
Ding, S., Mardziel, P., & Hu, X. (2020). Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networks. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW),111–119. https://doi.org/10.1109/CVPRW50498.2020.00020
WIdaningrum, I., Astuti, I. P., Mustikasari, D., Nurfitri, K., Az-Zahra, R. R., Vidyastari, R. I., & Selamat, A. (2025). Recognition of the Lima Pandawa Shadow Puppet characters utilizing Principal Component Analysis (PCA) for feature extraction and K-Nearest Neighbor (KNN) for classification. Indonesian Journal of Information Systems, 8(1), 68-77. https://doi.org/10.24002/ijis.v8i1.11032
Zhang, H., Torres, F., Sicre, R., Avrithis, Y., & Ayache, S. (2024). Opti-CAM: Optimizing saliency maps for interpretability. Computer Vision and Image Understanding, 248, 104101.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Muhammad Al-Husaini, Hen Hen Lukmana, Muhammad Sidik Asyaky, Ichsan Fauzi Rachman
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
