Comparative Study of CatBoost, XGBoost, Random Forest, and Decision Tree for Phishing Web Page Classification
Keywords:
CatBoost, Cybersecurity, Ensemble algorithms, Machine learning, Phishing detectionAbstract
Phishing is a fraudulent method in which attackers using fake websites steal user information such as login credentials and sensitive financial data. Therefore, this study compares four machine learning algorithms, namely CatBoost, XGBoost, Random Forest, and Decision Tree, in classifying phishing websites efficiently and accurately. In this study, the dataset used is the Web Page Phishing Dataset, which begins with exploration and preprocessing, which includes data cleaning, handling missing values, normalization, feature selection, and testing. Post-split. The data used has been divided into training data and test data, namely 80:20. The model was implemented using Python in Google Colaboratory. Model performance evaluation was measured in five main metrics, such as accuracy, precision, recall, F1-score, and AUC. The experimental results indicate that CatBoost achieved the best position with a performance of 89.57% in accuracy, 85.74% in F1-score, 88.73% in precision, 88.78% in recall, and 89.00% in AUC. XGBoost ranked second with a very competitive performance, followed by Random Forest, which was relatively stable with an accuracy value of 89.41% and an F1-score of 85.35%. On the other hand, the decision tree achieved the lowest performance with an accuracy of 88.69% and an F1-score of 84.10%. These performance results indicate limitations in handling complex data, as well as a tendency to overfit. Overall, ensemble boosting-based algorithms, especially CatBoost and XGBoost, outperform single trees in detecting phishing websites. These results will be benefical to?progress in the next generation for the construction of intelligent based phishing detection system under machine learning. In addition, the outcomes of this study will gain momentum for future works where hyperparameter optimization, larger datasets and real-time applications for phishing detection systems?can be focused. Furthermore, this work will contrast the application of ensemble?algorithm in the cybersecurity field.
Downloads
References
R. Zieni, L. Massari, and M. C. Calzarossa, “Phishing or Not Phishing? A Survey on the Detection of Phishing Websites,” IEEE Access, vol. 11, no. February, pp. 18499–18519, 2023, doi: 10.1109/ACCESS.2023.3247135.
A. F. Mahmud and S. Wirawan, “Deteksi Phishing Website menggunakan Machine Learning Metode Klasifikasi,” Sist. J. Sist. Inf., vol. 13, no. 4, pp. 2540–9719, 2024, doi: 10.32520/stmsi.v13i4.
V. A. Windarni et al., “Deteksi Website Phishing Menggunakan Teknik Filter Pada Model Machine Learning,” Inf. Syst. J., vol. 6, no. 1, pp. 39–43, 2023, doi: 10.24076/infosjournal.2023v6i01.
Y. Miftahuddin and M. M. Faturrahman, “Penerapan Data Standardization dan Multilayer Perceptron pada Identifikasi Website Phishing,” J. MIND J. | ISSN, vol. 7, no. 2, pp. 111–123, 2022, doi: 10.26760/mindjournal.v7i2.111-123.
P. Subarkah and A. N. Ikhsan, “Identifikasi Website Phishing Menggunakan Algoritma Classification And Regression Trees (CART),” J. Ilm. Inform., vol. 6, no. 2, pp. 127–136, 2021, doi: 10.35316/jimi.v6i2.1342.
A. Fajar, S. Yazid, and I. Budi, “Enhancing Phishing Detection through Feature Importance Analysis and Explainable AI: A Comparative Study of CatBoost, XGBoost, and EBM Models,” 2024, doi: 10.48550/arXiv.2411.06860.
R. Saputra and E. Hartati, “Deteksi Website Phising Menggunakan Algoritma Random Forest Dengan Optimalisasi Gridsearch,” JUTIM (Jurnal Tek. Inform. Musirawas), vol. 10, no. 1, pp. 55–67, 2025, doi: 10.32767/jutim.v10i1.2674.
S. Asiri, Y. Xiao, S. Alzahrani, S. Li, and T. Li, “A Survey of Intelligent Detection Designs of HTML URL Phishing Attacks,” IEEE Access, vol. 11, no. January, pp. 6421–6443, 2023, doi: 10.1109/ACCESS.2023.3237798.
Lukito and W. B. T. Handaya, “Deteksi Website Phishing Menggunakan Teknik Machine Learning,” J. Inform. Atma Jogja, vol. 6, no. 01, pp. 69–80, 2025, doi: 10.24002/jiaj.v6i1.11538.
M. R. Fatiha, I. Setiawan, A. N. Ikhsan, and I. R. Yunita, “Optimisasi Sistem Deteksi Phishing Berbasis Web Menggunakan Algoritma Decision Tree,” J. Ilm. IT CIDA, vol. 10, no. 2, p. 97, 2024, doi: 10.55635/jic.v10i2.212.
C. Opara, Y. Chen, and B. Wei, “Look before you leap: Detecting phishing web pages by exploiting raw URL and HTML characteristics,” Expert Syst. Appl., vol. 236, no. October 2020, p. 121183, 2024, doi: 10.1016/j.eswa.2023.121183.
R. Liu, Y. Lin, X. Yang, S. H. Ng, D. M. Divakaran, and J. S. Dong, “Inferring Phishing Intention via Webpage Appearance and Dynamics: A Deep Vision Based Approach,” Proc. 31st USENIX Secur. Symp. Secur. 2022, pp. 1633–1650, 2022.
L. Ouyang and Y. Zhang, “Phishing Web Page Detection with HTML-Level Graph Neural Network,” Proc. - 2021 IEEE 20th Int. Conf. Trust. Secur. Priv. Comput. Commun. Trust. 2021, pp. 952–958, 2021, doi: 10.1109/TrustCom53373.2021.00133.
H. Faris and S. Yazid, “Phishing Web Page Detection Methods: URL and HTML Features Detection,” IoTaIS 2020 - Proc. 2020 IEEE Int. Conf. Internet Things Intell. Syst., pp. 167–171, 2021, doi: 10.1109/IoTaIS50849.2021.9359694.
A. Rácz, D. Bajusz, and K. Héberger, “Effect of Dataset Size and Train/Test Split Ratios in QSAR/QSPR Multiclass Classification,” Molecules, vol. 26, no. 4, p. 1111, Feb. 2021, doi: 10.3390/molecules26041111.
I. H. Sarker, “Machine Learning?: Algorithms , Real ? World Applications and Research Directions,” SN Comput. Sci., vol. 2, no. 3, pp. 1–21, 2021, doi: 10.1007/s42979-021-00592-x.
C. I. Agustyaningrum, Y. Ramdhani, D. Purnama Alamsyah, and O. I. B. Hariyanto, “Deep neural networks and conventional machine learning classifiers to analyze thoracic survival data,” IAES Int. J. Artif. Intell., vol. 13, no. 3, pp. 3686–3694, 2024, doi: 10.11591/ijai.v13.i3.pp3686-3694.
V. Cutting and N. Stephen, “A Review on using Python as a Preferred Programming Language for Beginners,” Int. Res. J. Eng. Technol., vol. 08 Issue:, no. 08, pp. 4258–4263.
M. A. Hamid, D. Aditama, E. Permata, N. Kholifah, M. Nurtanto, and N. W. A. Majid, “Simulating the COVID-19 epidemic event and its prevention measures using python programming,” Indones. J. Electr. Eng. Comput. Sci., vol. 26, no. 1, pp. 278–288, 2022, doi: 10.11591/ijeecs.v26.i1.pp278-288.
J. Qi, R. Yang, and P. Wang, “Application of explainable machine learning based on Catboost in credit scoring,” Electrochem. Soc. J. Phys. Conf. Ser., p. https://iopscience.iop.org/article/10.1088/1742-65, 2021, doi: 10.1088/1742-6596/1955/1/012039.
D. Wang and H. Qian, “CatBoost-Based Automatic Classification Study of River Network,” ISPRS Int. J. Geo-Information, vol. 12, no. 10, p. 416, Oct. 2023, doi: 10.3390/ijgi12100416.
F. Yi et al., “XGBoost-SHAP-based interpretable diagnostic framework for alzheimer’s disease,” BMC Med. Inform. Decis. Mak., vol. 23, no. 1, p. 137, Jul. 2023, doi: 10.1186/s12911-023-02238-9.
Hanif Abdul Karim Afandi, M. Lazaro Fa. Al-Dzaki, Nurul Qomariasih, and Reza Aulia Wildana, “GuardSurfing?: Ekstensi Browser sebagai Alat Bantu Deteksi Website Phishing dengan Metode Klasifikasi XGBoost untuk Deteksi URL Phishing Berbasis Flask Framework,” Info Kripto, vol. 19, no. 2, pp. 73–85, Sep. 2025, doi: 10.56706/ik.v19i2.124.
K. Kirso and M. D. Anasanti, “Improving Alzheimer’s Disease Prediction Accuracy using Feature Selection, K Fold Cross Validation, and KNN Imputer Techniques,” Telematika, vol. 18, no. 1, pp. 75–90, 2025, doi: 10.35671/telematika.v18i1.3055.
M. C. E. Simsekler, N. H. Alhashmi, E. Azar, N. King, R. A. M. A. Luqman, and A. Al Mulla, “Exploring drivers of patient satisfaction using a random forest algorithm,” BMC Med. Inform. Decis. Mak., vol. 21, no. 1, p. 157, Dec. 2021, doi: 10.1186/s12911-021-01519-5.
M. R. Maskur A and A. Wibowo, “Taxpayer Awareness Classification Using Decision Tree and Naïve Bayes Methods,” J. Appl. Informatics Comput., vol. 8, no. 1, pp. 47–54, 2024, doi: 10.30871/jaic.v8i1.6654.
H. Syahputra, S. I. Naibaho, M. A. Maulana, I. Zulfahmi, and E. P. Sinaga, “Perbandingan Algoritma Support Vector Machine (SVM) dan Decision Tree Untuk Deteksi Tingkat Depresi Mahasiswa,” Bina Insa. Ict J., vol. 10, no. 1, p. 52, 2023, doi: 10.51211/biict.v10i1.2304.
M. Heydarian, T. E. Doyle, and R. Samavi, “MLCM: Multi-Label Confusion Matrix,” IEEE Access, vol. 10, pp. 19083–19095, 2022, doi: 10.1109/ACCESS.2022.3151048.
S. Parodi, D. Verda, F. Bagnasco, and M. Muselli, “The clinical meaning of the area under a receiver operating characteristic curve for the evaluation of the performance of disease markers,” Epidemiol. Health, vol. 44, pp. 1–10, 2022, doi: 10.4178/epih.e2022088.
A. M. Carrington et al., “Deep ROC Analysis and AUC as Balanced Average Accuracy, for Improved Classifier Selection, Audit and Explanation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 1, pp. 329–341, 2023, doi: 10.1109/TPAMI.2022.3145392.
H. E. Sadig et al., “Advanced time complexity analysis for real-time COVID-19 prediction in Saudi Arabia using LightGBM and XGBoost,” J. Radiat. Res. Appl. Sci., vol. 18, no. 2, p. 101364, Jun. 2025, doi: 10.1016/j.jrras.2025.101364.
Y. Cao, T. E. Schartel, D. C. Houghton, J. Ross, and D. M. Infante, “Ecological Informatics Evaluating machine leaning algorithms for accuracy , stability , and among-predictors discriminability in modeling species-richness across ten datasets,” Ecol. Inform., vol. 90, no. January, p. 103323, 2025, doi: 10.1016/j.ecoinf.2025.103323.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2026 Haryani Haryani, Cucu Ika Agustyaningrum
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
