Comparative Machine Learning Analysis for Sentiment Classification of Sumatra Disaster 2025

Nauval Alfarizi; Prima Lydia; Muhammad Syahputra Novelan; Adi Putra; Satria Sinurat

Penulis

Nauval Alfarizi Universitas Pembangunan Panca Budi Penulis
Prima Lydia Universitas Pembangunan Panca Budi Penulis
Muhammad Syahputra Novelan Universitas Pembangunan Panca Budi Penulis
Adi Putra Universitas Pembangunan Panca Budi Penulis
Satria Sinurat Universitas Pembangunan Panca Budi Penulis

Kata Kunci:

Sentiment Analyst, Naive Bayes, Support Vector Machine, K-Nearest Neighbor, Natural Disaster

Abstrak

Indonesia is highly vulnerable to natural disasters due to its geological position, resulting in extensive disaster-related news coverage that shapes public sentiment. This study presents a comparative machine learning analysis for sentiment classification of online news related to natural disasters in Sumatra during December 2025. The dataset was collected through web scraping from two major Indonesian news portals, like CNN Indonesia and Detik, and categorized into three sentiment classes: negative, neutral, and positive. Sentiment classification was conducted using Naive Bayes, Support Vector Machine (SVM), and k-Nearest Neighbors (KNN) algorithms. The results demonstrate that Naive Bayes achieved accuracy values of 0.57 on the CNN Indonesia dataset and 0.61 on the Detik dataset. However, its performance was highly biased toward the dominant negative class, as indicated by low macro-average F1-scores of (0.24) and (0.39). In contrast, SVM showed the most balanced performance by reducing class bias, achieving accuracies of (0.68) and (0.67) with macro-average F1-scores of (0.51) and (0.59), respectively. KNN demonstrated moderate performance, with accuracy values of 0.60 and 0.59, but remained less effective than SVM in handling minority sentiment classes.

Referensi

A. Baldah, A. V. Duarisah, and R. A. Maulana, “Clustering Daerah Rawan Bencana Alam Di Indonesia Berdasarkan Provinsi Dengan Metode K-Means,” J. Ilm. Inform. Glob., vol. 14, no. 2, pp. 31–36, 2023, doi: 10.36982/jiig.v14i2.3186.

S. J. Kim, “The role of social media news usage and platforms in civic and political engagement: Focusing on types of usage and platforms,” Comput. Human Behav., vol. 138, no. September 2022, 2023, doi: 10.1016/j.chb.2022.107475.

S. Kim and Y. C. Kim, “Attention to news media, emotional responses, and policy preferences about public health crisis: The case of fine dust pollution in south korea,” Int. J. Environ. Res. Public Health, vol. 18, no. 24, 2021, doi: 10.3390/ijerph182413325.

D. Erokhin and N. Komendantova, “Social media data for disaster risk management and research,” Int. J. Disaster Risk Reduct., vol. 114, no. October, p. 104980, 2024, doi: 10.1016/j.ijdrr.2024.104980.

J. Homepage, A. S. Dimas, P. Sinaga, and A. S. Aji, “Public Sentiment Analysis Towards Mayor Teddy Indra Wijaya with Logistic Regression Approach Analisis Sentimen Publik Terhadap Mayor Teddy Indra Wijaya dengan Pendekatan Logistic Regression,” vol. 5, no. 1, pp. 222–231, 2025.

E. Mayor, M. Miché, and R. Lieb, “Associations between emotions expressed in internet news and subsequent emotional content on twitter,” Heliyon, vol. 8, no. 12, 2022, doi: 10.1016/j.heliyon.2022.e12133.

Y. Mao, Q. Liu, and Y. Zhang, “Sentiment analysis methods, applications, and challenges: A systematic literature review,” J. King Saud Univ. - Comput. Inf. Sci., vol. 36, no. 4, p. 102048, 2024, doi: 10.1016/j.jksuci.2024.102048.

C. Baden et al., “Public Opinion Negotiations in a Digital Media Ecosystem: A Conceptual Framework,” Int. J. Public Opin. Res., vol. 37, no. 3, 2025, doi: 10.1093/ijpor/edaf049.

B. B. Budisusetyo, W. A. Dhitya, A. Nugroho, and G. Pangestu, “Sentiment analysis methods recommendation: A review of aibased techniques on social media analysis,” Procedia Comput. Sci., vol. 245, no. C, pp. 1120–1128, 2024, doi: 10.1016/j.procs.2024.10.341.

B. A. Maulana, M. J. Fahmi, A. M. Imran, and N. Hidayati, “Analisis Sentimen Terhadap Aplikasi Pluang Menggunakan Algoritma Naive Bayes dan Support Vector Machine (SVM),” MALCOM Indones. J. Mach. Learn. Comput. Sci., vol. 4, no. 2, pp. 375–384, 2024, doi: 10.57152/malcom.v4i2.1206.

Syahril Dwi Prasetyo, Shofa Shofiah Hilabi, and Fitri Nurapriani, “Analisis Sentimen Relokasi Ibukota Nusantara Menggunakan Algoritma Naïve Bayes dan KNN,” J. KomtekInfo, vol. 10, pp. 1–7, 2023, doi: 10.35134/komtekinfo.v10i1.330.

M. Syahputra Novelan and M. Iqbal, “Comparative Analysis of the Performance of K-Nearest Neighbor (K-NN) and Naive Bayes Algorithms on User Satisfaction Levels of the Tokopedia Application,” 2025.

H. Zhao, Z. Liu, X. Yao, and Q. Yang, “A machine learning-based sentiment analysis of online product reviews with a novel term weighting and feature selection approach,” Inf. Process. Manag., vol. 58, no. 5, p. 102656, 2021, doi: 10.1016/j.ipm.2021.102656.

Y. A. Hafiz and E. Sudarmilah, “Implementasi Web Scraping Pada Portal Berita Online Pendahuluan Informasi semakin pesat dan penting seiring dengan semakin canggihnya era digital ( Fathir et al ., 2023 ). Portal berita online adalah salah satu sumber informasi yang dapat memberikan manfa,” J. Ilmu Komput., vol. 12, no. 1, pp. 55–60, 2023.

N. Ukey, Z. Yang, B. Li, G. Zhang, Y. Hu, and W. Zhang, “Survey on Exact kNN Queries over High-Dimensional Data Space,” Sensors, vol. 23, no. 2, pp. 1–44, 2023, doi: 10.3390/s23020629.

J. Y. Guyt, H. Datta, and J. Boegershausen, “Unlocking the Potential of Web Data for Retailing Research,” J. Retail., vol. 100, no. 1, pp. 130–147, 2024, doi: 10.1016/j.jretai.2024.02.002.

M. Rodríguez-Ibánez, A. Casánez-Ventura, F. Castejón-Mateos, and P. M. Cuenca-Jiménez, “A review on sentiment analysis from social media platforms,” Expert Syst. Appl., vol. 223, no. March, 2023, doi: 10.1016/j.eswa.2023.119862.

A. Rafdi, H. Mawengkang, and S. Efendi, “Sentiment Analysis Using Naive Bayes Algorithm with Feature Selection Particle Swarm Optimization (PSO) and Genetic Algorithm,” Int. J. Adv. Data Inf. Syst., vol. 2, no. 2, 2021, doi: 10.25008/ijadis.v2i2.1224.

A. Widodo, B. A. Herlambang, and R. Renaldy, “Optimizing Support Vector Machine (SVM) for Sentiment Analysis of Blu by BCA Reviews with Chi-Square,” J. Appl. Informatics Comput., vol. 9, no. 5, pp. 2588–2597, 2025, doi: 10.30871/jaic.v9i5.10541.

M. Yusuf et al., “Memprediksi Dan Menghitung Tingkat Akurasi Data Cuaca Di Indonesia,” vol. 2, pp. 11–16, 2021.

T. Ahmed Khan, R. Sadiq, Z. Shahid, M. M. Alam, and M. Mohd Su’ud, “Sentiment Analysis using Support Vector Machine and Random Forest,” J. Informatics Web Eng., vol. 3, no. 1, pp. 67–75, 2024, doi: 10.33093/jiwe.2024.3.1.5.

R. Setiabudi, N. M. S. Iswari, and A. Rusli, “Enhancing text classification performance by preprocessing misspelled words in Indonesian language,” Telkomnika (Telecommunication Comput. Electron. Control., vol. 19, no. 4, pp. 1234–1241, 2021, doi: 10.12928/TELKOMNIKA.v19i4.20369.