Evaluating Accuracy and Temporal Consistency of Machine Learning Models for Land Use/Land Cover Mapping in the Cimanuk Watershed
Abstract
One of the critical watersheds in Indonesia is the Cimanuk watershed, which is recognized as a national priority watershed. Remote sensing plays a vital role in monitoring land use/land cover (LULC) in the Cimanuk watershed. Overall Accuracy (OA) and Kappa Accuracy (KA) are commonly used as primary measures of classification accuracy. In addition to these parameters, it is essential to examine the consistency and rationality of LULC classification over time. This study aimed to compare the accuracy and temporal consistency of three Machine Learning Models: Random Forest (RF), Support Vector Machine (SVM), and Classification and Regression Tree (CART) for LULC classification in the Cimanuk watershed using Sentinel-2 Multispectral Instrument (MSI) images for 2020 and 2025, processed in Google Earth Engine (GEE). The results indicate that RF has the highest accuracy, with 87.9% (OA) and 83.7% (KA) in 2025, and 83.6% (OA) and 77.9% (KA) in 2020. When examining inconsistent or irrational transitions between 2020 and 2025, CART accounted for 11.97% of these transitions, higher than RF and SVM. After applying temporal consistency correction to the 2020 LULC classification result, RF remains the best-performing classifier, achieving 90% (OA) and 86.5% (KA), followed by CART and SVM. These findings provide valuable insights into incorporating accuracy and temporal consistency assessments into time-series LULC analysis, serving as a reference for future LULC studies in watershed management and other geographic contexts.
Full text article
References
1. Krisnanta, A.D.; Hasibuan, H.S.; Tambunan, R.P. Analysis of Landcover Changes and Carrying Capacity of Coastal Cities North Java of Central Java Province, Indonesia. J. Pengelolaan Sumberd. Alam dan Lingkung. 2024, 14, 181–189, doi:10.29244/jpsl.14.1.181.
2. Bondansari; Widiatmaka; Machfud; Munibah, K.; Ambarwulan, W. Preserving Rice Fields and Domestic Rice Adequacy: A Case Study in Banyumas Regency, Central Java, Indonesia. J. Pengelolaan Sumberd. Alam dan Lingkung. 2025, 15, 154–165, doi:10.29244/jpsl.15.1.154.
3. Supangat, A.B.; Basuki, T.M.; Indrajaya, Y.; Setiawan, O.; Wahyuningrum, N.; Purwanto; Putra, P.B.; Savitri, E.; Indrawati, D.R.; Auliyani, D.; et al. Sustainable Management for Healthy and Productive Watersheds in Indonesia. Land 2023, 12, 1–34, doi:10.3390/land12111963.
4. Ambarwulan, W.; Yulianto, F.; Widiatmaka, W.; Rahadiati, A.; Tarigan, S.D.; Firmansyah, I.; Hasibuan, M.A.S. Modelling Land Use/Land Cover Projection Using Different Scenarios in the Cisadane Watershed, Indonesia: Implication on Deforestation and Food Security. Egypt. J. Remote Sens. Sp. Sci. 2023, 26, 273–283, doi:10.1016/j.ejrs.2023.04.002.
5. Nahib, I.; Ambarwulan, W.; Rahadiati, A.; Munajati, S.L.; Prihanto, Y.; Suryanta, J.; Turmudi, T.; Nuswantoro, A.C. Assessment of the Impacts of Climate and LULC Changes on the Water Yield in the Citarum River Basin, West Java Province, Indonesia. Sustain. 2021, 13, doi:10.3390/su13073919.
6. Setyorini, A.; Khare, D.; Pingale, S.M. Simulating the Impact of Land Use/Land Cover Change and Climate Variability on Watershed Hydrology in the Upper Brantas Basin, Indonesia. Appl. Geomatics 2017, 9, 191–204, doi:10.1007/s12518-017-0193-z.
7. Fakhrudin, M.; Daruati, D. Zonasi Resapan Air Hujan Sebagai Dasar Konservasi Sumber Daya Air DAS Cimanuk. LIMNOTEK Perair. Darat Trop. Indones. 2017, 24, 26–35.
8. Ridwansyah, I.; Yulianti, M.; Apip; Onodera, S. ichi; Shimizu, Y.; Wibowo, H.; Fakhrudin, M. The Impact of Land Use and Climate Change on Surface Runoff and Groundwater in Cimanuk Watershed, Indonesia. Limnology 2020, 21, 487–498, doi:10.1007/s10201-020-00629-9.
9. Mashala, M.J.; Dube, T.; Ayisi, K.K.; Ramudzuli, M.R. Using the Google Earth Engine Cloud-Computing Platform to Assess the Long-Term Spatial Temporal Dynamics of Land Use and Land Cover within the Letaba Watershed, South Africa. Geocarto Int. 2023, 38, doi:10.1080/10106049.2023.2252781.
10. Loukika, K.N.; Keesara, V.R.; Sridhar, V. Analysis of Land Use and Land Cover Using Machine Learning Algorithms on Google Earth Engine for Munneru River Basin, India. Sustain. 2021, 13, doi:10.3390/su132413758.
11. Gautam, L.; Rai, R. Land Use and Land Cover Change Analysis Using Google Earth Engine in Manamati Watershed of Kathmandu District, Nepal. Third Pole J. Geogr. Educ. 2022, 22, 49–60, doi:10.3126/ttp.v22i01.52560.
12. Gorelick, N.; Hancher, M.; Dixon, M.; Ilyushchenko, S.; Thau, D.; Moore, R. Google Earth Engine: Planetary-Scale Geospatial Analysis for Everyone. Remote Sens. Environ. 2017, 202, 18–27, doi:10.1016/j.rse.2017.06.031.
13. Tamiminia, H.; Salehi, B.; Mahdianpari, M.; Quackenbush, L.; Adeli, S.; Brisco, B. Google Earth Engine for Geo-Big Data Applications: A Meta-Analysis and Systematic Review. ISPRS J. Photogramm. Remote Sens. 2020, 164, 152–170, doi:10.1016/j.isprsjprs.2020.04.001.
14. Arpitha, M.; Ahmed, S.A.; Harishnaika, N. Land Use and Land Cover Classification Using Machine Learning Algorithms in Google Earth Engine. Earth Sci. Informatics 2023, 16, 3057–3073, doi:10.1007/s12145-023-01073-w.
15. Maxwell, A.E.; Warner, T.A.; Fang, F. Implementation of Machine-Learning Classification in Remote Sensing: An Applied Review. Int. J. Remote Sens. 2018, 39, 2784–2817, doi:10.1080/01431161.2018.1433343.
16. Prodromou, M.; Gitas, I.; Mettas, C.; Tzouvaras, M.; Danezis, C.; Hadjimitsis, D. Comparative Analysis of Supervised Machine Learning Algorithms for Forest Habitat Mapping in Cyprus. Sustain. 2025, 17, 1–32, doi:10.3390/su17136021.
17. Xie, G.; Niculescu, S. Mapping and Monitoring of Land Cover/Land Use (LCLU) Changes in the Crozon Peninsula (Brittany, France) from 2007 to 2018 by Machine Learning Algorithms (Support Vector Machine, Random Forest, and Convolutional Neural Network) and by Post-Classification Comparison (PCC). Remote Sens. 2021, 13, doi:10.3390/rs13193899.
18. Zafar, Z.; Zubair, M.; Zha, Y.; Fahd, S.; Ahmad Nadeem, A. Performance Assessment of Machine Learning Algorithms for Mapping of Land Use/Land Cover Using Remote Sensing Data. Egypt. J. Remote Sens. Sp. Sci. 2024, 27, 216–226, doi:10.1016/j.ejrs.2024.03.003.
19. Pan, X.; Wang, Z.; Feng, G.; Wang, S.; Samiappan, S. Automated Mapping of Land Cover in Google Earth Engine Platform Using Multispectral Sentinel-2 and MODIS Image Products. PLoS One 2025, 20, 1–21, doi:10.1371/journal.pone.0312585.
20. Heryani, N.; Kartiwa, B.; Sosiawan, H.; Rejekiningrum, P.; Adi, S.H.; Apriyana, Y.; Pramudia, A.; Yufdy, M.P.; Tafakresnanto, C.; Rivaie, A.A.; et al. Analysis of Climate Change Impacts on Agricultural Water Availability in Cimanuk Watershed, Indonesia. Sustain. 2022, 14, 1–18, doi:10.3390/su142316236.
21. Cortes, C.; Vapnik, V. Support-Vector Networks. Mach. Learn. 1995, 20, 273–297, doi:10.1109/64.163674.
22. Kasahun, M.; Legesse, A. Machine Learning for Urban Land Use/ Cover Mapping: Comparison of Artificial Neural Network, Random Forest and Support Vector Machine, a Case Study of Dilla Town. Heliyon 2024, 10, e39146, doi:10.1016/j.heliyon.2024.e39146.
23. Hay Chung, L.C.; Xie, J.; Ren, C. Improved Machine-Learning Mapping of Local Climate Zones in Metropolitan Areas Using Composite Earth Observation Data in Google Earth Engine. Build. Environ. 2021, 199, 107879, doi:10.1016/j.buildenv.2021.107879.
24. Savas, C.; Dovis, F. The Impact of Different Kernel Functions on the Performance of Scintillation Detection Based on Support Vector Machines. Sensors (Switzerland) 2019, 19, 1–16, doi:10.3390/s19235219.
25. Breiman, L.; Friedman, J.H.; Olshen, R.A.; Stone, C.J. Classification and Regression Trees; Chapman & Hall/CRC: Boca Raton, 1984; ISBN 9780412048418.
26. Abedinia, A.; Seydi, V. Building Semi-Supervised Decision Trees with Semi-Cart Algorithm. Int. J. Mach. Learn. Cybern. 2024, 15, 4493–4510, doi:10.1007/s13042-024-02161-z.
27. Tilahun, A.; Teferie, B. Accuracy Assessment of Land Use Land Cover Classification Using Google Earth. Am. J. Environ. Prot. 2015, 4, 193–198, doi:10.11648/j.ajep.20150404.14.
28. Gao, J.; O’Neill, B.C. Mapping Global Urban Land for the 21st Century with Data-Driven Simulations and Shared Socioeconomic Pathways. Nat. Commun. 2020, 11, 1–12, doi:10.1038/s41467-020-15788-7.
29. Bojago, E.; Tadila, G.; Masha, M. Monitoring Spatio-Temporal Changes in Land Use, Land Cover, and NDVI Using MODIS Data in Ethiopia’s Gambela Region. Discov. Appl. Sci. 2025, 7, 1–19, doi:10.1007/s42452-025-07879-1.
30. Cao, G.; Tsuchiya, K.; Zhu, W.; Okuro, T. Vegetation Dynamics of Abandoned Paddy Fields and Surrounding Wetlands in the Lower Tumen River Basin, Northeast China. PeerJ 2019, 2019, 1–17, doi:10.7717/peerj.6704.
31. Ardiansyah, M.; Nugraha, R.A.; Iman, L.O.S.; Djatmiko, S.D. Impact of Land Use and Climate Changes on Flood Inundation Areas in the Lower Cimanuk Watershed, West Java Province. J. Ilmu Tanah dan Lingkung. 2021, 23, 51–58, doi:10.29244/jitl.23.2.53-60.
32. Amin, G.; Imtiaz, I.; Haroon, E.; Saqib, N. us; Shahzad, M.I.; Nazeer, M. Assessment of Machine Learning Algorithms for Land Cover Classification in a Complex Mountainous Landscape. J. Geovisualization Spat. Anal. 2024, 8, 1–19, doi:10.1007/s41651-024-00195-z.
33. Al Farikhi, F.; Pramono, R.W.D. Perbandingan Algoritma Classification and Regression Tree (Cart) Dan Random Forest (Rf) Untuk Klasifikasi Penggunaan Lahan Pada Google Earth Engine. J. Spat. Wahana Komun. dan Inf. Geogr. 2023, 23, 170–179, doi:10.21009/spatial.232.09.
34. Pande, C.B.; Srivastava, A.; Moharir, K.N.; Radwan, N.; Mohd Sidek, L.; Alshehri, F.; Pal, S.C.; Tolche, A.D.; Zhran, M. Characterizing Land Use/Land Cover Change Dynamics by an Enhanced Random Forest Machine Learning Model: A Google Earth Engine Implementation. Environ. Sci. Eur. 2024, 36, 1–23, doi:10.1186/s12302-024-00901-0.
35. Khan, S.; Bhardwaj, A.; Sakthivel, M. Accuracy Assessment of Land Use Land Cover Classification Using Machine Learning Classifiers in Google Earth Engine; A Case Study of Jammu District. Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. - ISPRS Arch. 2024, 48, 263–268, doi:10.5194/isprs-archives-XLVIII-4-2024-263-2024.
36. Riaz, M.T.; Riaz, M.T.; Rehman, A.; Bindajam, A.A.; Mallick, J.; Abdo, H.G. An Integrated Approach of Support Vector Machine (SVM) and Weight of Evidence (WOE) Techniques to Map Groundwater Potential and Assess Water Quality. Sci. Rep. 2024, 14, doi:10.1038/s41598-024-76607-3.
37. Gülci, S.; Wing, M.; Akay, A.E. Land Use and Land Cover (LULC) Mapping Accuracy Using Single-Date Sentinel-2 MSI Imagery with Random Forest and Classification and Regression Tree Classifiers. Geomatics 2025, 5, 1–20, doi:10.3390/geomatics5030029.
38. Yang, G.; Fang, S.; Gong, W.; Zhao, Y.; Ge, M. Evaluating the Reliability of Time Series Land Cover Maps by Exploiting the Hidden Markov Model. Stoch. Environ. Res. Risk Assess. 2020, 35, 881–892, doi:10.1007/s00477-020-01915-9.
Authors
Copyright (c) 2026 Salis Deris Artikanur, Widiatmaka, Wiwin Ambarwulan, Irmadi Nahib, Darmawan Listya Cahya, Afifuddin, Yudi Setiawan
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
Article Details
How to Cite
