IMPROVING SEGMENTATION PERFORMANCE AND OBJECT IDENTIFICATION IN THERMAL ROOF IMAGERY USING ARTIFICIAL INTELLIGENCE
DOI:
https://doi.org/10.21837/pm.v24i44.2256Keywords:
Thermal imaging, Image segmentation, Boundary delineation, Rooftop inspection, Building assessmentAbstract
The segmentation performance for boundary delineation of rooftop areas and the identification of objects in thermal images is often inaccurate due to low contrast and thermal noise. Improving both boundary delineation and object identification is essential to support further analyses. Therefore, the research objective of this paper is to identify roof-level elements in thermal images and improve boundary delineation accuracy for segmentation. This research employed an experimental methodology, examining 100 thermal images from a case study done at Universiti Sains Malaysia. Thermal images were captured using an unmanned aerial vehicle (UAV) and subsequently processed by frame extraction, similarity-based filtering, and manual polygon-based annotation. The study uses an artificial intelligence (AI) framework to execute the analysis. The findings showed that the AI framework was able to identify roof related objects, such as solar panels, roof equipment, and roofs. The proposed Mask R-CNN model achieved a segmentation mAP (AP averaged over IoU 0.50–0.95) of 77.2%, an AP50 of 95.1%, and an AP75 of 90.2%, indicating strong segmentation performance in the present case-study setting. Compared with the benchmark studies reviewed in this paper, which reported performance levels ranging from 9.4% to 80.8% depending on the task and evaluation metric, the present study showed a markedly stronger reported segmentation result. The delineation boundary of the roof elements was also improved, leading to more accurate assessments of the roof's condition and better identification of potential maintenance needs.
Downloads
References
Acorsi, M. G., Gimenez, L. M., & Martello, M. (2020). Assessing the Performance of a Low-Cost Thermal Camera in Proximal and Aerial Conditions. Remote Sensing. https://doi.org/10.3390/rs12213591
Balaras, C. A., & Argiriou, A. A. (2002). Infrared thermography for building diagnostics. Energy and Buildings. https://doi.org/10.1016/s0378-7788(01)00105-0
Batchuluun, G., Lee, Y. W., & Park, K. R. (2020). Thermal Image Reconstruction Using Deep Learning. IEEE Access. https://doi.org/10.1109/access.2020.3007896
Boccalatte, A., & Chanussot, J. (2025). Quantifying urban solar potential losses from rooftop superstructures via aerial imagery and convolutional neural networks. Renewable Energy, 249, 123088. https://doi.org/10.1016/j.renene.2025.123088
Che Haron, R., & Muhd Zafir, N. B. (2025). Digital twin application in construction cost management. Planning Malaysia, 23(35). https://doi.org/10.21837/pm.v23i35.1665
Chicherin, S., Zhuikov, A., & Junussova, L. (2023). District heating for poorly insulated residential buildings—Comparing results of visual study, thermography, and modeling. Sustainability, 15(20), 14908. https://doi.org/10.3390/su152014908
Choi, J.-S., Kim, C., Jang, H., & Kim, E.-J. (2022). Dynamic thermal bridge evaluation of window-wall joints using a model-based thermography method. Case Studies in Thermal Engineering, 35, 102117. https://doi.org/10.1016/j.csite.2022.102117
Chu, F., Cao, J., Song, Z., Shao, Z., Pang, Y., & Li, X. (2024). Toward generalizable multispectral pedestrian detection. IEEE Transactions on Intelligent Transportation Systems, 25(5), 3739–3750. https://doi.org/10.1109/TITS.2023.3330155
Dai, X., Yuan, X., & Wei, X. (2020). TIRNet: Object detection in thermal infrared images for autonomous driving. Applied Intelligence, 51(3), 1244–1261. https://doi.org/10.1007/s10489-020-01882-2
Deng, R., Yang, Z., Chen, X., You, G., & Yan, K. (2020). Object-based thermal image segmentation for fault diagnosis of reciprocating compressors. Sensors, 20(12), 3436. https://doi.org/10.3390/s20123436
Dobbie, S., Strafford, H., Pickrell, O. W., Fonferko-Shadrach, B., Jones, C., Akbari, A., … Lacey, A. (2021). Markup: A Web-Based Annotation Tool Powered by Active Learning. Frontiers in Digital Health, 3. https://doi.org/10.3389/fdgth.2021.598916
Ejaz, N., Tariq, T. B., & Baik, S. W. (2012). Adaptive key frame extraction for video summarization using an aggregation mechanism. Journal of Visual Communication and Image Representation, 23(7), 1031–1040. https://doi.org/10.1016/j.jvcir.2012.06.013
Garrido, I., Erazo-Aux, J., Lagüela, S., Sfarra, S., Ibarra-Castanedo, C., Pivarčiová, E., Gargiulo, G., & Maldague, X. P. V. (2021). Introduction of deep learning in thermographic monitoring of cultural heritage and improvement by automatic thermogram pre-processing algorithms. Sensors, 21(3), 750. https://doi.org/10.3390/s21030750
Gou, Y., Li, Q., Yao, R., Chen, J., & Bai, T. (2023). Algorithm analysis of defect depth for PVC plate with flat-bottomed holes in active pulsed infrared thermography. Measurement Science and Technology, 34, 085404. https://doi.org/10.1088/1361-6501/acd39e
He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask R-CNN. In Proceedings of the IEEE International Conference on Computer Vision (ICCV) (pp. 2961–2969). https://doi.org/10.1109/ICCV.2017.322
He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 770–778). https://doi.org/10.1109/CVPR.2016.90
Hernandez, G., Navarro, M., & Perez-Cisneros, M. (2020). Failure Detection on Electronic Systems Using Thermal Images and Metaheuristic Algorithms. IEEE Latin America Transactions. https://doi.org/10.1109/tla.2020.9111672
Hou, Y., Chen, M., Volk, R., & Soibelman, L. (2021). An approach to semantically segmenting building components and outdoor scenes based on multichannel aerial imagery datasets. Remote Sensing, 13(21), 4357. https://doi.org/10.3390/rs13214357
Khan, N. A., & Bhattacharjee, B. (2021). Thermal and noise insulation performance interaction of building envelope during building simulation optimization in tropical climates. Building and Environment, 200, 107948. https://doi.org/10.1016/j.buildenv.2021.107948
Kim, H., Lamichhane, N., Kim, C., & Shrestha, R. (2023). Innovations in building diagnostics and condition monitoring: A comprehensive review of infrared thermography applications. Buildings, 13(11), 2829. https://doi.org/10.3390/buildings13112829
Krapf, S., Brabec, J., & Moser, M. (2022). RID—Roof information dataset for computer vision-based photovoltaic potential assessment. Remote Sensing, 14(10), 2299. https://doi.org/10.3390/rs14102299
Mastrucci, A., van Ruijven, B., Byers, E., Poblete-Cazenave, M., & Pachauri, S. (2021). Global scenarios of residential heating and cooling energy demand and CO2 emissions. Climatic Change, 168(3–4). https://doi.org/10.1007/s10584-021-03229-3
Mayer, Z., Kahn, J., Hou, Y., & Volk, R. (2021). AI-based thermal bridge detection of building rooftops on district scale using aerial images. In Proceedings of the EG-ICE 2021 Workshop on Intelligent Computing in Engineering (pp. 497–507).
Mayer, Z., Kahn, J., Hou, Y., Götz, M., Volk, R., & Schultmann, F. (2023). Deep learning approaches to building rooftop thermal bridge detection from aerial images. Automation in Construction, 146, 104690. https://doi.org/10.1016/j.autcon.2022.104690
Mohd Nazri, S. F. A., Hashim, K. S. H.-Y., & Mohd Suhaimi, M. S. N. (2025). BUILDING CONDITION ASSESSMENT FOR SCHOOL BUILDING. Planning Malaysia, 23(35). https://doi.org/10.21837/pm.v23i35.1692
Mondal, S. A., Bhattacharaya, T., Rai, A., Sodhi, G. S., Bansal, R., Mondal, A., & Gupta, A. (2025). AutoTag: A framework for generating training data. Progress in Artificial Intelligence, 14(2), 191–210. https://doi.org/10.1007/s13748-024-00360-x
Naiman, A., Farber, E., & Stein, Y. (2022). Edge detection and characterization of digitized images. Pattern Analysis and Applications, 26(1), 61–72. https://doi.org/10.1007/s10044-022-01097-w
Nardi, I., Paoletti, D., & Sfarra, S. (2016). U-value assessment by infrared thermography: A comparison of different calculation methods in a Guarded Hot Box. Energy and Buildings. https://doi.org/10.1016/j.enbuild.2016.04.017
Park, G., Lee, M., Jang, H., & Kim, C. (2021). Thermal anomaly detection in walls via CNN-based segmentation. Automation in Construction, 125, 103627. https://doi.org/10.1016/j.autcon.2021.103627
Pedranzini, F., Alloni, E., Ficco, G., & Frattolillo, A. (2024). Proposal of a new method for the characterization and operational air leakages assessment in HVAC systems. Energy and Buildings, 305, 113881. https://doi.org/10.1016/j.enbuild.2023.113881
Peng, Y., Zhao, S., & Liu, J. (2021). Segmentation of Overlapping Grape Clusters Based on the Depth Region Growing Method. Electronics. https://doi.org/10.3390/electronics10222813
Rosti, B., Jankovic, A., Goia, F., Mathisen, H. M., & Cao, G. (2025). Design and performance analysis of a low-cost monitoring system for advanced building envelopes. Building and Environment, 269, 112344. https://doi.org/10.1016/j.buildenv.2024.112344
Xu, Y., Chau, V., Wu, C., Zhang, Y., Zissimopoulos, V., & Zou, Y. (2023). A Semi Brute-Force Search Approach for (Balanced) Clustering. Algorithmica, 86(1), 130–146. https://doi.org/10.1007/s00453-023-01158-4
Zhang, J., Liang, X., Wang, M., Yang, L., & Zhuo, L. (2020). Coarse-to-fine object detection in unmanned aerial vehicle imagery using lightweight convolutional neural network and deep motion saliency. Neurocomputing, 398, 555–565. https://doi.org/10.1016/j.neucom.2019.03.102
Downloads
Published
How to Cite
Issue
Section
License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 Unported License.
Copyright & Creative Commons Licence
eISSN: 0128-0945 © Year. The Authors. Published for Malaysian Institute of Planners. This is an open-access article under the CC BY-NC-ND license.
The authors hold the copyright without restrictions and also retain publishing rights without restrictions.
