ارزیابی خودکار وضعیت روسازی جاده‌ای با استفاده از مدل شبکه مولد تخاصمی با جریمه گرادیان و بخش‌بندی مبتنی بر U-Net

نوع مقاله : مقاله پژوهشی

نویسندگان

دانشکده مهندسی عمران و محیط زیست، دانشگاه صنعتی امیرکبیر (پلی تکنیک تهران)، تهران، ایران

چکیده

توسعه مدل‌های یادگیری عمیق کارآمد در حوزه تشخیص و بخش‌بندی خودکار خرابی‌های روسازی آسفالتی با چالش جدی کمبود و عدم توازن داده‌های آموزشی مواجه است. پژوهش حاضر با هدف ارزیابی خودکار وضعیت روسازی و با فرضیه غلبه بر محدودیت‌های داده‌ای از طریق تولید داده‌های مصنوعی واقع‌گرایانه، رویکردی نوین را برای تشخیص و بخش‌بندی خودکار خرابی‌های آسفالتی ارائه داده است. در روش پیشنهادی، ابتدا از شبکه مولد تخاصمی واسرشتاین با جریمه گرادیان (WGAN-GP) به ‌منظور تولید تصاویر باکیفیت و متنوع ترک با استفاده از مجموعه داده عمومی کرک 500 استفاده شد تا ضمن جلوگیری از فروپاشی حالت، پایداری آموزش تضمین شود. سپس، یک مدل U-Net برای بخش‌بندی پیکسل ‌به ‌پیکسل بر روی مجموعه داده ترکیبی (واقعی و مصنوعی) آموزش داده شد. نوآوری اصلی این تحقیق در تلفیق معماری بهبودیافته GAN با مدل بخش‌بندی برای رفع مشکل بیش‌برازش و افزایش قابلیت تعمیم‌پذیری مدل در شرایط محیطی مختلف است. بررسی نتایج نشان می‌دهد که افزودن تصاویر مصنوعی، عملکرد مدل بخش‌بندی را به ‌طور چشمگیری افزایش داده و منجر به کسب ضریب دایس 961/0 و شاخص اشتراک روی اجتماع 925/0 شده است. همچنین، ارزیابی عملکرد مدل توسعه یافته حاکی از توانایی بالای آن در شناسایی ترک‌های ریز و پیچیده در مجموعه داده‌های دیگر می‌باشد. در نهایت، با ادغام خروجی‌های مدل در یک شاخص وضعیت سطحی، چارچوب پیشنهادی توانایی ارزیابی هوشمند، دقیق و کم‌هزینه شرایط روسازی را فراهم کرده است.

کلیدواژه‌ها

موضوعات

عنوان مقاله [English]

Automatic Assessment of Road Pavement Condition Using a Generative Adversarial Network Model with Gradient Penalty and U-Net-Based Segmentation

نویسندگان [English]

  • Mohammad Sedighian-Fard
  • Amir Golroo
Civil Eng Dept, Amirkabir University of Technology
چکیده [English]

Today, road Pavement Management Systems (PMS) require a transition from traditional methods to automated approaches to ensure safety and reduce maintenance costs. With the advancement of technology, including Autonomous Vehicles (AVs) and Intelligent Transportation Systems (ITS), the need for automatic detection and segmentation of asphalt pavement distress has become critical. However, developing deep learning-based models in this domain faces the critical challenge of the scarcity and imbalance of training data. This study presents a novel approach for the automated detection and segmentation of asphalt distress, aiming to assess pavement condition based on the hypothesis that generating realistic synthetic data can overcome data limitations. In the proposed method, a Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP) was first developed to generate high-quality and diverse crack images using the Crack500 dataset, ensuring training stability and preventing mode collapse. Subsequently, a U-Net model was trained for pixel-wise segmentation on the combined dataset (real and synthetic). The primary innovation of this research lies in integrating the improved GAN architecture with a segmentation model to address overfitting and enhance model generalization across various environmental conditions. Results demonstrated that adding synthetic images significantly enhanced segmentation performance, achieving a Dice coefficient of 0.961 and an Intersection over Union (IoU) of 0.925. Furthermore, qualitative assessment indicated the model's superior capability in detecting fine and complex cracks in other public datasets. Finally, by integrating the model outputs into a Surface Condition Index (SCI), the proposed framework provides an intelligent, accurate, and cost-effective capability for assessing pavement conditions.

کلیدواژه‌ها [English]

  • Pavement Management System
  • Wasserstein Generative Adversarial Network with Gradient Penalty (WGAN-GP)
  • Pavement Crack Segmentation
  • U-Net Model
  • Surface Condition Index (SCI)

مراجع

[1]         K.G. Murty, Letter to the Editor—An Algorithm for Ranking all the Assignments in Order of Increasing Cost, Https://Doi.Org/10.1287/Opre.16.3.682 16 (1968) 682–687. https://doi.org/10.1287/OPRE.16.3.682.
[2]         M. Sedighian-Fard, N. Solatifar, H. Sivilevičius, Calibration of regression-based models for prediction of temperature profile of asphalt layers using LTPP data, Journal of Civil Engineering and Management 29 (2023) 329-341–329–341. https://doi.org/10.3846/JCEM.2023.18611.
[3]         S. Mathavan, K. Kamal, M. Rahman, A Review of Three-Dimensional Imaging Technologies for Pavement Distress Detection and Measurements, IEEE Transactions on Intelligent Transportation Systems 16 (2015) 2353–2362. https://doi.org/10.1109/TITS.2015.2428655.
[4]         P. Marcelino, M. de Lurdes Antunes, E. Fortunato, Comprehensive performance indicators for road pavement condition assessment, Https://Doi.Org/10.1080/15732479.2018.1446179 14 (2018) 1433–1445. https://doi.org/10.1080/15732479.2018.1446179.
[5]         V. V. Silyanov, J.I. Sodikov, R. Kiran, A.I. Sadikov, An overview road data collection, visualization, and analysis from the perspective of developing countries, IOP Conf Ser Mater Sci Eng 832 (2020). https://doi.org/10.1088/1757-899X/832/1/012056.
[6]         M. Gouda, I. Chowdhury, J. Weiß, A. Epp, K. El-Basyouny, Automated assessment of infrastructure preparedness for autonomous vehicles, Autom Constr 129 (2021). https://doi.org/10.1016/j.autcon.2021.103820.
[7]         M. Fahmani, A. Golroo, M. Sedighian-Fard, Deep learning-based predictive models for pavement patching and manholes evaluation, International Journal of Pavement Engineering 25 (2024). https://doi.org/10.1080/10298436.2024.2349901.
[8]         M. Sezgin, B. Sankur, Survey over image thresholding techniques and quantitative performance evaluation, Https://Doi.Org/10.1117/1.1631315 13 (2004) 146–165. https://doi.org/10.1117/1.1631315.
[9]         N.J. Sucgang, M. Ramos, N.A. Arriola, Road Surface Obstacle Detection using Vision and LIDAR for Autonomous Vehicle, Lecture Notes in Engineering and Computer Science 2227 (2017) 260–264.
[10]       S. Chambon, J.M. Moliard, Automatic Road Pavement Assessment with Image Processing: Review and Comparison, International Journal of Geophysics 2011 (2011) 989354. https://doi.org/10.1155/2011/989354.
[11]       Y.-A. Hsieh, Y.J. Tsai, Machine Learning for Crack Detection: Review and Model Performance Comparison, Journal of Computing in Civil Engineering 34 (2020) 04020038. https://doi.org/10.1061/(asce)cp.1943-5487.0000918.
[12]       D. Arya, H. Maeda, S.K. Ghosh, D. Toshniwal, A. Mraz, T. Kashiyama, Y. Sekimoto, Deep learning-based road damage detection and classification for multiple countries, Autom Constr 132 (2021) 103935. https://doi.org/10.1016/J.AUTCON.2021.103935.
[13]       Y. Shi, L. Cui, Z. Qi, F. Meng, Z. Chen, Automatic road crack detection using random structured forests, IEEE Transactions on Intelligent Transportation Systems 17 (2016) 3434–3445. https://doi.org/10.1109/TITS.2016.2552248.
[14]       I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative adversarial networks, Commun ACM 63 (2020) 139–144. https://doi.org/10.1145/3422622.
[15]       J. Zhong, J. Huyan, W. Zhang, H. Cheng, J. Zhang, Z. Tong, X. Jiang, B. Huang, A deeper generative adversarial network for grooved cement concrete pavement crack detection, Eng Appl Artif Intell 119 (2023). https://doi.org/10.1016/j.engappai.2022.105808.
[16]       C. Han, T. Ma, J. Huyan, Z. Tong, H. Yang, Y. Yang, Multi-stage generative adversarial networks for generating pavement crack images, Eng Appl Artif Intell 131 (2024). https://doi.org/10.1016/j.engappai.2023.107767.
[17]       M. Arjovsky, S. Chintala, L. Bottou, Wasserstein GAN, ArXiv (2017) 1–32. https://arxiv.org/pdf/1701.07875 (accessed July 12, 2025).
[18]       I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, A. Courville, Improved Training of Wasserstein GANs, Adv Neural Inf Process Syst 2017-December (2017) 5768–5778. https://arxiv.org/abs/1704.00028v3 (accessed October 25, 2024).
[19]       M. Mirza, S. Osindero, Conditional Generative Adversarial Nets, (2014). https://arxiv.org/abs/1411.1784v1 (accessed December 8, 2024).
[20]       J.Y. Zhu, T. Park, P. Isola, A.A. Efros, Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks, Proceedings of the IEEE International Conference on Computer Vision 2017-October (2017) 2242–2251. https://doi.org/10.1109/ICCV.2017.244.
[21]       H. Maeda, T. Kashiyama, Y. Sekimoto, T. Seto, H. Omata, Generative adversarial network for road damage detection, Computer-Aided Civil and Infrastructure Engineering 36 (2021) 47–60. https://doi.org/10.1111/MICE.12561.
[22]       D. Ma, H. Fang, N. Wang, C. Zhang, J. Dong, H. Hu, Automatic Detection and Counting System for Pavement Cracks Based on PCGAN and YOLO-MF, IEEE Transactions on Intelligent Transportation Systems 23 (2022) 22166–22178. https://doi.org/10.1109/TITS.2022.3161960.
[23]       B. Xu, C. Liu, Pavement crack detection algorithm based on generative adversarial network and convolutional neural network under small samples, Measurement 196 (2022) 111219. https://doi.org/10.1016/J.MEASUREMENT.2022.111219.
[24]       K. Zhang, Y. Zhang, H. Da Cheng, CrackGAN: Pavement Crack Detection Using Partially Accurate Ground Truths Based on Generative Adversarial Learning, IEEE Transactions on Intelligent Transportation Systems 22 (2021) 1306–1319. https://doi.org/10.1109/TITS.2020.2990703.
[25]       Q. Mei, M. Gül, A cost effective solution for pavement crack inspection using cameras and deep neural networks, Constr Build Mater 256 (2020) 119397. https://doi.org/10.1016/J.CONBUILDMAT.2020.119397.
[26]       Y. Que, Y. Dai, X. Ji, A. Kwan Leung, Z. Chen, Y. Tang, Z. Jiang, Automatic classification of asphalt pavement cracks using a novel integrated generative adversarial networks and improved VGG model, Eng Struct 277 (2023) 115406. https://doi.org/10.1016/J.ENGSTRUCT.2022.115406.
[27]       T. Zhang, D. Wang, A. Mullins, Y. Lu, Integrated APC-GAN and AttuNet Framework for Automated Pavement Crack Pixel-Level Segmentation: A New Solution to Small Training Datasets, IEEE Transactions on Intelligent Transportation Systems 24 (2023) 4474–4481. https://doi.org/10.1109/TITS.2023.3236247.
[28]       S. Yu, S. Jia, C. Xu, Convolutional neural networks for hyperspectral image classification, Neurocomputing 219 (2017) 88–98. https://doi.org/10.1016/J.NEUCOM.2016.09.010.
[29]       A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, F.F. Li, Large-scale video classification with convolutional neural networks, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2014) 1725–1732. https://doi.org/10.1109/CVPR.2014.223.
[30]       J. König, M. Jenkins, M. Mannion, P. Barrie, G. Morison, Optimized Deep Encoder-Decoder Methods for Crack Segmentation, Digital Signal Processing: A Review Journal 108 (2020). https://doi.org/10.1016/j.dsp.2020.102907.
[31]       Z. Fan, C. Li, Y. Chen, P. Di Mascio, X. Chen, G. Zhu, G. Loprencipe, Ensemble of Deep Convolutional Neural Networks for Automatic Pavement Crack Detection and Measurement, Coatings 2020, Vol. 10, Page 152 10 (2020) 152. https://doi.org/10.3390/COATINGS10020152.
[32]       Z. Fan, C. Li, Y. Chen, J. Wei, G. Loprencipe, X. Chen, P. Di Mascio, Automatic Crack Detection on Road Pavements Using Encoder-Decoder Architecture, Materials 2020, Vol. 13, Page 2960 13 (2020) 2960. https://doi.org/10.3390/MA13132960.
[33]       W. Song, Z. Zhang, B. Zhang, G. Jia, H. Zhu, J. Zhang, ISTD-PDS7: A Benchmark Dataset for Multi-Type Pavement Distress Segmentation from CCD Images in Complex Scenarios, Remote Sens (Basel) 15 (2023). https://doi.org/10.3390/rs15071750.
[34]       CRACK500-20220509T090436Z-001 | Kaggle, (n.d.). https://www.kaggle.com/datasets/pauldavid22/crack50020220509t090436z001 (accessed September 5, 2023).
[35]       I.J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, Y. Bengio, Generative Adversarial Networks, Sci Robot 3 (2014) 2672–2680. https://arxiv.org/abs/1406.2661v1 (accessed April 22, 2025).
[36]       W. Weng, X. Zhu, U-Net: Convolutional Networks for Biomedical Image Segmentation, IEEE Access 9 (2015) 16591–16603. https://doi.org/10.1109/ACCESS.2021.3053408.
[37]       GitHub - milesial/Pytorch-UNet: PyTorch implementation of the U-Net for image semantic segmentation with high quality images, (n.d.). https://github.com/milesial/Pytorch-UNet (accessed April 22, 2025).
[38]       GitHub - GohVh/resnet34-unet: Segmentation model using UNET architecture with ResNet34 as encoder background, designed with PyTorch., (n.d.). https://github.com/GohVh/resnet34-unet (accessed April 22, 2025).
[39]       M.Y. Shahin, Pavement management for airports, roads, and parking lots: Second Edition, Pavement Management for Airports, Roads, and Parking Lots: Second Edition (2005) 1–572. https://doi.org/10.1007/B101538.
[40]       ASTM, Practice for Roads and Parking Lots Pavement Condition Index Surveys - ASTM D6433-20, (2020). https://doi.org/10.1520/D6433-20.
[41]       J. Ha, D. Kim, M. Kim, Assessing severity of road cracks using deep learning-based segmentation and detection, The Journal of Supercomputing 2022 78:16 78 (2022) 17721–17735. https://doi.org/10.1007/S11227-022-04560-X.
نشریه مهندسی عمران امیرکبیر
دوره 57، شماره 9
آذر 1404
صفحه 1667-1696

فایل ها

هم رسانی

ارجاع به این مقاله

آمار