تشخیص خواب آلودگی راننده با استفاده از تلفیق روش پردازش تصویر ویولا-جونز و آشکارسازی لندمارک های تصویر

نوع مقاله : مقاله پژوهشی

نویسندگان

1 گروه عمران، دانشکده مهندسی، دانشگاه زنجان، زنجان، ایران

2 دانشکده فنی و مهندسی، دانشگاه بین المللی امام خمینی، قزوین، ایران

چکیده

خستگی یکی از دلایل اصلی تصادفات رانندگی است که سالانه منجر به مرگ و میر بسیاری از مردم در جاده‌ها می‌شود. روش‌های متعددی برای تشخیص میزان خستگی ایجاد شده است. یکی از روش‌ها، تشخیص خستگی هوشمند و استفاده از پارامترهای مختلف (ابزار‎ها) در زمان طولانی و با استفاده از سابقه رانندگی است. این مزیت منجر به تشخیص خستگی در مراحل اولیه و فعال‎سازی زنگ هشدار قبل از وقوع تصادف رانندگی می‌شود. هدف این تحقیق تشخیص خواب‎آلودگی از روی حالت چهره می‌باشد. با استخراج و ردیابی نقاط کلیدی می‌توان حالت چهره شخص مانند حالت عادی یا خواب‎آلوده را تشخیص داد. در این مقاله روش جدید برای تشخیص خواب‎آلودگی از روی فریم‌های ویدئویی با استفاده از پایگاه داده YawDD می‌باشد. نحوه عملکرد سیستم پیشنهادی براساس ردیابی لندمارک‌های چهره استخراج شده از فریم‌های ویدئویی است. در ابتدا موقعیت اولیه چهره با استفاده از الگوریتم ویولاجونز مشخص می‌شود، سپس نقاط کلیدی توسط الگوریتم سیفت استخراج می‌شوند. این روش قادر است با دقت خوبی عملیات تشخیص خواب‎آلودگی راننده را انجام دهد. روش پیشنهادی قادر است با دقت خوبی عملیات تشخیص خواب‎آلودگی راننده را انجام دهد. از دیگر مزایای روش پیشنهادی می‌توان به اجرای بلادرنگ 47 فریم بر ثانیه و دقت 94 درصد و خطای 6% بر روی پایگاه داده YawDD اشاره کرد.

کلیدواژه‌ها

موضوعات


عنوان مقاله [English]

Detecting Driver Drowsiness by Combining Viola-Jones Image Processing Method and Image Landmarks Detection

نویسندگان [English]

  • Sanaz Eskandari 1
  • Amir Masoud Rahimi 1
  • Ehsan Ramezani Khansari 2
1 Department of Civil Engineering, Faculty of Engineering, University of Zanjan, Zanjan, Iran
2 Department of Transportation, Faculty of Engineering, Imam Khomeini International University, Qazvin, Iran
چکیده [English]

Fatigue is one of the main causes of traffic crashes which leads to the death of many people on the roads every year.  Several methods have been developed to detect the level of fatigue, one of which is intelligent fatigue detection based on driving history. This advantage leads to the detection of fatigue in the early stages and notifying before the crash. In this paper, a new method is applied to detect drowsiness from video frames using the YawDD database. The operation of the proposed system is based on the tracking of facial landmarks. The purpose of this paper is to detect drowsiness from facial expressions. By extracting and tracking key points, facial expressions such as normal or sleepy can be detected.  First, the initial position of the face is determined and then the key points are extracted using Viola-Jones and SIFT algorithms, respectively. The results showed that the proposed method is capable of detecting the driver's drowsiness with good accuracy. The proposed method executed 47 frames per second in real-time with an accuracy of 94% and an error rate of 6%.

کلیدواژه‌ها [English]

  • Car Accidents
  • Fatigue
  • Viola-Jones
  • Drowsiness
  • Facial Tracking
[1]        R. W. Wohleber, G. Matthews, G. J. Funke, and J. Lin, “Considerations in Physiological Metric Selection for Online Detection of Operator State: A Case Study,” in Foundations of Augmented Cognition: Neuroergonomics and Operational Neuroscience, vol. 9743, D. D. Schmorrow and C. M. Fidopiastis, Eds., in Lecture Notes in Computer Science, vol. 9743. , Cham: Springer International Publishing, 2016, pp. 428–439. doi: 10.1007/978-3-319-39955-3_40.
[2]        M. J. Parsa, M. Javadi, and A. H. Mazinan, “Fatigue level detection using multivariate autoregressive exogenous nonlinear modeling based on driver body pressure distribution,” Proceedings of the Institution of Mechanical Engineers, Part D: Journal of Automobile Engineering, vol. 236, no. 1, pp. 168–184, Jan. 2022, doi: 10.1177/09544070211014290.
[3]        R. E. P. Soares, “Driver monitoring systems of fatigue based on eye tracking,” PhD Thesis, 2017. Accessed: Apr. 24, 2024. [Online]. Available: https://repositorium.sdum.uminho.pt/handle/1822/54750
[4]        G. Matthews, R. Wohleber, J. Lin, G. Funke, and C. Neubauer, “Monitoring Task Fatigue in Contemporary and Future Vehicles: A Review,” in Advances in Human Factors in Simulation and Modeling, vol. 780, D. N. Cassenti, Ed., in Advances in Intelligent Systems and Computing, vol. 780. , Cham: Springer International Publishing, 2019, pp. 101–112. doi: 10.1007/978-3-319-94223-0_10.
[5]        H. Singh, J. S. Bhatia, and J. Kaur, “Eye tracking based driver fatigue monitoring and warning system,” in India International Conference on Power Electronics 2010 (IICPE2010), IEEE, 2011, pp. 1–6. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/5728062/
[6]        J. Xu, J. Min, and J. Hu, “Real‐time eye tracking for the assessment of driver fatigue,” Healthc. technol. lett., vol. 5, no. 2, pp. 54–58, Apr. 2018, doi: 10.1049/htl.2017.0020.
[7]        R. Bhardwaj, P. Natrajan, and V. Balasubramanian, “Study to determine the effectiveness of deep learning classifiers for ECG based driver fatigue classification,” in 2018 IEEE 13th international conference on industrial and information systems (ICIIS), IEEE, 2018, pp. 98–102. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8721391/
[8]        L. Wang, H. Wang, and X. Jiang, “A new method to detect driver fatigue based on EMG and ECG collected by portable non-contact sensors,” Promet-Traffic&Transportation, vol. 29, no. 5, pp. 479–488, 2017.
[9]        R. Bhardwaj, S. Parameswaran, and V. Balasubramanian, “Comparison of Driver Fatigue Trend on simulator and on-road driving based on EMG correlation,” in 2018 IEEE 13th International Conference on Industrial and Information Systems (ICIIS), IEEE, 2018, pp. 94–97. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8721431/
[10]      X. Hu and G. Lodewijks, “Exploration of the effects of task-related fatigue on eye-motion features and its value in improving driver fatigue-related technology,” Transportation Research Part F: Traffic Psychology and Behaviour, vol. 80, pp. 150–171, Jul. 2021, doi: 10.1016/j.trf.2021.03.014.
[11]      S. Ananthi, R. Sathya, K. Vaidehi, and G. Vijaya, “Drivers Drowsiness Detection using Image Processing and I-Ear Techniques,” presented at the 2023 7th International Conference on Intelligent Computing and Control Systems (ICICCS), IEEE, 2023, pp. 1326–1331.
[12]      M. Hossain, M. Hasan, K. Abid, and S. Ahmed, “Developing an Image Processing Based Real-Time Driver Drowsiness Detection System,” International Journal of Scientific and Research Publications, vol. 13, Apr. 2023, doi: 10.29322/IJSRP.13.04.2023.p13622.
[13]      J. Jose, J. S. Vimali, P. Ajitha, S. Gowri, A. Sivasangari, and B. Jinila, “Drowsiness Detection System for Drivers Using Image Processing Technique,” in 2021 5th International Conference on Trends in Electronics and Informatics (ICOEI), Jun. 2021, pp. 1527–1530. doi: 10.1109/ICOEI51242.2021.9452864.
[14]      V. Naren Thiruvalar and E. Vimal, “A comparative analysis on driver drowsiness detection using CNN,” International Journal of Nonlinear Analysis and Applications, Dec. 2021, doi: 10.22075/ijnaa.2021.5894.
[15]      K. Balasubramanian, G. D. K, and Ramya, “Drowsiness detection and safety monitoring using image processing”, doi: 10.6703/IJASE.202209_19(3).001.
[16]      Y. Albadawi, A. AlRedhaei, and M. Takruri, “Real-Time Machine Learning-Based Driver Drowsiness Detection Using Visual Features,” Journal of Imaging, vol. 9, no. 5, Art. no. 5, Apr. 2023, doi: 10.3390/jimaging9050091.
[17]      P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in Proceedings of the 2001 IEEE computer society conference on computer vision and pattern recognition. CVPR 2001, Ieee, 2001, p. I–I. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/990517/
[18]      P. Viola and M. J. Jones, “Robust Real-Time Face Detection,” International Journal of Computer Vision, vol. 57, no. 2, pp. 137–154, May 2004, doi: 10.1023/B:VISI.0000013087.49260.fb.
[19]      Z. Kalal, K. Mikolajczyk, and J. Matas, “Forward-backward error: Automatic detection of tracking failures,” in 2010 20th international conference on pattern recognition, IEEE, 2010, pp. 2756–2759. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/5596017/
[20]      D. G. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, Nov. 2004, doi: 10.1023/B:VISI.0000029664.99615.94.
[21]      N. Neshov and A. Manolova, “Drowsiness monitoring in real-time based on supervised descent method,” in 2017 9th IEEE International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), IEEE, 2017, pp. 660–663. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/8095173/
[22]      B. Akrout and W. Mahdi, “Yawning detection by the analysis of variational descriptor for monitoring driver drowsiness,” in 2016 International Image Processing, Applications and Systems (IPAS), IEEE, 2016, pp. 1–5. Accessed: Apr. 24, 2024. [Online]. Available: https://ieeexplore.ieee.org/abstract/document/7880127/
[23]      C.-H. Weng, Y.-H. Lai, and S.-H. Lai, “Driver Drowsiness Detection via a Hierarchical Temporal Deep Belief Network,” in Computer Vision – ACCV 2016 Workshops, vol. 10118, C.-S. Chen, J. Lu, and K.-K. Ma, Eds., in Lecture Notes in Computer Science, vol. 10118. , Cham: Springer International Publishing, 2017, pp. 117–133. doi: 10.1007/978-3-319-54526-4_9.
[24]      R. M. Salman, M. Rashid, R. Roy, M. M. Ahsan, and Z. Siddique, “Driver Drowsiness Detection Using Ensemble Convolutional Neural Networks on YawDD,” Dec. 19, 2021, arXiv: arXiv:2112.10298. Accessed: Apr. 24, 2024. [Online]. Available: http://arxiv.org/abs/2112.10298.
[25] S. Eskandari, A.M.Rahimi, E.R.Khansari, “Study of cognitive variables for driver fatigue to reduce severity and prevent accidents,” Master's Thesis, Department of Engineering, University of Zanjan, pp.34-42, Aug 2023.