The developed pipeline should be able to recognize driver's drowsiness using real-time video of the driver's face and trigger an alert if necessary. Additional functionality may allow to collect driver's state data from the whole workday, marking the the most drowsy or distracted states on timeline. Application is run using Streamlit library.
The first thing to look for in determining a driver's attention span is drowsiness. A basic sign of drowsiness is more infrequent and longer blinks. To detect blinks, you we use the EAR, which allows you to count how much your eyes are open at a given moment.
The EAR formula returns a single scalar quantity that reflects the level of eye opening.
Using a Mediapipe library Face mesh pipeline, we can extract facial landmarks referred to eyes.
Since we are focusing on detecting driver drowsiness, we need only the landmark points belonging to the eye regions out of the 468 points. The eye regions have 32 landmark points (16 points each). To calculate the EAR we need only 12 points (6 for each eye).
Using the above image as a reference, the following 12 landmarks are chosen:
Left eye: [362, 385, 387, 263, 373, 380]
Right eye: [33, 160, 158, 133, 153, 144]
Landmarks above represented from P1 to P6.
Landmarks are available in default system configuration JSON file and can be adjusted if necessary.
Using a pre-calculated threshold (also available in configuration file), we may simply compare it with EAR and determine whether eyes are closed or not. Attaching a timer to this functionality allows us to detect drowsiness.
Head position ("Eyes on road")
Application uses pre-trained Intel head-pose-estimation-adas-0001 model (OpenVINO inference). Using the face’s ROI, we feed a head-pose detector to track driver's view direction. Analizing the output of that model we can easily detect when the face is not centered or not looking to the front. Analyzing the output of that model we can easily detect when the face is not centered or not looking to the front.
