Positioning and determining pupils being open and closed and its role in accident reduction

Despite wide spread advances in the car manufacturing industries around the world, the death toll from car accidents is worrying. This concern is heightened when reports of a road accident website reports that the main reason in 25% of road accidents in particular and 60% to road accidents resulting in death or injury. Therefore, there has been a great deal of researches and functions for the automatic and machine detection of drowsiness that of them have reached the production stage. The present study is a method for positioning the pupil of the eye, determining the opening and closing of the pupil in real time and in unlimited environments, using the color feature in the first step, in Ybcbr color space and with the help of Gaussian function and Euclidean distance detection and then the area of the eye is positioned using the Viola jones algorithm. Finally, in order to locate the pupil and detect its openness, we have used two parallel Kalman filters. If the pupils are closed, they follow the Harris Eye Detection algorithm to identify the drowsiness of the driver and use this system to minimize the deaths caused by fatigue (tiredness) while driving. Detection is performed more precisely in the case of face rotation, different lighting conditions, and the eyes being closed or missing one of the eyes, the presence of glasses, beards, makeup, hijab, or obstruction of the eyes. The low computational complexity and maximum stability, real-time and unrestricted environment are other advantages of this method, all because the filters operate in parallel and do not even require high-resolution images.


INTRODUCTION
One of the most important contributors to traffic accidents, especially on intercity roads, is fatigue, drowsiness and lack of focus [1-3]. Fatigue and drowsiness reduce the driver's perception and ability to control the car. Researches show that the driver is usually tired after one hour of driving. But in the early afternoon after lunch, and also at midnight, the driver feels drowsy(tired) for less than an hour. Of course, in addition to natural causes, alcohol consumption, drugs and medications that lead to decreased alertness can also affect on the driver's drowsiness [4]. Most of the accidents that are caused by fatigue or lack of concentration occur on intercity roads and for heavy vehicles.
Most of these accidents occur around 2-6 or 15-16 o'clock. Drowsiness is one of the factors affecting the severity of road accidents that cause a large number of casualties to the community every year. Recognizing the drowsiness is very important and failure to detect it in time will not work [5,6]. Drowsiness detection methods can be classified into two types of surveillance and vehicle-based approaches. In the surveillance mode, the main means are sensors and cameras that record the physical signs of the driver. These signals are then sent to a computer and after processing, the driver's consciousnes gets estimated. The physiological index sensor, based on the natural physiological effects of humans are divided in two groups.
Changes in physiological signals such as brainwaves, palpitations, blink of an eye and measurement of physical changes such as tilting, leaning and tilting, anteing, eyes being open and closed [7-9]. The camera of these systems is usually positioned at the top of the steering wheel to get a good picture of the driver's face.
The figure below shows an overview of a driver's face monitoring system and how the camera is positioned inside the vehicle. Of course, because of its simpler and more accurate mathematical and engineering analysis, this method is more expansive and versatile than vehiclebased methods [10]. But since these sensors are connected directly to the driver's body, they cause driver annoyance. In addition, for long periods of time, the transpiration of the driver's body reduces the accuracy of the sensors. The eye is an important and sensitive part of the human body and exhibits many forms such as fatigue and drowsiness, sorrow, gloom, drunkenness, and so on. So, it can be used in cars to detect driver's drowsiness.

RESEARCH METHODS
eye is the most important member of the face, where symptoms of fatigue and lack of focus appear.
For this reason, many driver face monitoring systems only detect driver's fatigue and lack of focus based on features extracted from the eye. Steps of an Eye Tracking system: 1)Imaging 2) Face decryption 3) Eye decryption 4) Face tracing (detection)

Imaging
Imaging section includes lighting, camera and optical filter, image card and controller if necessary.
The camera captures an image with a camera with a PHILIPS video input in a Omnidirectional way with a resolution of 160 x 120.
And all clips are made at 15fps (frame rate 15 frames per second).
In the clips, situations such as blindfolding, driver's head condition, repeated yawning of the driver's face Software Basics Recognize these situations by applying image processing techniques and video frame processing.
Its hardware includes an electronic board that, as soon as it receives visual data showing the driver's unconscious, failure to recognize the driver's face by lowering the head, repetition of the yawning, it warns.

Face decryption Step
The most common way to decrypt (detect) faces is to use color attributes [11][12][13]. Because the color is constant compared to the face rotation, and on the other hand, the facial color is processed faster than other facial features.
Rgb color space is a standard and commonly used color space for rendering color images, but due to the use of factors such as brightness and saturation to determine the color code of a pixel one should look for a space that is not dependent on these components to be used for facial recognition. As we know, the intensity of light and facial light varies from person to person and from environment to environment.
Ycbcr chromatic color space is obtained from Rgb using Equation 1:

RGB
Cr Cb YCBCR Figure 1. Image of a person in a different color space

Getting a facial skin color model:
Having a collection of different skin types with different colors and textures and converting them into YCbCr color space is necessary.

Figure 2 -Examples of different facial skin colors
We obtain the skin color model using covariance and mean values according to Equation 2: We then use the Gaussian function and the Euclidean distance to reveal the face model to obtain Euclidean equation 3: The input image used here has a complex background and it has no limitations and the cervical region can rotate in any way and at any angle. Or the eyes can be closed and the person can be with glasses, beard, hijab, makeup Also, the ownership of each pixel from the image to the obtained model is specified here.

3) Eye detection method
Eye detection methods [14,15] are: 1) Shape and structure-based methods 2) Feature-based methods 3) Apparatus based methods 4) Combined method Shape-based methods: It is about having a particular shape and form that has an eye This model can be oval or have a complex structure [16].
Feature-based methods: They use a number of eye features such as pupil and light reflection, eyelashes, sclera [17] The method based on the appearance or pattern matching, also known as the general method, is based on the appearance of the eye, directly detecting and tracing the eyes.
These methods are independent objects and can model any other object next to the eye [18].
Combined techniques whose main purpose is to combine the different advantages of eye models in one system to overcome the relative limitations of each other. [19] Due to the differentiation of eye color from other parts of the face, we can use horizontal and vertical facial image based on the color feature.
In the horizontal image, we obtain the summation of the pixel intensities in each row and in the vertical image the summation of the pixel intensities in each column [20].
We use the Viola-Jones method to diagnose: This algorithm is proposed by Paula and Michael Jones, It is generally used to detect objects, but its use in Its use to find the location of the eye is very pervasive.
Training process of this algorithm is very time consuming but its detection process is very fast. We use cascading classification to expedite the detection.

Extracting the eye range
This range starts from the top of the eyebrows to the bottom of the eyelid It includes two eyebrows and their distinctive features such as eyelids, corners of the eyes Using the horizontal face image obtained from Equation 5: On the other hand, the horizontal image also has two vertices or peaks.
The first peak is forehead (No. 1). Second Peak -Intervals of the nasal septum to the under eye (bottom of the eye area NO. 5) Because of the more brightness than the eyes themselves. We use these peaks to extract the eye area Since the second peak is located above nose septum it does not cause problems even if one person has a beard. Vertical and horizontal points can be used to divide the eye area into the left and right areas.

Figure A Analysis
There are two valleys above. The first valley is number 2 that is related to the eybrows and the second valley number 4 is related to the pupil.
This subject is true for both the left and right regions, and This is due to the common property of the darkening of the eyebrows and pupil, which results in the formation of valley.

Finding pupil
One of the most common usages of image processing is pupil positioning.
Pupil positioning has many usages in medicine, behavioral science, biometrics, and human-computer interaction.
By having a pixel of the iris or pupil and having the pupil and using the Harris algorithm and the corners of the eye properties, you can calculate the pupil center.
We use the intersection method to find the center of the pupil In this section, only one row and column are processed instead of the entire eye area.
The pupil center is obtained from Equation 7: Yc= ; =

4)Eye tracking
There are 3 categories available for eye tracking [13] -Knowledge-based methods: Based on the laws obtained from the results of studies and researches on the components of the face that are defined and developed, they must establish rules that define the characteristics of the face and the relationships between them (Y.Tian, 2000).
-Educate training-based techniques in three categories of neural networks, Adaboost clusters, and support vector machines [14] Estimation and positioning of eyes in consecutive frames according to cervical movements, eye movements, independent eyelid movements. If the first stage is correctly estimated, the accuracy and efficiency of the next steps will increase [20] The Kalman filter method is more suitable for this step because of its lower computational complexity and higher speed.
The Kalman filter is a recursive algorithm that estimates the location of the eye in subsequent time frames.
That it means where to look for pupil and how much of the image in the next frame should be searched around estimated locations to find pupil with high confidence. This is accomplished by the state vector (matrix of equation 9) of the system, which contains the various components of motion and the components of velocity (pace) in the x and y direction. + = + (8) Wt Gaussian white noise with mean zero, xt + 1 state vector at time t + 1, xt is the state vector at time t (acceleration), t the distance between frames = (9) Since the method used in this article is a combination of two Kalman filters in parallel and the detection of the corners of the eye that will reveal each eye separately, if both eyes are closed, the Kalman filter is not capable of accurate tracing and it becomes disordered.
To fix this problem, in addition to the Kalman, specifications of the corners of the eyes should also be used For this purpose, the two corners of the left and right should be recognized.
We use the Harris algorithm (Equation 10) to find the corners In this algorithm, At each point in the image, we move the frame that by moving it the average change in the brightness of the image in each window compared to the main window gets calculated and the minimum value of this variable is considered as the response.
Depending on the movement and displacement of the frame, we have three impressions: 1)If the framed area is homogeneous in brightness, all displacements lead to a small change and the corner response has an small amount.
2)If the boxed area is located on an edge, the displacement in the direction perpendicular to the edge has the highest change and the displacement along the edge has the least change, so the function in this case is also small.
3)If the boxed area is on one corner, the displacement in all directions will cause a big change So the corner response function on the edges of the image will have the maximum value. , = ∑ , , | + , + − , | , (10) W is the frame applied to the image that is considered as circle with a factor of 1and I is the image gradient and E are the changes made to the (X, Y).

Proposed Algorithm Results
If the Kalman filter algorithm estimates the position of the eye, the estimated position is compared with the previous position, and their coordinates exceed an empirically obtained threshold level, it indicates that the tracing is wrong and that the position of the corners of eyes needs to be found.
However, the hardware used in this method has increased compared to previous methods Due to an added filter for each eye, but tracing in the face-to-face rotation, different lighting conditions and the eyes being closed or missing one of the eyes, the presence of glasses, beards, makeup, hijab, eye obstruction in this method is more accurate Another advantage of this method is the low computational complexity and maximum stability, realtime environment without limitation. And all are because of the filters that work in parallel and don't even need high-resolution images.