Application Of Driver State Monitoring Systems (Dms)

Application Of Driver State Monitoring Systems (Dms) The term ‘driver state’ has no specific universal definition and is loosely used in the fields of Psychology and Engineering. In general, driver state refers to a set of conditions that affect the driver in a specific instance. Drivers in an optimal state do not suffer from any condition. Vehicle control transition is very important for the success of Highly Automated Driving (HAD). In an ideal scenario, human driver will be attentive to the Driving scene during Take Over Request (TOR). Recent advancements, along with higher levels of automation has made it possible for the driver to be in distracted, inattentive, or even fatigued state. Under similar driving scenario and automation behavior, a driver with some sort of impairment will have poorer performance tackling the situation than when in ideal state. Researches showed that distracted drivers had higher difficulty to adjust speed to a critical event than attentive drivers. Driver State Monitoring Systems (DMS) monitors and collects observable information on the driver, which can be used to determine his/her driving state. It is used, among other things, for active safety, adaptive Human Machine Interface (HMI), and annoyance reduction for false positive notifications in HAD. Electrodermal Activity Electrodermal activity (EDA) is the autonomous changes in electrical properties of the skin, of which, skin conductance is the most widely used property. To measure skin conductance, an electrical potential is applied between two points on the skin, and the resulting current flow between them is measured. This includes both background tonic (Skin Conductance Level or SCL) and rapid phasic components (Skin Conductance Responses or SCRs) that result from sympathetic neuronal activity. It is useful in measuring sympathetic arousal that are tractable to emotional and cognitive states. It is associated with autonomic emotional and cognitive processing. Autonomous nervous system (ANS) generally controls the body’s unconscious actions. Sympathetic nervous system (SNS) prepares body for intense physical activity and is responsible for fight or flight responses whereas parasympathetic nervous system (PSNS) is generally associated with homeostasis and when the body is at rest, while being responsible for rest and digest functions. Some emotional responses may occur without conscious awareness or cognitive intend. EDA can be used to examine such responses (i.e., threat, anticipation, salience, novelty). Recent research has shown that EDA is also a useful indicator of attentional processing per-se, where salient stimuli and resource demanding tasks evoke increased EDA responses. There are two main components to EDA. The slower acting components the overall level, slow climbing, slow declinations over time) are known as general tonic EDA. Skin Conductance Level (SCL) is the most common measure for this and changes in the SCL are thought to reflect general changes in autonomic arousal. The faster changing elements of the signal correspond to the Phasic component, also known as Skin Conductance Response (SCR). SCRs are generally associated with startle reflex or startle response. Recent evidence suggests that both components are important and may rely on different neural mechanisms. Empatica E4 wrist band is the common device used in ITS Leeds for psychophysiological data collection, giving EDA.csv files with Unix timestamp and sampling frequency of 4 Hz. Post processing of the same is done using MATLAB R2016a and Ledalab v3.4.9. For the first study, no pre-filtering was implemented. Although it can be done to smoothen the raw signal, using EDA explorer or a low bypass filter like Butterworth filter. In EDA signal, there is generally a latency of 1-3 seconds (i.e. a delay of around 1-3 seconds from when the event occurs to when you see the change in SC levels). Continuous decomposition analysis (CDA) is generally used instead of Discrete Decomposition analysis. This method extracts the phasic (driver) information underlying EDA and aims at retrieving the signal characteristics of the underlying sudomotor nerve activity (SNA). SC data is deconvolved by the general response shape which results in a large increase of temporal precision. Then data is decomposed into continuous phasic and tonic components. It is the method generally recommended for the analysis of skin conductance data. It features the computation of several standard measures of phasic EDA. Moreover, straightforward measures such as the average (or integrated) phasic driver activity are provided. To find Event Related SCR (ER-SCR) and separate it from Non-Specific SCR (NS-SCR), the event files are loaded in and the Ev ent related SCR activity is exported. The most common minimum threshold amplitude is 0.01  µS. Peaks with amplitude below this value is not considered significant. Another key factor is to standardize the values so that it can be compared across participants. For SCL the standardization is done using the formula: ((SCL ã€â€" SCLã€â€"_min) )/((ã€â€"SCLã€â€"_max –ã€â€" SCLã€â€"_min)) equation 2.2 Where ã€â€" SCLã€â€"_min is the baseline SCL which is to be measured while keeping the participant at rest doing nothing for at least 10-15 mins and ã€â€"SCLã€â€"_max is the maximum value computed when the participant is aroused using loud noise/music for a short period of time. This is done to get the SCL range of the given participant. Since this was not done for the SM study, it can’t be implemented. This can be circumnavigated by finding the minimum non-zero SCL score and the max SCL score during the study for each participant and use this as a baseline. Some studies recommend transforming SCRs into Z-scores. This requires the mean and standard-deviation to be used instead of a hypothetical maximum (from the other methods above). This navigates around the problems associated with determining the maximum SCR response from range-corrected methods / maximal correction methods. Here each raw SCR, a mean SCR value and standard deviation of SCRs, are used to compute the Z-score which is normally distributed, has an average of 0 and a standard deviation of 1. From here one can transform these Z-scores into T-scores, which have a mean of 50 and standard deviation of 10 (thus removing minus scores). The advantage to this approach here is that the resultant z-scores are based on unambiguous mathematical factors that represent the participants typical response level and not on unwarranted assumptions about maximum SCRs. Researchers further suggested that another useful transformation might be to divide each raw SCR, by the participants mean SCR thus providing a kind of standardized ratio. Final output is to obtain the ER-SCR activity window’s average SCR in T-score format, so that it can be compared across all the participants. From this, it might be possible to deduce suggestions/reasons for certain behavior during failure. Shimomura, et al. (2008) showed in his study that frequency domain analysis enabled detection of small differences in mental workload that could not be detected by traditional amplitude domain analysis. Here the signal is transformed from amplitude domain to frequency domain using techniques such as Fast Fourier Transforms (FFT). This technique enables real-time automatic analysis. Motion artifacts can cause the EDA signal to be quite noisy. If 90% of EDA value is zero or close to zero within a 5s window (a lower bound threshold of 0.001 µS can be used, it is probably caused by the sensors losing contact with the skin during that period and they can be removed. Generally, EDA levels are found to be not changing by more than 20% while increasing and 10% while decreasing, within a 1 second time window. A moving one second median filter can be used for initial interpolation to even out the signal. The resultant acceleration from accelerometer sensor can be used to find points/periods of high movements, and the EDA signal during that same period can be checked for motion artifacts, which can be removed if necessary. A bi-cubic interpolation of the signal can be done after the filtering to account for missing data points from the removal of motion artifacts. To summarize, skin conductance generally increases with arousal, stress, salience, mental workload, anticipation and overall increase in body temperature or physical activity. But measurements from palm of the hand or feet are generally highly sensitive emotional responses and can be distinguished from skin conductance spikes due to body temperature or physical activity. Heart rate variability Heart rate (HR) is the number of times the heart beats in a minute or â€Å"bpm†, and resting heart rate is the heart rate measured while being relaxed, but awake. Amongst the general population a resting heart rate of 60-80bpm is considered pretty good. But heartbeat intervals are irregular and there is variation in time between each heartbeat. Heart rate variability (HRV) is simply the measurement of variation between heartbeats. In general, a healthy functioning body will display a greater variability between beats than a poorly functioning one. Heart period is the time interval between two successive heart beats. The Sinoatrial node (SAN) can be seen on the top left side of the figure and the Atrioventricular node (AVN) to the right of SAN. The depolarization of SAN and AVN provides the electrical driving force that triggers the contraction of the heart. SAN’s spontaneous depolarization speed is typically faster than that of AVN, which is why it’s called natural pacemaker. Electrical impulses generated by SAN stimulates each beat of the heart, thereby dictating its rhythm. Sympathetic (SNS) and Parasympathetic (PSNS) branches of the autonomic nervous system are what mainly influence SAN. Hormone and immunity can have a role as well. According to Berntson, et al. (1997) â€Å"sympathetic activity tends to increase HR and decrease HRV, whereas parasympathetic activities tends to decrease HR and increase HRV†. HRV is closely related to emotional arousal. In HRV spectrum, there are both high-frequency (HF) and low-frequency (LF) signals. HF activity generally decreases under pressure, stress, strain, focused attention etc. High stress can cause reduction in LF values as well. Individuals who worry more have shown reduction in HRV. In PTSD patients, HRV and its HF component is reduced whilst the low-frequency (LF) component is elevated. Decrease in PSNS activity or increase in SNS activity results in reduced HRV. HF activity (0.15 to 0.40 Hz), especially, has been linked to PSNS activity. LF activity (0.04 to 0.15 Hz), which is generally associated with a mixture of both SNS and PSNS. So, it’s safe to summarize that during rest periods, HF HRV tends to be higher than when the driver is engaged, stressed, focused or strained/tired. To analyze HRV, either time domain or frequency domain analysis can be implemented. Time domain methods include â€Å"measures of the variance among heart period, the variance of the differences among heart periods, and geometric methods based on the shape characteristics of heart period distributions†. The most common method used to compute heart rate variability amongst time domain methods is the square root of the mean squared successive heart period differences or the RMSSD (Root Mean Square Successive Difference) statistic. It is based on the differences between adjacent heart periods and is nominally independent of basal heart period, although heart period level and heart period variability are themselves physiologically correlated. Because of the differences between adjacent heart periods sample HRV over relatively short periods of time (the duration of a heart period), the RMSSD resolves the total variance by filtering out LF signals. Consequently, the RMSSD has been a pplied as a measure of HF based HRV. The properties of RMSSD, including its cut-off frequency and its frequency-dependent transfer function vary as a function of basal heart period. A more systematic parsing of heart period variance into specific frequency components can be achieved by frequency domain methods. There are mainly two ways to measure HRV and HR. They are Electrocardiogram (ECG) recordings and Photoplethysmogram (PPG) recordings. ECG recordings are collected by placing electrodes on the chest (near the heart), which measure electrical impulses for each cardiac cycle. QRS complex is the defining feature of ECG signal. QRS complex is the three graphical deflections seen on a typical ECG, which is Q wave (downward deflection right after P wave, which is IV.), R wave (upward deflection after Q wave) and S wave (downward deflection after R wave) which is represented by I., II. The heart’s electrical activation is measured directly by ECG recordings. It also generally shows a strong QRS complex presence in the resulting signal. Motion artifacts caused by sensor displacement due to participant movement is a common source of noise in ECG signals. These tend to fall in the same frequency range as the QRS-complexes, which can make it difficult to filter them without deforming the QRS complex. Photoplethysmogram (PPG) recordings are a less invasive method to study cardiac cycle. They generally measure the discoloration of the skin as blood perfuses through the arteries and capillaries with each heartbeat, using optical sensors. PPG is typically measured at the fingertip or at the wrist. The PPG Heart Rate Analysis generally consists of a systolic peak, a dicrotic notch, and a secondary peak called a diastolic peak (2.5b-III). In recordings with very low amplitude the diastolic peak may be absent. The main advantages of PPG over ECG are low cost, ease of setting up and non-invasive methods. Ways of obtaining the PPG signal contactless through cameras have been proposed, further reducing intrusiveness. There is generally more amplitude variation over short time-intervals, more variation in waveform morphology, as well as more noise from various sources as opposed to ECG measurements. This makes analysis of PPG more difficult. The heart signal is often split into heart rate (HR) and heart rate variability (HRV) measures. The distance between the detected heart beats (the RR-intervals, named because in the ECG, the largest amplitude peak is called the R-wave) are used to calculate them. The heart beats are represented by the peaks in both signals. Even though the measurement technique vary quite considerably between ECG and PPG, a high correlation (median 0.97) between RR-intervals extracted from ECG and PPG signals has been reported. This makes the PPG a valid alternative for human factors studies that require non-intrusive heart rate measurements, and hence will be the focus of this research. Researchers talked on how to remove motion artifacts and filter it out of EDA signal. This is explained in the last paragraph of the previous section. The same methodology can be applied to remove motion artifacts from the HR/HRV signal as well.