Noncontact evaluation of autonomic nervous system activity during exercise by using video analysis

TL;DR

This paper introduces a noncontact method using video analysis to monitor autonomic nervous system activity during exercise.

Contribution

A novel noncontact technique using facial video and G/B ratio to evaluate autonomic nervous system activity during physical exertion.

Findings

The G/B ratio decreased during exercise with increased heart rate, indicating sympathetic nervous system activation.

Post-exercise, the G/B ratio increased as heart rate decreased, reflecting parasympathetic recovery.

The G/B ratio effectively tracked capillary dilation and contraction linked to autonomic nervous system activity.

Abstract

Autonomic nervous system activity (ANSA) plays a crucial role in the physical condition experienced during exercise and prolonged physical activity. In other words, ANSA is related to exercise performance and physical condition. Therefore, it is important to continuously monitor ANSA during high-intensity and sustained exercise. To this end, an uncomplicated and noncontact measurement system is preferable. Hence, in this study, we propose a method for the noncontact measurement of capillary contraction and dilation state, representative of ANSA, using a common commercial camera.

Specifically, we focused on alterations in the green value of facial video images, from which we derived the green-to-blue (G/B) ratio as an indicator of blood vessel dilation and contraction, and to facilitate assessment of their activity. We performed a validation experiment involving exercise tasks using an ergometer in 10 healthy adults (23 ± 1.6 years old). The G/B ratio shows the state of contraction and expansion of facial capillaries, and it was evaluated using heart rate as ground truth data of the fluctuation of autonomic nerve activity.

We observed an increase in heart rate with decreased G/B ratio during exercise in all subjects. Postexercise, the heart rate decreased but the G/B ratio increased.

During exercise, characterized by dominant sympathetic NSA, the G/B ratio decreased, and recovered after exercise when parasympathetic NSA was dominant. In this way, noncontact evaluation of ASNA was achieved by using the G/B ratio. In the future, this measurement system will be applied to functional tests for heat acclimation.

Figures

Cross-sectional structure of the skin and light wavelength penetration.

Absorption properties of hemoglobin.

Relationship between capillary dilatation and contraction, as well as the cross-sectional area and absorbance, of the blood vessel.

Relationship between autonomic nervous system activity, facial capillaries, and green light.

Schematic diagram of the experiment.

Overview of image processing methods.

Coordinates of each feature point in the face.

Changes in heart rate and G/B ratio during exercise.

(a) Variation in heart rate. p < 0.001 between resting-exercise and exercise-after exercise. p = 0.001 between resting-after exercise. There were significant differences in all combinations of rest-exercise and exercise-postexercise and rest-postexercise. (b) Variation in the G/B ratio. p = 0.004 between resting-exercise, p = 0.226 between exercise-after exercise, and p = 0.175 between resting-after exercise. There was a significant difference only between resting-exercise.

(a) Variation in RMSSD. p < 0.001 between resting-exercise, p = 0.794 between exercise-after exercise, and p < 0.001 between resting-after exercise. There was a significant difference only between resting-exercise and resting-after exercise. (b) Variation in the G/B ratio. p < 0.001 between resting-exercise, p = 0.457 between exercise-after exercise, and p < 0.001 between resting-after exercise. There was a significant difference only between resting-exercise and resting-after exercise.

Changes in body motion, G/B ratio, and heart rate.

Tables

Table 1. Relationship between autonomic nervous system activity, facial capillaries, and green values.

Item	Sympathetic nervous system activity predominates	Parasympathetic nervous system activity predominates
Facial capillaries	Expansion	Contraction
Blood volume	Increase	Decrease
Amount of hemoglobin	Increase	Decrease
Green value	Decrease	Increase
G/B ratio	Decrease	Increase

Table 2. Subject information and target heart rate of each experimental subject.

Number of experimental subjects	Age	Height [m]	Weight [kg]	Resting HR	Target HR
sub1	21	1.66	62	74	124
sub2	23	1.68	55	69	120
sub3	22	1.77	58	70	121
sub4	23	1.86	77	75	124
sub5	22	1.69	60	72	122
sub6	25	1.66	54	74	122
sub7	22	1.7	61	75	124
sub8	24	1.75	58	98	137
sub9	24	1.78	64	75	124
sub10	19	1.74	65	78	127

Full text

Introduction

1

Exercise is an important contributor of health. In recent years, the proliferation of home exercise equipment has made at-home workouts increasingly popular (1). However, engaging in self-directed exercise is associated with the risk of injuries and adverse physical effects (2). Autonomic nervous system activity (ANSA) and cardiovascular status are closely related to physical well-being, emphasizing the importance of monitoring these factors during exercise for safety (3, 4). Adjusting exercise intensity and duration based on an individual's daily cardiovascular status, which is influenced by ANSA, has been demonstrated to help managing physical well-being. When exercise intensity is appropriately calibrated, it can strengthen immune function and increase resistance to infections (5). However, excessive or overly strenuous exercise is counterproductive, highlighting the need to monitor one's physical state and adjust exercise intensity accordingly.

Furthermore, exercise training considering ANSA and cardiovascular status contributes to improved performance (6–8). Sustained optimal performance can extend an athlete's longevity, resulting in a healthier and more enduring exercise regimen. Further, maintaining normal cardiovascular activity and ANSA can reduce the risk of heat stroke (9).

In recent years, the heart rate measured by wearable devices has been mainly used to evaluate ANSA during exercise. However, wearing them during exercise often causes discomfort. In addition, if the core temperature exceeds 40°C, the risk of heat stroke is very high. Body temperature is an important index from the viewpoint of evaluating exercise performance, because elevated body temperature leads to muscle weakness and decreased reaction time. Contact thermometers are mainly used to measure body temperature, but they are complicated during exercise. Non-contact thermography has a long time constant for measurement and cannot be evaluated simultaneously with the heartbeat. In addition, since ANSA has both cardiovascular and thermoregulatory functions, it is necessary to evaluate the two functions.

We noted that the dilatation and contraction information of the peripheral blood vessels of the face contains this information (10). Peripheral blood vessels play a key role in regulating body temperature, enabling heat retention or dissipation through their dilation and contraction (11). Capillaries also fulfill a circulatory role by transporting blood and oxygen. In a healthy state, the deep body temperature increases during exercise, and skin vessels dilate (12). Therefore, a novel indicator of ANSA, particularly focusing on facial peripheral blood vessel dilation and contraction, is essential. In this study, we focused on changes in facial color that accompany the contraction and dilation of facial capillaries. When facial capillaries dilate, the increasing blood volume in the face results in a red face. In contrast, facial capillary constriction decreases blood volume, resulting in a pale face (13). This pallid appearance correlates with lowered blood pressure. These facial color characteristics are also used for medical diagnosis, underscoring the high significance of facial color information (14, 15).

Face images captured by common cameras contain facial color information. This information has been widely used in applications such as the acquisition of photoplethysmography signals and blood pressure estimation (16, 17). These applications are based on blood flow extraction and hemoglobin quantification to obtain data on blood vessels. Using green information from a standard red, blue, and green (RGB) camera, heart and respiratory rates can be inferred from the cheeks and lips (18). Further, an RGB camera can also be used together with radar to estimate heart and respiratory rates (19). Moreover, this information can be used to estimate emotions (20). Thus, green light information was used to estimate indices such as heart rate and blood pressure. However, because they are very sensitive to body movement, measuring them during exercise is difficult. These measurements are difficult to measure unless face movement is eliminated by placing the jaw on a table and fixing the jaw, for example. Therefore, a new evaluation method is needed to evaluate ANSA during exercise.

In this study, we aimed to develop a method to extract the facial color information associated with blood vessel dilation and contraction during exercise using images captured with a common web camera. Then, the verification experiment was carried out to evaluate the method.

Methods

2

Characteristics of color information in face images

2.1

Human skin is composed of three primary layers, starting from the surface, there are the epidermis, dermis, and subcutaneous tissue. The epidermis contains melanin pigment, and the dermis is lined with capillaries. Especially, the blood in the capillary contains hemoglobin. Skin color is mainly determined by two light-absorbing substances, melanin and hemoglobin (21). Arteries and veins in the subcutaneous tissue deliver nutrients to the skin and remove waste products, with a significant portion of this tissue comprising subcutaneous fat (22). There are no significant light-absorbing materials in this layer, and since minimal light reaches the subcutaneous tissue, we can assume that only total or wavelength-independent reflection occurs.

The penetration of light irradiated on the skin varies depending on its wavelength. Longer wavelength penetrates deeper into the skin. As such, short-wavelength blue light penetrates only the epidermis, while green light reaches the dermis, and longer-wavelength red light even reaches the subcutaneous tissue. Essentially, green and red light provide information about blood vessels. Figure 1 shows the correlation between the skin's cross-sectional structure and the penetration of light at different wavelengths.

Cross-sectional structure of the skin and light wavelength penetration.

Hemoglobin, a crucial component of blood, binds to oxygen in the lungs, where oxygen pressure is high, and releases it in tissues with lower oxygen pressure. Oxygen-bound hemoglobin, oxyhemoglobin (oxyHb), is bright red in arterial blood. Conversely, hemoglobin not bound to oxygen, deoxyhemoglobin (Hb), produces the dark red of venous blood. Figure 2 shows the absorbance characteristics of oxyhemoglobin and deoxyhemoglobin under light exposure (23). The absorbance properties of the two differ. However, oxyhemoglobin constitutes approximately 85% of the hemoglobin in blood vessels. Therefore, the absorbance properties of the fluid in entire vessels are similar to those of oxyhemoglobin. Figure 2 depicts the relationship between the light wavelength and the absorbance coefficient of hemoglobin. The larger the vertical axis of the molecular absorption coefficient, easier light is absorbed; this implies that hemoglobin absorbs green light more readily than red light.

Absorption properties of hemoglobin.

Green light is readily absorbed by the hemoglobin in the pigmented dermis. So that green light could carry information about the vessels' cross-sectional area. Figure 3 denotes the relationship between capillary dilatation and contraction, as well as the cross-sectional area and absorbance, of the blood vessel. The cross-sectional area of facial capillaries and the amount of green light absorbed are defined as V and I, respectively. When sympathetic nervous system activity dominates and facial capillaries expand, the cross-sectional area of facial capillaries and the amount of green light absorbed are defined as $[eqn]$ and $[eqn]$ . Conversely, when parasympathetic nervous system activity becomes dominant and facial capillaries contract, the cross-sectional area of capillaries and the amount of green light absorbed are defined as $[eqn]$ and $[eqn]$ . In this case, $[eqn]$ holds. With increasing cross-sectional area of capillaries, the amount of hemoglobin in the blood and the green light absorbed increases. Therefore, $[eqn]$ holds. Thus, a relationship between the cross-sectional area of capillaries and the amount of green light absorbed exists. Figure 4 shows the relationship between ANSA, facial capillaries, and green light. Light penetrates the human skin as depicted in Figure 1. When the sympathetic nervous system dominates, facial blood vessels dilate (24) because of an increasing fluid volume and the amount of hemoglobin in the capillaries. As shown in Figure 2, the amount of green light reflected from the face decreases because green light is easily absorbed by hemoglobin. Conversely, when the parasympathetic nervous system is activated, facial blood vessel contract. This results in a decrease in fluid volume and the amount of hemoglobin in the capillaries, thereby increasing the amount of green light reflected from the face. Because green light passes through the skin surface, it has information on the skin surface in addition to information on hemoglobin. The information on the skin surface must be reduced because it can become noisy during exercise. Green light penetrates the epidermis while blue light only carries information about the skin's surface. Therefore, by calculating the green-to-blue (G/B) ratio, it is possible to extract vascular information. Table 1 shows the relationship between facial capillaries and green light. The G/B ratio differs from the image pulse wave. Imaging pulse waves measure pulse rate, whereas the G/B ratio extracts information about blood vessel contraction and expansion. Imaging pulse wave technology calculates the pulse from face images (25). However, no technique allows capturing the contraction and expansion of face capillaries. Moreover, the noncontact measurement of the ANSA index is being developed. However, many can be measured only in a stationary state because body movements become noisy (26, 27). In addition, image pulse wave technology is weak against body movement; thus, it cannot be measured without fixing the measurement position. Moreover, processing steps such as image analysis and frequency analysis are necessary. The proposed evaluation method is noncontact and unaffected by the body movement being a very useful for evaluating ANSA. As such, this measurement method can directly evaluate contraction and expansion of facial capillaries and instantaneous ANSA.

Relationship between capillary dilatation and contraction, as well as the cross-sectional area and absorbance, of the blood vessel.

Relationship between autonomic nervous system activity, facial capillaries, and green light.

Verification experiment

2.2

We developed a noncontact evaluation system for ANSA during exercise. To confirm that the system can evaluate the ANSA during exercise, we conducted a verification experiment. To validate the validity of the G/B ratio for the development of a system to detect facial blood vessel information during exercise using a common camera, we conducted an exercise task using an ergometer (indoor cycling IC4, LIFEFITNESS). We chose a cycling exercise using an ergometer because it involves physical movement and exercise intensity is easy to maintain. The experimental subjects were 10 healthy adults (23 ± 1.6 years old). We excluded individuals with a history of upper or lower limb disease, sensitive skin, alcohol allergy, hypertension, diabetes, any preexisting medical condition such as hypertension or diabetes mellitus, fear or anxiety that rendered them unable to perform the exercise task in this study, and any physical condition (e.g., inability to sleep) that rendered them unable to perform the task. We also excluded individuals who were not in good physical condition (e.g., unable to get enough sleep). As we anticipated that makeup could interfere with accurately obtaining changes in face color, we asked those who wearing makeup to remove it before participating in the experiment. This study was conducted in accordance with the ethical principles of the Declaration of Helsinki. All participants provided informed consent, oral and written. This study was approved by the Ethics Committee of Ritsumeikan University (approval number: BKC-LSMH-2022-065).

In this experiment, room brightness posed a challenge as it introduced noise into the experimental data. Room brightness has both long-term effects, influenced by weather and time of day, and short-term effects due to the flow of clouds. Both factors significantly impacted this experiment. Furthermore, the intensity of the circulatory response of the autonomic nervous system varies between the morning and afternoon, necessitating a fixed experiment time. Therefore, to conduct this experiment, it was crucial to create an environment unaffected by changes in room illumination due to weather or day time. To solve these problems, the experiment was conducted in a room completely shielded from outside light. The laboratory was covered with a light-shielding curtain, and the brightness was kept constant in the laboratory using LED lights. The LED light used (HRL-9168, Harvest Japan) emitted white daylight with a luminance of 6,500 K. When the color temperature of the LED is set to a yellowish color such as a light bulb color, it is difficult to evenly cover all the frequency components of visible light. In such an environment, the necessary wavelength of light may be weakened. The brightness of the LED was set to 6,500 K to measure under a white light environment close to natural sunlight. The room dimensions were 4,000 mm × 1,000 mm × 2,000 mm, providing ample room to perform the cycling exercise. The laboratory was bright enough to exercise.

A TAWARON-HDC1 camera (TAWARON) was used to capture facial images. The camera was positioned approximately 1.3 m from the participant's head to integrate the pixel values in the images. The white balance and brightness were set to a constant value to keep the shooting conditions constant. The resolution was taken at 1,920 × 1,080. A versatile bio amplifier, PolymateV (Miyuki Giken, Japan), was used to measure electrograms (ECG) at a sampling frequency of 1,000 Hz. The intensity of the participants' exercise during the experiment was self-regulated while observing their real-time heart rate. Therefore, since it was necessary for the experimental subject to check his/her own heart rate during exercise, a wristwatch-type wearable device, $[eqn]$ (Google, USA), was attached to the participant's arm to allow them to inspect their heart rate during exercise. Skin blood flow is easily influenced by ambient temperature. For human comparison, the optimal conditions are a room temperature of 22°C–26°C and a humidity of 40%–60% (28). In this study, the experiment was conducted under conditions of 22.9°C ± 0.8°C room temperature and 46.0% ± 8.7% humidity. Figure 5 shows a schematic of the experiment.

Schematic diagram of the experiment.

The experimental procedure comprised an initial 15-min resting period, followed by 15 min of exercise at 40% intensity and 10 min of postexercise rest. No special instructions were given during the exercise task, and the subjects performed cycling exercise freely. The subject's heart rate during the exercise task on the aero bike was considered as the target heart rate and determined using the Karvonen method, see Equation 1 (29).

$$[eqn]$$

The experimental subjects performed the exercise task with a wristwatch-type wearable device affixed to their arm to monitor their pulse rate. The device displayed the heart rate in real time, and the subjects were able to exercise at their target heart rate while watching the displayed heart rate.

Analysis methods

2.3

A region of interest (ROI) was detected from a face image, to obtain the G/B ratio. Figure 6 shows an overview of the image processing method described below. In Figure 6, processes (1) and (5) surrounded by red lines are processes for eliminating the influence of body motion.

• (1)A mask image was created by adjusting the hue saturation value (HSV) range to isolate skin-colored portions. Focusing on these regions, prevented changes in the background due to body movements, ensuring consistent color information acquisition. Upper and lower HSV value limits for creating the mask were set using the software's track bar. It was possible to remove the change of the background color by the body motion by eliminating the color except for the face, and it always succeeded in extracting only the color of the face.
• (2)For further image processing, the video was divided into frames. Since the video used was 30 fps and the experiment duration was 40 min, we obtained a total of 72,000 images.
• (3)Processing was applied to each image acquired in step (2).
• (4)Face detection was performed using the Center Face, a nimble model renowned for swiftly and accurately predicting face box locations and landmarks (30). The Center Face enabled the retrieval of coordinates for each facial feature point, as shown in Figure 7.
• (5)ROI images were defined by cropping the images, focusing on the nose and cheeks. The cheek region exhibits a prominent capillary response. However, a ROI limited to the cheeks might be too small to detect capillaries during body movements. To mitigate this, the nose area was included to facilitate measurements unaffected by body movements. By making this range, the capillary information would be able to be measured under the motion. The ROI image was then trimmed into a rectangular shape with specific upper left and lower right coordinates. Here, if the coordinates of the upper left corner of the rectangle are $[eqn]$ and the coordinates of the upper right corner are $[eqn]$ , the coordinates would be calculates as indicated in Equations 2–5. Let us assume that the number of pixels in the ROI image is a * b pixel.

$$[eqn]$$ $$[eqn]$$ $$[eqn]$$ $$[eqn]$$

• (6)We acquired color information in the ROI area. The average value of red, green, and blue values for each pixel in the ROI region was calculated for each frame. If the matrix of green values in the image is $[eqn]$ and the matrix of blue values is $[eqn]$ , and the green and blue values at the coordinates $[eqn]$ will be $[eqn]$ and $[eqn]$ . If $[eqn]$ were the matrix of the mask image created after adjusting the HSV in (1), and $[eqn]$ were the value at the coordinates $[eqn]$ . In the matrix $[eqn]$ , the skin color is set to 1 and the others to 0. Then, if the matrices of green and blue when only the skin color is extracted are $[eqn]$ and $[eqn]$ , and the values at the coordinates $[eqn]$ are $[eqn]$ and $[eqn]$ , the green and blue values at each coordinate are extracted as in Equations 6, 7.

$$[eqn]$$ $$[eqn]$$

Overview of image processing methods.

Coordinates of each feature point in the face.

Therefore, if the average of green and blue values at this time is $[eqn]$ and $[eqn]$ , the respective values are provided by Equations 8, 9.

$$[eqn]$$ $$[eqn]$$

• (7)To set the ROI area and acquire color information for all frames, steps (3) to (6) were repeated for each frame.
• (8)Filtering was applied after acquiring data from the entire video. The obtained RGB values often contained noise from body motions, pulse waves, and respiratory components. To address this, a low-pass filter with a cutoff frequency of 0.1 Hz was used to eliminate noise and extract information regarding vasoconstriction and dilation.
• (9)We calculated the G/B ratio by determining the ratio between the green and blue values derived from the RGB values obtained in (8). ECG data was analyzed using MATLAB software (MathWorks, USA). The first step involved detecting the R wave based on the ECG data measured by the bio amplifier. Next, the RR interval was calculated based on the detected R waves; the RR interval is the interval between the detected R and R waves. Then, the heart rate was derived as reference data for evaluating the G/B ratio. Heart rate variability (HRV) is often used to evaluate ANSA (31, 32). In this study, SDNN and RMSSD were calculated as HRV indices. Clinically, SDNN <50 ms is generally considered to reflect a significant increase in sympathetic activity, and RMSSD <25 ms represents a significant parasympathetic depression (33).

After calculating the G/B ratio and heart rate, we derived time-series data and the mean value of each value. Average values were calculated for 1 min starting 14 min after the beginning of the resting period, 1 min starting 10 min after the beginning of the exercise period, and 1 min starting 1 min after the end of the exercise period. We then derived the changes in each condition using the resting state as a reference. The reason for using the average 10 min after the beginning of exercise as the average during exercise was that all experimental subjects had stable heart rates and other parameters during exercise, and the noise of the G/B ratio was relatively low. Then, heart rate and G/B ratio were expressed as fluctuation values with resting state as 1. One-way analysis of variance (ANOVA) was used for statistical analysis, and Tukey's method was used as a subsequent test. The significance level was set at 0.05. Hypothetically, when sympathetic nervous system activity becomes dominant during exercise, heart rate increases, facial capillaries dilate, and the G/B ratio decreases. However, when parasympathetic nervous system activity dominates postexercise, heart rate decreases, facial capillaries contract, and the G/B ratio increases.

In addition, to investigate the effect of body motion on G/B ratio, the difference between frames was calculated in the video taken during exercise. The total number of pixels between frames calculated as the difference is treated as body motion and evaluated.

Results

3

As a result, heart rate data and G/B ratio data were obtained from 10 subjects. The results are shown in Figure 8 and Table 2. However, sub9 did not reach the target heart rate during exercise. Therefore, Sub9 was excluded from the analysis. First, Table 2 shows the target heart rates, computed using the subjects' physiological data, including age and resting heart rate. For most subjects, the resting heart rate was 60–80 bpm. However, sub8 showed a high resting heart rate (98 bpm). The subjects then engaged in an exercise task based on their respective target heart rates. Time series of G/B ratio and heart rate data for all subjects are shown in Figure 8. During exercise, the heart rate increased over time, whereas the G/B ratio progressively decreased. Postexercise, the opposite was true. These patterns were observed in most subjects. Additionally, Figure 9 presents box-and-whisker diagrams for G/B ratio and HRV, with the average resting value normalized to 1. Figure 10 shows RMSSD and SDNN for 9 subjects.

Changes in heart rate and G/B ratio during exercise.

(a) Variation in heart rate. p < 0.001 between resting-exercise and exercise-after exercise. p = 0.001 between resting-after exercise. There were significant differences in all combinations of rest-exercise and exercise-postexercise and rest-postexercise. (b) Variation in the G/B ratio. p = 0.004 between resting-exercise, p = 0.226 between exercise-after exercise, and p = 0.175 between resting-after exercise. There was a significant difference only between resting-exercise.

(a) Variation in RMSSD. p < 0.001 between resting-exercise, p = 0.794 between exercise-after exercise, and p < 0.001 between resting-after exercise. There was a significant difference only between resting-exercise and resting-after exercise. (b) Variation in the G/B ratio. p < 0.001 between resting-exercise, p = 0.457 between exercise-after exercise, and p < 0.001 between resting-after exercise. There was a significant difference only between resting-exercise and resting-after exercise.

Our results indicated that the heart rate rapidly increases immediately after the start of exercise decreasing rapidly immediately after the end of exercise. Correspondingly, the G/B ratio decreased with increasing heart rate immediately after the start of exercise and increased when the heart rate decreased immediately after the end of exercise. There was a significant difference in the heart rate at all intervals. There was a significant difference in G/B ratio between resting and exercise phases. RMSSD decreases during exercise, and the mean value for all subjects during exercise is less than 25 ms, indicating that parasympathetic activity is significantly suppressed. The mean value for SDNN for all subjects during exercise is less than 50 ms, indicating that sympathetic activity is significantly increased.

Figure 11 shows the time-series data of the number of pixels (body movement), G/B ratio and heart rate of sub1. After the start of exercise, body movement increases because the body moves. During exercise, the movement of the body is constant, so the amount of movement is constant. At that time, the G/B ratio decreases, but there was no change in the body movement related to the G/B ratio during exercise.

Changes in body motion, G/B ratio, and heart rate.

Discussion

4

The results of RMSSD and SDNN show that sympathetic activity increases and parasympathetic activity decreases during exercise. In all subjects, heart rate increased and G/B ratio decreased during exercise. At the beginning of dynamic exercise, the heart rate increases rapidly, primarily because of decreased cardiac vagal regulation (34). With increasing exercise intensity, vagal modulation is further reduced, and heart rate increases due to accompanying sympathetic activation (35). Facial blood vessels dilate with predominating sympathetic activity. The increase in heart rate and decrease in the G/B ratio observed with exercise are considered the result of these ANSA. Therefore, the features of contraction and expansion of the capillary were extracted from the moving image of the face with body movement. To our knowledge, this is the first study to extract features of capillary constriction and dilatation only from a common RGB camera in a noncontact manner during exercise with body movement. The G/B ratio shows features of direct facial capillary constriction and dilatation.

The G/B ratio decreased with increasing heart rate during exercise in all subjects, indicating that the G/B ratio captures vasodilation caused by sympathetic activation. In many subjects, the G/B ratio was lower during the postexercise recovery period than during the preservice period. Conversely, the heart rate was higher during the postexercise recovery period than during the preservice period. These results suggest that the heart rates did not sufficiently recover 10 min after exercise. A previous study on heart rate recovery at different exercise intensities (36) indicated that when the heart rate immediately before the end of exercise was >160, the heart rate at 5 min after the end of exercise was approximately 120 bpm, and at 30 min, it was approximately 100 bpm. For a heart rate of approximately 120 bpm immediately before the end of the exercise, the heart rate 5 min after the end of exercise was approximately 90 bpm, and 30 min after the end of exercise was approximately 85 bpm. In such cases, the heart rate 30 min after the end of exercise was higher than the resting heart rate. These results are consistent with those of our study.

A previous study reported an increase in facial skin blood flow during exercise (37), which aligns with our findings of a decreased G/B ratio during exercise due to vasodilation. Regarding the relationship between heart rate and G/B ratio fluctuations, the G/B ratio also decreased significantly where heart rate increased significantly during exercise, and the G/B ratio fluctuation also tended to decrease where heart rate fluctuation was small. This suggests that the variability in heart rate and capillary dilation/contraction are controlled by the same ANSA.

There are two main mechanisms for autonomic nervous system regulation during exercise: the exercise pressor reflex, in which receptors are stimulated and autonomic nervous system activity is regulated via feedback control, and central command, in which motor commands from higher brain centers directly regulate autonomic nervous system activity (38). Additionally, humans possess a function that allows external changes to alter the internal environment of the body, known as allostasis (39). During exercise, the command for central command, which is a feedforward control, is sent in conjunction with the intention to move, from just before the onset of movement to immediately after its initiation. In other words, by recognizing “exercise,” the brain predicts the upcoming movement and triggers allostasis, thereby issuing central command signals. Since vasodilation and vasoconstriction before and after exercise onset are regulated by central command, changes in the G/B ratio begin responding earlier than body movements (40). In this study, the exercise performed caused a rapid change in heart rate, suggesting a significant influence of central command. Therefore, the change in the G/B ratio is considered reasonable. Also, fluctuations in heart rate and G/B ratio occur almost simultaneously. Vascular expansion and contraction before and after the start of exercise are controlled by the central command (40). Since the G/B ratio may move predictively according to the prediction of the central command, it can be said that there is no significant delay in heart rate. Thus, it can be said that changes in heart rate follow changes in G/B ratio.

Recent advancements in image pulse wave measurement technology have allowed to estimate the heart rate from an RGB camera with high accuracy (41). By taking a moving image of the face only with a camera and combining it with the vascular information measurement system proposed in this study and other high-precision imaging pulse wave measurement techniques, it will be possible to manage physical condition during exercise more accurately in a completely noncontact manner. Heart rate can be used to evaluate optimal exercise intensity, whereas vascular information can be used to assess exercise safety from a cardiovascular perspective. Because the system is entirely noncontact, it eliminates the inconvenience of wearing equipment, allowing for physical condition management in a more natural state and facilitating appropriate exercise evaluation. It is possible to evaluate ANSA only with this system, but by combining it with various indices, it is possible to observe the condition of the body with more detail and accuracy.

The G/B ratio reflects circulatory and thermoregulatory functions. Since there are individual differences in these balance and regulation methods, the variation of the G/B ratio does not perfectly correspond to the variation of heart rate (42, 43). However, the G/B ratio tended to decrease when sympathetic nervous system activity was dominant. In the future, it is necessary to comprehensively evaluate ANSA and to verify whether the G/B ratio is appropriate as an evaluation index of ANSA.

One limitation of this study was the impact of individual differences. For some subjects we had difficulties capturing changes in the G/B ratio due to variations in cardiovascular and autonomic responses, even at the same exercise intensity. Individual differences were also observed in the recovery period. Therefore, it was not possible to eliminate individual differences in response to exercise in this experiment.

In addition, our study was conducted in an environment shielded from external light, which is the ideal experimental environment for the test. Further research is needed to determine whether the same system can be used in environments outside the ideal experimental environment. For practical application, constructing a system that can measure under an environment with rapidly changing external light, such as under sunlight, is necessary. Additionally, constructing an image processing system considering the effect of the change of external light is vital. The camera used in this study had a 30-fps frame rate and a resolution of 1,920 × 1,080. Because our verification experiments were conducted using only a camera with these specifications, it is necessary to examine whether similar experimental results can be obtained using cameras with different specifications.

Conclusion

5

Until now, evaluating the ANSA during exercise contactless was difficult. Therefore, we used facial capillaries as an ANSA index and detected contraction and expansion of facial capillaries using only a common RGB camera. We could extract features of contraction and expansion of facial capillaries through image processing during exercise with body movement. There are few restrictions because the measurement method is noncontact. This is an unprecedented evaluation method for ANSA that directly extracts features of contraction and expansion of facial capillaries. Because facial capillaries are controlled by ANSA, they can be considered a new ANSA index. In this study, we proposed a method for measuring capillary vasoconstriction and dilation characteristics during an exercise task using a common camera. Our verification experiment, which involved an exercise task, allowed us to extract these values from all subjects. By combining the camera with other indicators such as heart rate measurements, we could obtain more detailed information about the body function. This enables the sensitive management of physical conditions, potentially leading to healthier lifestyles. This method can be applied in various ways. For instance, it could be used to test heat acclimatization because capillaries dilate with heat acclimation (44). This application will be investigated in future research.