Association Between Self-Reported Dietary Intake Questionnaires and Objective Measures in an Inpatient Cross-Sectional Study

TL;DR

This study found that self-reported dietary questionnaires have weak correlations with actual food intake, especially for women and low-fat foods.

Contribution

The study reveals sex-specific differences in the accuracy of self-reported dietary data for low-fat food groups.

Findings

Self-reported dietary intake showed weak positive correlations with actual intake (r ≤ 0.25).

Males showed significant associations for low-fat/high-protein and low-fat/high-carb food groups.

Females had little to no association between self-reported and actual low-fat food intake.

Abstract

Background/Objectives: Measuring dietary intake through self-reported questionnaires can be inaccurate and influenced by sex, eating behavior, and the environment. Here, we compare self-report dietary intake questionnaire responses to objectively measured ad libitum dietary intake in a large, diverse population, and assess differences by sex and food-group composition. Methods: In our inpatient study, from 1999 to 2023, (n = 279) participants completed three different questionnaires assessing different aspects of food intake. Each questionnaire contained the same 77 food items belonging to one of six groups. Groups were either high-fat (HF) or low-fat (LF), then high complex carbohydrate (HCC), high protein (HP), or high simple sugar (HSS). Intake was measured based on the average percent group (PctGrp) intake over three days of ad libitum intake. General linear models, adjusted for relevant covariates and a PctGrp by sex interaction, assessed the relationship between PctGrp intake and questionnaire scores. Results: We found a weak positive correlation between PctGrp intake and food rating (all r ≤ 0.25). There was an interaction between LF/HP and LF/HCC with sex (significant slopes in males only, p = 0.0078, p ≤ 0.0001, respectively). Conclusions: This large study demonstrated little association between self-report dietary questionnaires and intake, especially in females with regards to low-fat foods.

Figures

Full text

1. Introduction

The ability to accurately measure habitual dietary intake is a cornerstone of nutrition research. One objective measure of free-living dietary intake is doubly labeled water (DLW), which uses stable isotopes to measure energy expenditure and habitual dietary intake over the study period [1]. However, DLW is expensive, time-consuming, and requires a return visit from participants. In the absence of objective free-living food intake measures, self-reported measures of dietary intake are often used as they are inexpensive and easily implemented by research staff [1]. Questionnaires, such as a food frequency questionnaire (FFQ), are among the most widely used and validated self-report measures [1].

The subjective nature of self-report measures of dietary intake has inherent limitations. When questionnaires such as FFQ are compared to DLW, their correlation coefficients are consistently under r = 0.25 and are prone to systematic error and underreporting of intake, which has been shown to be between 20 and 30% compared to DLW [1,2,3]. Specific demographic, behavioral, and environmental characteristics, including age, dietary restraint, weight status, income, and education level, may account for the measurement errors and inaccuracy of dietary intake questionnaires [1,2,4,5,6,7]. The impact of sex on accuracy of self-reported intake is uncertain. Some studies find that both men and women underreport similarly [6]. In contrast, others suggest that females underreport their intake to a greater degree, leading researchers to hypothesize that females may be more conscious of what they “should” report [2,5].

To overcome these challenges, dietary intake can also be measured under controlled, laboratory conditions. Though resource-intensive, this approach allows for precise, objective measurement of dietary intake. It is also well-established that people modify their behavior when they know they are being observed, yet the ability to collect accurate, detailed information on dietary intake under controlled conditions provides critical insight into eating behaviors that is not possible in free-living environments [8]. While some degree of under- and over-reporting still occurs in these controlled settings, the magnitude is substantially lower than what is observed in free-living conditions, as mentioned above [8]. Under observation, females in one study underreported their intake (around 10%) to a greater extent than their male counterparts [9]; however, the study was conducted with a small sample size (<60 individuals) and performed limited analysis on the types of foods that were misreported [9]. In fact, many studies assessing the difference between objectively measured and subjectively recorded food intake rely on small sample sizes and provide limited analyses of which types of foods are likely to be misreported [8].

In this analysis, we compare participant answers on three self-report questionnaires measuring dietary intake prior to admission to an objective measure of ad libitum dietary intake using our vending machine paradigm [10]. These questionnaires capture what participants state they eat, what they prefer to eat, and what they selected for the ad libitum paradigm. Our primary aim is to assess and quantify the discrepancies between self-reported and objectively measured dietary intake. We will also explore whether these discrepancies differ by sex, hypothesizing that females’ relationship between self-report and intake will be less strongly associated than that of their male counterparts. Finally, we add a novel exploration into whether patterns of under- or over-reporting vary by food group, and whether these patterns continue to differ by sex. Our analysis is novel as it represents a unique research design in which participants completed these questionnaires prior to objectively measured consumption of foods in a large, diverse dataset.

2. Materials and Methods

2.1. Subjects and Recruitment

From 1999 to 2023, healthy subjects were invited to participate in a nine-day inpatient study (ClinicalTrails.gov, identifier NCT00342732) at the NIDDK clinical research center in Phoenix, AZ. This study investigated the behavioral and physiological determinants of ad libitum food intake. An oral glucose tolerance test (OGTT) was used to determine whether participants were free from diabetes. In total, 314 individuals participated in the original parent study. For this analysis, participants were excluded if they had missing OGTT, body composition data through dual X-ray absorptiometry (DXA, DPX-1, Lunar Radiation Corp, Madison, WI, USA), or were diagnosed with type 2 diabetes (n = 22). A total of 13 individuals were removed for missing or incomplete dietary intake questionnaires, resulting in n = 279 individuals being included in this analysis. The NIH IRB approved this study, and all participants gave informed consent.

2.2. Study Design

For this study, participants were inpatients for nine days. Participants completed dietary intake questionnaires within the first two days of admission (described in detail below). For the next three days, participants received a weight-maintaining diet based on unit calculations [11,12]. The macronutrient distribution of the calculated weight-maintaining diet was 50% carbohydrates, 30% fat, and 20% protein. Over the three days following the weight-maintaining dietary intervention, participants ate ad libitum using a validated vending machine paradigm (described in more detail below).

2.3. Dietary Intake Questionnaires and Food Groups

As this study began recruitment in 1999, the Geiselman questionnaire was selected as the best instrument due to superior test–retest correlation (r = 0.90) and strong correlation with the block food frequency questionnaire (r > 0.70), which was seen as the gold standard at the time [10].

Prior to ad libitum intake on the vending machine, participants completed three 77-item dietary intake questionnaires [10]. A complete list of the food items and their Geiselman Food Group classification is available in Supplemental Table S1. The three questionnaires (“Food Selection Questionnaire (Selection),” “Food Frequency Questionnaire (Frequency),” and “Preferred Food Frequency Questionnaire (Preference)”) contained the food items listed in this Supplemental table. The responses to the selection questionnaire determined the foods stocked in each participant’s personal vending machine. Participants rated each item on a scale from 1 (extremely dislike) to 9 (extremely like), with 5 being neutral. Then, 40 foods scored in the range of 4–8 were selected to fill the vending machines. Vending machine size restricted the selection to a maximum of 40 food items. The frequency questionnaire asked about the frequency of consumption of these foods at home. Participants then rated foods on a scale of 1 (never eaten) to 9 (more than once a day), with 5 being once a week. The preference questionnaire asks, “How much would you eat of this food if time, convenience, or cost were not a factor?” with the same response scale as the frequency questionnaire.

For subsequent analyses, food items on the questionnaires were grouped according to macronutrient content, which remained constant throughout the study period [10]. Food groups are characterized by fat content, either high-fat (fat ≥ 45% of total kcals) or low-fat (<20% total kcals), then either high complex carbohydrate (≥30% total kcals), high protein (≥13% total kcals), or high simple sugar (≥30% total kcals), resulting in the following subgroups: high-fat, high-complex-carbohydrate (HF/HCC); high-fat, high-protein (HF/HP); high-fat, high-simple-carbohydrate (HF/HSS); low-fat, high-complex-carbohydrate (LF/HCC); low-fat, high-protein (LF/HP); and low-fat, high-simple-carbohydrate (LF/HSS). For each questionnaire (selection, frequency, and preference), the “food ratings” were calculated using the average food-group score in each of the 6 food subgroups.

2.4. Ad Libitum Dietary Intake

Each patient was given exclusive access to one of two vending machines on our unit. As mentioned above, these vending machines were stocked with 40 food items based on their answers to the Food Selection Questionnaire (FSQ). Our unit’s vending machine paradigm is a highly reproducible (ICC = 0.90) and validated measure of objective ad libitum dietary intake [12,13]. The vending machines are located on the inpatient floor in a separate room, where participants were given 23.5 h of free access to enter the room whenever they like, with a half-hour restocking period [13]. Food was weighed before being placed in the vending machine and reweighed every 24 h. Once participants removed a food item from the vending machine, leftovers or waste were placed back in the machine to be reweighed. Participants ate ad libitum from their vending machine for the final three days of their inpatient stay. Food intake was then entered into one of two food-processing software programs depending on the date—CBORD from 1999 to 2007 (CBORD Professional Diet Analyzer Program, CBORD Inc., Ithaca, NY, USA) and Food Processor from 2007 to the present (version 10.0.0; ESHA Research, Salem, OR, USA)—to calculate total caloric and macronutrient intake as well as the percentage of total caloric intake from each of the six food groups averaged over their three days of ad libitum vending machine intake (PctGrp). More details about the vending machine protocol can be found in (Venti et al., 2010) [13].

2.5. Covariates

Race was categorized into four groups: Indigenous Americans and Alaska Natives (AI/AN), White, Black (AA), and Other (Hispanic, Asian, and people who identified as multiple races). Fat-free mass (FFM; fat-free mass (kg)/height^2^ (m^2^)]) and fat mass (FM; [fat mass (kg)/height^2^ (m^2^)]) indices consider height in addition to mass, and were determined to be the best indicators of body composition [4]. To ensure consistency of the results, sensitivity analyses were performed on all models, replacing the FFM index and FM index with BMI or percent fat, and results remained the same.

2.6. Statistical Analysis

Statistical analyses were completed using SAS (version 9.4, SAS Institute Inc., Cary, NC, USA). Mean ± standard deviation was used to express normally distributed data, while non-normally distributed data were expressed as the median and interquartile range (IQR). Absolute counts and percentages were used to express categorical data. An alpha of 0.05 was set as the significance level for all tests. Sex differences were assessed using independent-samples t-tests.

To assess our primary outcome, relationships between PctGrp intake and food ratings were assessed by Pearson’s correlation coefficients (r). The relationship between PctGrp intake and food ratings was further tested using general linear models (GLMs) adjusted for sex, age, race, FFM index, and FM index, and separated by questionnaire. To assess our exploratory outcomes of whether sex moderated the relationship between intake and food ratings, interaction terms were added to the GLMs (food rating × sex). For models with significant interaction terms, simple slopes for females and males were performed to interpret the directionality of food rating effects on intake by sex.

3. Results

3.1. Demographics and Descriptives

After the exclusions noted above, 279 participants were included in this analysis. As seen in Table 1, more participants were male (n = 172, 62%), AI/AN (n = 164, 58.78%), and had increased adiposity (mean BMI, 31.71 ± 8.13). Total intake (kcals) was significantly higher in males when compared to females, as expected given males’ significantly higher fat-free mass (p ≤ 0.0001). For PctGrp intake, there were no sex differences across all groups except HF/HP (p = 0.046), in which males ate more.

Food ratings in Table 2 were similar between males and females except in the frequency questionnaire, where males rated HF/HSS foods significantly higher (p = 0.04), and the selection questionnaire, where females rated LF/HCC foods significantly higher (p = 0.0014). The top-rated foods across the questionnaires included pizza, cheeseburger, orange, eggs, fried chicken, potato chips, cheddar cheese, and apples, with no significant differences between sexes (Supplemental Table S2).

3.2. Correlations with Intake of Food Groups by Questionnaire

Overall, there were weak, positive correlations between the intake expressed as PctGrp and questionnaire food ratings (Figure 1). The HF/HSS food group had the highest association between PctGrp intake and food ratings across all three questionnaires (selection: r = 0.25, p < 0.001; frequency: r = 0.23, p < 0.001; preference: r = 0.19, p < 0.001). Food ratings were correlated with each other; the correlation between selection and frequency for LF/HCC food-group ratings was the lowest (r = 0.42), and between the preference and frequency HF/HP food-group ratings was the highest (r = 0.71).

When separated by sex (Supplemental Figure S1), associations between PctGrp and food ratings for the HF food groups were consistent with the whole group. In males, the correlations between PctGrp and food ratings for LF food groups became generally larger, while in females, the correlations between PctGrp and food ratings for LF food groups were absent.

3.3. Adjusted General Linear Models with Intake of Food Groups by Questionnaire

Significant interactions between food ratings and sex on PctGrp were found for LF/HP frequency (p = 0.016) and LF/HCC preference (p = 0.0053) (Table 3). Simple slope analyses revealed LF/HP frequency questionnaire food rating was positively associated with LF/HP PctGrp intake for males (Figure 2A; male β = 1.18, p = 0.0078) but not for females (Figure 2A; female β = −0.63, p = 0.30). LF/HCC preference questionnaire food rating was positively associated with LF/HCC PctGrp intake for males (Figure 2B; male β = 1.52, p < 0.0001) but not females (Figure 2B; female β = −0.26, p = 0.61).

4. Discussion

Our analysis of 279 participants revealed a weak correlation between self-reported dietary questionnaires (food ratings) and objectively measured ad libitum intake, as represented by the designated groups (PctGrp). The three questionnaires demonstrated a medium-to-strong correlation within each food group (r = 0.42–0.71), showing consistency between their self-described usual intake, preferences, and ratings for foods to select for the vending machine paradigm. As expected, while statistically significant, the objectively measured percent group intake for each food group was weakly correlated with each food group (r = 0.01–0.25). The questionnaires were weakly associated with intake of foods in the HF group. For the LF group, we found that the relationship between dietary questionnaires and intake differed by sex, such that there was an association between reported intake, preference, and selection in men but not in women.

Overall, our analysis demonstrated a weak correlation between questionnaires and intake. Given that the selection questionnaire was used to stock the vending machines with items the participant indicated a liking for, we would have expected stronger correlations. However, this is in line with the literature demonstrating that questionnaires are often not an accurate measure of intake. The associations between FFQ responses and measures of absolute intake, typically biomarkers and DLW, are typically low (r ≤ 0.25) [3,7]. In a systematic review by Freedman et al., the average correlation coefficients were 0.21 for total energy intake and 0.29 for protein intake in FFQs [7]. This may be mainly due to the high propensity for participants to underreport dietary intake in FFQ [1,2]. In fact, participants are more likely to underreport on questionnaires when compared to other forms of self-reported dietary intake [1,2]. Correlations were stronger when comparing responses within questionnaires regarding a particular food group, indicating people tended to be consistent in their answers on the questionnaires. However, this does not translate to actual intake. After adjustment for covariates, questionnaire scores were associated with the HF/HSS group (all questionnaires) and HF/HP group (frequency and preference questionnaires only). Thus, even after adjustment for potential confounders, questionnaires are poorly reflective of intake.

In the analysis by sex, interactions were demonstrated in the LF/HP group in the frequency questionnaire and the LF/HCC group in the preference questionnaire. In the simple slopes analysis, males had a positive slope, while females demonstrated no association. This indicates that males were more consistent in their questionnaire responses and consumption of low-fat foods, while females may have indicated preferences but did not consume such foods. One potential explanation is females’ documented sensitivity to stating what they “should eat” [14,15]. According to Barebring et al., on food-related questionnaires, females are more concerned about being healthy and are more likely to select foods based on perceived health [14]. In a systematic review of studies with questionnaires validated with DLW, females tended to underreport more frequently than their male counterparts [2,5]. However, other reports show underreporting across the study population, not just from females [6]. Nevertheless, our data appear to support a mismatch between reported and preferred intake of low-fat foods (perhaps deemed healthier foods) and subsequent consumption in females.

Lastly, we acknowledge that this cross-sectional analysis spans more than two decades (1999–2023), during which secular changes in dietary intake patterns could theoretically influence our findings. Although other studies have noted substantial shifts in macronutrient intake in the United States over a similar period, a previous analysis from our group within the same cohort observed relatively modest annual changes of −10, −27, and −22 kcals/year for protein, fat, and carbohydrates, respectively [16,17]. While these values may appear meaningful over time, it is important to note that participants were substantially overconsuming energy at baseline (mean intake of 3896 kcal/day) [16]. Moreover, these declines were not accompanied by corresponding reductions in body weight. Therefore, we concluded that the observed reduction in energy intake likely reflected behavioral changes associated with study participation and increased public health messaging around excessive energy consumption, rather than true population-level shifts in dietary patterns in our study population.

We acknowledge both strengths and limitations in this analysis. In recent years, the Leeds Food Preference Questionnaire (LFPQ) has become a well-established food preference questionnaire as it takes into account the measurement of liking and wanting certain foods [18]. However, due to the long-term nature of this study, we deemed it more important to preserve the continuity of measurements, and thus, the Geiselman questionnaire remained our questionnaire for measurement of food preference. The inpatient study design, while allowing for greater experimental control, limits the external validity of our findings as it further separates the study setting from free-living conditions. With regards to the vending machine paradigm, although highly reproducible (ICC = 0.90), this may not accurately represent a participant’s free-living intake. Additionally, there may have been a misalignment between the participants’ expectations of what food would be presented to them and what foods were offered. For example, they might have filled out the questionnaire anticipating that fresh spinach would be presented to them, not realizing they would be provided with canned spinach. A statistical limitation in our analysis is that these large GLM models may have led to a loss of power in the simple slopes analyses, thus potentially losing some significant interactions.

To our knowledge, this is the largest study of its kind, with most studies directly assessing self-reported versus objective dietary intake having less than 100 participants [8]. Another major strength comes from our use of the vending machine paradigm. Many studies, such as Freedman et al., use biomarkers to assess the participant’s intake in relation to their responses to the questionnaires [7]. While this may demonstrate more habitual dietary intake than the vending machine, biomarker use is limited to the number of dietary intake measurements that can be assessed. Using a vending machine paradigm to assess dietary intake, we can objectively test all the macronutrients, gaining more information on the type of diet someone may have.

5. Conclusions

Despite internal consistency among the questionnaires themselves, we found weak or no associations between our food intake questionnaires and objective measures of dietary intake from the vending machine. Our findings are interesting because the questionnaires were completed right before an ad libitum consumption period in such a large and diverse group of participants. Moreover, despite supplying participants with foods they preferred, their consumption did not generally mirror their stated usual or preferred intake. In fact, in women, there was no association with intake of low-fat foods, despite their questionnaires expressing a preference and selection for these foods. Collectively, these results underscore the continued importance of incorporating objective measures of dietary intake in nutrition research. Additionally, these results further highlight the disconnect between free-living dietary behaviors and those observed under controlled laboratory conditions. Future studies should consider the context of the dietary intake data they wish to collect when selecting measurement tools. Given established challenges inherent to accurately capturing dietary intake, and the limitations of both inpatient and free-living approaches, future efforts should aim to integrate the strengths of controlled ad libitum paradigms with real-world assessment strategies to improve the validity and interpretability of dietary intake data.