Trends in physical and mental health needs across generations in Australia

TL;DR

This study shows that health needs are changing across generations in Australia, especially for women, and traditional planning methods may not be enough.

Contribution

The paper introduces a new approach to health service planning by considering generational differences in health needs.

Findings

Recent-born female cohorts have higher prevalence of long-term health conditions compared to earlier-born cohorts.

Increases in mental disorders among females are not offset by decreases in physical impairments, unlike in males.

Traditional planning models may underestimate future mental health service requirements when ignoring cohort effects.

Abstract

Despite evidence of changes in age-specific incidence and prevalence rates for chronic conditions and disease, future health resource planning is often based on historical age- and gender-specific service use, neglecting changes in the need for care within age groups between generations. This paper studies differences in health needs by age and gender across birth cohorts in Australia and considers the implications for future health service planning. Whilst controlling for age and period effects, we find that more recent-born female birth cohorts have higher prevalence rates of long-term health conditions than earlier-born cohorts, whereas we don’t find an effect for males. The increase for females corresponds with an increase in probable mental disorders, and while we also find an increase in probable mental disorders among males, decreases in physical impairment rates among both genders offset the overall rates of long-term health conditions among males but not among females, where increases in probable mental disorders are larger. Comparing projections of mental health service requirements that integrate cohort effects, as opposed to those that do not, shows that traditional planning models may underestimate health service requirements for the future. Our findings suggest that health service planners should relax assumptions about constant age-specific use.

Figures

Plotted birth cohort effects of long-term health conditions for females and males after controlling for age and periodTable 1Age and/or cohort effects on long-term health conditions among male and female respondents in HILDA between 2001 and 2021Female sampleMale sample(1)(2)(3)(4)Age onlyAge, Cohort, PeriodAge onlyAge, Cohort, PeriodAge group*Reference group: Respondents below 22 years of age*22—280.019* (0.004)0.029* (0.005)0.014* (0.004)0.016* (0.004)29—350.040* (0.005)0.061* (0.006)0.039* (0.005)0.041* (0.006)36—420.060* (0.006)0.088* (0.007)0.066* (0.006)0.065* (0.007)43—490.114* (0.006)0

Plotted birth cohort effects of physical impairment and mental disorder prevalence rates for females and males after controlling for age and period. Notes: The Figures show the values of the cohort dummies plus the constant of our age, period cohort regression model, following Deaton and Paxson [[11](#CR11)] with physical impairment and probable mental disorder as the dependent variables and as independent variables a full set of age and transformed year dummies. Birth cohort 1922–1929 is the reference. The light lines display 95% confidence intervals.Source: HILDA data

Full text

Introduction

Health service planning is pivotal in making the most effective use of available and future health care resources and in order to plan appropriately for recruitment and training of health care professionals to meet future population needs. An aspect that is often neglected in health service planning models is the consideration of changes in age-specific prevalence and incidence rates of disease and disability over time, i.e., changes in health care needs by age. For example, 65-year-olds in 2023 have very different health profiles than 65-year-olds in 1950. But despite this evidence, most governments base their health service plans on historical age-specific health service use estimates and apply this number to future population estimates in the respective age groups [26, 27]. In other words, it is assumed that previous age-specific use rates will continue.

Planning based on historical use is problematic for two reasons. First, the population’s current use of health services does not necessarily meet that population’s needs. In fact, there is much evidence of unmet health care needs for certain conditions, or subgroups of people, such as those on lower incomes or from disadvantaged backgrounds as well as for people living in more rural as opposed to urban areas [15, 22]. At the same time, it has been shown that in other areas health services are being overutilised [6]. A mismatch of demand and supply in health care has been recognised by economists for several decades [17]. Using historical use estimates for planning for the future will therefore perpetuate current problems into the future, instead of improving unmet care needs.

Second, even if current health service use were to meet the needs of populations perfectly, the need for health care is changing, over time, both within and between age groups. For example, while overall dementia prevalence rates are predicted to increase over the next decades, due to an increase in population numbers in old age, age-specific dementia incidence rates are declining in more recent birth cohorts [13, 18, 24]. At the same time, some evidence suggests that more recent birth cohorts will have a shorter life expectancy than the current generations due to the impacts of obesity and sedentary lifestyles, changing health care needs at different ages [2]. We argue that incorporating these changing health care needs into health service planning is crucial to better align health care supply with needs in the future.

In the existing literature, advantages of needs-based planning models have been discussed in England and Canada, where the authors highlight the importance of accounting for birth cohort effects when estimating future health care needs [21, 30]. Lenzen and Birch [19] discuss the advantages of dynamic needs-based health service planning models in the Australian context, but do not account for period effects in their analysis. In addition, when attempting to incorporate changes in need over time and between cohorts, gender specific differences in need should be taken into consideration. Research has shown that prevalence estimates of disease and disability do differ substantially by gender. For example, while females are more likely to develop conditions such as dementia in old age [20], the prevalence of coronary heart disease is much higher among males than females [31]. Neglecting any of these gender-specific differences in health needs and taking into consideration the greater longevity of females substantially affects future health service use estimates.

We add to the literature in several important ways. Foremost, this is the first study to investigate changes in health needs across birth cohorts, stratifying need estimates not only by age but also by gender, and the first study to do so for the Australian population. Using data over 21 years from a representative sample of the Australian population, we are able to address the following research questions: How do health needs differ across birth cohorts after controlling for age and period? How does incorporating gender-specific cohort effects into future health need predictions compare to traditional age-constant models? In addition, we are the first to provide needs-based predictions for future mental health service requirements by estimating future prevalence rates of mental disorders, applying dynamic age- and cohort-specific predictions, and multiplying these with service requirements.

Methods

Empirical methodology

Health service planning models

In conventional health service planning models, future health service requirements are based on historical age-specific use estimates and projected onto the future population size. In other words, future health service requirements ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthCar{e}_{t+1})$$\end{document} are a function of the future projected population size \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(Populatio{n}_{a,t+1})$$\end{document} in each age group (a) and the current or past mean level of health care use per capita \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(HealthCar{e}_{a,t})$$\end{document} in that age group:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthCar{e}_{t+1}=\sum\limits_{a=1}^{N}Populatio{n}_{a,t+1}*HealthCar{e}_{a,t}$$\end{document}

where a represent age groups (a = 1, 2, 3, …, n). t indicates the current year, and t + 1 any future years. In practice, levels of health care use by age group can be estimated using individual level survey data and linear regression models of health care use as the dependent variable, regressed on age dummies and a random error term ( \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\upepsilon }_{\mathrm{i},\mathrm{t}}$$\end{document} ) as follows:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthCar{e}_{i,t}={\updelta }_{0}+{\upbeta }_{1}agegrou{p}_{a,i,t}+{\upepsilon }_{i,t}$$\end{document}

where i indexes individuals and t indexes time.

While some planning models include more sophisticated versions of this model, such as focusing on previous trends in use, their estimates remain based on utilisation rates rather than need, thus perpetuating a continuation of the potential mismatch between use and needs [27].

As a solution to this problem, we suggest an approach in which future health service requirements are based on predicted future age- and cohort-specific health needs, as follows:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthCar{e}_{t+1}=\left(\sum\limits_{c=1}^{{N}{\prime}}\sum\limits_{a=1}^{N}Populatio{n}_{a,t+1}*\widehat{HealthNee{d}_{a,c,t+1}}\right)*\frac{HealthService{s}_{t+1}}{HealthNee{d}_{t+1}}$$\end{document}

here c represents corresponding birth cohorts  \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$(c=1,2,3...n')$$\end{document} and  \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${Health\widehat{Need}}_{a,c,t+1}$$\end{document} are predicted age (a) and birth cohort (c) specific estimates of health needs based on prevalence rates of health conditions or in our example health need proxies.  \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\frac{{HealthServices}_{t+1}}{{HealthNeed}_{t+1}}$$\end{document} represent the predicted amount and type of services that are required to meet the population’s predicted health needs. The aggregated amount of health service requirements is thus a function of health needs estimates per age group, multiplied by population predictions of the respective age group and service requirements to meet that population’s health needs.

Predicting health needs

While Eq. (3) is straightforward, understanding changes in health needs across age and cohort is complex and tackling these methodological issues is the focus of our analysis. We start by estimating age-specific health needs, similar to what is done in traditional health service planning models, but instead of using health care as dependent variable we use health needs proxies:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthNee{d}_{i,t}={\updelta }_{1}+{\upbeta }_{2}agegrou{p}_{a,i,t}+{\upepsilon }_{i,t}$$\end{document}

Changes in the need for health care by age groups over time though are not incorporated in Eq. (4) and therefore as a next step, we include a time trend or year dummies \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{i,t}$$\end{document} as follows:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthNee{d}_{i,t}={\updelta }_{2}+{\upbeta }_{3}agegrou{p}_{a,i,t}+{\upgamma }_{1}{d}_{i,t}+{\upepsilon }_{i,t}$$\end{document}

While this model represents an improvement over model (4), it still does not account for birth cohort-specific trends. However, as pointed out in the Introduction, health care needs might not only vary by age and over time but also between birth cohorts. In particular, birth cohort effects might occur due to environmental factors that are related to health but are unequally distributed across a population and across time. For example, younger people today may have higher access to processed foods than younger people of previous generations. Birth cohort effects can also occur because the impact of environmental factors on health may differ across age groups. For example, exposure to high levels of air pollution may affect younger people more than older people. At the same time, health changes over the life course, as people age (age effects); or due to events that happen in a particular year, such as the recent Covid-19 pandemic or medical advances (year effects). Including all three of these variables, however, is problematic, because of the age, period, and cohort identification problem. Even with panel data and a survey period that spans many years, cohort effects are difficult to identify because they are correlated with the age and period effects. In other words, it is not possible to control for age, birth-year, and period/year in the same equation, because age is a perfect linear combination of birth-year and survey period. The challenge we face is how to control for all three dimensions at the same time. We propose to overcome this challenge using two leading approaches in the literature.

First, to resolve any issues of collinearity between age, period and cohort, Deaton and Paxson [11] and Deaton [10] proposed an approach in which the year dummies are transformed to estimate unbiased coefficients on age and cohort. Since we are primarily interested in the cohort effects and not so much in any year or period effects when estimating future healthcare requirements, we find the approach suitable for our research questions. More specifically, Deaton and Paxson [11], proposed that the year dummies be transformed to follow a trend around zero, which still allows to account for any fluctuations over the survey periods, without potentially biasing the age and/or cohort estimates. We follow this approach and transform year dummies in the following way:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{i,t}^{*}={d}_{i,t}-\left[\left(surveynumber-1\right){d}_{i,surveywave2}-\left(surveynumber-2\right){d}_{i,surveywave1}\right]$$\end{document}

where \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{i,t}^{*}$$\end{document} is the transformed year variable, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{i,t}$$\end{document} is a binary year variable for individual i at survey year t. surveynumber is the survey number. By construction, \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{i,surveywave1}^{*}$$\end{document} and \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${d}_{i,surveywave2}^{*}$$\end{document} will equal zero and are thus omitted from the regression. If we include this transformed year variable in the estimated equation for health needs, prevalence rates can be estimated as a function of age, cohort effects, and year effects, reducing potential bias caused by specific time/period effects that may influence some cohorts and ages more than others:

\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$HealthNee{d}_{i,t}={\updelta }_{3}+{\upbeta }_{4}agegrou{p}_{a,i,t}+{\theta }_{1}cohor{t}_{c,i,t}+{\gamma }_{2}{d*}_{i,t}+{\upepsilon }_{i,t}$$\end{document}

Second, we will employ a further methodology that has been used in birth cohort analysis. The proxy variables approach uses one or more proxy variables to substitute either the age, period or cohort variable. This is a popular approach because of its intuitive rationale. After all, the age, period, and cohort variables serve as proxies for different sets of unmeasured correlates. In the literature, several papers have used among others, the unemployment rates [14, 28] and GDP growth rate [12, 14, 25, 28] as proxy variables for period effects. In this context, it is important that proxy variables are not linearly related to cohorts or periods since, otherwise, they will be highly collinear with the other two effects, just as the effects they are intended to replace in the models [16].

We propose to follow the literature and use GDP growth rates as a proxy for period effects in robustness checks. Globally, a clear strong positive correlation between health and GDP exists. Countries with better health status tend to have higher incomes than countries with worse health status, a relationship known as the “Preston curve” (see [23]). Similarly, within countries, people with higher incomes tend to have better health than those with lower incomes. This relationship has also been observed over time. While the average life expectancy in 1960 was just over 70 years in Australia, it is now well over 80 years. At the same time, GDP has grown substantially over the same time period [27]. Clearly, this relationship alone does not allow us to conclude that higher income or better economic growth leads to better health outcomes, since the main forces that drive economic growth, such as technological progress, education, and physical capital accumulation also promote population health, but we do not need to be able to disentangle the drivers of this relationship, as long as we believe that they are correlated with GDP.

Estimating future health needs

Using previously estimated age and cohort effects, we now estimate future health needs by age group. More specifically, we compare health care needs estimates using (i) only the age effects plus the constant, i.e., \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\updelta }_{1}+{\upbeta }_{2}$$\end{document} , following Eq. 4, with (ii) both the age as well as cohort effects plus the constant, i.e., \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$${\updelta }_{3}+{\upbeta }_{4}+{\theta }_{1}$$\end{document} , following Eq. 7. The latter accounts for changes in health needs across birth cohorts and is our preferred specification. To estimate the aggregate number of people who will require health care in the future, we multiply the age specific health need prevalence rates with official Australian population projections from the Australian Bureau of Statistics in the respective age group [1]. These estimates can be found in the Appendix, Tables A2 and A3.

Estimating future health service requirements: a case of probable mental disorders

As a next step, we use estimated future health needs to predict future health service requirements. Since health service requirements vary depending on the disease or condition, ideally one would want to calculate future health care requirements for every condition separately. While this is out of the scope of a single research paper, instead, we present a case study for probable mental disorder, as a proof of concept. We focus on mental health disorders because among the three health need proxies, we find that probable mental disorders change the most and are therefore most relevant to future health service planning. In addition, HILDA respondents are specifically asked about mental health service use, allowing us to estimate historical use.1

First, we follow traditional planning methods and use these historical age-specific use estimates from HILDA and apply them to predicted probable mental disorder prevalence rates for each age group from the previous section. Next, we compare these traditional projections with projections that apply recommended service use based on clinical guidelines. Here, we follow the recent review of evidence-based psychological interventions in the treatment of mental disorders, published by the Australian Psychological Society [4].2

Data

Our analysis uses data from the HILDA Survey which has been conducted annually since 2001. The survey sample is an area-based clustered sample of households initially interviewing almost 14,000 persons from 7,682 households. In 2011, another 4,000 persons from 2,153 households were added to the sample [29]. HILDA is divided into a face-to-face interview as well as a self-completion supplement, including questions about family life, income, and employment, as well as health, and health behaviours.

With the goal to study populations’ ‘overall’ health service needs, we start our analysis with a health need proxy measure that captures whether respondents experience any ‘long-term health condition, impairment or disability that restricts them in their everyday activities, and has lasted or is likely to last, for 6 months or more’ (see Appendix, Table A1). We assume that this measure is most predictive of the overall need for health care, due to its ‘objective’ nature, and because it is most similar to proxies used in previous studies, such as limiting long-standing illness [21, 30]. When answering this question in the survey, respondents were provided with a list of conditions, including among others sight, hearing and speech problems, difficulty learning or understanding things, limited use of arms, or legs and conditions that impact physical activity or physical work, pain, shortness of breath and any mental illness which requires help or a nervous or emotional condition which requires treatment.

The choice of an appropriate measure of overall health needs is a crucial cornerstone of this paper and one may argue that some health needs are not picked up by measuring only long-term health conditions. We therefore also use two quality-of-life measures as health need proxies. Here we use information from the SF-36 questionnaire that is part of the self-completion part of the HILDA survey. The SF-36 instrument is an internationally recognised diagnostic tool for assessing functional health status and well-being. More specifically, the SF-36 consists of eight scaled scores, which are the weighted sums of the questions in their section. Dimensions include vitality, physical functioning, bodily pain, general health perceptions, physical role functioning, emotional role functioning, social role functioning and mental health. Each scale is directly transformed into a 0–100 scale on the assumption that each question in the respective dimension carries equal weight. We focus on the SF-36 subscales measuring (1) physical functioning, and (2) mental health problems, thus picking up one physical health as well as one mental health dimension. We use dichotomous measures of the variables. Physical impairment is derived from the physical functioning subscale, identifying someone as being in poor physical health if the respondent scores below 68 on the transformed scale. This equals one standard deviation below the mean and has been used in previous studies [8]. Probable mental disorder has been defined as respondents scoring below 52 on the mental health scale in several previous studies, which we use in our analysis by creating a dichotomous measure of mental health [7, 9].

Having 21 years of data, we divide our sample into 12 seven-year age groups (15–21 up to 92 and above) and 12 seven-year birth cohorts (1923–1929, … 2000–2006). With seven being a divisor of 21, we are able to split our sample into these equally sized age-cohort combinations. The sample sizes for people born before 1923 are very small and are therefore not part of our analyses. Tables A4 and A5 provide sample sizes for all other birth cohort age group combinations over the study period for our female and male sample, respectively.

We estimate the rate of health needs, i.e. prevalence rates of health need proxies, in each age group for the years 2000, 2007, 2014, 2021, 2028, and 2035. The years are chosen based on corresponding birth cohort-age-group combinations in our dataset. For example, 22- to 28-year-olds in 2000 were born between 1972 to 1978. An overview of these matching year, age group and cohort combinations can be found in Table A6. Since we are missing cohort estimates for people aged 70 and above in 2000, 77 and above in 2007 and 85 and above in 2014, our results will focus on comparing 2021, 2028, and 2035 estimates. This is in line with our research question, considering that we are interested in health service projections for the future. For the youngest age.

group in 2035, we do not have a cohort effect estimate either, so we replace it with the cohort effect of 2000 − 2006.

Results

Descriptive statistics

Our full sample contains 318,608 observations with 151,133 male and 167,475 female observations. Tables A7 and A8 contain the percentage of females and males in each age-group-cohort combination that have at least one long-term health condition. Statistics for physical impairment and mental disorder prevalence rates by age and cohort are provided in Tables A9 to A12 in the Appendix. As expected, particularly younger age groups have low shares of their population experiencing long-term health conditions, and that increases starting from age 50–60. For example, the average prevalence of respondents with long-term health conditions is around 24 percent among 43 to 49-year-old females, but this increases to 40 percent among 57 to 63-year-old females. The reverse is true for mental disorders, though, where prevalence rates are higher among younger age groups. Among young females, the prevalence of probable mental disorders is 17 percent for 15 to 21-year-olds, compared to 11 percent among 57 to 63-year-old females. We observe lower rates among males, where the prevalence is 10 percent among 15 to 21-year-olds and eight percent among 57 to 63-year-olds.

Cohort Effects

We now present the results of our main analyses. Figure 1 plots the estimated cohort effects based on Eq. (7), using long-term health conditions as a proxy for health care needs for females and males, respectively. The corresponding regression results can be found in Table 1 for the models (i) age only, based on Eq. 4 and (ii) age, cohort, and period, based on Eq. 7. Columns (1) and (2) present results for females, whereas columns (3) and (4) present results for males. The plotted cohort effects correspond to columns (2) and (4) in Table 1. For all models, the age group 15–22 and the birth cohort 1922–1929 are the reference categories. The prevalence of long-term health conditions increases among females born more recently, after controlling for age and period. Except for the cohort closest to the reference, all other cohort coefficients are statistically significant. For males, the signs are negative but statistically insignificant, except for the 1979 to 1985 cohort, which shows a statistically significant reduction of long-term health conditions by about five percent. Our results are robust, when using the proxy variable method, rather than the Deaton and Paxson [11] method. However, the reduction in long-term health conditions for males among more recently born birth cohorts is statistically significant for most cohorts using the proxy variable approach. Results can be found in the Appendix, Table A14.Fig. 1. Plotted birth cohort effects of long-term health conditions for females and males after controlling for age and periodTable 1Age and/or cohort effects on long-term health conditions among male and female respondents in HILDA between 2001 and 2021Female sampleMale sample(1)(2)(3)(4)Age onlyAge, Cohort, PeriodAge onlyAge, Cohort, PeriodAge groupReference group: Respondents below 22 years of age22—280.019*** (0.004)0.029*** (0.005)0.014*** (0.004)0.016*** (0.004)29—350.040*** (0.005)0.061*** (0.006)0.039*** (0.005)0.041*** (0.006)36—420.060*** (0.006)0.088*** (0.007)0.066*** (0.006)0.065*** (0.007)43—490.114*** (0.006)0.147*** (0.009)0.114*** (0.006)0.111*** (0.009)50—560.187*** (0.007)0.224*** (0.010)0.172*** (0.007)0.164*** (0.010)57—630.266*** (0.007)0.308*** (0.011)0.251*** (0.008)0.239*** (0.011)64—700.335*** (0.008)0.383*** (0.012)0.330*** (0.008)0.313*** (0.013)71—770.447*** (0.009)0.501*** (0.014)0.414*** (0.010)0.393*** (0.015)78—840.582*** (0.011)0.639*** (0.017)0.557*** (0.012)0.535*** (0.018)85—910.728*** (0.017)0.791*** (0.022)0.682*** (0.020)0.660*** (0.024)92—older0.836*** (0.037)0.907*** (0.040)0.772*** (0.063)0.751*** (0.065)Birth yearReference group: Respondents born before 19301930—19360.031(0.021)0.016 (0.022)1937—19430.053**(0.021)0.030 (0.022)1944—19500.054*** (0.021)0.018 (0.022)1951—19570.078*** (0.021)−0.024 (0.023)1958—19640.055*** (0.021)−0.027 (0.023)1965—19710.070*** (0.021)−0.030 (0.023)1972—19780.086*** (0.022)−0.023 (0.023)1979—19850.082*** (0.022)−0.046** (0.023)1986—19920.105*** (0.022)−0.028 (0.023)1993—19990.142*** (0.022)−0.017 (0.024)2000—20060.163*** (0.024)−0.003 (0.025)Constant0.136*** (0.004)0.029(0.021)0.138*** (0.004)0.162*** (0.023)Observations167,475167,475151,133151,133Models displayed in column (2) and (4) also include the transformed period effects, which are provided in the Appendix, Table A15. Standard errors in parentheses; * p < 0.10, ** p < 0.05, *** p < 0.01.

Notes: The Figures show the values of the cohort dummies plus the constant of our age, period cohort regression model, following Deaton and Paxson [11] with long-term health condition as the dependent variable and as independent variables a full set of age and transformed year dummies. Birth cohort 1922–1929 is the reference. The light lines display 95% confidence intervals. Sources: HILDA data.

In Table 2 we present results for our two alternative health need proxies, (i) physical impairment in columns (1) and (2), and (ii) probable mental disorder in columns (3) and (4) for females and males respectively. Results show an increase in the prevalence of probable mental disorders among more recently born birth cohorts and at the same time a reduction in physical impairment prevalence rates. Particularly among females, the mental disorder estimates seem to be quite pronounced among the last two birth cohorts of our sample, those born between 1993 to 1999 and 2000 to 2006, with higher magnitudes than the reduction in physical impairment among those cohorts, which may explain the overall increase in long-term health conditions. The opposite is true for males, the reduction in physical impairment prevalence rates is larger than the corresponding increase in mental disorder prevalence rates. The number of observations is slightly reduced in these samples since not all respondents completed the self-completion questionnaire. Cohort effects are plotted in Fig. 2. Table 2. Age and cohort effects on physical impairment and probable mental disorder among male and female respondents in HILDA between 2001 and 2021Physical ImpairmentProbable Mental Disorder(1)(2)(3)(4)FemaleMaleFemaleMaleBirth yearReference group: Respondents born before 19301930–1936−0.018(0.022)−0.013(0.027)0.002(0.013)0.002(0.012)1937–1943−0.062*** (0.023)−0.059** (0.026)0.003(0.012)0.017 (0.012)1944–1950−0.102*** (0.022)−0.076*** (0.025)0.012(0.012)0.030** (0.012)1951–1957−0.106*** (0.022)−0.138*** (0.025)0.028** (0.012)0.027** (0.012)1958–1964−0.142*** (0.022)−0.139*** (0.025)0.040*** (0.013)0.032** (0.013)1965–1971−0.132*** (0.022)−0.155*** (0.025)0.051*** (0.013)0.042*** (0.013)1972–1978−0.143*** (0.022)−0.155*** (0.025)0.063*** (0.013)0.057*** (0.013)1979–1985−0.159*** (0.022)−0.168*** (0.025)0.076*** (0.014)0.071*** (0.014)1986–1992−0.156*** (0.022)−0.149*** (0.025)0.098*** (0.014)0.088*** (0.014)1993–1999−0.155*** (0.022)−0.156*** (0.026)0.166*** (0.015)0.106*** (0.015)2000–2006−0.155*** (0.023)−0.175*** (0.026)0.230*** (0.018)0.130*** (0.017)Age groupReference group: Respondents below 22 years22–28−0.011*** (0.003)−0.011*** (0.004)0.012** (0.005)0.022*** (0.004)29–35−0.007(0.005)−0.013*** (0.005)0.029*** (0.006)0.036*** (0.006)36–420.003(0.006)−0.000(0.006)0.041*** (0.007)0.045*** (0.007)43–490.042*** (0.007)0.015** (0.007)0.049*** (0.007)0.053*** (0.007)50–560.086*** (0.008)0.045*** (0.008)0.055*** (0.008)0.055*** (0.008)57–630.142*** (0.009)0.092*** (0.009)0.056*** (0.009)0.052*** (0.008)64–700.202*** (0.011)0.120*** (0.011)0.046*** (0.009)0.044*** (0.009)71–770.312*** (0.013)0.201*** (0.013)0.050*** (0.010)0.053*** (0.010)78–840.476*** (0.016)0.348*** (0.017)0.063*** (0.011)0.068*** (0.011)85–910.615*** (0.022)0.553*** (0.027)0.078*** (0.013)0.095*** (0.014)92 + 0.663*** (0.041)0.618*** (0.071)0.093*** (0.029)0.130*** (0.050)Constant0.262*** (0.022)0.272*** (0.025)0.038*** (0.014)0.013(0.014)Observations151,101132,770152,127133,993All regressions also include the transformed period effects, which are provided in the Appendix, Table A15. Standard errors in parentheses; * p < 0.10, ** p < 0.05, *** p < 0.01.Fig. 2. Plotted birth cohort effects of physical impairment and mental disorder prevalence rates for females and males after controlling for age and period. Notes: The Figures show the values of the cohort dummies plus the constant of our age, period cohort regression model, following Deaton and Paxson [11] with physical impairment and probable mental disorder as the dependent variables and as independent variables a full set of age and transformed year dummies. Birth cohort 1922–1929 is the reference. The light lines display 95% confidence intervals.Source: HILDA data

Future prevalence estimates

In Table 3, we compare predicted future prevalence rates of long-term health conditions among females and males using (i) age estimates, and (ii) age and cohort estimates, applied to population projections from the Australian Bureau of Statistics. We find that the projected number of people with long-term health conditions is lower among males and larger among females when using age and birth cohort estimates together, compared to when applying age effects only. While the difference seems small in relative size, with seven percent by 2028 for the female sample and one percent for the male sample, when multiplying this with population numbers, focusing on age-only estimates, would lead to an underestimation of 269,525 females and an overestimation of 36,478 males experiencing a long-term health condition by the year 2028. This increases for females and reduces for males by 2035. Table 3. Predicted prevalence rates of persons with long-term health conditions for 2021,2028 and 2035FemalesMales202120282035202120282035Population projections9,765,53410,857,30311,926,9719,356,18210,381,04611,383,295Estimates of persons with long-term health conditionAge effects only3,250,8673,704,1584,161,1592,943,6993,345,1433,728,761As % of total population0.3330.3410.3490.3150.3220.328Age and cohort effects3,357,0663,973,6834,605,4152,906,7743,308,6663,700,577As % of total population0.3440.3660.3860.3110.3190.325Difference in pop numbers106,199269,525444,256−36,925−36,478−28,184Difference as percentage0.0320.0730.107−0.013−0.011−0.001Source: Australian Bureau of Statistics, Population Projections, Australia, 2017–2066 and author’s calculation based on the HILDA data

Tables 4 and 5 show results for the predicted prevalence rates of persons with physical impairment and probable mental disorder respectively, comparing (i) age only with (ii) age and cohort estimates. Incorporating not only age but also cohort effects into future prevalence estimates results in a decrease in physical impairment estimates and an increase in mental disorder prevalence estimates. Prevalence rates of physical impairment are predicted to be lower if including cohort effects compared to focusing on age effects only, with a reduction of 10.3 percent points for females and 14.4 percent for males by 2035. At the same time, particularly for females, mental disorder prevalence rates are predicted to be larger when adjusting for cohort effects. Predictions for 2035 are 45.2 percent larger compared to when only using age effects, resulting in 741,899 more females who are expected to suffer from mental disorders by 2035. It is 27.4 percent, i.e. 351,721 males by 2035. Table 4. Predicted prevalence rates of persons with physical impairment for 2021,2028 and 2035FemalesMales202120282035202120282035Population projections9,765,53410,857,30311,926,9719,356,18210,381,04611,383,295Estimates of persons with physical impairmentAge effects only2,500,8812,872,5703,250,4751,990,2142,279,2692,562,861As % of total population0.2560.2650.2730.2130.2200.225Age and cohort effects2,367,7542,633,6922,915,1621,855,0282,024,7372,194,066As % of total population0.2420.2430.2440.1980.1950.193Difference in pop numbers−133,127−238,879−335,313−135,186−254,532−368,795Difference as percentage−0.053−0.083−0.103−0.068−0.112−0.144Source: Australian Bureau of Statistics, Population Projections, Australia, 2017–2066 and author’s calculation based on the HILDA dataTable 5Predicted prevalence rates of persons with probable mental disorder for 2021,2028 and 2035FemalesMales202120282035202120282035Population projections9,765,53410,857,30311,926,9719,356,18210,381,04611,383,295Estimates of persons with probable mental disorderAge effects only1,346,1351,492,9671,639,5791,036,5601,155,8481,281,574As % of total population0.1380.1380.1370.1110.1110.113Age and cohort effects1,454,4421,899,4362,381,4781,098,3601,359,7231,633,294As % of total population0.1490.1750.2000.1170.1310.143Difference in pop numbers108,307406,468741,89961,800203,876351,721Difference as percentage0.0800.2720.4520.0600.1760.274Source: Australian Bureau of Statistics, Population Projections, Australia, 2017–2066 and author’s calculation based on the HILDA data

Estimating future health service requirements: the case of mental disorders

Estimating future mental health service requirements for 2028 and 2035, we discuss the results of applying (i) age only, versus (ii) age- and cohort- prevalence predictions, to (iii) historical use, versus (iv) evidence-based care resource guidelines. Table A13 shows the age- and gender-specific prevalence of respondents who visited a psychiatrist or psychologist at least once in the previous 12 months based on data from the HILDA sample in columns (1) and (2) as well as data from the National Study of Mental Health and Wellbeing (NSMHW) in columns (3) and (4) [3]. Despite varying age groupings between the surveys, the use rates are very similar, providing confidence that we can apply the average number of consultations as historical use estimates.

In Table 6 columns (1)—(4) these historical use estimates are multiplied by the estimated number of people who are predicted to require mental health services for 2028 and 2025. Using age-only predictions, we estimate that in 2028, 1.5 million females and 1.2 million males are expected to visit a mental health professional. Applying the average number of consultations of 5.7 from the NSMHW this results in a total of 8.5 million and 6.6 million consultations, for females and males respectively. Applying guideline best care requirements, this number almost doubles to 14.9 and 11.6 million consultations in 2028. Table 6. Mental health service use predictions by age group, gender and year for constant use versus needs-based modelsAge-constant modelDynamic needs-based modelFemaleMaleFemaleMale(1)(2)(3)(4)(5)(6)(7)(8)Age groups20282035202820352028203520282035Estimated number of people who need care22–28198,314216,947155,843170,617382,951418,932233,764255,92629–35222,827227,558174,093178,638350,802456,654234,647278,05436–42215,808231,449162,130178,300261,630388,391209,477258,77243–49185,846214,358142,857162,795213,330279,270174,745223,84350–56162,391183,285122,514137,573180,950221,251140,497175,43757–63149,760153,848105,487110,314162,054180,249111,754133,25164–70123,369127,55487,14788,554130,751147,17787,14798,50471–77100,505115,57772,90583,27799,646126,44472,12986,82078–8478,14392,48557,20765,12670,08094,68760,47666,98785–9140,87957,73628,88442,05234,74752,78727,35243,96392 + 15,12618,78646,78264,32612,49515,6357,73511,736Total number of people1,492,9671,639,5791,155,8481,281,5741,899,4362,381,4781,359,7231,633,294Historical use estimatesTotal number of consults8,509,9129,345,6006,588,3347,304,97210,826,78513,574,4257,750,4219,309,776Guidelines best care estimatesTotal number of consults14,929,67016,395,79011,558,48012,815,74018,994,36023,814,78013,597,23016,332,940Difference in consultsHistorical use vs guidelines6,419,7587,050,1904,970,1465,510,7688,167,57510,240,3555,846,8097,023,164Age only vs age-cohort4,064,6907,418,9902,038,7503,517,200As share0.270.450.180.27

Using our age- and cohort- estimates of probable mental disorder prevalence rates, multiplied with population projections in columns (5) to (8), we estimate that in 2028, 1.9 million females and 1.4 million males will need to see a mental health professional. Applying historical use estimates that result in future service predictions of 10.8 and 7.8 million consultations, compared to 8.5 and 6.6 million if using age-only models, an increase of around 18 to 27 percent. Using current guidelines that recommend ten consultations with a mental health professional, this would result in service requirement estimates of 14.9 and 11.6 million consults when using age-constant models compared to 19.0 and 13.6 million consults using age-cohort models. An increase of 75 percent. We find that the difference between age only and age- and cohort estimates widens by 2035, leading to an underestimation of 45 percent for females and 27 percent for males, presented in columns (3)—(4) and (7)—(8).

Discussion and conclusion

In this paper, we outline the flaws of planning methods that rely on current or past health service use by age group and gender and project these estimates onto future population numbers when planning health service requirements in the future. Using Australian survey data, we provide evidence that this method results in a substantive over- or overestimation of health service need requirements, depending on the condition, compared to when using a method that integrates age and birth-cohort changes in health needs and combines these estimates with evidence-based care guidelines.

Our results show that the relationship between health needs and age is not constant across birth cohort, even when accounting for random effects and period effects. We find significant health differences across birth cohorts. While some conditions seem to be declining in age-specific incidence rates among more recently born birth cohorts, others are increasing. Most importantly, when comparing the predictions of our needs-based model to a model based on age-specific use only, we find that estimates differ substantially between methods and that age-constant use methods either over- or under-estimate future health needs, depending on the condition studied. The advantage of our needs-based approach is that it accounts for predicted changes in needs, not only between age groups but also between birth cohorts.

Our results are comparable to findings in Canada and England, where estimates between models that incorporated birth cohort changes in health needs compared to those that assume age constant need, differ substantially [21, 30]. However, while results in these countries indicated a decline of long-standing illnesses among more recently born birth cohorts, we find that the prevalence of long-term health conditions is increasing among females. One reason could be the way survey questions are framed and whether respondents identified mental health conditions as ‘long-standing illnesses’. In the case of HILDA, these are specifically classified as long-term health conditions and seem to be the driving factor in the prevalence increase. This is in line with recent findings that study generational differences in mental health trends in Australia [5]. Using the same dataset, the authors find that mental health is deteriorating among people born in the 1990 s and 1980 s and that mental health is not necessarily worsened by age.

There are a number of limitations to the approach taken. First, we applied evidence-based care guidelines to predict future health service needs. Guidelines often change and advancement in the delivery of care, e.g. through new technology or updated evidence, will need to be incorporated into future planning. In addition, although in a best-case scenario, those in need also take up service offerings, this is unlikely to be the case, and some services may be under-utilised in practice. Second, using responses from the SF-36 questionnaire to identify respondents who are likely to suffer from a mental health disorder has the disadvantage that QoL instruments are subject to reporting heterogeneity, particularly between generations, which may bias our cohort estimates. An analysis that corrects for generational differences in reporting of the latent mental health state would be a further innovation to make in the future. Third, while people with mental disorders also often suffer from co-morbidities, we use a simplified planning model for mental health services and neglect services to other health care professionals. Once planning for other services, such as GP visits or other specialists, controlling for co-morbidities will be important. Fourth, while we try our best to find a health need proxy that is independent of use, we recognize that QoL measures as well as self-reported long-term health conditions may potentially be influenced by previous health care use. In particular for mental health conditions, anti-depressant use may result in relatively high QoL scores, while at the same time, some people might not know they are in need of mental health services. Lastly, our approach relies on the availability of longitudinal, age-specific health data from nationally representative surveys. While such data are not yet available in every country, they are increasingly being collected, and comparable datasets already exist in several settings. In this context, elements of the needs-based planning approach have already been explored in countries such as Canada and the UK [21, 30].

Besides these limitations, our study has several strengths. Compared to previous studies in the field, we go one step further and provide an illustrative application to mental health services, estimating resource requirements in the future, using data from HILDA, and combining it with guidelines best care requirements, and population projections. We find that age-constant use models underestimate mental health service requirements by 2028 and 2035 substantially, whereas the needs-based models show that care needs will most likely be 18 to 45 percent times larger than predictions from traditional health service planning methods. While we provide a proof-of-concept case study using mental disorders to predict future mental health service requirements, we are not able to cover all or even several conditions in a single research paper. Ultimately, planners should repeat this estimation for each condition or illness and then aggregate the findings to plan for future health service and workforce requirements.

In summary, this paper highlights a critical limitation in current health service planning: the reliance on historical, age-specific service use that fails to account for generational shifts in health needs. By incorporating birth cohort effects into our model, we demonstrate that future health service demands, may be significantly over- or underestimated if planners assume stable age-specific patterns of need. Our approach offers a more responsive and forward-looking framework by integrating longitudinal trends and generational differences, allowing for more accurate projections of future service requirements. This model provides a valuable tool for policymakers and planners aiming to align health services with evolving population needs, ensuring that resource allocation keeps pace with demographic and epidemiological change.