Determination of Offset Values in Binary Regression Models to Adjust for Misclassification Errors

TL;DR

This paper introduces a method to correct biases in public health research caused by using inaccurate proxy measures instead of gold-standard data.

Contribution

The novel approach determines offset values in binary regression models to adjust for misclassification errors using validation samples.

Findings

Offset values eliminate biases in risk difference, relative risk, and odds ratio estimates.

Simulation studies verified unbiased point estimates and standard errors using the proposed method.

Abstract

Public health relevance—How does this work relate to a public health issue?
Use of unbiased gold-standard measures is limited in public health or epidemiologic research.Surrogates or proxy measures are more often used despite their vulnerability to measurement or misclassification errors.

Use of unbiased gold-standard measures is limited in public health or epidemiologic research.

Surrogates or proxy measures are more often used despite their vulnerability to measurement or misclassification errors.

Public health significance—Why is this work of significance to public health?
Misclassification errors induce biases in research results.Development of methods for corrections of biases pertinent to given circumstances is thus strived for.

Misclassification errors induce biases in research results.

Development of methods for corrections of biases pertinent to given circumstances is thus strived for.

Public health implications—What are the key implications or messages for practitioners, policy makers and/or researchers in public health?
Biased research results may misdirect public health policies.Careful assessment of the extent of misclassification errors is critical for unbiased results.

Biased research results may misdirect public health policies.

Careful assessment of the extent of misclassification errors is critical for unbiased results.

In randomized clinical trials and observational studies alike, it is difficult and challenging to collect gold-standard outcome measures for all participants. Although it would be ideal to use gold-standard measures, the costs and logistics of collecting them are often prohibitive. Therefore, surrogate or proxy measures or screening survey instruments are more often used to mitigate such difficulties, yet at the expense of misclassification errors and consequent biased statistical inferences. In this paper, when misclassification errors of proxy measures in comparison to a gold-standard measure are available through external or internal validation samples, we determined appropriate offset values in generalized binary regression models as a function of the proxy measure to eliminate biases of estimated effects in terms of risk difference, relative risk, and odds ratio that are incurred due to misclassification errors. Simulation studies were conducted to empirically demonstrate and verify the approach using appropriate offset values specific to each binary effect measure for estimating unbiased effects. Both point estimates of all effect measures and standard errors of regression coefficients obtained from the proposed offset-adjusted binary models were shown to be unbiased.

Full text

1. Introduction

Misclassification errors are inevitable in any research area, especially when gold-standard measures are not available or used [1]. Even if gold-standard measures may not be free of misclassification or misdiagnosis errors, depending on research areas, they are often expensive, time-consuming and logistically challenging to collect to an extent that many studies, large-scale studies in particular, may not be able to afford [2]. Therefore, less expensive, more convenient, and reliable proxy or surrogate measures, screening tools, or self-reported survey instrument responses are often instead used for research investigations.

Examples include the diagnosis of various diseases, such as cancer and major depressive disorder (MDD), to mention only a couple. For instance, a gold-standard diagnosis of prostate cancer can be made by biopsy guided by a transrectal ultrasound or MRI imaging [3,4]. A surrogate diagnosis could be made based on the prostate-specific antigen (PSA) level in the blood [5], which can serve as a screening tool. In addition, there are symptom-based self-administered survey instruments for prostate cancer diagnosis, such as the International Prostate Symptom Score (IPSS) [6]. For the diagnosis of MDD, a gold-standard diagnosis can be made through clinical interviews using the Structured Clinical Interview for DSM (SCID) [7,8]. Instead, the symptom-based Hamilton Depression Rating Scale (HDRS) [9] or Patient Health Questionnare-9 (PHQ-9) scale [10] is more often used. After all, none of these surrogate screening instruments may be free of misclassification errors compared with the gold-standard measures. Therefore, statistical inferences based on proxy or self-reported data will suffer biases in effect estimates of exposures, interventions, or treatments on study outcomes [11,12,13].

Numerous epidemiological and statistical approaches have been developed to adjust for misclassification and measurement errors, thereby mitigating potential biases in inferences, as introduced and compiled in several books [14,15,16,17,18]. This reflects the fact that measurement errors are difficult issues to appropriately handle and affect analysis results in broad research fields, including but not limited to epidemiology, public health, and statistics. We attempted to propose a relatively simple and straightforward method to adjust misclassification errors through building appropriate offset values in binary models to estimate unbiased effect sizes in terms of risk difference, relative risk, and odds ratio. The offset values specific to each effect measure will be determined based on the probabilities of misclassification errors of proxy measures compared to a gold-standard measure, which is often available through external or internal validation samples. We believe that the proposed approach to correcting biases through offset adjustment is novel and has not been previously studied. We presented a hypothetical randomized study example below, in which two groups were compared using several different measures of outcomes, and simulation studies were conducted to empirically verify the proposed approach in terms of unbiased point estimates and standard errors.

2. Materials and Methods

2.1. Setting and Notations

Suppose a randomized clinical trial is designed to compare adequate adherence to a medication for treating a disease between two study arms. The most rigorous and valid way to accurately measure and ascertain adherence would be to directly observe on a daily basis a participant’s ingestion of the medication in the presence of a healthcare professional treating the participant, which is called directly observed therapy (DOT). The daily DOT adherence measure could be considered a gold-standard measure. However, it would be extremely difficult and costly in many circumstances for both the patients and the health care professionals to ingest and observe, respectively, at the same time in a clinic or another appointed place on a daily basis.

To mitigate that kind of logistical burden and difficulties, the study considered dispensing medications in electronic devices that can capture and timestamp the retrieval of medications. For instance, an electronic blister pack containing one or two weeks worth of direct-acting antiviral (DAA) medications for treating hepatitis C virus (HCV) in seven or fourteen blisters can record the day and time when a blister containing a medication was opened up for retrieval of the medication inside [19]. Such blister packs have been used in several studies, including but not limited to SIMPLIFY [20], D3FEAT [21], PREVAIL [22], and HERO study [23]. Although this is also a rigorous way of measuring daily adherence and can serve as a close proxy measure of the DOT gold standard, the electronic blister packs are not able to record or validate whether the participant indeed ingested the retrieved medications. In addition, self-reported adherence was also collected to estimate the extent of validation of self-reported data, which would be the most convenient and least costly way of measuring adherence in this kind of study and beyond.

Suppose that the target binary outcome, i.e., adequate adherence, is declared if more than 75% days during the medication period are adherent to the medication regimen. Let us denote (1) the “true” adequate adherence by Y* when declared based on the directly observed ingestion of medications; (2) a proxy adequate adherence of Y* by Y when declared based on the electronic blister pack records; (3) a convenient proxy adequate adherence of Y* by Z when declared based on the self-reported adherence. An indicator variable representing the two arms is denoted by X.

2.2. Assumptions

All measures, Y*, Y and Z, in addition to the random assignment indicator X, are assumed to be scored 0 or 1 on a binary scale. We assume that (A1) P(Z|X) is hypothesized and therefore known; (A2) the study measured both Y and Z and therefore P(Y|Z) is known; (A3) P(Y*|Z) is known from external validation studies or from an internal small validation sample; (A4) conditional probabilities of Y given Z and Y* given Y are both independent of X, i.e., Y|Z⊥X and Y*|Y⊥X by virtue of random assignments of X; that is, P(Y|Z, X) = P(Y|Z) and P(Y*|Y, X) = P(Y*|Y), which is referred to as non-differential misclassification (with respect to X) [24].

Based on these assumptions, derivations of P(Y*|X) and P(Y|X) are possible as follows. For instance, P(Y|X) = P(Y|Z = 0, X)P(Z = 0|X) + P(Y|Z = 1, X)P(Z = 1|X) = P(Y|Z = 0)P(Z = 0|X) + P(Y|Z = 1)P(Z = 1|X) owing to the law of total probability for the first equation and to assumption A4 for the second. Once P(Y|X) is derived this way, P(Y*|X) can similarly be derived based on assumptions A3 and A4.

2.3. Effect Measures

The goal of the statistical inference is to build a statistical model concerning P(Y*|X) and thereby estimate the following “true” binary outcome effect measures:

$$[eqn]$$ $$[eqn]$$

and

$$[eqn]$$

2.4. Statistical Models

A generalized linear model for the true binary outcome Y* can be expressed in terms of X as follows:

$$[eqn]$$

where g(·) is called a link function. Specifically, g(·) = I(·) = ·, called the identity link function, for estimating RD = β_1.g_ = β_1.RD_, g(·) = log(·), called log-link function, for estimating RR = exp(β_1.g_) = exp(β_1.RR_), and g(·) = logit(·) = log(·/(1 − ·)), i.e., log-odds and called logit-link function, for estimating OR = exp(β_1.g_) = exp(β_1.OR_).

Given that Y* is not available for all participants in reality, we attempt to build a generalized linear model for P(Y|X) that will be equivalent to model (1) by constructing appropriate offset values specific to each effect measure above as a function of Z (i.e., free of both Y* and X). Specifically, we attempted to estimate β_1.g_ in model (1) in particular by estimating ξ_1.g_ in the following model for Y with offset_g_(Z) as a function of Z and a link function g(·),

$$[eqn]$$

where

$$[eqn]$$

We will compare the performances of this model with model (1) to assess bias of ξ_1.g_ or exp(ξ_1.g_) in comparison with β_1.g_ or exp(β_1.g_), in terms of RD, RR and OR when offset_g_(Z) is appropriately determined.

Model (1) will also be compared with the following models as well to assess bias of effect measures due to misclassification when Y or Z is regressed on X without any adjustment using offset values.

$$[eqn]$$

and

$$[eqn]$$

Similar to model (X), when g(·) = I(·), φ_1.g_ = φ_1.RD_ and η_1.g_ = η_1.RD_ for estimating RD; when g(·) = log(·), exp(φ_1.g_) = exp(φ_1.RR_) and exp(η_1.g_) = exp(η_1.RR_) for estimating RR; and when g(·) = logit(·), exp(φ_1.g_) = exp(φ_1.OR_) and exp(η_1.g_) = exp(η_1.OR_) for estimating OR. In the following, let us denote π_0_* = P(Y* = 1|Z = 0), π_1_* = P(Y* = 1|Z = 1), π_0_ = P(Y = 1|Z = 0) and π_1_ = P(Y = 1|Z = 1).

2.5. Derivations of Offset Values

2.5.1. Offset for RD Model

For estimating the “true” RD = β_1.RD_, when g(·) = I(·), the offset(Z) can be determined by taking the following steps. First, we consider the following identity link model:

$$[eqn]$$

Second, we consider identity link-specific correction factors, ζ_0.RD_ and ζ_1.RD_, depending on Z, so that P(Y* = 1|Z = 0) + ζ_0.RD_ = δ_0.RD_ + ζ_0.RD_ = P(Y = 1|Z = 0) and P(Y* = 1|Z = 1) + ζ_1.RD_ = δ_0.RD_ + δ_1.RD_ + ζ_1.RD_ = P(Y = 1|Z = 1). It follows that

$$[eqn]$$

and

$$[eqn]$$

Therefore, if we determine the identity link-specific offset as

$$[eqn]$$

in the following model

$$[eqn]$$

then we will have γ_0.RD_ = δ_0.RD_ and γ_1.RD_ = δ_1.RD_. Therefore, if we fit the following model,

$$[eqn]$$

then we will have ξ_0.RD_ = β_1.RD_ and ξ_1.RD_ = β_1.RD_.

2.5.2. Offset for RR Model

To estimate the “true” RR = exp(β_1.RR_), when g(·) = log(·), first, we consider the following model:

$$[eqn]$$

Under this log-link model, we consider log-link-specific correction factors ζ_0.RR_ and ζ_1.RR_, depending on Z so that log(P(Y* = 1|Z = 0)) + ζ_0.RR_ = δ_0.RR_ + ζ_0.RR_ = log(P(Y = 1|Z = 0)) and log(P(Y* = 1|Z = 1)) + ζ_1.RR_ = δ_0.RR_ + δ_1.RR_ + ζ_1.RR_ = log(P(Y = 1|Z = 1)). It follows that

$$[eqn]$$

and

$$[eqn]$$

Therefore, if we determine the log-link specific offset as

$$[eqn]$$

in the following model

$$[eqn]$$

then we will have γ_0.RR_ = δ_0.RR_ and γ_1.RR_ = δ_1.RR_. Therefore, if we fit the following model,

$$[eqn]$$

then we will have ξ_0.RR_ = β_1.RR_ and ξ_1.RR_ = β_1.RR_.

2.5.3. Offset for OR Model

To estimate the “true” OR = exp(β_1.OR_), when g(·) = logit(·), we first consider the following logit-link model

$$[eqn]$$

Under this model, we consider logit-link specific correction factors ζ_0.OR_ and ζ_1.OR_, depending on Z, so that logit(P(Y* = 1|Z = 0)) + ζ_0.OR_ = δ_0.OR_ + ζ_0.OR_ = logit(P(Y = 1|Z = 0)) and logit(P(Y* = 1|Z = 1)) + ζ_1.OR_ = δ_0.OR_ + δ_1.OR_ + ζ_1.OR_ = logit(P(Y = 1|Z = 1)). It follows that

$$[eqn]$$

and

$$[eqn]$$

Therefore, if we determine the log-link specific offset as

$$[eqn]$$

in the following model

$$[eqn]$$

then we will have γ_0.OR_ = δ_0.OR_ and γ_1.OR_ = δ_1.OR_. Therefore, if we fit the following model,

$$[eqn]$$

then we will have ξ_0.OR_ = β_1.OR_ and ξ_1.OR_ = β_1.OR_. All of these models, regression equations, coefficients, offset functions, and notations are summarized in Table 1 below.

2.6. Simulation Studies

We consider 8 scenarios for simulation evaluations of biases of model (2), in particular, in comparison with model (1), with prespecified probability presented in Table 2, which also includes derived probabilities and true effect measures thereof, following the steps described in the Section 2.2 above. For each scenario, we ran 1000 (i = 1 to 1000) simulations with N = 500 (j = 1 to 500) samples for each simulation. For the i-simulation for any scenario, we randomly generated variables in the following sequence: (1) Xij for the j-th participant with a 1:1 or 2:1 ratio between X = 1 and X = 0; (2) Zij based on Xij and the prespecified P(Z|X); (3) Yij based on Zij and the prespecified P(Y|Z); and (4) Y**ij* based on Yij and the prespecified P(Y*|Y). With observed data simulated in this way, we estimated observed probabilities of π_0i, π_1i*, π_0i_ and π_1i_ for the i-th simulation based on estimates from N = 500 observations and then determined offset_g_(Z)i* = ζ_0.gi(1 − Z) + ζ_1.giZ, following Equations (6) for RD, (7) for RR and (8) for OR. We note that in practice, both π_0i* and π_1i* would be estimated based on a smaller sample size than N = 500 for internal validation, but here we used all observations to have more accurate estimates for verification purposes. Finally, we fitted model (2) with the observed offsets for the i-th simulation. Let us denote estimated effect measures from the i-th simulated data as follows: f(ξ_1.RDi) = ξ_1.RDi for RDi, f(ξ_1.RRi) = exp(ξ_1.RRi) for RRi and f(ξ_1.ORi) = exp(ξ_1.ORi) for ORi. In addition, we also fitted the “true” model (1), and we denote the estimated standard errors obtained from models (1) and (2) by SE(β_1.gi) and SE(ξ_1.gi), respectively, for the i-th simulated data set. SAS v9.4 (SAS Inc. Cary, NC, USA) simulation code with proc genmod is provided in the Supplementary File.

2.7. Expected and Empirical Bias and %Bias

The expected bias of the effect measures estimated based on models (4) and (5) was calculated in comparison to the “true” effect measures in model (1) that are presented in Table 2. For instance, biases were calculated as bias(f(φ_1.g_)) = f(φ_1.g_) − f(β_1.g_) and bias(f(η_1.g_)) = f(η_1.g_) − f(β_1.g_), where f(·) = I(·) for RR and f(·) = exp(·) for both RR and OR. Subsequently, their corresponding %biases were calculated as %bias(f(φ_1.g_)) = bias(f(φ_1.g_))/f(β_1.g_) × 100%, and %bias(f(η_1.g_)) = bias(f(η_1.g_))/f(β_1.g_) × 100%.

As for the simulation-based empirical biases of effect measures of model (2), we first obtained an average of 1000 estimated f(ξ_1.gi), say Avg(f(ξ_1.gi)), and then bias(f(ξ_1.gi)) was calculated as bias(f(ξ_1.g)) = Avg(f(ξ_1.gi)) − f(β_1.g) and %bias(f(ξ_1.g_)) as %bias(f(ξ_1.g_)) = bias(f(ξ_1.g_))/f(β_1.g_) × 100%. We also compared the empirical standard errors (SEs) of the regression coefficients for models (1) and (2), since the SEs were not theoretically derived in closed forms, and thus we estimated them from the simulated data. The biases of SE(ξ_1.g_) in comparison to SE(β_1.g_) were defined as bias(SE(ξ_1.g_)) = Avg(SE(ξ_1.gi)) − Avg(SE(β_1.gi)) and %bias as bias(SE(ξ_1.g_)) = bias(SE(ξ_1.gi))/Avg(SE(β_1.gi)) × 100%.

3. Results

3.1. Expected Bias of Models Without Offset Adjustment

Table 3 presents the extents of the biases of effects expected from models fitting Z (4) or Y (5) with X alone in comparison to the model fitting Y* with X, the “true” model (1). The effect measures—RD, RR, and OR—were computed based on the prespecified probability, P(Z = 1|X), and derived probabilities. P(Y = 1|X) and P(Y* = 1|X), as presented in Table 2. Across all eight scenarios, compared to the true effect measures in Table 1, %biases of expected effect measures from models fitting Z with X were much greater than those fitting Y with X. Regardless of models and scenarios, however, the sizes of %bias were substantially large, and the %bias was greatest for RD followed by OR and RR, which had the smallest %bias. Specifically, for model (4), the range of %bias RD was 108.3–316.7%, that of RR was 32.7–52.0% and that of OR was 53.1–162.2%. For model (5), the range of %bias RD was 25.0–66.7%, that of RR was 5.0–9.5% and that of OR was 10.2–21.5%. It follows that all effect measures calculated based on models (4) and (5) markedly overestimated the true effect measures calculated based on model (1).

3.2. Empirical Bias of Estimated Effect Measures from Models with Offset Adjustment Assessed by Simulations

Table 4 presents simulation-based empirical %biases of effect measures estimated based on model (2) with offset adjustment (3). Across all scenarios, the empirically estimated RD = ξ_1.RD_ tended to overestimate the true RD = β_1.RD_, with %bias ranging from 5.5% to 13.6%. However, the sizes are much smaller by far than the %biases of RDs based on models (4) and (5) presented above in Table 3. In contrast, the %biases of both the empirically estimated RR = exp(ξ_1.RR_) and OR = exp(ξ_1.OR_) are negligible without clear direction of over- or underestimation of the true effect measures of RR = exp(β_1.RR_) and OR = exp(β_1.OR_) obtained from model (1), respectively, ranging from −4.0% to 3.8% for the former and −2.2% to 0.9% for the latter. These %biases are again much smaller compared to the %biases of RRs and ORs based on models (4) and (5), presented above in Table 3.

3.3. Empirical Bias of Standard Errors Estimated from Model with Offset (2) in Comparison to Those from the True Model (1)

Table 5 presents simulation-based empirical %biases of SE(ξ_1.g_) of regression coefficients of model (2) with offset adjustment (3). Across all scenarios, the empirically estimated SE(ξ_1.RD_) and SE(ξ_1.OR_) tended to overestimate, albeit negligibly, the SE(β_1.RD_) and SE(β_1.OR_) of the true model (1). However, the direction of the bias(SE(ξ_1.RR_)) depended on the scenario, but the extent of the bias was similarly almost ignorable.

4. Discussion

We demonstrated that appropriate determinations of offsets reduced or eliminated biases of binary effect measures—RD, RR, and OR—that incur when proxy or surrogate measures with misclassification errors are modeled and fitted without adjustment of offsets. Although the %bias of RD was not completely eliminated, its absolute bias was <0.018 across all scenarios. On the other hand, %biases of RR and OR were very close to zero, yielding unbiased estimates. The estimated standard errors are also shown to be little biased, regardless of the effect measure. Collectively, inferences about or tests of the significance of any effect measures based on the estimates and standard errors obtained from model (2) would be valid with little bias, regardless of allocation ratios.

Although model (2) was simple, many simulations failed to fit with a log link for RR estimation. Nonetheless, unbiased estimates were yielded based on averages of fitted simulations, as shown in Table 4. The number of failed fits depends on the simulation scenarios (ranging from 5% to 81%), but the underlying reasons are unknown. However, there was no failed fit for estimation of RD despite non-trivial empirical %biases, but their absolute biases appear negligible. No failed fit was observed for the estimation of OR with a logit link, and model (2) yielded unbiased estimates. Therefore, model (2) with the canonical logit-link function would result in the most reliable unbiased estimates of OR in view of the simulation results.

Our approach, while addressing a specific situation with both binary outcomes and exposures, is distinctive from existing approaches such as regression calibration [25], the Berkson error model [26,27], and diverse Bayesian approaches [16]. While these methods may be applicable to broader situations, including studies with continuous multiple covariates, their measurement error assumptions often assume a distributional form for the exposure variable X, whereas our approach addresses misclassification of binary outcome measures. Regardless, to the best of our knowledge, no existing methods have used offset approaches.

It would be more practically useful to determine appropriate offsets when only Z or Y (but not both) is available in studies, and when internal or external validation of either P(Y*|Z) or P(Y*|Y) is available. However, in either case, it is not possible to determine offsets similarly to model (2) and offset values (3). It is fundamentally because an offset must be defined as a function X, which requires “direct” knowledge of P(Y*|X) at every individual level. However, if this is the case, model (1) can be fitted without any adjustment with the offset to estimate “true” binary effect measures.

Although the non-differential misclassification assumption (A4) in particular would be acceptable for randomized studies, it might be relatively strong and unacceptable for observational studies in general unless an instrumental variable mimicking randomization is available, which could be achieved through a Mendelian randomization approach [28] if genetic information is available. In practice, therefore, careful assessments of the validity of assumption (A4) should be carried out to determine and apply the proposed processes for determining offset values to reduce biases. Nonetheless, assumption (4) enables investigators at a study design stage to assess the extent of potential biases based on derived probabilities as presented in Table 1 and Table 2 with hypothesized probabilities of misclassification errors, P(Y*|Y) and P(Y|Z). In addition, the availability of unbiased estimates of both π_0_* and π_1_* would be crucial for obtaining unbiased estimates of effect measures by including offsets in a generalized linear model. However, if the validation sample size is relatively small, the estimates of π_0_* and π_1_* would have wider confidence intervals even if their point estimates are unbiased. In this case, it would be worth conducting sensitivity analyses with varying estimates of π_0_* and π_1_* within the confidence interval. As long as the non-differential assumption holds, the offset-adjusted models will yield unbiased estimates of the effects and standard errors of the regression coefficients corresponding to each of the varying estimates.

There are several potential extensions that deserve further study. First, when the non-differential misclassification assumption (A4) is violated, its impact on estimates from the offset-adjusted model (2) can be examined. If the impact is sizable, depending on the extent of the violation or deviation, it would be worth examining whether further adjusted offset functions specific to the combination of at least X and Z would yield unbiased estimates. In addition, estimates of the conditional probabilities, such as P(Y*|X, Z), would be required, perhaps again based on external or internal validation samples. Second, it will be helpful to extend the approaches to determining offset values as a future study when covariates (let us say W collectively) should be included in model (1) and (2) adjustments. For instance, it should be examined what kind of assumptions concerning relationships of W with other measures should be made, whether offsets should be a function of W as well, and lastly, whether it would ever be possible to determine appropriate offsets in the presence of W in a model. Third, extension of the proposed approach to studies that compare more than two groups would also represent an interesting future study and so would be the extension of the proposed approach to ordinal or nominal outcomes with >2 categories. Fourth, it is unknown in this study why RD shows a larger % bias. Therefore, large- or finite-sample theoretical derivations would be needed to derive asymptotic or finite-sample properties of the RD (and RR and OR as well) estimate under the offset-adjusted models to examine this issue. Lastly, applying the proposed methods to real data analysis would be more illustrative.

5. Conclusions

In conclusion, biases of binary effect measures can be substantial when misclassification errors are unavoidable. However, the biases could be reduced or eliminated by including appropriate offset values in generalized binary regression models. The proposed approach also yields unbiased standard errors of regression coefficients and has potential applicability across diverse areas of the health sciences and beyond, helping mitigate concerns about inferential biases and the significance testing of estimated effects of interventions, treatments, or exposures due to misclassification errors in binary outcomes.