So What?

Abstract

Figures

Logistic fit plot of the incidence of instrumental delivery (blue line) associated with the duration of epidural labor analgesia in 1,231 parturients undergoing a trial of labor. Chi-square=6.5, P=0.0110. hr, hour.

Tables

Table.. Adjusted Risk Differences for Body Mass Index and Duration of Labor Analgesia and the Risk of Instrumental Delivery in 1,231 Parturients

Term	Standardized Risk Difference	95% CL	Std Error	Log-Ratio Chi-Square	P Value
Intercept	0.09	0.01 to 0.17	0.04	5.4	0.0207
Delivery body mass index	–0.001	–0.004 to 0.001	0.002	1.0	0.3261
Duration of labor analgesia	0.004	0.001 to 0.007	0.002	5.2	0.0221

Full text

INTRODUCTION

Historically, the statistical approaches we use to assess the results from clinical studies have been based upon the concept of null hypothesis significance testing.^1^ The frequentist tests—t tests, chi-square tests, analysis of variance, linear and logistic regression analyses—are used with the assumption that the null hypothesis is true (no difference observed between the parameters of interest). This assumption then allows selection of the alternate hypothesis once the calculation of the frequentist test statistic falls below a preassigned cut point, usually P <0.05.^2^ However, these tests can be misleading in suggesting a clinical effect, as frequentist tests do not provide 2 important pieces of information: the magnitude of the effect of the intervention and the precision of that effect.^1,3^

Clinicians want to apply the best information obtained from clinical studies. However, when medical researchers only use frequentist tests to investigate their results, statistically significant results may or may not have clinical importance.^3^ Rather than using frequentist analyses, researchers should examine the degree of clinical difference with measures of effect size—risk or proportion differences in bivariate analyses and adjusted or standardized risk differences in multivariable analyses—and then determine if those differences are clinically important.^3-5^ We demonstrate the test differences on the same dataset used in Chai et al^6^ with the use of additional data collected but not reported in the main article.

EXAMPLE

Concerns have long been expressed that epidural analgesia may delay the process of maternal labor, leading to an increase in the incidence of instrumental delivery including the need for cesarean.^7-9^ Over the years, a low-dose epidural analgesia technique^10-13^ has been developed and was used for patients included in the study by Chai et al.^6^ We examined the association of the duration of epidural labor analgesia (in hours) to the incidence of instrumental delivery, first using frequentist testing and then using risk differences to analyze the same data.^4,5^

Chi-square analysis with the classical statistical significance cut point of P <0.05^2^ showed that the duration of epidural labor analgesia was statistically associated with the incidence of instrumental delivery (chi-square=6.5, P=0.0110). The blue incidence line in the Figure increases

during the time period of interest and leads us to suspect a clinical association exists because the P value is less than the traditional cut point of 0.05.^2^

Logistic fit plot of the incidence of instrumental delivery (blue line) associated with the duration of epidural labor analgesia in 1,231 parturients undergoing a trial of labor. Chi-square=6.5, P=0.0110. hr, hour.

Using the same data set, we calculated the risk difference of the magnitude and precision of the duration of epidural labor analgesia with the incidence of instrumental delivery and obtained a mean risk value of 0.3% (CI 0.04%-0.6%) increase per hour of labor. As an example, the mean duration of labor was 8.5 hours, and the incidence of instrumental delivery increased from a baseline of 6.5% after the initial 2 hours of labor to an incidence of instrumental delivery of 8.7% after 10.5 hours of labor. This 2.2% change allows clinicians to interpret the importance of this association.

Additionally, we can investigate the interactions of additional clinical variables of interest by readjusting or standardizing their risk differences.^14^ We added delivery body mass index (BMI) to the analysis as it was the chief independent predictor of interest in Chai et al.^6^ The readjusted risk differences are shown in the Table. The addition of BMI to the model now increased the incidence of instrumental delivery to 0.4% (CI 0.1%-0.7%) per hour of labor, a minimal additive effect. The determination of whether this calculated effect size is clinically relevant depends upon the experience and professional practice of the clinician, as it should be.^4,5^

CONCLUSION

Although P values obtained from frequentist tests may suggest a clinical effect, the value does not reveal the magnitude or the precision of that effect. The use of measures of effect size can quantify this clinical influence. Properly conducted research studies will improve our delivery of health care when they answer the clinically important question: “So what?”