Effect Size Meta-analysis

Case-control studies of continuous outcomes (e.g. serum creatinine) may be investigated with respect to overall size of effect of an intervention. Meta-analysis may be used to investigate the combination or interaction of a group of independent studies, for example a series of effect sizes from similar studies conducted at different centres. This StatsDirect function examines the effect size within each stratum and across all of the studies/strata.

There are a number of statistical methods for estimating effect size, StatsDirect uses g (modified Glass statistic with pooled sample standard deviation) and the unbiased estimator d (Hedges and Olkin, 1985):

- where ne is the number in the experimental group, nc is the number in the control group, μe is the sample mean of the experimental group, μc is the sample mean of the control group, σe is the sample standard deviation for the experimental group, σc is the sample standard deviation for the control group, N = ne + nc, J(m) is the correction factor given m and Γ is the gamma function.

For each study StatsDirect gives g with an exact confidence interval and d with an approximate confidence interval. An iterative method based on the non-central t distribution is used to construct the confidence interval for g (Hedges and Olkin, 1985).

The pooled mean effect size estimate (d+) is calculated using direct weights defined as the inverse of the variance of d for each study/stratum. An approximate confidence interval for d+ is given with a chi-square statistic and probability of this pooled effect size being equal to zero (Hedges and Olkin, 1985).

StatsDirect also gives the option to base effect size calculations on weighted mean difference (a non-standardized estimate unlike g and d) as described in the Cochrane Collaboration Handbook (Mulrow and Oxman, 1996).

The inconsistency of results across studies is summarised in the I² statistic, which is the percentage of variation across studies that is due to heterogeneity rather than chance – see the heterogeneity section for more information.

Please note that the results from StatsDirect may be slightly different from the results you obtain using other packages or from those quoted in papers; this is due to the use of exact bias correction calculated from the gamma distribution in StatsDirect.

DATA INPUT:

You may enter number, mean and standard deviation for control and experimental groups of each study. Alternatively you may just enter numbers in experimental groups, numbers in control groups and effect size g (nb. please make sure you use g only as defined above!).

Example

From personal communication from Dr N. Freemantle.

The following data represent test outcomes for six studies in which an educational intervention was investigated:

 Experimental Control Trial N Mean SD N Mean SD Kottke 27 18.50 14.90 17 5.40 17.30 Levinson 16 60.00 29.20 15 55.00 29.20 oliver intensive 25 10.72 6.46 66 6.92 6.83 oliver standard 62 9.20 6.16 66 6.92 6.83 sulmasy 9 3.75 2.55 22 1.05 2.12 white 63 60.00 13.30 40 46.30 18.60 Wilson 23 9.24 5.35 23 5.33 4.48

To analyse these data in StatsDirect first prepare them in four workbook columns and label these columns appropriately. Alternatively, open the test workbook using the file open function of the file menu. Then select effect size from the meta-analysis section of the analysis menu, select the option to use mean, n and sd, and then select the columns ’Exptal. number’, ’Exptal. mean’, Exptal. SD’, ’Control number’, ’Control mean’, ’Control SD’ and ’Trial’ as prompted.

For this example:

 Study J(N-2) g Exact 95% CI 1 0.982 0.8261 0.19 1.453 kottke 2 0.9739 0.1712 -0.536 0.8759 levinson 3 0.9915 0.5644 0.0954 1.0303 oliver intensive 4 0.994 0.35 2.945E-17 0.6986 oliver standard 5 0.9739 1.2017 0.3581 2.0267 sulmasy 6 0.9925 0.8804 0.4639 1.2928 white 7 0.9828 0.7924 0.187 1.3893 wilson

 Study N (exptal.) N (control) d Approximate 95% CI 1 27 17 0.8113 0.1812 1.4413 kottke 2 16 15 0.1668 -0.5389 0.8724 levinson 3 25 66 0.5597 0.0923 1.0271 oliver intensive 4 62 66 0.3479 -0.0013 0.6972 oliver standard 5 9 22 1.1703 0.3419 1.9987 sulmasy 6 63 40 0.8738 0.46 1.2876 white 7 23 23 0.7788 0.1794 1.3783 wilson

Fixed effects (Hedges-Olkin)

Pooled effect size d+ = 0.612354 (95% CI = 0.421251 to 0.803457)

Z (test d+ differs from 0) = 6.280333 P < 0.0001

Non-combinability of studies

Cochran Q = 7.737692 (df = 6) P = 0.258

Moment-based estimate of between studies variance = 0.020397

I² (inconsistency) = 22.5% (95% CI = 0% to 67.1%)

Random effects (DerSimonian-Laird)

Pooled d+ = 0.627768 (95% CI = 0.403026 to 0.85251)

Z (test d+ differs from 0) = 5.474734 P < 0.0001

Bias indicators

Begg-Mazumdar: Kendall's tau = 0.238095 P = 0.5619 (low power)

Egger: bias = 1.439076 (95% CI = -2.5809 to 5.459051) P = 0.3997

Here we can say with 95% confidence, assuming a random effects model, that the true size of the effect was at least 0.4 greater for the group who received the educational intervention compared with those who did not. Assuming a fixed effects model a slightly stronger inference could be made about an effect size of 0.42 (the lower confidence limit) but the high inter-study variation makes the fixed effects model less appropriate.