15

An important aspect of study design and analysis is sample size. An appropriate sample size achieves certain desirable properties for statistical inferences including descriptive (one-sample) and comparative (two-sample) studies. These can either be for the purpose of estimation of unknown parameters or testing. Table 15.1 briefly reports descriptive and comparative quantities of interest which are suitable for sample size calculations. Sample size is influenced by three major factors: (1) design features, (2) analysis features, and (3) logistics as they shape the problem at hand. Analysis is the focus in this chapter, as design has been previously discussed and logistics are realities that should be incorporated into design and, to an extent, analysis. Table 15.2 provides a brief summary of the three factors.

Table 15.1 Categorization of various quantities of interest, suitable for sample size calculations

	Descriptive	Comparative
Binary	Proportion	RD, OR, RR, HR, etc.
Continuous	Mean	Mean difference, correlation coefficient, etc.

Abbreviations: RD, risk difference; OR, odds ratio; RR, relative risk; HR, hazard ratio.

The motivation for sample size calculations is often to achieve one of two major goals: avoid exceeding a maximum acceptable (as scientifically meaningful) width for a confidence interval, or achieve a predefined power for a study. Intuitively, one should expect that larger sample sizes provide more information and, thus, yield better estimates (narrower confidence intervals) leading to correct decisions in hypothesis tests (adequate power). One enhances the accuracy of estimation through confidence intervals, and the other increases the study power. In either case, the study could be descriptive (one sample) or comparative (two or more groups). Each of these features in various combinations is discussed with examples throughout the chapter.

Sample Size Estimation Based on Confidence Intervals

The first sample size calculations described are those relating to confidence intervals. The general form of a confidence interval for the true mean is given by the following formula:

where is the sample mean, is the (positive) critical value from the normal distribution corresponding to the 100 (1 − α)% confidence interval, σ is the population standard deviation, and n is the size of the sample. From this formula one can tell that the interval is centered around , and extends a length in both directions from the center, giving a width of . The width decreases with increasing values of n. Instead of discussing the full width of the interval, it is common to discuss just the half-width of intervals, , how far the interval extends from the center, and is called the margin of error. Therefore, the confidence interval can then be written as

Table 15.2 Factors influencing sample size

Design	Analysis	Logistics
• Cross-sectional/cohort/case-control	• Scale of outcome	• Drop-out
• Descriptive/analytical	• Effect measure	• Missing data
• Parallel/cross-over	• Confidence interval	• Funding
• Repeated measures	• Test of hypothesis	• Nature of disease
• Bioequivalence	• Directional vs. nondirectional	• Scope of survey
• Adjustment for confounders

Both the margin of error and the width of the interval decrease proportionally with , as illustrated in Fig. 15.1.

Note that this interval width decreases toward 0 as the size of the study sample increases. If m represents the margin of error of the confidence interval, one can very simply rearrange the formula to obtain

Here, the impact of various factors on sample size is explicit. In particular, to achieving a small margin of error, m, requires a large sample size, n. There are two other major components which impact the width of a confidence interval: the total remaining tail area outside of the interval which defines the confidence interval, α, and the standard error (when the central limit theorem is used). The standard error itself is obviously impacted by the standard deviation of the population, σ, and the sample size, n.

Fig. 15.1 Relationship between confidence interval width and sample size for a 95% confidence interval and known unit variance.

Unlike the hypothesis testing approach, the confidence intervals approach is agnostic to hypotheses. It acts as a descriptive quantity with a particular level of precision. As an example, investigators may want to know how large a study’s sample should be so that the sample mean has, on average, less than a 10% chance of being 3 units away from the true population mean. In this case, the half-width of the confidence interval is 3 units, so the full width of the interval is 6 units, and α = 0.1 for a 90% confidence interval. Missing here is an estimate of the population’s standard deviation. This may be estimated through a pilot study. The larger the estimate of this standard deviation, the larger the sample size required to ensure that there is less than a 10% chance of the sample mean being 3 units away from the true population mean. That is, a larger sample size is required to ensure that the confidence interval’s width does not grow beyond 6 units. Thus, one may think of the confidence interval of the predefined width as now having a smaller value of α, which means that there is an even smaller chance the sample mean strayed from the true mean than what was originally expected. Typically investigators consider conservative approaches for estimation of sample size, preferring to err on the side of using more resources (larger samples) to ensure that certainty and precision of study results are not compromised.

Sample Size Estimation Based on Hypothesis Tests (One-sided)

Hypothesis testing can be another focus for estimating an appropriate sample size for a study. Recall that power is the probability of rejecting the null hypothesis when it is indeed false, and so it is desirable for this to be as close to 100% as much as possible. If investigators use the normal distribution and a predefined power that they were interested in achieving for their study, then the formula for sample size is

where Z_1−β is the critical value associated with the power 1 − β (β is the probability of making a type 2 error), and μ₁ is the “true” mean used to estimate the detectable effect size of interest from the hypothesized mean, μ₁ − μ₀, and other symbols denote the same quantities as in confidence intervals. Here, one sees that the sample size is influenced as before by the variance of the population, σ², namely larger variances demand larger sample sizes, but also by the effect size of interest to detect, and the power. The larger the effect size of interest, the easier it should be to detect, so larger effect sizes decrease the required sample size. Finally, larger values of power correspond to larger values (in magnitude) of Z_β so this extra certainty in testing requires greater sample size. Fig. 15.2 illustrates the sample size required for a one-tailed test where the null hypothesis is that the true mean, μ, is equal to the hypothesized mean, μ₀: μ = μ₀ when in truth μ = μ₁ ≠ μ₀ for some μ₁.

Unlike the one-sided hypothesis test, there is no formula in the form n = … for sample size for two-sided tests. Instead, numerical approximations calculated by software are used. These will be investigated directly within examples and exercises.

Variants of formulae for estimating appropriate sample size scenarios are discussed in this chapter, including when standard deviation is estimated rather than known for continuous measurements (using the t distribution rather than a normal), when the outcome is binary, and performing inference on proportions (using the normal distribution suitable approximations when proportions of interest are not near 0 or 1). Other variants of formulae for sample size estimation arise by making adjustments in the estimate for standard error. The number of variants beyond a simple case is too large to cover exhaustively in this chapter, and usually involve calculations that are tedious, although not difficult. For these and other scenarios, existing software is particularity useful. In this chapter some of the formulae and theory behind these concepts are omitted in favor of illustrating solutions provided by R functions.

Diagnostic Test Performance Sample Size Calculations

For sample size estimations of the area-under-the-curve (AUC) and reliability of diagnostic test readings, one should refer to the results of existing methodological work. Because of the more involved calculations, approximations are often used for such estimations. This chapter illustrates approaches that are implemented in R as well as approaches that use tables. Some of these tables, and the means of constructing tables, have been described elsewhere.1

Fig. 15.2 Relationship between sample size and power.

Reliability can be measured by different ways, and so the sample size calculations for each can differ. For example, hypothesis testing on AUCs can be used to assess whether two raters are indicating that the same images are abnormal. The appropriate hypothesis test for this scenario is the Wilcoxon rank sum test,2,3 measuring if one rater ranks images “higher” than another rater, beyond what is expected due to chance. In an example in musculoskeletal imaging, the raters may assess the same images and assign a score to each case according to the amount of cartilage degeneration in the joint. If classifying a set of images to only two categories (e.g., “normal” or “abnormal”), two raters may agree by chance and this should be incorporated into calculations. The interrater agreement between two raters when rating images dichotomously can be measured by Cohen’s kappa coefficient,⁴ which achieves a value of 1 if the two raters completely agree and a value of 0 if the raters only agree as much as expected simply due to chance. If there are more than two possible classifications, say raters are rating the severity on something more like a Likert scale, the intraclass correlation coefficient (ICC) is suitable. The ICC is a ratio (proportion) of variances: the variance between images compared to the total variance, and is the same idea as an ANOVA. If this ratio is close to 1, then most of the variation in ratings is because of the variety of images, meaning that very little is due to variation between raters. The ICC, being a proportion, lies between 0 and 1, but has more sophisticated calculations for sample size as this measures more than just a simple frequency. When incorporating power the numerical methods are also difficult. Although exact methods have been found⁵ a useful approximation may be more feasible in practice and only requires reading off published tables.¹ For reliability measured with kappa or ICC, software and tables are referred over formulae.

Examples Estimating Sample Size

Only gold members can continue reading. Log In or Register to continue

Share this:

• Click to share on Twitter (Opens in new window)
• Click to share on Facebook (Opens in new window)

Related posts:

Obtaining Ethics Approval, Data Management, and Budget Formulation Steps for Using Research as a Success Tool in Academia Decision Analysis Descriptive Statistics Linear and Logistic Regression Measurements: Validity, Reliability, and Responsiveness

Stay updated, free articles. Join our Telegram channel

Join