With this concept, we can then calculate the sensitivity and specificity of the technique. The sensitivity is the true-positive rate, and the specificity is the true-negative rate. A ROC curve is a presentation of sensitivity v 1-specificity or false-positive rate. A perfect test will have a 100 % true-positive and true-negative value. A test which has a poor discrimination will have 50 % false results and 50 % true results. This is seen as the line at 45° on the ROC curve. The better a technology or measurement, the closer the curve to the top left corner of the graph. One way two techniques can be compared is to calculate the area under the curve (AUC).

A free web-based application for evaluating ROC curves can be found at: http://​www.​rad.​jhmi.​edu/​jeng/​javarad/​roc/​JROCFITi.​html

However, before we discuss this further, a second differentiation has to be considered: whether the assessment of disease is based on truly quantitative assessments (like a linear measurement, such as joint space narrowing in osteoarthritis or brain volume in Alzheimer’s disease) or a radiological interpretation of an image into a semiquantitative scoring system, like the Genant score for vertebral deformity [5] or Sharp score for rheumatoid arthritis [6].

In general (and this is a generalization, since there are examples that do not follow this distinction), the greater the numerical spread, the “better” the instrument at differentiating between treatments whether it be between active comparators or between active and placebo, particularly for measurement techniques. The definition of normal is different for each therapeutic area. For example, when considering bone densitometry as assessed by dual energy X-ray absorptiometry (DXA), bone ultrasonometry, or quantitative computed tomography (QCT), “normality” is defined by a population of young healthy individuals aged between 20 and 40 without any history of bone disease or medication usage likely to affect bone. The “normal” population should also be drawn from a geographically diverse population to avoid local regional differences. Owing to phenotypic variation, a separate population needs to be assessed for each of the major ethnic groups (e.g., Caucasian, African, and Asian) which has to be further divided by gender. The accepted definition of osteoporosis as defined by the World Health Organization (WHO) is an individual having bone mass less than 2.5 standard deviations (−2.5 SD) below the young normal mean [5]. The definition is also specific to the anatomical area being imaged with different values meeting the criteria for osteoporosis at the hip and at vertebrae. This has come about due to the inverse relationship between DXA measurement and the age-related increase in fracture risk. Once past the menopause there is a decrease in bone mineral density caused by an uncoupling of bone remodeling due to the decrease in circulating estrogens in women. There is a more gradual but age-related loss of bone in men due to the decrease of circulating sex steroids.

By the very definition of the disease being based on a normal population curve, as individuals stay alive they develop age-related bone loss, and a larger percentage of the population will become osteoporotic. However, it is critically important to appreciate that normality is gender, race, and anatomy specific, which creates further challenges when combining populations in clinical trials. However, the concept still remains – the greater the difference between normal and abnormal, the better the overall ability to diagnose with clarity but more importantly in clinical trials, to see a change caused by therapeutic intervention. This is further related to the precision of the measurement which will be discussed further in the next section.

With the so-called semiquantitative measurements, a scaling and graduation scoring system has to be used which is generally a numerical system with 0 being normal or healthy. The common ones are often used in musculoskeletal disease, as already mentioned. The challenge is to have a scoring system which has sufficient granularity to distinguish change by therapeutic intervention or to demonstrate worsening disease without being so complex as to make the scoring system unusable. A great example is the Sharp scoring system, modified by van der Heijde for the evaluation of joint space narrowing and erosions in rheumatoid arthritis [7]. On initial view, it seems very complex, with a total score of 428 (280 for maximum erosion score and 168 for maximum joint space narrowing score), with a score of 0 being a subject with no observable disease radiographically on hands and feet radiographs. However, the score is broken down into a 0–5 (in most instances) for each of 26 joints per hand and feet being evaluated for erosions and again for joint space narrowing, with a slightly different evaluation. On a joint perspective, this makes a relatively simple and reliable scoring technique. Furthermore while there are reader interpretive differences on a per visit basis, these are decreased by evaluating the “change score” between time points. An alternative method that has been used in a number of trials is the Genant score, which has many similarities to the novice evaluator [8]. However the scoring system is reduced to a total of 312 (176 and 136 for maximum erosions and joint space narrowing, respectively). On initial review this would suggest a decrease in sensitivity to detect change, but a potential increase in the reproducibility of the readers to score. There is a subtle twist – the score can be in 0.5 increments! Debate regarding the merits of each scoring system has existed and will continue to do so, and it takes the trialist carefully evaluation as to which is the most pertinent to their trial for other reasons. After all, these scoring systems are not used for routine clinical practice. The final say probably now goes to the regulators; the European Medicines Agency (EMA) has come out with a new guidance for rheumatoid arthritis and stated that the van der Heijde modified Sharp scoring system should be used unless there is a good rational to use another methodology [9].

One further concept has to be considered when evaluating end point in clinical trials: the smallest detectable difference (SDD) or smallest detectable change (SDC). “The SDD expresses the smallest difference between two independently obtained measures that can be interpreted as “real” – that is, a difference greater than the measurement error” [10]. The SDC is a concept that has grown out of the field of rheumatoid arthritis, where there is a dichotomous group of subjects that can be observed in a clinical trial. The majority of subjects show little or no change in the 2 years of observation in their radiological scores, where there is a small subgroup which undergoes a large change in joint damage. Furthermore, since the reads are conducted with a number of time points presented all at one time, the images are not truly independent in the statistical sense. The read design and methodology will be covered more fully in Chap.​ 5. However, the key difference is that SDD is based on the concept that a patient’s disease is progressing and SDC is based on the concept of how the much less disease is progressing. Either assessment is the determination of the degree of progression above the measurement error that can be statistically determined. In rheumatoid arthritis trials, Bruynesteyn and colleagues conclude that SDD over estimates the number of patients required for the study [10]. SDC has now been used for in other rheumatoid diseases such as ankylosing spondylitis [11]. However, when reviewing the literature using Google Scholar, SDD is the preferred methodological approach in all other therapeutic areas when it comes to radiological interpretations.