This is the second in a two-part series discussing item statistics. By understanding what makes an item good statistically, we can produce better items.
A key statistic used when evaluating items is "R" or the "point-biserial correlation." This is the degree to which getting the item right predicts performance on the examination. R can range from -1 (getting the item right means you do worse on the exam) to 1 (getting the item right means you do better on the exam). For legal and practical reasons, we cannot put items on our examination that are negative. Ideally, we like items to have an R of .2 or higher.
There are several things that can reduce an item's R; by taking care to avoid them, we can raise our R values. First, we want to make sure an item is not too easy or too hard, because that negatively affects the R value. For more on that, please refer to last month's article about P-values here. We also want to avoid asking questions that are not related to entry level performance of a technologist. Just as if we are trying to measure mathematical ability, we would not ask art history questions; we want to make sure our questions pertain to the topic we are trying to measure. We also want to make sure that distractors are not technically true; only the key should be true. Finally, we want to make sure distractors are not so obviously wrong that no one would pick them.
Unlike p-values where the ideal range is somewhere in the middle, higher is always better for R.
