IB Reliability and Diagnosis

Next, we need to think about the reliability of the different versions of the DSM and ICD. But, what does “a reliable diagnosis” mean and how would we measure this?

  • What does reliability mean when we use it to think about research findings in psychology?
  • What would it mean to say a diagnosis was reliable?
  • How could we determine whether a diagnosis is reliable?
  • Why do you think it might be hard for practitioners to reach a reliable diagnosis? Think about how we extract information from service-users? Why might different people come to different decisions about the person?
  • If 100 is a perfect agreement between practitioners regarding a particular diagnosis and 0 is no agreement at all; what number do you think would be acceptable for us to say that a particular diagnosis can be made reliably?

Time to work on some terminology. Make sure you understand the following terms, look them up in your textbooks and other sources as required:

  • inter-rater reliability
  • test-retest
  • PPV scores
  • Cohen’s Kappa

Be sure to use these terms when describing, explaining, applying and evaluating issues regarding reliability and diagnosis.

Before you go any further make sure you have answered the following questions, using your textbook:

  1. What Kappa score should be obtained according to Cooper if a diagnosis is seen to be reliable?
  2. How as this changed since 1974?
  3. Which disorders seem to be most/least reliably diagnosed using DSM5?
  4. Why does Kupfer say that it is difficult to make a reliable diagnosis sometimes?
  5. Why does Cooper argue that problems with reliability may not be as worrying as they first appear?

Now we have some idea about how psychologists talk about reliability and diagnosis, let’s see what research evidence there is on this topic:

Chop up the studies in the following worksheet and sort them into 2 piles according to whether you think they are about reliability or validity of diagnosis. Then resort the pile of studies that you think are about reliability into whether they suggest diagnosis can be made reliably or not:

Validity and Reliability – Studies for sorting activity: studies-on-reliability-and-valdity-for-sorting-and-commenting

We will come back to the validity pile later! As you consider each of the studies on reliability, think about possible GRAVE points that you could make about these studies.