Dog Show, again

Dr. Mark takes up the Dog Show Challenge with this ‘guest’ post. (I love it when he talks Psychology.)

Well OK I will respond then. I love talking about judgment and decision making behavior.

So let’s take an example that appears in the literature, judging beef cattle. There are 15- 25 dimensions that can be used to decide what grade of beef a given cow will likely end up as. The research shows that the judges of such things really only use 3 or 4 criteria and maybe as many as 7 in rare cases. What one really wants in such a situation is a panel of judges whose collective determinations predict the outcome well (safe beef in the meat isle.) Technical jargon: The inter-rater reliability can even be 'moderate' as long as each of the 'raters' use at least some dimensions that help predict the criterion.

What you have in the dog show is only one judge, and many more criteria.

The dog show is so much more complicated a judgment task that I can only laugh at it. There are 125 recognized breeds, each of which has on the order of 50 criteria: color, weight, shape of all the different body parts, blue tongue etc. etc. The overall judge for best in show has no idea which dog will win each of the 7 categories so we are to believe that he knows and can evaluate a potential pool of 125 breeds times 50-some pretty bizarre multi-faceted criteria in the time that spans about 3 commercial breaks. The standards don’t necessarily represent any real dog, so where is the example for each of the judges to use as a point of reference? Some of the criteria are mutually exclusive, can’t really appear together so how to weight each of them appropriately? So, best in show is supposed to be the one dog that best exemplifies that specific breed of dog better than all of the other entrants. I don’t buy it even if the head judge takes on faith that the judges of the individual categories did their job.

(Your radiologist too is evaluating x-rays relative to some “normal” standard that basically is there in his head, by the way.)

Anyway, it all comes down to which of these 7 dogs do I, the judge simply like the best, and thankfully I don’t have to explain the criteria and probably couldn’t. I want a job like that; can wear a tux and look dignified and won’t (can’t) be questioned. What a racket. Oh yeah and then how does one determine that one “judge” actually judges better than another? What would the criteria look like for that?

-- Dr. Mark