Are You a German Spy? A Blackadder Inspired Rasch Approach to Detecting Social Desirability Bias in Self-Report Data

In the 1989 BBC sitcom Blackadder episode General Hospital, the titular character is tasked with identifying a German spy at a WW1 field hospital. His ever helpful, yet dim-witted sidekick, Private Baldrick, suggests asking every patient, “Are you a German spy?” unaware that ‘honesty is not always the best policy’.

Self-report instruments, often using Likert scales, are commonly used to assess various psychological constructs. However, these instruments face criticism due to Social Desirability Bias (SDB), where participants may over-report positive traits, behaviours and values or under-report negative ones to appear more favourable. Like Baldrick’s simplistic plan, asking people to candidly self-assess may generate misleading results. Despite this limitation, in the absence of external evaluation (an inherently time-consuming process) self-report continues to be a predominant research method, particularly in language learning contexts.

This study introduces a method to identify and quantify Social Desirability Bias (SDB) using Rasch Infit MNSQs in conjunction with analyses of response strings. Utilizing Likert scale data from a personality survey administered to 86 native English-speaking university lecturers in Japan, the study estimates the degree to which participant responses may have been compromised by SDB and describes the data signals indicative of its presence. The method offers researchers a way to enhance the integrity of studies that rely on self-report questionnaire data.