Research paper
Addiction screening and diagnostic tools: ‘Refuting’ and ‘unmasking’ claims to legitimacy

https://doi.org/10.1016/j.drugpo.2015.08.016Get rights and content

Highlights

  • Undertakes a searching analysis of tools used to screen for and diagnose addiction.

  • Scrutinises the validity and reliability claims made about the tools.

  • Critically analyses validation as a concept in itself.

  • Shows how tools participate in making addiction and shaping affected individuals.

Abstract

Human practices of all kinds – substance use, gambling, sex, even eating – are increasingly being reframed through the language of addiction. This ‘addicting’ of contemporary society is achieved, in part, through the screening and diagnostic tools intended to identify and measure addiction. These tools are a key element in the expert knowledge-making through which realities of addiction emerge. Promoted as objective and accurate, the tools are given legitimacy through application of scientific validation techniques. In this article, we critically examine the operations of these validation techniques as applied to substance addiction tools. Framed by feminist and other scholarship that decentres the epistemological guarantees of objectivity and validity, we structure our analysis using Ian Hacking's (1999) concepts of ‘refuting’ (showing a thesis to be false) and ‘unmasking’ (undermining a thesis). Under ‘refuting’, we consider the methodological validation processes on their own terms, identifying contradictory claims, weak findings and inconsistent application of methodological standards. Under ‘unmasking’, we critically analyse validation as a concept in itself. Here we identify two fundamental problems: symptom learning and feedback effects; and circularity and assumptions of independence and objectivity. Our analysis also highlights the extra-theoretical functions and effects of the tools. Both on their own terms and when subjected to more searching analysis, then, the validity claims the tools make fail to hold up to scrutiny. In concluding, we consider some of the effects of the processes we identify. Not only do these tools make certainty where there is none, we contend, they actively participate in the creation of social objects and social groups, and in shaping affected individuals and their opportunities. In unpacking in detail the legitimacy of the tools, our aim is to open up for further scrutiny the processes by which they go about making (rather than merely reflecting) the disease of addiction.

Introduction

Human practices of all kinds – substance use, gambling, sex, even eating – are increasingly being reframed through the language of addiction. This ‘addicting’ (Fraser, Moore & Keane, 2014) of contemporary society is achieved, in part, through the standardised screening and diagnostic questionnaires (hereafter referred to as ‘tools’) intended to identify and measure addiction. Critical public health and science studies scholars have argued that diseases, including addiction, are constituted through the scientific knowledge-making processes used to identify, measure and diagnose them (Duffin, 2005, Fraser et al., 2014, Mol, 2002). Addiction screening and diagnostic tools are a key element in the expert knowledge-making through which realities of addiction emerge. Promoted as accurate and objective, the tools are given legitimacy and authority through application of scientific validation techniques. In this article, we critically examine the operations of these validation techniques as applied to substance addiction tools. Doing so is of pressing importance because the knowledges produced via tools carry with them important political implications. They contribute to influential statements about substance users and ‘addicts’, are used to justify public policies and programs and directly inform decisions about resource distribution and service delivery.

We begin with a background section with three parts. First we conduct a brief review of critical literature on addiction tools and on addiction diagnosis, identifying a need for further work in this area. We then outline the objectivity and accuracy claims made about the tools and discuss the statistical techniques of ‘validity’ and ‘reliability’ testing that are deployed to legitimate these claims. This is followed with a discussion of our theoretical approach and methods. The second section of the article presents our analysis of the validation processes applied to the tools, which we structure using Hacking's (1999, pp. 53–4) concepts of ‘refuting’ and ‘unmasking’. Developed by Hacking to perform specific analytical tasks, these two terms have a particular purpose in this article. In the ‘Refuting’ section we consider the validation processes on their own terms, identifying contradictory claims, weak findings and inconsistent application of methodological standards. In the ‘Unmasking’ section we go further, critically analysing validation as a concept in itself. Here we identify two fundamental problems with the validation processes: symptom learning and feedback effects; and circularity and assumptions of independence and objectivity. Our analysis also highlights the extra-theoretical functions and effects of the tools. In concluding, we consider some of the effects of the processes we identify, for the meanings ascribed to addiction, the forms it is said to take and the individuals it is thought to affect. We close by noting that our critique raises pressing questions about the effects of these tools, the politically loaded categories they create and legitimise, and the justifiability of the ‘kinds’ (Hacking, 1998) of persons they produce.

Section snippets

Background

To date the validity and reliability claims made about the tools have received little examination from a critical social science perspective. Room (2006) raises important questions about validity when he points out the differential cross-cultural performance of addiction tools. He draws attention to the cross-cultural variation in interpretations and meanings of individual tool items, as well as in underlying concepts of intoxication, harmful use and addiction. Midanik, Greenfield and Bond

Method

As noted, the analysis in this article proceeds from our broader project examining addiction screening and diagnostic tools (for further detail see Dwyer & Fraser, forthcoming). For this article, we confine our analysis to four key alcohol tools. We focus on alcohol tools in contrast to tools aimed at screening and diagnosing addictions to other substances for several reasons. Firstly, more alcohol tools have been developed. Secondly, the uptake of alcohol screening tools is strongly encouraged

Analysis

The validity and reliability of these tools is widely promoted in the scientific literature aiming to characterise and quantify alcoholism or alcohol addiction. But what precisely is the basis for these claims? In this section, we examine the operations and logics of the validation processes, asking how sound they are. Our argument here is in two parts, representing the two stages of our analysis, refuting and unmasking. In the first part, ‘Refuting’, we assess the validation processes on their

Conclusion

Screening and diagnostic tools for addiction (or alcoholism) have considerable potential to shape subjectivities, introduce or alleviate stigma, and direct resources – in short, to ‘make up’ people, communities and societies. They are fundamental building blocks in the process of ‘addicting’ currently at work in the West (Fraser et al., 2014): in all we know about what constitutes addiction, who experiences addiction problems, where they are located, how they may be engaged, and how widespread

Acknowledgements

The research on which this article was based was supported by an Australian Research Council Future Fellowship (FT120100215). The National Drug Research Institute receives core funding from the Australian Government Department of Health and Ageing. This article was prepared with assistance from Adrian Farrugia, who contributed to the collection of the addiction screening and diagnostic tools and supporting scientific literature.
Conflict of interest

We wish to confirm that there are no known

References (66)

  • G. Andrews et al.

    The psychometric properties of the composite international diagnostic interview

    Social Psychiatry and Psychiatric Epidemiology

    (1998)
  • T.F. Babor et al.

    AUDIT. The Alcohol Use Disorders Identification Test. Guidelines for use in primary care

    (2001)
  • A.H. Berman et al.

    Evaluation of the Drug Use Disorders Identification Test (DUDIT) in criminal justice and detoxification settings and in a Swedish population sample

    European Addiction Research

    (2005)
  • M.M. Berner et al.

    The Alcohol Use Disorders Identification Test for detecting at-risk drinking: A systematic review and meta-analysis

    Journal of Studies on Alcohol and Drugs

    (2007)
  • E.S. Carr

    Scripting addiction: The politics of therapeutic talk and American sobriety

    (2011)
  • C.J. Cherpitel

    Brief screening instruments for alcoholism

    Alcohol Health & Research World

    (1997)
  • R. Clements

    A critical evaluation of several alcohol screening instruments using the CIDI-SAM as a criterion measure

    Alcoholism: Clinical and Experimental Research

    (1998)
  • C. de Meneses-Gaya et al.

    Alcohol Use Disorders Identification Test (AUDIT): An updated systematic review of psychometric properties

    Psychology & Neuroscience

    (2009)
  • J. Deeks

    Systematic reviews of evaluations of diagnostic and screening tests

    BMJ

    (2001)
  • Department of Health and Human Services [DHHS]

    Alcohol and drug treatment services. Fact sheet: Screening for complexity

    (2014)
  • S. Dhalla et al.

    The CAGE questionnaire for alcohol misuse. A review of reliability and validity studies

    Clinical and Investigative Medicine

    (2007)
  • J. Duffin

    Lovers and livers. Disease concepts in history. The 2002 Joanne Goodman lectures

    (2005)
  • R. Dwyer et al.

    Making addiction in screening and diagnostic tools used in AOD and other health settings

    (2015)
  • J. Ewing

    Detecting alcoholism. The CAGE questionnaire

    JAMA

    (1984)
  • D. Fiellin et al.

    Screening for alcohol problems in primary care: A systematic review

    Archives of Internal Medicine

    (2000)
  • S. Fraser

    Beyond the ‘potsherd’. The role of injecting drug use-related stigma in shaping hepatitis C

  • S. Fraser

    A thousand contradictory ways: Addiction, neuroscience, and expert autobiography

    Contemporary Drug Problems

    (2015)
  • S. Fraser et al.

    Habits: Remaking addiction

    (2014)
  • S. Fraser et al.

    Making disease, making citizens. The politics of hepatitis C

    (2011)
  • S. Fraser et al.

    Substance and substitution: Methadone subjects in liberal societies

    (2008)
  • E. Georgaca

    Social constructionist contributions to critiques of psychiatric diagnosis and classification

    Feminism and Psychology

    (2013)
  • P. Haber et al.

    Guidelines for the treatment of alcohol problems

    (2009)
  • Cited by (14)

    • Making multiple ‘online counsellings’ through policy practice: an evidence-making intervention approach

      2018, International Journal of Drug Policy
      Citation Excerpt :

      Such discourses assume that an intervention will have a singular, predictable effect if implemented correctly, such that promises can be made about an intervention’s likely effectiveness prior to its implementation on the basis of evidence generated in other intervention contexts. However, several scholars have critiqued the assertion that AOD clinical tools and interventions are stable objects with predictable effects through critically-informed empirical analyses of, for instance, the implementation and use of AOD diagnostic and outcome monitoring tools (Dwyer & Fraser, 2015; Dwyer & Fraser, 2017; Savic & Fomiatti, 2016), online AOD screening and automated feedback interventions (Savic, Barker, Hunter & Lubman, 2016), and methadone (Rhodes et al., 2016; Fraser, Moore, & Keane, 2014; Fraser & valentine, 2008). This body of work illustrates that clinical tools and interventions are not stable or singular but are made (and made multiply) through their interaction with networks of other human and non-human actors in local contexts.

    • Engendering drug problems: Materialising gender in the DUDIT and other screening and diagnostic ‘apparatuses’

      2017, International Journal of Drug Policy
      Citation Excerpt :

      As we have argued elsewhere, knowledges produced by these tools are used to justify public policies and programs and direct resource allocations and service delivery. Tools also have considerable potential to shape subjectivities and to influence people’s encounters with social services, the law and other regulatory agencies (Dwyer & Fraser, 2015, 2016). Drawing on science and technology studies (STS), which proposes that expert-knowledge making practices produce realities, rather than merely investigating them (Fraser, Moore, & Keane, 2014; Law & Mol, 2002), we have elsewhere explored how the logics and assumptions embedded in addiction screening and diagnostic tools work to constitute and stabilise particular realities of addiction (Dwyer & Fraser, 2016).

    • The future of ‘addiction’: Critique and composition

      2017, International Journal of Drug Policy
    • Challenging the addiction/health binary with assemblage thinking: An analysis of consumer accounts

      2017, International Journal of Drug Policy
      Citation Excerpt :

      The major international tools currently being used to screen for and diagnose ‘addiction’ (or dependence or disorder) have also been the subject of recent critique. Dwyer and Fraser (2015) deconstruct the validation processes used to establish the legitimacy of addiction screening and diagnostic tools as well as question their claims to independence and objectivity. In a related analysis, the same authors interrogate the logics and assumptions of such tools and argue that they ‘work to establish, standardise and reify addiction as an independent, pre-existing affliction located in individuals’ (Dwyer & Fraser, 2016: 1).

    View all citing articles on Scopus
    View full text