Comparative policy analysis for alcohol and drugs: Current state of the field

Abstract

Background

A central policy research question concerns the extent to which specific policies produce certain effects – and cross-national (or between state/province) comparisons appear to be an ideal way to answer such a question. This paper explores the current state of comparative policy analysis (CPA) with respect to alcohol and drugs policies.

Methods

We created a database of journal articles published between 2010 and 2014 as the body of CPA work for analysis. We used this database of 57 articles to clarify, extract and analyse the ways in which CPA has been defined. Quantitative and qualitative analysis of the CPA methods employed, the policy areas that have been studied, and differences between alcohol CPA and drug CPA are explored.

Results

There is a lack of clear definition as to what counts as a CPA. The two criteria for a CPA (explicit study of a policy, and comparison across two or more geographic locations), exclude descriptive epidemiology and single state comparisons. With the strict definition, most CPAs were with reference to alcohol (42%), although the most common policy to be analysed was medical cannabis (23%). The vast majority of papers undertook quantitative data analysis, with a variety of advanced statistical methods. We identified five approaches to the policy specification: classification or categorical coding of policy as present or absent; the use of an index; implied policy differences; described policy difference and data-driven policy coding. Each of these has limitations, but perhaps the most common limitation was the inability for the method to account for the differences between policy-as-stated versus policy-as-implemented.

Conclusion

There is significant diversity in CPA methods for analysis of alcohol and drugs policy, and some substantial challenges with the currently employed methods. The absence of clear boundaries to a definition of what counts as a ‘comparative policy analysis’ may account for the methodological plurality but also appears to stand in the way of advancing the techniques.

Introduction

The study of alcohol or drugs policy can take many forms – one of which is comparative policy analysis. Comparative policy analysis (CPA) is a sub-set of all policy analysis. CPA aims to explicate the relationship between policies and their effects. It is the systematic study and comparison of policy(ies) in different jurisdictions, and hence explicitly includes two or more different ‘states’ which are compared with reference to the policy under study (Cacace et al., 2013, Schmitt, 2013). In this paper we use the term ‘state’ to refer to a country/nation, state, province, county or other discrete geographical unit. CPA may be conducted at a single point in time, or make use of policy changes over time in the states under study.

There are a number of reasons why CPA is important (Marmor, Freeman, & Okma, 2005): firstly in order to learn ‘about’ policies in different countries, focussing on knowledge accumulation regarding similarities and differences; secondly to learn ‘why’ they take the form that they do: seeking to explain developments, trends and so on; and thirdly to learn ‘from’ in terms of outcomes and effectiveness. All three of these are valuable and indeed, intertwined. Already in this simple three-fold description of the purpose of comparative policy analysis we see the emergence of distinct methodologies. In learning ‘about’, descriptive methodologies may be most appropriate; in learning ‘why’, qualitative analyses may take place, and in learning ‘from’, quasi-experimental quantitative and mixed methods designs may best suit.

Comparative policy analysis can generate important insights. For example, the decriminalisation of drug possession in association with a range of other measures in Portugal in 2000 has been the subject of intense interest, concerning how it affected drug use and drug-related problems in that country. The Hughes and Stevens (2010) comparative policy analysis which compared changes over time in Portugal with changes over the same time periods in Italy and Spain (where policies were unchanged), has been very important in the debate about whether Portugal's experiment was successful.

Comparative policy analysis is a diverse set of activities, undertaken by many different disciplines, all with their own approaches – it is not a unified field of study. Hence the diversity in methodological approaches, tailored for different types of policy questions and each with strengths and weaknesses. Brans and Gouglas (2015), editors of the Journal of Comparative Policy Analysis explicitly note the divide between the social sciences comparative policy analysis approaches and the “policy analytical comparative studies” that come from public policy schools (some other terms used with reference to this ‘split’ between the two schools include: policy science/public administration/technocratic policy-making/empirical problem-solving (Lasswell, Wildavsky, etc.) and post-positivism/interpretive turn/critical policy analysis/argumentation and discourse (Fischer, Hoppe, Hajer, etc.). In this paper, and as a multi-disciplinary group of authors, we endeavour to transcend those disciplinary divides.

Twenty-two years ago, MacCoun and colleagues (1993) raised a number of issues and challenges for comparative drugs policy analysis. While noting the strengths of the approach, such as providing variations in policies of interest and opportunity for states “to learn from each other” (p. 106), they also pointed to the main challenges: operationalising the drug policies to be compared, taking into account contextual socio-cultural factors, measuring outcomes, study design challenges and statistical or analytic problems. They also commented that there were relatively few such comparisons.

In this paper we seek to scope the current state of comparative alcohol and drugs policy analysis and explore the varieties of CPA and how they are represented across a sample of recent literature. As all the authors are engaged with forms of comparative policy analysis, but from diverse disciplinary perspectives, we aimed to assess the diversity of approaches and methods. Another aim was to examine the differences between CPA conducted for alcohol policy and CPA for illicit drugs policy, as well as examine the kinds of policies that have been subject to CPA and those that are largely missing. As the first published analysis of the state of CPA literature in alcohol and drugs policy, we hope that this work will provide a foundation for more and better analyses of CPA methods, and serve as a resource for those undertaking CPA and seeking to place their chosen methods within a broader array of approaches.