Publicações Recentes

ABELSON, J.; GAUVIN, F. Assessing the Impacts of Public Participation

ABELSON, Julia; GAUVIN, François-Pierre. Assessing the Impacts of Public Participation: Concepts, Evidence and Policy Implications. Canadian Policy Research Networks, 2006.

ABSTRACT

The degree of attention being given to expanding ordinary citizens’ roles in the policy process underscores the need to consider what effects these processes might and will have on policy decisions and on those who participate in them. This paper explores what is known about the extent to which the goals of public participation in policy have been met. It also examines the extent to which research evidence has been used by policy makers and public participation practitioners to design and improve public participation. The authors review the current state of knowledge about the impact of public participation on policy and civic literacy and identify different conceptual and methodological approaches to evaluation and their associated challenges. In addition to theoretical and conceptual literature, the authors also review published (English and French) empirical public participation evaluation literature and incorporate the results of key informant interviews with policy makers and public participation practitioners. The paper concludes with a discussion of the limits to evaluation and its uptake, and recommends strategies for promoting further practice and methods of public participation evaluation.

Some progress has been made in improving the rigour with which public participation evaluation is undertaken, chiefly through the development of more explicit and agreed upon evaluation criteria that have both process and outcome evaluation properties. However, much of the progress made has been focused on improving what are still largely formative evaluation studies (i.e. efforts to improve on existing practice and to assess public participation against an a priori set of objectives for what constitutes successful public participation).

There are a plurality of evaluation approaches and methods but it is possible to identify three main approaches: 1) user-based which assumes different participants have different goals and that evaluation must take these into account; 2) theory-based which is driven by theories and models of public participation and applies normative evaluation universally to any public participation effort; and 3) goal-free evaluation which is not constituted by any stated goals and is conducted in the absence of any theory. Most evaluation studies to date fall under the user-based category. Another distinguishing feature of public participation evaluation is the emphasis on either process or outcome evaluation.

Evaluation processes face a number of theoretical and practical challenges. First, the task of defining the end-point of a participation exercise for purposes of measuring effectiveness is often unclear. The ability to measure the institutional and societal impacts of the process, which can take many years, and may be difficult to disentangle from other events that are influential to the policy process, may be limited. Second, the public participation process may be well run according to some criteria but not others. How do we determine how much credence to give to an apparently acceptable, democraticallydriven recommendation? The third big challenge lies with measurement criteria. Participant satisfaction is routinely used as a measure of success despite problems associated with its interpretation. The issue of perceived versus actual impacts is problematic in any evaluation. Likewise the absence of properly tested measurement tools is another area in need of attention. The authors offer a conceptual map of public participation evaluation that includes context evaluation (public policy process), process evaluation (public participation process) and outcome evaluation (public policy, decisionmakers and participants/general public) as an aid in understanding the different types and approaches to evaluation.

The research evidence from empirical evaluation literature is grouped under processoriented research and outcome-oriented studies. The former reveals that “process matters” and that different types of public participation should be designed for different types of issues, decision-making conditions and groups of participants. What is much less clear is which of these contextual variables matter most and which processes are better suited to each of these different sets of arrangements.

Most of the outcome-oriented empirical research has focused on assessing the impacts of public participation processes on a range of citizen participant attributes. These studies have reported: increased levels of interest and knowledge of public issues; improved capacity for future public involvement; increased propensity for social bond formation; and improved trust of fellow citizens. There has been less research about the direct impacts of public participation on the policy process and political decision-making. What has been produced offers mixed and ambiguous results (e.g., public deliberation can produce outcomes that influence policy but the conditions under which this occurs are not easily identifiable). However, more recent literature finds a strong association between the broad acceptance of the decision outcomes and ‘processes in which agencies are responsive, participants are motivated, the quality of deliberation is high, and the participants have at least a moderate degree of control over the process’ (Beierle and Cayford 2002).