In “The Impact Of Experience On How We Perceive The Rule Of Law” (BSE Working Paper No. 1139), Benito Arruñada investigates the role experience plays in perceptions of institutional quality. By exploring indicators of performance in four areas of the rule of law, he shows that experienced individuals are more critical of institutional quality than inexperienced individuals, and performance indexes built with experienced subsamples substantially alter country rankings. Somehow surprisingly, experience is more influential than alternative variables often considered as proxies of knowledge, such as education, gender, wealth or age. The findings support eliciting the additional knowledge held by experienced individuals.
Interaction, experience and knowledge
Legal institutions are key elements of developed society so evaluating them is both relevant and necessary, as shown by numerous indicators compiled by the World Bank (Governance Indicators and Doing Business), the World Justice Project (Rule of Law Index), and Transparency International (Corruption Perception Index). Many of these measures rely on estimates and perceptions obtained from surveys of experts and citizens.
Even though all citizens benefit from institutions such as the political system or the rule of law, it is likely that some citizens are better informed on how these institutions work because information is costly and hard to transfer. The present paper is based on the assumption that those who interact most closely with an institutional function are well informed about its performance.
For instance, people who experience contractual conflicts which might be litigated are expected to be well-informed on the performance of contract law. Similarly, both victims and criminals are supposed to know more than the rest of the public on the functioning of the police and the courts and, therefore, on criminal law; those who most often deal with public bureaucracies should know more on administrative law; and parties to employment contracts should know more on labor law. In this sense, the paper postulates experience as a proximity to the organizations and institutions being evaluated and, therefore, more experienced people should have greater knowledge about them.
With this in mind, the author focuses on answering two main questions: how does experience affect perceptions of institutional quality? How does considering experienced subsamples modify performance indicators, both at the individual level and in terms of country scores, and what drives such changes? To do so, he uses the World Justice Project (WJP) general surveys and its indicators of performance in four areas of the rule of law — contract, criminal, administrative and labor law.
Experts versus the masses
As a first step, the author explores how individual perceptions on rule-of-law performance differ between the general public and experienced individuals. He identifies for each legal area at least one experienced and allegedly better-informed group of respondents, develops performance measures for each legal area, and estimates the “impact” of experience on such measures. He shows that experience is negatively correlated with stated perceptions of performance in all areas. Moreover, the former is the main observed driver of perceived performance, well above other popular variables, such as education, wealth, gender, age and participation in economic activity, some of which are shown to exert their influence only through inexperienced individuals.
Experience is associated with statistically significant negative evaluations of performance, which are also substantively significant. For example, having suffered a contractual conflict in the last three years lowers the perception of contract law performance in 0.128 standard deviations. Some interactions of experience with the dependent variables are also significant The results suggest that the positive effect that education and age seem to have on how respondents evaluate the performance of contract law is concentrated among subjects without experience. (Download the paper to view associated tables.)
The country-ranking effect of experience
Given the aforementioned findings, the author proceeds to analyse the impact of considering experience on country scores for institutional quality, and how this effect varies across countries. He finds that differences between performance scores based on the responses given by the general population and by the corresponding experienced subsample substantially change how countries are ranked by both sets of indexes. Moreover, the alleged informational advantage of experienced subsamples increases with some plausibly exogenous country features, such as legal systems originating in France and regulatory quality, and positively with ethno-linguistic fractionalization. In addition, while the estimated effect of experience does not vary with the percentage of experienced respondents in each country sample, the estimation error decreases significantly as this percentage rises.
The effect of experience also change with economic development. Countries with greater GDP show better institutional performance (except for labor law). And experience has a significantly more negative impact in high-GDP countries. The effects of other variables, such as education or relative wealth, are also substantially different for poor and rich countries. (Download the paper to view associated tables.)
Experience as a channel for institutional improvement
In sum, the findings support the existence of an “experience effect”: individuals who have interacted more with institutions are better informed on how they work and evaluate them more negatively than the general population. Furthermore, this effect has two important consequences: indexes based on the assessments of experienced individuals rank countries very differently; and other proxies for knowledge, such as education, gender, wealth or age, are less influential for citizens’ institutional evaluations.
The author argues that these results shed light on how institutional performance measures should be constructed: they should focus more on experienced individuals’ reports, imitating the established practice in all sorts of organizations—from business firms to churches to universities—which mainly collect the opinions of users and customers. In fact, not considering experience might cast doubt on reality since perceptions by the inexperienced might be reflecting stereotypes that experts and the media create and educated individuals help to disseminate.
