This special issue of the Journal of Community Informatics explores the promise and practice of Open Government Data (OGD). In the last decade, OGD has risen from being a niche cause in a few developed countries to becoming pervasive in the policy agendas of governments across the world. In the recently launched Open Government Partnership (OGP), discussions around OGD have become the focus of a significant networked movement of technologists, activists, the private sector, and civil society actors. The online publication of structured datasets by governments is seen as playing an important role in driving the transparency and accountability of states, enabling new forms of civic participation and action, and stimulating economic growth and development. Over 100 OGD initiatives are active across the globe, ranging from community-led OGD projects in urban India, to a World Bank sponsored OGD programme in Kenya, government-led developments in Brazil, civil-society initiated work in Russia, and a World Wide Web Foundation supported programme in Ghana.
While OGD has become a focus of global attention, we are still to understand how it plays out in national, sub-national, and local community contexts. The papers and notes in this Special Issue make a contribution towards a more nuanced understanding of OGD policies and practices. They explore particular historical trajectories of government policies with respect to openness, data management and data-use, and look at different approaches to publishing, creating and using datasets of relevance to the processes of governance. In doing so, they provide a sharper understanding of the key challenges of OGD: the practical details that advocates of OGD need to engage with when they recognize that government data cannot simply be treated as a neutral and uncontested resource. The papers also suggest to OGD advocates that they must account for the differences in people's capacities to access and use data because these differences eventually shape, at least in part, the impacts, outcomes and distribution of benefits of OGD. The papers also question an implicit orthodoxy that the consequences of OGD will inevitably be "good developments" (Yu and Robinson, 2012) and therefore, the conventional wisdom as to who the winners and losers of OGD policies might be. To raise such questions is not to argue for or against openness, but to suggest that approaching OGD as a tool of pro-social change necessitates a critical understanding of how OGD creation and use operates in practice and in diverse settings, and that advocates and architects of open data need to account for these complexities when deciding how to act.
This issue is offered as a constructive contribution to the developing debates and practices of OGD - as global infrastructures, technical architectures, and grassroots movements co-evolve. It was published as a rolling issue, with an initial set of papers made available in April 2012 in advance of the first plenary meeting of the Open Government Partnership, and a further set of papers and field notes added up until the end of 2012. Our goal in adopting this approach was to combine the benefits of the peer-review and editorial processes involved in putting together a journal special issue, with the demand for practical and policy relevant analysis in a rapidly emerging and evolving field, where developments far outpace the speed of conventional scholarly publishing. This introductory essay was revised with the closure of the rolling issue in December 2012. The earlier version is archived here.
Many discussions of OGD focus on it as a new phenomena: a technologically enabled discontinuity from the past. An exclusive emphasis on OGD as part of "the next big thing" can mask the historical processes which have shaped the current advocacy, practices and tools of OGD. What counts as government data emerges from the rich history and long-standing practices of state record-keeping and archiving systems, and the deployment over time of various technologies of governance. These historical practices point us to the state's changing political rationalities regarding openness and secrecy of government information. The meaning of "open" in OGD draws upon successive experiments with openness across a number of fields of endeavor. While tracing a complete historical trajectory of OGD merits a standalone paper, in this editorial essay we want to discuss how visions of openness have evolved over time and informed OGD advocacy and policies in different parts of the world. In the course of this discussion, we will refer to the papers and notes that feature in this Special Issue, and explore how they variously challenge, build on, and provide new framing for OGD policies, projects and advocacy.
"Open data" is just one of a number of high-profile labels with the prefix "open". Open government, open access, open innovation, open education and open knowledge are some of the other initiatives and movements in this area. Many of these draw from the emergence of "open source" as the inspiration for their development (Willinsky, 2005; Berry, 2008, Lathrop and Ruma, 2010). Yet, "open" is an incredibly broad and multi-dimensional term. Wittgenstein’s notion of "family resemblance" concepts (Wittgenstein, 1953) is useful here to capture the overlapping intentions and meanings of openness. When you look at open access, open source and open data, for example, there is an evident resemblance, like looking at members of a family, but this resemblance cannot be put down to some simple property all members of the family have in common – there is a much more complex set of similarities and dissimilarities at play.
In many settings, specific notions of "open" are primarily articulated in opposition to some "closed" sets of arrangements that are being challenged. For example, the application of Intellectual Property Rights (IPRs) to software, emerging from the 1970s onwards, in environments where peer-to-peer norms of source code sharing were already established, stimulated the articulation of "open source" (and free software) as the preservation of existing relationships between programmers (Berry, 2008). This oppositional construction can allow many parties unhappy with the status quo to collaborate in calling for change, whilst not necessarily specifying in full what the goals of that change are. However, as the note from Asne Kvale Handlykken in this issue outlines, advocacy for openness may need to be prepared for the long-haul, as short-term policy wins can quickly be eroded by non-implementation or counter-lobbying from vested interests. Handlykken paper uses a case study of the adoption of the Open Source Policy in South Africa to draw out a cautionary tale for those who would see the existence of open data policies as the end of an openness campaign, suggesting some of the political and practical obstacles that a policy commitment to open data could face in a complex political context.
Whilst the connection between open source and open data is often drawn, the strongest equation in many national discourses on open data is with open government. Distinct from “open data”, narratives of "open government" have generally been understood as a reaction to long-standing cultures of governmental secrecy, and, more recently, to the limited scope for citizen participation in policy making (Yu and Robinson, 2012). The latter position has emerged, in part, as a response to the gap between government practice and new or evolving articulations of the democratic rights of citizens. Open government is conceptually linked to democracy, and seen to be an intrinsic good for modern states. A significant number of OGD interventions in countries such as the UK and India have initially been justified on the basis of the contribution open data can make to ‘open government’, emphasising citizen entitlements over the state, and the need to foster greater transparency and accountability in decision-making and resource allocation processes. However, other advocacy for open data draws more upon technological narratives of openness as facilitating new modes of production, enabling more efficient delivery of services, or as supporting the role of competitive market forces in the operation of government services.
Jo Bate's paper in this issue asks about the different understandings that OGD advocates in the UK have of the open data agenda. It questions whether open data movements risk being co-opted as part of a neo-liberal project of state deregulation rather than acting in the interests of social progress and democratic futures. Drawing on interviews with participants from across the UK's open data movement, the paper looks at how political differences and differing visions of the goals of openness have been downplayed in the advocacy for government data: "the result of converging pressures and interests". Bate's paper also explores the tensions emerging in OGD, mainly between practices driven by a logic of citizenship and participation in government and those driven by "hegemonic capitalist institutions" and an interest in market over social provision of services. The paper ends with recommendations for a deeper dialogue and broader debate informed by perspectives from community informatics, compelling us to think where we are headed in the development of OGD.
Katleen Janssen’s paper takes a specific look at the relationship between the Right to Information movement, which has strong connections to transparency and accountability conceptions of open government, and the emerging Open Government Data movement. Janssen asks whether OGD is set to increase effective access to information, or whether a focus on OGD could displace attention on the legally binding right to information, and could lead to concerns about the intellectual accessibility of information being neglected as OGD releases rely on technically skilled intermediaries to turn data into accessible information.
One route to an understanding of how openness manifests itself in contemporary OGD initiatives is to draw a distinction between the openness of processes and openness of artifacts. Smith et. al present an account of openness in ICT for Development (ICT4D) which focuses on process. They describe openness as "a way of organising social activities that favours: a) universal over restricted access, b) universal over restricted participation, and, c) collaborative over centralized production" (Smith et. al, 2008), emphasising openness as a multidimensional continuum in which some data or some governmental activity could be more or less open. Openness here is presented as incorporating a set of values and ways of working. By contrast, the widely used OpenDefinition.org (OKF, n.d.) (which draws inspiration from the Open Source Definition (OSI, n.d.)) offers a binary definition of what it means for an artifact to be open. It states that to be open, data must be accessible online, published without technical restrictions to re-use, and provided under a license that allows the data to be re-used without limitation, including across different "fields of endeavour" (i.e. commercial and non-commercial alike).
The legal status of the artifact is generally regarded as particularly important by open source and open data movements in order to facilitate the creation of a common pool of data that can be freely "mashed together" without concerns about incompatible restrictions that may have been placed on different datasets by their owners. Whether this "common pool" of data constitutes a true commons or public good, accessible to all, is a key question to ask in the development of open (government) data. Berry has explored how the common pool of source code sought by open source advocates may function more like a "club good" than a commons, promoting the freedom of the virtual guild of technically skilled developers, but doing little to promote the freedom of non-technical actors and communities (Berry, 2008). Openness as a process generally needs to be rooted in local communities, whereas in a world of increasingly aligned intellectual property regimes, openness of artifacts is more prevalent.
Bhuvaneswari Raman's paper in this issue explores the power relations underlying specific OGD initiatives linked to land ownership information in the local settings of Bangalore. By looking at the forces that have driven the move towards open data and identifying the groups to whom data is being made open and accessible in practice, Raman highlights the potential risks of opening data in the context of unequal access vis-à-vis the capacity to use that data. This raises a significant challenge for the construction of a common pool of freely licensed data, where a binary logic that regards data as open or non-open dominates. In practice, however, "open" and "closed" are not absolutes. These concepts are relative in that both state institutions and communities continuously negotiate the boundaries of entitlements, legal interference and autonomy through various historical, social and tacit practices of opening and closing information.
Analysing the state of OGD in government institutions, Rolie Cole's note suggests that in practice, institutions have always negotiated a space between closed and open access to data, adopting numerous strategies to accommodate degrees of openness. Cole predicts that such strategies will stay with us for a long time to come, with OpenDefinition notions of openness continuing as one of the many ways in which data will likely be made open.
Government data is not always a pre-existing, complete entity. "Datasets" to be made open often exist in the form of paper records, surveys, reports, entries in registers, and files. When demands for this government data are made, officials have to collate it from their archives as well as from different departments that are responsible for providing the same services or resources. Anne Thurston’s note, based on the experience of the International Records Management Trust, argues that even when data is available, if the underlying records from which it is generated are not well managed, the potential for that data to be used to effect change are diminished.
The incompleteness, or what is referred to as the "mess" of government data, stems to a substantial degree from the historical practices of maintaining records, and what (and how) government officials interpret as explicitly constituting "facts" in these records. Further, officials often bear a good deal of "government" and "public" data on their person and in their memories. Data is often "embodied" in this way because officials may only be able to provide services and develop infrastructures by overlooking or bypassing the regulatory frameworks which prevent straightforward fulfilment of claims (Bawa, 2011). If they were to "officially" maintain such records, they would at least implicitly be implicated in the very "illegalities" around which the data is to be collected and which the state is trying to curb. Further, as historian Ann Stoler (2002) argues, governments classify some information as "secret" because the state itself is unable to resolve the problems posed by the information. Thus, government officials rarely maintain records of street vendors in cities and the state of sanitary infrastructure because, among other reasons, they are unable to resolve the challenges posed by these conditions. The need to protect their interests or those of their constituencies, constitute additional reasons why officials maintain some data as secret or inaccessible.
This partial nature of government archives is highlighted in Nithya V. Raman's field note as she details efforts to access data on bus routes and public toilets in Chennai. Raman's note highlights not only the practical challenges of getting hold of datasets from diverse institutions, but that frequently the datasets available may not offer the most authoritative information for a given issue. Information may exist instead in the head of a key actor, or "informally" recorded in notes and paper records, which are not comprehensively encoded as structured data. In focusing on getting structured datasets, we gain a lot in terms of the share-ability of formally structured information. But there is also a potential loss or missed opportunity of engaging with the depth of tacit knowledge which is not easily translated, encoded and deciphered in open datasets. It is then important to ask whether absolutely complete, true and verifiable information is the primary tool for interactions between citizens, communities and the state, or whether open information, available in any form, is the key to fostering negotiations and bargaining between government officials and citizen groups?
The same technologies that support the opening of government data also provide citizens with the opportunity to scrutinize their governments. Many datasets (such as public toilet or bus route and timing data) play a dual role in supporting service provision (finding the next bus or public toilet) and enabling citizen scrutiny of public services (i.e., analysing whether provision of services meets the commitments made by politicians, or identifying corruption where records of payment made for providing a service do not match local realities). This accountability may require a combination of government data, secured either through the government (as in, for example, the release of local government public spending data in the UK) or through Right to Information (RTI) acts, enabling grassroots data-sourcing supported by technically capable civil society groups.
In this way, civic OGD impacts may not result simply from national policy or from a pool of data dominated by government produced data. As Raman concludes in her note, a combination of approaches to access data and pursue accountability are required. At the same time, it may be useful to question whether it is accountability that is desired in all circumstances, or whether responsiveness and trust are elements that need to be present for certain rights to be fulfilled, and for particular socio-economic groups to claim political space in state and society. An answer to this question necessitates that we attend to the specificities of contexts, claims and histories of state-citizen interactions to concretely understand the impacts of OGD.
Early narratives of OGD implied that simply releasing open data would be enough to promote improved policy making and to increase government accountability. However, rather than a cohort of citizen "armchair auditors" using data to directly influence government decisions (McClean, 2011; Maguire, 2011), OGD feeds into decision making in more subtle (though not less significant) ways. The paper from Simon McGinnes and Kasturi Muthu Elandy questions whether more data is always better, looking at unintended consequences of publishing performance data. Focussing on how data can be used by citizens and policy makers to make choices over public services, or to punish and reward service providers, they explore the discontinuities at work when data is released. Their findings suggest that whilst gains transparency may bring benefits, there is a complex rather than linear relationship between more data and greater pro-social impact.
Sharadini Rath's paper in this issue discusses the importance of communicating analyses of OGD to government officials and elected representatives, and engaging them in interpreting the data. Rath describes how she was able to take a range of government datasets and use them to produce analyses and digital graphical representations (in this case, maps) in order, as a party outside government, to provide an evidence-based input into policy and planning processes. However, on their own, the statistics generated did not provide answers. It was in taking this data to local elected representatives and initiating a dialogue around them that Rath uncovered the meaning of the data. Local officials were able to bring "their deep knowledge of local conditions and possibilities" into conversation with the data, although even the insights gained from this process did not necessarily drive politicians to take action. Rath's paper highlights that the extent to which the exploitation of underused state data resources can drive change depends not only on the arguments and ideas that data can support, but also on the political landscape and the extent to which informed local actors can influence the institutional locations where policy and resource allocation decisions are made.
While Rath describes working with government provided data, producing data to inform government decision-making can be an important form of action for communities, especially when this data is provided in open ways that allow it to be combined with other sources. The same technological shifts that break government monopolies on analysing and managing information can also shift the balance of power when it comes to producing information. Shashank Srinivasan's note describes a process of co-producing data on the Tso Kar basin in Ladakh with the Changpa people in order to support more informed decision-making about development activities in the area. When combining grassroots information with existing top-down data structures, sensitivity is required from those encoding the data to ensure that local knowledge and practice remains visible and is not discounted in the process. Shashank describes the challenge of translating the Changpa community's perceptions of their landscape into a structure that could be fed into a Geographic Information System (GIS). Shashank's note also raises key questions about the role and extent of involvement of external facilitators and agencies in creating and actualizing OGD.
Although many of the factors that influence the potential impacts of OGD are influenced by histories of government data and the politics of openness, OGD actors are also constructing new spaces, platforms, tools and practices that seek to create new relationships between states and citizens. Each of the hundreds of OGD initiatives taking place across the world need to make decisions about which technical platforms to adopt, the processes by which to choose data, and the extent to which an OGD initiative provides support for particular groups to engage with open data resources. A number of notes in this issue address the practical choices involved in the development of an OGD initiative.
Open data portals and directories that catalogue discrete datasets have become the public face of many OGD initiatives. These portals and directories vary in the extent to which they provide a space for citizens to request, engage with and even collaborate on producing data. Sometimes, they are simply a place where government hosts the data it chooses to make public. Wolfgang Both's note describes the process of creating a data catalogue in the City of Berlin. Here, city officials did not simply want to replicate the catalogue of contents that other cities or states were opening up. Instead, they collaborated with the Department of Economics and Technology in an online survey to identify priority datasets to be released. The survey, which attracted primarily more technologically savvy respondents, highlighted some of the datasets that citizens were most interested in, although the final decision of what to release first was also driven by a pragmatic focus on which datasets were easier to release.
Tom Demeyer's note describes how in Amsterdam the process of opening up government data was driven by an "Apps Competition", inviting developers to build prototype web applications using datasets. App contests have played an important role in giving form to the potential of open datasets, providing tangible examples of what data can do. However, they also face limitations, with few resulting in applications that are sustained long-term. Demeyer describes how the Amsterdam app competition incentivised release of government datasets, providing a means to engage with different government departments. In evaluating the competition, Demeyer raises a number of key issues - from the need to pay attention to privacy when opening government data, to the challenges of securing an ongoing supply of data instead of one-off releases of datasets which can quickly become outdated.
The construction of technological platforms and intermediary tools to provide access to open (government) data is an area of rapid development. Many open source and community projects are working to create data-specific tools, such as openspending.org which is developing a standard platform to capture and analyse country budgets and other public spending information. As these tools develop around specific datasets, there is a need to connect them with bottom-up community issues, and to identify the interventions that are needed for both, the architecture of OGD initiatives as well as the development of grassroots capacity to avoid the emergence of new data divides (Gurstein, 2011).
Fiorella Di Cindio's paper addresses the design of online platforms for open data from both technical and social perspectives. Drawing upon learning from over 10 years of designing online civic engagement, Di Cindio's paper provides a model for thinking about the development of deliberative digital habitats: online spaces where data can form part of civic dialogue. She outlines the importance of considering technological and social dimensions of these online spaces as well as being attentive to issues of ownership. Di Cindio also suggests paying attention to whether data created through civic dialogue is being managed in truly open spaces, or whether it is being lost to proprietary systems. This shift in perspective, from OGD driving functional "apps" to OGD feeding into wider civic discourse highlights the importance of tracing the ways in which increased access to data will, in practice, impact decision making.
We published this as a "rolling issue" of the journal with paper published between April and December 2012. We did this in order to support a conversation around the meanings and practices of OGD from critical and community perspectives. Even in this short time, the conversation has been developing, although note always towards a more nuanced understanding of openness. The papers in this issue explore how the technological possibilities of open data can be both enabling and disabling, depending on the specific contexts of communities and their relationships with information and the state, and the approaches taken to open up the data.
Further research and critical practice continues to be needed to provide the insights that can guide OGD in directions that will work to the advantage of the most marginalised and excluded. This is work we hope to contribute to over 2013 and beyond on a project entitled ‘Exploring the Emerging Impacts of Open Data in Developing Countries’ as part of the Web Foundation and IDRC Open Data Research network. In the April 2012 version of this introduction we set out give five key themes emerging from the issue that have been instrumental in shaping our own research plans, and that we hope can also provide you with directions to explore in considering the powerful potential, and significant challenges, of making OGD work as part of progressive change:
Grounded descriptions of local practice: A number of the papers and notes in this issue show both the challenges, and opportunities of working with OGD at a local level. Description of diverse cases of OGD use in practice and accounts that go beyond anecdotes are vital to support the design of open government data policies and practices. We have seen that OGD is not a panacea, but it can be a useful tool for creating new ways of working, and for negotiations, bargaining and claiming political space.
Identifying the impacts of technological structures: The digital architectures and data standards involved in OGD, whether created inside government, by corporations, or by open-source community efforts, all impact upon what can be done with data: what is recorded and made visible, and what is hidden or hard to access. It is therefore necessary to excavate and explain how these "back-end" systems of technology and information curating processes impact the "front-end" of what appears as "data" to the rest of the world. This would also involve analysing proprietary and openly available technologies, and how each of these tools and platforms affords/disables the possibilities of opening information in particular ways.
Understanding framings and frameworks of openness: OGD initiatives are interventions in political space. Whilst much OGD advocacy has sought to downplay politics, the new political formations created by shared interests in open data should not be ignored. Research needs to understand OGD in the context of relationships between communities and power.
Exploring the relationship between OGD and law: Writing and documentation have been deemed as powerful tools in history. Documenting and publishing information changes the status of the information. Published information acquires the status of evidence, over time, as it circulates in different spatial and temporal contexts. At the same time, OGD advocates conceive of data obtained through RTI and Freedom of Information (FOI) legislations as "evidence" of infractions committed by the state. It now becomes necessary to explore this "evidentiary paradigm" (Ginzburg, 1989) that underlies published information, and to understand the emerging relationship between OGD and law. The implications of this relationship extend to state-citizen interactions, notions of "openness" and "secrecy", and the manner in which technology will be deployed for opening data.
Imagining and exploring alternative OGD practices: The OGP and other formal and informal global collaborations are promoting shared knowledge around OGD practice, often drawing on templates from developed countries. Jo Bates' paper ends with a recommendation for "civil society, local communities, and domain experts" to develop "tools and resources that empower people to imagine alternative futures". There are many possible futures for OGD policies, platforms and practice. Imagining and exploring these alternatives is a key part of identifying the approaches that can connect OGD with the concerns of communities, and with serving the interests of those most marginalised.
Bawa, Zainab. (2011). Where is the State? How is the State? Accessing Water and the State in Mumbai and Johannesburg. Journal of Asian and African Studies, 46(5).
Berry, D. M. (2008). Copy, Rip, Burn: The Politics of Copyleft and Open Source. Pluto Press.
Ginzburg, Carlo. (1989). "Clues: Roots of an Evidential Paradigm", in Clues, Myths and the Historical Method. Baltimore: Johns Hopkins University Press. pp. 96–125.
Gurstein, M. (2011). Open data: Empowering the empowered or effective data use for everyone? First Monday, 16(2).
Lathrop, D., & Ruma, L. (2010). Open Government: Collaboration, Transparency, and Participation in Practice. O'Reilly Media.
Maguire, S. (2011). Can Data Deliver Better Government ? The Political Quarterly, 82(4).
McClean, T. (2011). Not with a Bang but a Whimper The Politics of Accountability and Open Data in the UK.
OKF - Open Knowledge Foundation. (n.d.). Open Knowledge Definition. Retrieved March 4, 2010, from http://www.opendefinition.org/
OSI - Open Source Initiative. (n.d.). Open Source Definition. Retrieved from http://www.opensource.org/osd.html
Smith, M., Engler, N. J., Christian, G., Diga, K., Rashid, A., & Flynn-Dapaah, K. (2008). Open ICT4D.
Stoler, Ann. (2002). Colonial Archives and Arts of Governance. Archival Science, 2(87-109)
Willinsky, J. (2005). The unacknowledged convergence of open source, open access, and open science.First Monday, 10(8)
Wittgenstein, L. (1953). Philosophical investigations. (P. M. S. Hacker & J. Schulte, Eds.)
Yu, H., & Robinson, D. G. (2012). The New Ambiguity of "Open Government."