Over the last two days I've posted some thoughts on the government's ambitions for open data and what is currently being delivered.

One of the key aims, particularly with the release of spending data, has been to encourage 'armchair auditors' to keep a check on wasteful spending.

But perhaps the closest to 'armchair auditing' has come not from the release of this data but from the crowd-sourced spending challenge website. It invited comments from public sector employees and the public, and contains more examples of inefficient spending than any of the spending data that has so far been released.

This is interesting because it points to the issue that a lot of the inefficiency comes not from individual spending items (which all need to comply with Treasury guidelines on public spending anyway) but in the processes which drive spending decisions.

This in turn raises a question about whether publishing, for example, all items of spending over £500 is genuinely going to bring about a step-change in efficiency.

'Armchair auditing' takes place at a micro level. It is scrutiny of individual decisions and individual services run by particular institutions.

But it is not scrutiny of the big picture policy decisions.

Knowing whether some bodies are more frugal or implement a policy more successfully than others does not reveal whether it was the optimum policy in the first place.

How was it used?

It would therefore be much better if the data that is being released came with information about how it was used inside government.

That information is currently kept hidden because it often relates to policy formulation.

But to make the most of what is being published, citizens and experts alike will need to assess what role the data had in making policy.

If any analysis reveals something of interest in the data, a series of questions will surely follow:

  • Does the government already know this?
  • If so, then how did it use that information?
  • What other political factors may have been judged to be more important than the data analysis?

So the logical consequence of open data would seem to be that the policy-making process has to be opened up in tandem.

Open data and freedom of information

Being more open about the use to which data is put inside government also links to some of the ideas I wrote about in this post on eDemocracy and lessons from the Iraq war.

"But great that the Freedom of Information Act is, and promising as the government's pledges on open data are, there is still a gap in what the public is allowed to know about how they are governed.

"We know that ministerial decisions involve trade-offs, that some things are considered and discarded, and no doubt generally for sound reasons.

"But we are not allowed to know what the available alternatives were or why they were rejected.

"The result of this is that nearly all political debate is framed in black and white, with the favoured decision 'good' and all the other options 'bad'.

"This infantilises the debate, and hides from the public the subtleties of decision-making and the nature of the choices that have to be made.

"A politician who really wanted to raise the level of political debate, and who felt confident in the choices they were making, would commit to publishing policy advice."

Need to know more

And as one of the comments on this post on Tom Watson's blog says, "data is meaningless without context".

Andrea Di Maio has also suggested:

"Open government is not a way to outsource problem solutions to others. It is a means to collectively solve problems more effectively and efficiently. Government and society are both data suppliers and users: they have to find the right blend for that data to create public value."

So if the solutions are to be found more effectively, government transparency needs to go a big step further than is yet being contemplated.

This applies on two levels. First, as usefully noted on the MeanderingMammal blog, when it comes to spending data "we need to see 'why' the money was spent, not just how much and where".

And second, more information is needed about Whitehall's analysis of the other data it is releasing and the uses to which it was put.

Given that no post on transparency is complete without a quote from Lawrence Lessig's Against Transparency article for the New Republic, I'll add one here:

"The Web will show us every possible influence. The most cynical will be the most salient. Limited attention span will assure that the most salient is the most stable. Unwarranted conclusions will be drawn, careers will be destroyed, alienation will grow. No doubt we will rally to the periodic romantic promising change (such as Barack Obama), but nothing will change. D.C. will become as D.C. is becoming: a place filled with souls animated by – as Robert Kaiser put it recently in his fine book So Damn Much Money – a 'familiar American yearning: to get rich.'

"But if the transparency movement could be tied to this movement for reform – if every step for more transparency were attended by a reform that would disabuse us of the illusion that this technology is just a big simple blessing, and set out to make transparency both good and harmless – then its consequence could be salutary and constructive. When transparency and democracy are considered in this way, we may even permit ourselves to imagine a way out of this cycle of cynicism."

Is it possible that releasing not just the information but details of the (hopefully) sensible ways in which it is used inside government could go some way to addressing Lessig's issues with transparency. Context can make for informed discussion.

As Carl Malamud wrote in a response to Lessig:

"Merely revealing data is not enough. One must work with it, work with policy, and monitor effects. Transparency without a long-term commitment to policy is transparency without context, transparency that is merely naked."

It is really too early to be judging the success of open data policies. But I feel a mixture of hope and fear about what they could achieve.

I fear that open data as currently conceived cannot deliver all of what is expected of it. And I fear that this will cause it to be labelled at least a partial failure.

But I hope the debate can be refocused on what open data really does do. And I hope it will be possible to find ways of making this even more effective, to the benefit of the whole political system.

Politics Blog
, ,