January 2008
Transcript of presentation delivered by CCAF Director of Capacity Development, Geoff Dubrow, at the Conference Board of Canada's conference entitled, 2007 Public Sector Financial Leadership: Strategy, Accountability, and Performance, June 2007, Ottawa.
Let’s realign public performance reporting
By Geoff Dubrow
Geoff Dubrow is Director of Capacity Development at CCAF. Before serving as parliamentary affairs advisor to a federal Cabinet Minister, Geoff was a program director at the Parliamentary Centre, where he ran international parliamentary development and governance programs and provided expertise abroad on Canadian parliamentary experience.
I want to set the context by talking about the Conference Board Report entitled Strategy of Accountability: the Mover of the CFO. I was very taken by the idea that performance information is integral to building strategic financial management – and that performance information and the strength of performance management is one of the three strategic levers for strategic financial management.
Canadians want evidence that tough spending choices by governments are based on high quality reliable financial and performance information. Public performance reporting is really the nexus between strategic financial information, strategic financial management and citizen engagement.
Public performance reporting is an experiment that started about ten years ago, it is an experiment in accountability and transparency. Performance reports tend to be used for internal planning purposes, but at the same time they are shared with external audiences.
The external users of these reports, in theory at least, include Parliamentarians, the media, and non-governmental organizations.
CCAF has been fortunate (and I personally have been fortunate) to be able to travel across the country and talk to different jurisdictions about their performance reporting experiences. One of the things that has become clear is that the public performance reporting experiment is not going according to plan.
Shawn Murphy, the chairman of the House of Commons Public Accounts Committee, has said that Parliament has to rethink and entirely re-conceptualize public performance reporting.
And I think this is the case in other Canadian jurisdictions too – this is not a federal government issue, this is a pan-Canadian issue. Western jurisdictions, for example, are starting to say, “We need to rethink this, it’s not working exactly as we thought it would.”
I’d like to allude to some of the reasons for this and explore some possible solutions.
All things to all people
The Conference Board Report I just mentioned includes an excellent citation saying, basically, that performance reports lose their usefulness both to external and internal users when the information in those reports is watered down.
A central issue or theme around performance reporting is that performance reports cannot be all things to all people.
Can you take an internal document that is critical for the purpose of contributing to learning, and then use that same document for external purposes without facing a political critique?
We have a very political spin-oriented environment, not only at the federal level but at the provincial level as well. Spin is starting to take control of our political debates. It undermines good government and our democratic system. This is one of the reasons why we’re seeing the difficulties associated with trying to write one report for multiple audiences.
User consultation
CCAF is carrying out research on what makes for a good performance report. We have found that even organizations considered to be leaders in performance reporting are doing little in terms of consulting users about their reports.
Some consulting goes on in advance about the selection of performance indicators, but that’s more of a technical discussion. One of the things we are finding is that the discussion is descending to a technical level, it’s not focusing on the big picture of whether users are finding reports readable and usable in their work. Very little feedback is solicited by government organizations after the publication of performance reports.
Making reports understandable
The Public Sector Accounting Board’s Statement of Recommended Practice for public performance reporting is very much focused on some of the characteristics of user friendly reports.
Users want understandable reports. But we have been told by members of Parliament that they don’t really understand public performance reports because everything is in such technical and bureaucratic language.
Some jurisdictions make an effort to ensure reports are understandable. Oregon’s Progress Board, for example, encourages user friendly reports. People often cite Alberta’s Measuring Up document as an example of a report that is made for an audience to be able to read and understand.
Comparing actual and planned results
Parliamentarians say bureaucrats are great at telling them what they want to accomplish. But one of the complaints we hear from Parliamentarians is that they are not told how the results measure up to the plans.
The performance reports of Her Majesty’s Prison Service in the UK talk about targets achieved and targets not achieved. In one particular report (if I’m not mistaken) more targets were not achieved than achieved. But nonetheless they are talking about it.
Addressing different audiences
Some organizations prepare their performance reports in multiple formats for different audiences. In the UK there is an executive summary of reports, a summary of financial statements and a full report. These reports are laid out so that you can go directly to the concise information, but you can also drill down for more information if you need it.
So there are practices out there that can make performance reports more usable.
Political leadership and political will
Performance reporting is cyclical rather than linear in the sense that it seems to require constant renewal of political will. The absence of that political will ruins the performance reporting experiment.
In our research we are finding a concern in a number of Canadian jurisdictions that departments do not want to stick their necks out and be vulnerable. It takes courage to say, “You know what, we screwed up or we tried to do this but it didn’t quite work out.” That leadership has to come from a number of levels; one of them is of course management.
In our conversations with director-level-and-below performance reporting people, we’re hearing that by the time a draft report gets to the top level, i.e. the deputy, there is a sort of funnel effect – certain information gets processed out.
So people who are doing the performance measurement and performance indicator work tend to feel very frustrated. They say a lot of the information they generate will be put aside if it’s damaging to the government or damaging to the department. A good performance report requires a leader who is willing to be vulnerable and state, “We tried to reach this target, it didn’t quite work out this year, and this is what happened.”
This is not only a bureaucratic issue. At a recent conference, a public service participant said to four Public Accounts Committee members, “Your colleagues are ministers, why don’t you talk to them and ask them to start providing more accurate information?”
Consistency
Another criticism we hear from users of performance reports is that the information changes from year to year. They can’t tell if a program has moved from department X to department Y, for example.
Canada Revenue Agency at the federal level provides what they call cross-walks that allow readers of CRA’s performance report to relate performance to previous cycles. If there are changes to the data from one year to the next, CRA explains the changes and makes the link to the previous year’s indicators.
In New Zealand, the indicators are static. If an agency wants to change its indicators, it needs approval by Parliament. Once the indicators are in place, the agency has to report on them. This practice could increase consistency and rigour in reporting.
CCAF’s performance reporting program
In June 2006 CCAF launched our Improved Public Performance Reporting Program with the support of the Sloan Foundation in the United States. The program aims to realign performance reporting with the needs of user groups.
We’re not just there to raise awareness about performance reports, because we have tried that. We’ve gone to Parliamentarians and tried to convince them using the reports would be really good for them in the Hansard process, for their committee work, etc. – but that hasn’t gotten us very far. So what we are trying to do now is to realign supply with demand.
Thus we’re working with both users and governments.
First, we’re getting user feedback on how to improve the readability and usability of performance reports. There seems to be a reticence, with the exception of one or two jurisdictions, for central agencies and other government departments to consult directly with users about what users require.
In many jurisdictions, the deputy minister has to sign off on any plan to consult Parliamentarians or members of the Legislative Assembly, so the stakes are high and departments seem generally reticent to consult.
We’re doing that for them; we’re trying to get feedback on what members of Parliament, and other user groups as well, are looking for in reports. Then we provide governments with this feedback on the readability and usability of performance reports.
The other thing we are doing is providing governments and other institutions of government with good public performance reporting practices. I’ve referred to some of these good practices above. I think ‘good practices’ in Canada is a code word for, “We really want to do better but we’re not really sure what to do, so what are other jurisdictions doing?”
There does seem to be interest in numerous jurisdictions throughout the country in improving public performance reporting, but it’s a multi-stakeholder issue and that’s part of the problem, part of the challenge.
In the work CCAF is doing to strengthen Public Accounts Committees, we’re really only working with the Legislative Auditor on the one hand and the Public Accounts Committee on the other, and maybe with legislators. That’s the scope of our interaction.
But when you talk about improving public performance reporting, you’re talking about governments, Parliamentarians, legislative auditors, comptrollers (in some cases) and other stakeholders. It really is a complex issue. There is no one locale (for lack of a better word) in a jurisdiction that could lead the charge. A positive change cannot be brought about by the Treasury Board alone, or the Auditor General alone, or Parliamentarians alone.
So I think there are a lot of challenges in terms of re-examining performance reporting after ten years of experimenting. We should examine where it has gone, how well it has worked and what we should do next.
I will close with one final observation. We talk a lot about value for money. Now people are starting to ask, “Is performance reporting itself good value for money?” Are we just going through the motions here? Are we producing information that Parliamentarians can use in their estimates process? Are we producing information that can be used to hold a government to account? Are we producing information that non-governmental organizations can use to track and understand government policies? Are we preparing reports the media can use?
Some legislative auditors at the provincial level, mostly in western provinces, are starting to say, “We’ve been auditing reports, we’ve been reviewing performance reports, we’ve set criteria, we’ve set standards, we’re not seeing any improvement here. We’re spending a lot of our money, time and resources doing this work, but we’re not seeing any marked improvement.”
In fact, in some jurisdictions the provincial auditors are saying we’re actually seeing a step backwards. I think that is part of the reason why, after ten years of performance reporting in Canada, we are starting to get calls for a rethink, like we heard from Shawn Murphy, the Chair of the Public Accounts Committee of the House of Commons.
Are we just going through the motions or can we actually improve public performance reports so that there truly is a nexus between strategic financial management and citizen engagement?