May 2008
Good Practices Report: Interview with the Authors
Interviewer: I have with me Chris Hyde and Greg Gertz, the lead authors of the recently released CCAF Good Practices Report — What Can We Learn From Effective Public Performance Reporting?: Good Practices for Central Agencies, Legislators, and Auditors, and Report Producers. First question: how did the Good Practices Report (GPR) come about and where does it fit into CCAF’s research agenda?
Chris Hyde: The GPR came about as part of CCAF’s Improved Public Performance Reporting Project. It was designed with the feedback we received from the program’s task force. The task force wanted us to study current public performance reporting practices in Canada, in the United States, and abroad.
With the help of the BC Ministry of Finance, Greg and I looked at organizations and agencies recognized for the quality of public performance reporting to users. By users, we mean elected officials — members of a legislative assembly or members of parliament — and all those who serve on standing committees, such as public accounts committees.
These agencies also reached out to the media, the public, and to non-governmental organizations (NGOs). Basically, they report in a manner that’s understood by both government and by those outside of government who are using these reports to hold government to account for public expenditures.
Greg Gertz: As we looked at the information available and where we would focus our research and our analysis, we became more and more excited. When we started looking at external practices we weren’t sure if there were going to be a lot of them. It soon snowballed so that we had more jurisdictions than we knew what to do with.
Interviewer: It’s great that you had so much choice among leading jurisdictions and that you could make selections from there.
Greg Gertz: It was quite a process!
Chris Hyde: Interestingly, we had a lot of help selecting the jurisdictions and entities were not defined here at CCAF. Instead, we asked the leading central agencies or legislative audit offices within those jurisdictions to identify for us those entities that had reported well to the external audiences. Ultimately, we went on what their users had said about them. They had a better knowledge of what was being done than we would from Ottawa.
Interviewer: Was that a unique feature for soliciting input and advice at all the different stages?
Chris Hyde: Absolutely. I think it was crucial that we — from an early stage — consulted not only experts from BC, but the US Governmental Accounting Standards Board, Barry Leighton from the Office of the Auditor General of Canada, Lee McCormack from the Treasury Board of Canada Secretariat (TBS), Tim Wilson of TBS, and many others. They provided great feedback and direction during those crucial first days. We were really glad to have them providing us with advice.
Greg Gertz: I’ll mention one other point: the role of the Sloan Foundation, which provided funding to CCAF for research in this area. They have a strong interest in helping government organizations meeting the needs of users in their reporting. One of the key questions we addressed in the study was “How do we meet the needs of users?”
Interviewer: Why do you think that CCAF was well placed to lead the GPR?
Chris Hyde: One reason is CCAF’s mandate, which is bringing together three different communities: the government management community, the legislative auditor community, and the political community. So with the perspectives of all three of those groups, we have a unique perspective on performance reporting. We see it not just from the producer’s perspective or an auditor’s perspective but also from the governance perspective. Another reason is that CCAF has already done a lot of work in this area. For example, CCAF produced the nine principles for good performance reporting. Those points were adopted by a number of governments and generated a lot of interest internationally.
Interviewer: This question of meeting the needs of users is quite a new one in this field, is it not?
Greg Gertz: One of the things that surprised me was that there wasn’t much happening in trying to address the needs of users. We knew that the Sloan Foundation was very much focused on user needs, which spurred us in that direction.
We also knew that there was a general dissatisfaction — that’s not too strong a word — with performance reporting in Canada. What was really needed was a new perspective on performance reporting and how to make reports more useful. If you’re going to produce a report for somebody, then it makes sense to talk to those people and consider what they need.
Interviewer: Greg, you mentioned one finding that surprised you most — the lack of user engagement in the production of reports — was there anything else that you found surprising?
Greg Gertz: One observation that struck us as we got into this project was the important role of central agencies. In the past, there’s been a tendency in exercises like this to focus on the role of the producer and say, “Producers should do this, they should consult the users, they should make sure that the pictures are nice, and the graphs should be consistent.” Producers are important, but behind the producer is a central agency giving direction.
Chris Hyde: The role of the central agency is really important in the production of good performance reporting. In fact, 11 of our practices are geared to central agencies. So it’s really important for the central agencies to look at what they can do to help.
Interviewer: Governments over the past ten years have consulted stakeholders — the public, the private sector, and NGOs — to a degree where we’re now hearing about “consultation fatigue.” What are your observations on why that hasn’t happened with public performance reporting as much as it has with policy development?
Chris Hyde: How many people are aware of these reports outside of government, or even beyond the elected officials who are mandated to look at these reports? The number of people who are aware of them is very slim. Most people, on average, have no idea what they are, have no idea how they’re created, and they often have incorrect assumptions of what they are used is for.
Greg Gertz: With these policy consultations, people are not really thinking about how the government should report or, from the government’s perspective, how should it report back to the public so that we all know what has been accomplished. It is about linking the consultation on performance reporting with the policy consulting.
Interviewer: So you suggest having a more explicit kind of discussion about public performance reporting? Do you think that taking that step would increase awareness of public performance reporting?
Chris Hyde: Absolutely. If you were able to align policy discussions with public performance reporting and if you were able to illustrate to the public, the media and other stakeholders how a government is reporting on the policy discussion of today or last week or tomorrow, that would definitely get people’s interest. If the information was available, there’s a better chance that you’d be able to get more people involved in the process.
Interviewer: If internal reporting systems are different from the external ones, does that contribute to a disconnect between the policy consultation and the public performance reporting consultation?
Chris Hyde: It should be seen as a contributing factor. The needs of the internal users are always very important. They need a well placed, well functioning form of program management that ensures that the program is achieving the results in an economical, effective and efficient manner. When a government reports to an external audience, it communicates what it feels are the important criteria for that department in meeting its goals. At the same time, it has to take into account what the public feels is important. This could mean you’re speaking two different languages, so it’s hard to come to a consensus on a topic. When it’s a complex matter, reporting as a federal department, with say over a dozen programs, it’s hard to decide which one of those the public is going to be interested in.
Interviewer: Chris, at your presentation on the GPR to the Performance and Planning Exchange (PPX) Forum, you mentioned that aligning internal and external reporting systems was one of the contributing factors in Export Development Canada’s win of the Auditor General’s Prize for Excellence in Annual Reporting.
Chris Hyde: That is one of the many things that Export Development Canada is viewed as doing well with their performance reports. They are able, in a complex line of business that is not widely known by the public, to effectively communicate to their external audience what their priorities are and to report on those priorities and outcomes in an extremely effective manner (GPR page 31). They are also able to use their performance reports to convey the information required by their internal planners and managers.
Interviewer: We now have more outsourcing, insourcing, public/private partnerships and privatization — all of which shows the changing role of government, the private sector, and that of not-for-profit organizations. What are some of the good practices that you’ve identified that reflect these changes in program delivery and how is this trend important to performance reporting?
Chris: In some ways, your question goes beyond what we were looking at. There was one organization in Australia we talked to that was partially owned by the Crown and the rest was owned by private interests. It was a shared ownership where they were able to meld their process of developing objectives and reporting measures together to meet the needs of these two different groups in a very effective way (GPR page 34). So there are ways to do it. It’s not impossible.
Interviewer: Who do you think will benefit the most from the Good Practices Report and how will they be able to benefit from it?
Chris Hyde: We believe all users will benefit, as that was the focus of our effort. Many of our good practices focused on how to engage the user. For example, there was a recommendation about involving communications directors as their job is to connect with the public and they’re supposed to understand how to reach the public and what the public is interested in (GPR page 21). So the recommendation was to bring them in early in the process and to get the whole team thinking about the needs of users.
Interviewer: To pick up on your point of involving communications directors, is there a fear that the public might perceive performance reports as becoming spin documents?
Chris Hyde: I would say no. You need to distinguish between the political staff and the bureaucratic staff and our practice is to include the bureaucratic staff responsible for communications. Their role should be to make the document more credible, not more political.
Interviewer: Another topic is technology. With the change in program delivery and program development methods, technology has become an important force in public administration. How has technology come to bear on public performance reports and what role might it play in how public performance reports are produced or consumed?
Chris Hyde: I think one of the most interesting things we’ve seen is how central agencies and the departments themselves can use technology to reach out to users. Central agencies can use a web-based interface to present the information found in public performance reports in a manner that’s citizen-friendly and friendly to parliamentarians. They can read them, access the most succinct and necessary data, and lay it out for the public to understand (GPR pages 41-43).
If you’re a parliamentarian or even a citizen or a reporter, you don’t have the time to read a 200 or 300 page document. There are some good organizations out there that take the information and customize the way it’s delivered to certain key user groups. The State of Oregon Progress Board in the US is one example. It tries to facilitate a dialogue that involves the information that’s produced by these entities in a manner that the public and the parliamentarians can use.
There are departments that have taken that route and we saw examples of how to best interact with the public. Some of them, especially the British Library in the UK, have innovative online reports that virtually do away with the question of PDF or HTML format. It’s done with Adobe Flash, it has Windows Media, and it has audio. It attempts to take the paper and the reading out of it. Instead, they’re trying to provide an interesting, informative way to get at the information in a 10 to 15 minute timeframe.
Greg Gertz: In addition to the use of technology to improve service to the user, there’s also the collection side. This is not something we looked at in any detail, but we are aware that technology can be used in increasingly effective ways to collect and process the information used for performance reports.
Interviewer: So with the GPR out there and people digesting it, what is the next step? What are people going to do with this?
Greg Gertz: Going back to the Program for Improved Public Performance Reporting, we want to see changes made to the content to make reports that are more usable by their intended audiences — not just the internal audience but also the external audience. We want the media, legislators, NGOs, and even the public to take more interest in the material that’s being produced. Governments invest a great deal of effort in reporting. These reports are important tools for the public to hold governments to account. That’s the basic use for performance reports.
We hope governments will understand and take the necessary steps to reach out to the public. We hope central agencies will continue to advocate for externally user-friendly reports. If those central agencies see something that departments are not doing or haven’t done, they should take some interest in that. And we’d like external users to prick up their ears and understand that governments are making an effort. We want users to provide some feedback and make themselves available for consultation. There are many jurisdictions and entities out there who are trying to reach out to the public, and we’d like to see something coming from the other side.
Interviewer: So the process of improving public performance reporting is more of an incremental approach than revolutionary?
Chris Hyde: Yes. It’s a marathon, not a sprint. To make these improvements or any type of major change takes time. Performance reports are not intended to catch somebody for doing something incorrectly. They’re not documents to be used to end somebody’s career or to get someone in trouble in a political sense. They’re documents to let the public, to let the parliamentarians, to let the media, to let everybody, including those inside government, know what’s working well, what is not, and what can be improved. We should be constantly looking for ways to improve how we get things done.
Greg Gertz: I will try and reflect on this from a broad perspective. If you reflect back on how performance reporting was approximately ten years ago in Canada, it seemed to progress and then reach a plateau. This was why CCAF’s program to improve performance reporting was launched — to get off of that plateau.
In a sense, you could be discouraged that we hit that plateau but, on the other hand, ten years really isn’t that long for such an important change in the way people think about governance. There’s still a lot of room for progress and this report shows that, if you look at other jurisdictions, there is a lot of progress being made everywhere.
Interviewer: What do you think the next phase of this ongoing evolution will be? Are there any early indications from CCAF research of how applying some of the good practices you’ve identified would help jurisdictions get there?
Greg Gertz: That’s a difficult question. As a user, I see areas where governments are attempting to use technology to interface with me. They’re attempting to find new ways to take the essential content out of an arbitrary report format that the government produces and put it into something they can use to interact with citizens and to solicit feedback from them. I take encouragement from that.
Chris Hyde: There are two things that our report highlighted as good practices. One is more central agency leadership in performance reporting. I think we’re seeing that in a couple of places. A central agency in British Columbia assisted us with the GPR. We’ve also seen leadership by central agencies in Alberta and the Government of Canada.
The other practice we highlighted is the role of elected officials, particularly Public Accounts Committees. In New South Wales, the Public Bodies Review Committee, a committee of the legislature, conducted a major study on performance reporting, issued an excellent report, and then began an annual award — the Premier’s Award — presented by the Committee (GPR page 24). That’s an excellent indication of leadership. It shows to government that this is important and that, if you do a good job, we’re going to reward you for it. We see too little of this in Canada.
Interviewer: Do either of you have any closing observations that you’d like to make? For example, how the GPR can make a difference for producers or what is the “take away” for producers?
Chris Hyde: I used the line before and I’ll use it again: it’s a marathon and not a sprint. These changes are going to be incremental and producers will create reports that might not always solicit a response from the people they want to hear from. That doesn’t mean they should give up. It just means they should continue to work and reach out to users and find new ways to bring citizens into this process. Like it or not, public performance reporting is not going away so we need to find a way to bring all of the parties together.
Greg Gertz: I agree. In talking to producers from all over the world and reading all these performance reports, we saw just how hard it is to produce these reports well. There are so many demands, sometimes legislated, of what must be included. There’s the need to balance the political and the bureaucratic, which is very sensitive and very difficult to do. There’s trying to take numerous highly complex programs and report on them in a few pages, which is extremely difficult to explain at a level that makes sense while providing enough detail. In the end, performance reports will never be perfect, and they’ll always be difficult. But as we saw, there are significant opportunities for Canadian jurisdictions to improve their reporting.
Interviewer: Chris and Greg, thank you very much for your time and for this Good Practices Report.
Greg Gertz: Thank you.
Chris Hyde: Thanks.
For more information on CCAF’s Improved Public Performance Reporting Project or to download the GPR, visit: www.performancereporting.ca. For more information on the study, please consult Chris Hyde at chyde@ccaf-fcvi.com or by telephone at 613-241-6713 x 231.