Legislators

In theory, the information in government public performance reports (PPRs) should, first and foremost, provide a sound basis for helping the legislature hold the government to account. This would include, for example, scrutiny of government budgets and government performance, developing new policies and initiatives and rethinking existing ones. The information in PPRs also offers a potential means for legislators to communicate with their constituents on what results government has achieved.

Legislators say that PPRs rarely reflect their perspective as users or their very specific needs and concerns. PPRs, according to legislators, tend to lack both credibility and information that is directly relevantto them. Legislators also state that PPRs often are produced in a language and format that they as users find difficult to understand and access, although some examples of new reporting trends and formats are emerging.

Legislators and government producers of PPRs have an opportunity to create partnerships of trust, to create more relevant public reports that resonate with users. Legislators could increase the likelihood of obtaining what they need from PPRs by actively creating “demand” for quality performance reporting and then visibly using the information to hold government to account. Public accounts committees, for example, can use PPRs to hold governments accountable.

Projects involving legislators

In British Columbia, CCAF is working with members of the Public Accounts Committee of the BC Legislative Assembly to design a non-partisan agenda to help the committee work effectively. One of the key suggestions is for the PAC to hold a series of hearings on public performance reporting in BC, culminating in the tabling of a committee report.

The CCAF’s project with the legislative auditor in BC to review the evolution of public performance reports over a 10 year period (described below) has a tie-in with legislators. The retrospective will be utilized by CCAF and decision-makers in BC to better understand the historical role of users in providing feedback to government on reports, the role of the Public Accounts Committee as a driver for change and the impetus for the adoption of performance reporting in the first place. CCAF plans to present the findings to the Public Accounts Committee in 2007.

Auditors

According to legislators, PPRs tend to lack both credibility and information that is directly relevant to them. If users do not have confidence in the objectivity of the data in PPRs, they will not use these reports. 

To address this problem, legislators (and other users) stress the need for an independent assurance function to attest to the quality of the data in PPRs. Legislative audit offices might audit, attest to, assure or certify the quality of PPRs and the related data. The internal auditor might do the same, but for the deputy minister. The role of both might also include attesting to the quality of databases, even where no report is produced.

In some jurisdictions, the presence of legislative champions who support performance measurement and reporting has led to progress. Auditors General could actively nurture these legislative champions.

To reinforce the importance of using PPRs, auditors might also consider assessing the extent of use of PPRs and integrating the concept of use into their assessments of PPR quality. Additionally, audit offices will need to adapt to non-traditional forms of PPRs, as producers move to redesign reporting to meet the needs of intended users.

Projects involving auditors

CCAF is working with the Office of the Auditor General of BC on a retrospective of performance reporting over the last ten years. The focus is on the evolution of BC’s performance reporting during this time, and highlights possible areas of improvement. The project also includes case studies of strong performance reporting in BC drawn from ministries and Crown Corporations.

The Public Reporting Advisory Group is a working group of the Canadian Council of Legislative Auditors. PRAG’s mandate is to advance the theory, practice and state of public performance reporting in Canada. PRAG contains representatives from each of the legislative audit offices in Canada, except from the province of Prince Edward Island. CCAF is working with the group to provide a strong voice in advancing the needs of users in regards to public performance reporting practices.

Media

CCAF research indicates that the media have not shown a great interest in writing articles about government performance. Consequently, they do not generally use PPRs as a source of information in their reporting.

We found several reasons for this. The economics of the media have led to less investigative, analytical journalism, and more demand for the immediate story; to fewer expert journalists and more generalists. Reporters today tend towards limited research and use government documents or reports relatively little.

The media people to whom we spoke commented candidly about their lack of trust in the public reports governments produce on their own performance. They said they might, at best, use PPRs for statistical information, but they would not depend on them for substance.

PPRs need to explain why the government has either embarked on certain activities, or decided to provide particular services. PPRs also need to explain the rationale for the objectives that the government has established for these activities and services. Broad social indicators would help in assessing programs and their effectiveness.

CCAF encourages governments to make PPRs more user-friendly, visible and accessible to the media, easier to read and understand and more useful in terms of the information they contain. We also note that the media have an opportunity (and perhaps an obligation) to ask more probing questions of government about the quality and cost-effectiveness of its significant programs.

Projects involving media

CCAF conducted a study to assess and stimulate the use of performance reports by members of the media. The study documented media perception on a series of performance reports by organizations that are considered leaders in effective performance reporting at the federal and provincial level in Ontario.

The intention is to replicate the study in other jurisdictions.   

Non-governmental organizations (NGOs)

In 2006, CCAF invited a number of national non-governmental advocacy groups to participate in a roundtable discussion on the use of government performance information. The NGOs indicated that they use PPRs for five key purposes:

• Policy development and advocacy
• Tracking government follow-up on commitments or recommendations
• Assessing government preparedness and responsiveness to emerging issues
• Marketing research
• Monitoring the use of government programs.

The NGOs said they do not find annual reports as useful as they would like. Information on horizontal issues is lacking, and performance measures and information are often reported inconsistently across departments and jurisdictions, and are not comparable.

At the roundtable, it was suggested to us that governments might work more closely with NGOs to seek their input when developing PPRs. For example, a government might ask unions what information they need, design the report accordingly, and seek feedback afterward. Governments could also brief NGOs when reports are published, aiming to create more demand for particular reports.

The participants, as consumers of PPRs, suggested a number of things that they (as opposed to government) could do to improve these documents and make them more useful. Suggestions involved strategies such as becoming more active in persuading or influencing government to deal with specific concerns, and directing more effort to providing input to PPRs.

Projects involving NGOs

CCAF intends to re-engage NGOs involved in the previous roundtable. Two forums will be held during the second and third years of the project. The focus of the forums will be to work with NGOs to build demand for public performance reports that meet their needs and to help them utilize that information.

We will also launch an initiative related to NGO use of performance reporting with the Centre for Voluntary Research (located at Carleton University within the Faculty of Public Affairs), which is the secretariat for the National NGO Network.

Producers

Government organizations are the producers of public performance reports. Our work with the users of performance reports uncovered a number of ways governments could create more relevant public performance reports that connect with users. For example, governments could:

• consult with legislators on the content and presentation of public performance reports
• make the design of public performance reports more user-friendly
• report measures of performance that matter to ordinary people
• make public performance reports more visible and accessible to the media.

Projects involving producers

CCAF is collaborating with the Alberta Ministry of Finance and the Treasury Board on a three-year project to strengthen public performance reporting. The goal of the project is to produce new models of PPRs that better meet the needs of legislators, the media and the public. With the support of CCAF, the Government of Alberta will document and assess the initiatives undertaken to date to improve PPR. CCAF will conduct interviews and focus groups comprised of legislative members, journalists and the general public. Findings from the focus group will be presented to the two Alberta government entities in order to facilitate discussions on how to better align the reports of the users.

CCAF has conducted a good practices study to identify leading performance reporting practices in Canada and internationally. The study produced important information on the evolution of performance reporting, and highlighted user-focused practices in performance reporting. The good practices research documented efforts by Canadian and international entities in creating useful performance reports that are user-friendly, both in terms of information provided and in terms of readability.

In Ontario, CCAF is working with the provincial Treasury Board to improve its performance reporting guidelines. In a pilot project, the Ministry of Government Services will use the amended guidelines to improve the readability of its reports.

At the local level, a number of BC municipalities will gather for a one-day round table on performance reporting. Challenges and successes to date will be discussed with the CCAF presenting an overview of tools used in the IPRP. The aim is to develop a strategy for supporting the work of BC municipalities.

CCAF has been engaged by the Canadian International Development Agency (CIDA) to assist in developing its first annual performance report to Parliament. (This annual report is separate from CIDA’s regular departmental performance reporting requirements.)  CCAF is reviewing draft versions of CIDA’s report and providing feedback from parliamentarians as well as from a communications and performance reporting perspective.  CCAF will then make recommendations for improving the second annual performance report.