Measurement and Evaluation: Integrating the Functions at PPX

from: Performance and Planning Exchange (PPX) - November 28, 2011

This morning I attended another excellent PPX sharing and learning event at which senior performance measurement and evaluation managers from TBS and several line departments gave refreshingly candid presentations on the progress of measurement and evaluation integration.

By way of background, the Government of Canada recognizes the close relationship between measurement and evaluation in the context of its broad agenda for results-based management from a whole-of-government perspective.   This means developing fact-based decision processes that align policies, objectives, activities/programs, and results at all levels—as well as horizontally. 

The vision for the future is fairly clear. 

However, it is generally recognized that there is a long way to go in achieving it—the main barriers being the chronic difficulty of obtaining adequate performance data and an organizational culture that tends to resist exposure to rational decision processes out of fear that there will always be winners and losers.

The ‘Take-Aways’

The presentations from this morning’s session were both positive and realistic concerning progress made to date. 

Without imputing any specific opinions to the speakers—at risk of misrepresenting them—the following are some of my personal ‘take-aways’ from what was said.First, the Central Agencies have realistic expectations regarding the pace of change and they are mindful of the time needed to embed the performance culture while avoiding the compliance mentality that works against this.  They appear to be willing to drop high-level reporting requirements that either don’t add value (e.g. output measures) or are impossible to meet (e.g. high level attribution of outcomes).  However, they do expect departments to stop fiddling with Program Activity Architectures (PAA’s) that are good enough, and get on with developing performance measures that align with objectives—where standardization and re-use of measures will be key to success.

(It is worth reaffirming that performance measurement is the responsibility of program managers and not the heads of evaluation.)

Second, departments and agencies appear to be committed to the functional alignment of performance measurement with evaluation, and they have made a promising start at formalizing and standardizing the evaluation process and/or the key performance indicators.  For example, one group is using the familiar MAF assessment categories (attention required, opportunity for improvement, etc.) to assess progress on ten specifically defined attributes of quality performance measurement systems.

Other signs of progress include:

• A very high-level Evaluation Committee that includes the DM, senior managers and some external expertise;
• An effort to unify the data collection process for multiple purposes, including planning, reporting and evaluation;
• The use of ‘communities of practices’, for example, to move away from the compliance mentality and embed the value of performance measurement and evaluation in the culture.

Many of us question whether the 100% coverage requirement of the new Evaluation Policy is sustainable. 

On the basis of what I heard this morning, I would say that it is—provided that implementation includes: common sense, flexibility, and an organizational climate of trust.

What do you think—are we making progress toward a real culture of performance-based management?

Download our new eBook:

Evaluation - A Primer on Evalution in the Government of Canada