Outcome Measures

I mentioned Outcome Measures in a previous article (For Profit vs. Not for Profit) about opportunities for project managers to gain some valuable coaching and mentoring and volunteer their time to a charitable organization at the same time. This article will explore these opportunities in a little more detail in the area of measuring the outcomes of projects and programmes.

Outcome Measures is an initiative that actually began with government agencies changing the way they deliver social services. Governments wanted to be able to measure the effects their services were having on their clients. There are/were agencies that governments fund in partnership with organizations such as United Way and those agencies were forced by the governments to implement outcome measures in order to retain their funding. United Way became involved, both because outcome measurement is a good approach to delivering value for donor money and because funded agencies needed help to implement outcome measurement.

The SMART Fund in Vancouver, British Columbia, Canada describes Outcome Measurement as "… an approach to planning, managing, and evaluating projects that encourages us to be clear about what our projects are DOING and what they are CHANGING”. The way that Outcome Measurements help organizations do this is by ensuring that programme objectives support the agencies strategy and that the goals and objectives of component projects support the programme’s objectives. Further, measurements are put in place to determine the project’s success in meeting stated goals and objectives. Outcome Measurement goals go further:

• To gather the right information to know whether the project is achieving the results you want.
• To know how to improve project activities based on the information gathered.
• To identify opportunities and maximizes them and identifies obstacles and helps remove them.
• To communicate plans and achievements clearly to the team and other organizations.
• To gain from the knowledge, experience, and ideas of the team.
• To provide accurate and convincing information to support applications for funding.

There are other goals that are specific to the Not for Profit sector (as is the support of funding applications) but the ones stated here seem to me to be applicable to any project, no matter which sector it is in. Even the goal of providing accurate and convincing information to support funding applications is applicable. Isn’t a Business Case a form of funding application?

There are clearly many areas where the discipline arising from a successful Outcomes Measurement programme could help a project manager from the for profit sector. The key difference between the not for profit and for profit sectors is the focus on helping people in the not for profit sector vs. helping a company’s bottom line in the for profit sector. This gap may not be that great though, particularly in the area of software. Software systems are designed to be used by people to perform a job, to enter and process orders, or manage support calls, or sell software over the internet for example. The objective of the system is to help the user community to do their jobs or help them do them more efficiently. One of the problems with software development projects is the view that the overarching goal of the project is to deliver a software system; it shouldn’t be. The overarching goal should be to enable the user community to do their jobs, or enable them to do them more efficiently! Exposure to a project team that focuses on delivering a benefit to people as opposed to delivering a product will help project managers to get more value from their projects and in doing so, they improve their impact on the bottom line.

The experienced project manager can contribute to the not for profit’s Outcome Measurements programme. Our experience managing risks and opportunities are, or should be, well developed; this is a knowledge area that we should have lots of experience in. Opportunities and risks are no different in the not for profit project than in a for profit one. The management of those opportunities and risks requires the same analysis, prioritization, and mitigation strategies that takes place in the for profit project so this is an area that the seasoned risk manager can be of help as a volunteer in a not for profit organization.

Outcome Measurement uses indicators to determine the project’s success at meeting goals and objectives. The indicators used are divided into 2 categories: quantitative and qualitative. Quantitative indicators use metrics for measurement. The goal might be to deliver meals to shut-ins and the indicator might be the number of meals delivered. The indicator might be an interim one, measuring progress towards a long term goal. Outcome Measurement uses target indicators as well, so if the goal is delivery of meals the target indicator might be 10,000 meals and the indicator might be the number of meals or the percentage of the target indicator. Project managers do this every day, using MS Project (or whatever other PM tool they use to manage their schedules). Qualitative indicators approach measurement from the personal angle. A qualitative indicator might be an increase in self-confidence, or in the case of the meal deliveries, it might be a degree of satisfaction with the service. Any project manager who has participated in a customer satisfaction survey will have a head start in this area. Project managers tend to have a fairly restricted view of indicators. Most will come from the MS Project tool, those that don’t usually come from a trouble reporting system, or a time tracking system. Outcome Measurements uses a set of criteria to determine the feasibility of an indicator:

• Accuracy – does it measure the result?
• Can the information be gathered in a cost effective way?
• Does it give useful information with which to make management decisions?
• Will the information communicate well to stakeholders, including funding bodies?

You may want to apply these same criteria to choosing performance indicators for your project.

Outcome Measurements offers different approaches to information gathering. These approaches are suited to the service delivery nature of the not for profit sector but can be adapted to the for profit sector as well. They are:

• Interviews, surveys, and questionnaires.
• Observation
• Documents and records
• Sample or focus groups (subsets of the stakeholder community)
• Data sampling – surveying a subset of the stakeholder community

We’re all pretty well acquainted with collecting information from documents, records, and other Project Management Information Sources (PMIS), but how about surveys? I would advise any project manager to get well acquainted with the not for profit agency’s interviewing, survey, and questionnaire techniques. Try borrowing those techniques to perform a client satisfaction survey with project performance.

There are 2 approaches the seasoned project manager can take to volunteering. They can look for an agency that has yet to implement their Outcome Measurements programme (or one that is struggling to do so), or one that has successfully implemented and has project managers and other workers who are experienced. Project managers who choose to volunteer to the novice agency can bring their for profit experience to bear on the problem and help the agency to a successful implementation. There are also learning opportunities here. There are many resources available to an agency that wants to reach out, including coaching/mentoring from more experienced organizations. This could be your opportunity to learn from that coach or mentor.

Choosing an agency that has experience with Outcome Measurements will provide the project manager with new insights into defining project objectives that fulfill a human need. It will also provide you with new insights into how to choose measurements and the information to perform those measurements, in particular measurements that are meaningful to stakeholders. There is also an opportunity to learn about putting together the business case. Funded agencies are very focused on proving their programmes are effective in meeting their clients needs and being able to verify with reports. The ability of these reports to communicate successful results means the difference between receiving funding and going out of business.

Look for opportunities in your community to volunteer with a charitable organization. Opportunities abound and your offer will be acted on. It’s a win-win situation: the charitable organization gets your volunteer services and your project management expertise; you get exposure to their outcome measurement processes. There is one other benefit you should experience. One of the areas that charitable organizations are measured in is their ability to attract, retain, and utilize volunteers. Most successful charitable organizations are very good at putting your experience to good use and making you feel appreciated.