Project Scorecard
Senior executives use Project Scorecards, also known as
Balanced Scorecards, to ensure project activity aligns with the strategies and
visions of the organization. The scorecard is a little like putting the reader
in the driver’s seat of a car. They need a view through a clear windshield to
determine the direction the project is headed in and instrumentation such as
the speedometer, tachometer, oil pressure gauge, and water temperature gauge to
ensure the car is performing correctly and not in danger of breaking down or
crashing. By the way, the reason these scorecards are often referred to as
"balanced scorecards”? Previous to their introduction, executives had only a
view of the financial performance of operations or projects. A need was
identified for a more "balanced” view of activities, one that would include
measurements of other aspects of performance.
Project Scorecards should satisfy 2 project
requirements: the need for a vehicle to communicate project performance and
health to busy executives and the need to compare performance across multiple
projects. Organizations that have Project Management Offices (PMOs), or Project Management Centres (PMCs)
may already have templates for the scorecard. Organizations that don’t still
need to communicate performance to senior executives and a standard means of
measuring that performance. A carefully crafted project scorecard will meet
these needs.
No two project scorecards will be the same, nor should
they. The organizations that they serve aren’t the same so their needs for
project information won’t be the same. The objective of this article is to
provide some tips and techniques for defining the information to be captured in
the scorecard, and the best format for presenting that information.
The tips and tricks described in this article will draw
upon the project management best practices described in the PMBOK (Project Management
Body of Knowledge). You can acquire these best practices by taking a quality
PMP course, or PMP exam preparation training.
Balanced
Scorecard (BSC)
Balanced Scorecards, or BSC, were devised and
introduced by David P. Norton and Robert S. Kaplan in 1992 to correct the
one-dimensional view of organizational performance provided by measurement
instruments of the past. Performance was measured in terms of finances up to
that point and the downside of measuring financial performance was that it lagged
behind other elements of performance. For example, an organization that
undertook a $5M project to capture a 10% increase in market share worth $4M per
year would look great if it were completed for just $4.5M, but ROI could not be
measured until at least the first year after project completion.
Norton and Kaplan proposed measuring organizational
performance in 3 additional areas: Customer, Internal Business Processes, and
Learning and Growth to gain a more balanced view of organizational performance.
Norton and Kaplan believed that an organization performing well in these areas
could expect to perform well financially as well. Performance measurement in
these areas would also allow executives to be pro-active in correcting problems
that would lead to poor financial performance, rather than reacting once the
organization experienced the poor performance.
Balanced Scorecards were meant to measure performance
in all areas of the organization, not just project performance. A project could
fall under any one of the 4 areas of performance, or even under several, but
will only count for one facet of organizational performance. The use of BSC by
the organizational will definitely influence the design of the project
scorecard, but the project scorecard cannot duplicate the format of the BSC
because the information available does not satisfy the requirements of the BSC.
Key
Performance Indicators (KPI)
Key Performance Indicators, or KPIs, is an acronym
frequently used in conjunction with Balanced Scorecards. Balanced Scorecards
use 5 or 6 metrics in each of the 4 areas of organizational performance as
measurements. The 5 or 6 metrics could be any one of thousands which have been
measured, but choice is constrained by the nature of the area (financial,
customer, business processes, learning and growth), the nature of the
organization, and the nature of the processes and tools in place to capture the
metrics. These metrics are called Key Performance Indicators, or KPIs.
This article is more concerned with giving you practical
advice on composing your project scorecard, and choosing the metrics to support
it, than in the theory of balanced scorecards and KPIs and for that reason I
will avoid the use of these acronyms as they don’t directly help you build your
scorecard.
Choosing
Metrics
Goals
and Objectives
Keep the quality tenet "quality is a product’s ability
to meet customer requirements” in mind when selecting the metrics to base your
scorecard on. The first question a senior executive might ask regarding your
project is "What strategic goal or objective will this project help me meet?”
The answer to this question should be found in your Business Case and/or
Project Charter. There should be one or two over arching objectives described
in those documents which answers that question. Strategic objectives will be
those targets or goals that your organization will be able to meet as a result
of your project delivering its goals and objectives.
Your project objectives might be to produce an
engineering order and configuration system that will store up to 100,000
historical software orders and up to 10,000 configurable software packages,
process a customer order and produce a software operating system for the
customer’s central office telephone switch. Your organization’s strategic
objective might be to increase market share 10% by offering customers the
ability to perform configuration management of their software using your new
software system. Note that it is not the project manager’s responsibility to
achieve a 10% increase in market share, simply to produce the software order
and configuration management system that senior executives believe will enable
them to achieve that goal.
Choose the high level project goals and objectives
which support the organization’s strategic objective and identify them with
that objective. For example, you could make the strategic objective the main
bullet and the project objectives sub-bullets:
- Increase
the Company’s market share by 10%
- Introduce
an engineering order and configuration system which will:
- Store up to 100,000
historical software orders
- Manage up to 10,000
configurable software packages per order
- Process customer orders and
build an operating system
- Manage software library rules governing configuration
- Manage marketing rules
- Identify the software configuration of each customer's switch
The main bullet should be a strategic objective and no
more than one page/slide worth of supporting project goals and objectives
should accompany it.
Overall
Project Performance
The overall project performance indicator should be
derived from 3 or 4 indicators of the project’s health. These are: performance
to budget, performance to schedule, performance to scope, and quality. The
overall project performance indicator is a subjective evaluation of project
health; there is no single metric which can be used.
Red, yellow, and green are commonly used to indicate
overall project health with red being used to indicate a project that is
performing poorly and in need of intervention from sponsors, or steering
committee to put it back on the rails. Yellow is used to indicate a project
that is not performing to established standards but which can be recovered
using resources currently available to the project. Green indicates that the
project is performing well.
Regardless of which means you use to indicate overall project
performance, you should use performance indicator of the project area which is
performing the most poorly to indicate overall project health. If performance
to schedule is yellow then overall performance cannot be green.
Performance
to Budget
Earned Value Management (EVM) provides valuable and
universally accepted metrics for measuring a project’s performance to budget.
The objective is to determine whether you have accomplished the project worked
you planned for the budget you planned. Budget will include the cost of all
goods, services, resources (human and non-human), and administrative services
consumed by the project. There are several different ways of mining the metrics
needed to perform these measurements. The simplest is the MS Project file for the project, which can track all these. MS Project shows money being spent at exactly the same rate
as activity completion. The budget for the C++ programming is exactly 72%
consumed when the C++ programming is 72% complete. This may not be what your
audience is looking for.
Check for financial reports being issued by your
finance organization that measure budget consumption for your project. These
reports are likely to be ones read by senior executives and issuing a financial
report of your own which does not align with the one from the finance
department could lead to hours spent reconciling the 2 reports. Some things to
consider when you gather your metrics:
- Are there any time tracking
tools used by the finance department to track labor costs?
- When does the finance
department consider labor budgets spent? When the cost is incurred? When the
cheque is written? When the cheque is cashed?
- When does the finance department consider the
cost of materials incurred? When the materials are purchased? When they are
delivered? When they are used? When the cheque is issued?
Use reports from the finance organization to base your
reports on and explain how you used them, if possible. If financial reports
based on the project are not available to you, learn the methods used by finance
for their reports and make your metrics conform to your organization’s
financial reporting methods.
There are several different EVM performance indicators
which use financial metrics. The Cost Variance is the simplest. It simply
compares the Actual Cost of Work Performed to the Budgeted Cost of the Work
Performed (CV = BCWP – ACWP). A negative amount indicates a project that is
over budget, a positive amount indicates a project that is under budget. Earned
Value is actually the EVM term for the Budgeted Cost of Work Performed (CV = EV
– ACWP).
The Cost Performance Index, or CPI, is another
indicator of a project’s financial health. The CP is an absolute value in
currency while the CPI is a comparator and can be used to compare a project’s
performance to budget in one period against another period, or against another
project. The CPI is calculated by comparing the variance to 1, with 1 being
exactly on budget. The formula for calculating the CPI is BCWP/ACWP. A CPI
greater than 1 indicates a project that is under budget, a CPI less than 1
indicates a project that is over budget.
Burn rate is a third indicator used to indicate project
performance to budget. The burn rate of the project is simply the rate at which
the project budget is being spent. Faster than the plan? Slower than the plan?
Or exactly to plan? The burn rate is the inverse of the CPI so the formula for
calculating the burn rate becomes: Burn Rate = 1/CPI. A burn rate of greater
than one means your project is consuming budget faster than planned and will
exhaust the budget before all the work is completed. A burn rate of less than
one indicates your project is consuming budget slower than planned and will
complete before the budget is exhausted.
Performance to Schedule
Performance to schedule is the
measurement of how quickly the work of the project is being done. Is it being
done on time? Ahead of time? Behind time? Your project may be performing well
to schedule but performing poorly to budget so the indicators used to measure
cost performance cannot be used to measure schedule performance. Your project
may be ahead of schedule because you spent budget on overtime work to achieve
your head start.
Your MS Project file will be your only source for the
metrics you need to measure schedule performance. The file
should capture schedules in terms of time – hours, days, weeks, or months. The
base measurement for schedule performance is Schedule Variance, or SV. SV can
be calculated using monetary units (as dictated by EVM), or in units of time as
long as you stick with one measure and use it consistently throughout the
project. The EVM formula for calculating SV is BCWP – BCWS (Budgeted Cost of
Work Performed – Budgeted Cost of Work Scheduled). You can use monetary units,
hours, days, weeks, or months as units of measure for BCWP and BCWS. A positive
SV indicates a project ahead of schedule and a negative SV indicates one that
is behind schedule.
Schedule Performance Index, or SPI, is the schedule
equivalent of the CPI and is calculated with the formula SPI = BCWP/BCWS. An
SPI of greater than 1 indicates the project is ahead of schedule and an SPI
less than 1 indicates the project is behind schedule.
Scope
Scope can be viewed through 2 different lenses. The
alignment of the project to the original set of deliverables identified for it
and the amount of time or cost of the approved changes to the scope. Show your
project’s performance to scope by showing the planned deliverables for the
project, the deliverables created, and the cost of the new features approved
for the project. Limit the list of deliverables to key deliverables and
indicate whether they are planned or built.
Changes to scope can be shown as deltas to the
project’s planned scope. Show additional features and functions and their
related costs and also show features and functions that were removed from the
plan with their costs.
Quality
Quality performance can be measured in a variety of
ways. Your principal source of quality metrics should be the issue or trouble
tracking tool used to record bugs found by the QA (Quality Assurance) group.
This tool should be capable of producing just about any report you care to ask
of it and be able to break the recorded issues down according to the following
categories:
- By severity
- By Cause (software bug, data,
question, etc.)
- By area or application
- By status (open, interim,
fixed, closed, etc.)
- By owner (software
development, database administrator, etc.)
- Re-opened tickets
There are 2 metrics that will be of interest when QA
activities are underway: the number of tickets opened in a period and the
number of tickets closed in a period. You may also want to report on the number
of errors found per 1K lines of code, or any other indicator of the quality of
the development work. Just remember that the ultimate goal of Quality
Management is to produce a product that fits the clients requirements so
reporting on a large volume of opened tickets should not be a cause for alarm. Many
more tickets being opened than are closed in a period, or a large volume of
tickets being re-opened may be cause for alarm.
Accuracy
The value of your scorecard will very much depend on
the accuracy of the metrics it consists of. Use discretion when choosing
metrics to report. Only choose those which you can verify. Take care in the
capture and storage of the information you use; start by keeping your MS Project file accurate and up to date.
Your data probably won’t be 100% accurate no matter how
careful you are so inform your audience of your assessment of the data accuracy
and of how the data is captured and retrieved for the scorecard report.
Media
Do not attempt to create a scorecard report using only
text. The report won’t be read. Use a media that best supports the metric being
reported. You may want to stick with the car dashboard/windshield analogy in
which case a traffic light showing red (stop), yellow (proceed with caution),
or green can be an effective visual aid to the overall project performance
indicator. A speedometer showing a range of CPIs with the needle pointing to
the current CPI is also a great visual indicator (if a little gimmicky).
Bar graphs are an ideal means to show historical
information such as trends. Showing the project’s current CPI or SPI tells the
viewer if the project performed at, under, or over for the reporting period but
showing a rolling window of 10 reporting periods worth of CPIs or SPIs tells
the viewer the same thing plus whether performance is improving or degrading.
Trend lines will make the picture clearer and a line at the 1.0 index will show
where the project should be performing.
Scatter diagrams are useful for showing the causal
relationship between two variables, one of which is under the control of the
experiment. The control, or independent variable is plotted along the
horizontal axis and dependant variable is plotted along the vertical axis. This
type of chart is useful in showing the results of a process change on a quality
metric.
There are many off the shelf tool sets that will take
the metrics you wish to report and turn them into a great visual show. You can
use one of these tools and experiment with it’s features, choosing a
combination that suits your audience, or you can use the visual aids you have
available to you to custom make your own.
The tips and tricks described in this article
implement some of the best practices promoted by the PMI (Project Management
Institute). These are taught in most PMP® courses
and other PMP® exam preparation training products. If you haven't
been certified as a PMP® http://threeo.ca/pmpcertifications29.php.
three O Project Solutions also offers a downloadable software based training
tool that has prepared project managers around the world to pass their
certification exams. For more information about this product, AceIt, visit the
three O website at: http://threeo.ca/aceit-features-c1288.php. (Project Management Professional) by the
PMI and would like to learn more about certification, visit the three O Project
Solutions website at:
|