by Bob Stephen
In response to challenges facing the industry, many health care organizations are pursuing performance improvement programs. When discussing these programs, the linked terms “performance measurement” and “benchmarking” will often come up. Performance measurement utilizing benchmarking is an important part of the performance improvement “toolbox.” But it is important to understand both how and when to use this tool.
What do we mean by “benchmarking”?
Benchmarking is the process of comparing how your organization performs a specific activity relative to other organizations. Benchmarking applications include:
- Comparing your organization’s performance with others in similar operating and market environments
- Identifying industry best practices
- Setting and applying standards for performance
- Learning from best practice organizations in relevant industries and organizations
For most of these applications, benchmarking occurs with the use of performance measures. In other words, the goal is not one-time learning, but ongoing tracking of performance.
What is performance measurement?
“Performance measurement” covers a lot of ground. It refers to the regular collection, reporting, discussion and analysis of data that is tied to some measure of process improvement. This data can tie to resources used by the organization, accomplished production, and specific outcomes.
It is often assumed that performance measurement is limited to clinical quality, but the concept applies to all areas of an organization, including tracking progress on strategy, following financial outcomes, and monitoring operational efficiency, customer satisfaction, employee development and satisfaction, and patient quality and safety.
Within this broad category, there are common elements of performance measurement. Each performance measure needs the following:
- Definition (What is the measure?)
The definition explicitly details where the measure comes from, who is responsible, when it is collected, how it is manipulated (if at all), and explains the link to the other measured items.
- Objective (Why are we measuring?)
An objective is critical because it forms the reason for measuring. Typically, we find three types of objectives: performance or quality improvement initiatives (targeted projects to improve specific outcomes or processes), compliance (ensuring certain organization outcomes or processes meet obligations – bond covenants or core quality indicators are examples), and strategy management (includes tools such as the Balanced Scorecard which link measurement to strategy). To be effective, performance measures should have an explicit link to an objective. This link must explain the reason for measurement and how improvement in the measure will lead to the objective.
- Targets and Alarms (How will we know our status?)
Good measures have targets to tell us when we have achieved our goals and alarms to warn when the measure is approaching an unacceptable level. Targets and alarms alert the organization to take action – for example, correcting an issue before it results in unacceptable performance levels or introducing a new objective when current goals are met. Benchmarks – whether internal or external – help drive the definition of targets and alarms.
- Guiding Benchmarks (How do we check our performance?)
Benchmarks provide guidance on current or desired performance. Prior to setting stretch targets, an organization might use external benchmarks to make sure that their current performance is in line with industry peers or to make sure that their targets are set high enough, and not merely meeting an industry average.
- Data (What is the actual performance?)
Each measure must have actual data that documents performance. The data must be appropriate for the objective and must reference a timeframe that is that is relevant to the objective or strategy being measured. While this dimension of performance measurement reflects the definition, it is important to note the issue of quantitative vs. qualitative data. In most cases, an organization will want to use quantitative data in performance measurement. This allows for unambiguous analysis and relatively un-biased measurement. But there are instances, particularly when the objective is strategic management, where quantitative data does not currently exist (and may never exist) and qualitative measures are appropriate. When using qualitative data, internal benchmarking may be preferred.
Benchmarking is an issue that many organizations struggle with relative to performance measurement. These struggles relate to benchmark sources. In some cases, there will be a search for the best comparisons. In other cases, the source may exist, but the organization must decide if the benchmarks are valid or applicable. In reality, the first step for an organization should be to determine if external or internal benchmarks are appropriate. There is a time and place for each; the challenge is knowing when to use one or the other.
If external benchmarks are sought, the question then turns to identifying the right sources for the desired information.
External benchmarks
External benchmarks are summaries for a given performance measure, typically from a similar group of health care organizations – usually peers in terms of size, revenues, markets, etc. Benchmark data may include the mean, median, minimum, and maximum value for the group of peers. External benchmarks can serve to influence the setting of targets and alarms, to give context for organization data, and can provide a learning vehicle in process improvement.
While many external benchmarks allow for tracking over time and are based on a peer group, industry guidelines may also be used. These guidelines may come from clinical licensing, bond rating agencies or accreditation organizations. With guidelines, it is best to use these as a basis for setting measure alarms. (Note: Be sure to set the alarm above the guideline level, so that the organization will have time to react prior to reaching the guideline.)
Advantages
- Provide insights and learning about outcomes achieved at similar organizations – great for incremental improvement.
- Allow the organization to target based on “best practices.”
- Allow the organization to set alarms based on the peer group average.
- Tend to be available for measures that are commonly used or required.
- Provide context for organization performance (are we good, bad, middle of the pack?)
Disadvantages
- Provide limited learning opportunity without information on the unique operating circumstances of other organizations.
- May give unattainable best practices (each organization in the peer group produces results that are an outcome of its unique processes).
- May not match up with the measures the organization should be using.
- Provide an excuse to avoid performance measurement (“we are different so that does not apply”).
- May cause the organization to miss opportunities for significant “breakthrough” improvements.
When are external benchmarks most beneficial?
External benchmarks are most appropriate when the goal is process or quality improvement or compliance. In both instances, the goal is to ensure that performance is above a certain level – aiming for the median of a peer group is a good start. There is also a better chance that performance measures will be identical or similar to those contained in benchmarking groups. For compliance, the benchmarks available will be identical.
It is important to note that external benchmarks are most valuable when the goal is incremental improvement, or a project is in the beginning stages. Typically, in the beginning of a performance improvement project, the first step is to determine the relative performance of the organization - are we in the “ballpark” with our peers? If the organization measure indicates a significant gap between current performance and the median or best practices, then the first step prior to any major improvements is stabilization; it is important to understand what other organizations are able to achieve.
Figure 1 graphically depicts the difference in “incremental” versus “bold stroke” approaches to strategy and performance. “Incremental” refers to strategies and activities which, over time, only change in terms of the amount of resources and efforts applied to the situation. In contrast, “bold stroke” refers to a fundamental shift in the quality of the activity or strategy and represents a significant change in the quality and benefit of the underlying activity. In other words, “bold stroke” means fundamental strategic revitalization, not just doing more of the same thing.
In many cases, “bold stroke” improvement is just not feasible or necessary – a process or outcome may have little overall impact or room for change – in these cases, external benchmarks provide valuable context.
In cases such as the one described in Figure 1, when a bold stroke change is desired, a different sort of external benchmarking may be helpful; looking outside of peer groups and even the industry for ideas on creating breakthrough performance. While Southwest Airlines could not look to other airlines for breakthrough ideas, they could look to racing pit crews. Southwest looked at the traits – teamwork and process innovation – that allowed these racing pit crews to service their cars in rapid fashion. There were no hard numbers to compare, but Southwest was able to learn and apply observations to their industry. This is a concept that should be familiar to many health care organizations, as Disney is often cited as a “best practice” for customer service. Many in health care have observed Disney practices, attended seminars, or read the book If Disney Ran Your Hospital: 9 1/2 Things You Would Do Differently (Fred Lee, Second River Health care Publishing).
One final word about external benchmarks involves their use in special improvement projects, such as Six Sigma and Lean Process Improvement. In both of these types of projects, external benchmarks can be beneficial, but not essential. Experts in both of these areas bring up two specific warnings:
- Benchmarking can create its own “process waste” if too much time and effort is focused on collecting measures and finding external sources. Often this is a distraction from improvement opportunities which are already known.
- Benchmarking direct competitors or close peers can lead to “inbreeding” where less than best practices are studied and opportunities for breakthrough improvement are missed. It is beneficial to look for the truly exceptional organizations in and out of the industry.
Obviously, there are times when external data benchmarks are not the best solution for an organization. These cases tend to relate to cases where “bold stroke” improvement is desired or the performance measures relate to strategy.
Figure 1
Internal benchmarks
As the name implies, internal benchmarks are generated within the organization. For a given measure, an organization may develop benchmarks through:
- Historical comparison – these benchmarks represent improvement over a previous time period (e.g., percent increase in outpatient volumes).
- Link to business projections – these benchmarks are developed from required outcomes or drivers within financial models (e.g., percent increase in gross revenues).
- Organization requirements – similar to business projections, requirements are measures that tie to financial or legal covenants (from a bond issue or merger agreement). These indicate levels at which the organization must operate or face possible default on obligations (e.g., debt service coverage ratio; nosocomial infection rate/100 patient days).
- Management Estimates – Finally, for measures with little history or basis for quantification, benchmarks can be developed using management expertise. Typically, this expertise is used to set targets and alarms (e.g., waiting times for scheduled outpatient visits).
Internal benchmarks can be just as valuable (if not more) than external benchmarks because they are set against a known community or internally developed standard. It is important to note that benchmarks tying to requirements usually drive an alarm setting. But, just as with industry guidelines, the alarm should never be set at the requirement level – there must be room for action before arriving at the required level. For example, a organization with a bond covenant requiring 75 days cash on hand, would want to set their alarm higher (90 days or more) so that the alarm will sound and action can be taken to avoid defaulting.
Just as with external benchmarks, there are advantages and disadvantages for using internal benchmarks.
Advantages
- Match the exact measure used by the organization; this allows the organization to pick the best measures for each objective
- Reflect the uniqueness of the organization (not possible to claim “we are different”).
- Support breakthrough strategies and improvement
- Are easy to obtain and very timely (do not have to wait for compilation by another organization).
Disadvantages
- May show improvement but still be below standards without external context.
- Reinforce existing behaviors without aggressive target settting.
When are internal benchmarks most beneficial?
Internal benchmarks are valuable when it is important to quickly set a comparative standard that fits with the organization, and a specific strategy.
Often, a performance measurement initiative will come to a grinding halt as the organization starts to search for “appropriate” benchmarks. The search may take a while, but even if benchmarks are quickly identified, the issue of whether peer groups are similar or different will often be debated. How many times have you heard the phrase “But we’re different” in you organization?
With internal benchmarks, the information is readily available. In the early stages of a project, internal benchmarks may make sense until pertinent external benchmarks can be found and incorporated.
“Fit” within the organization refers to the fact that internal benchmarks already reflect what is important to the organization. Rather than changing a performance measure to fit a benchmark that may be used by your peers, it is better to keep the existing measures and use an internal benchmark.
Finally, evaluation of strategic performance usually requires internal benchmarking. No matter how similar two organizations are, their strategies will differ. This reflects the unique markets, cultures, and competitive situations for every organization. In cases where performance measures are tied to a strategic objective, it is not appropriate to set targets and alarms based on benchmarks built from other organizations. Internal benchmarks are usually the best approach.
Benchmarking and balanced scorecards
The balanced scorecard is often a tool used to align organization strategy to operations. This differs from a “dashboard” or indicator report. Within a Balanced Scorecard, many of the included measures will be strategic and unique to a specific organization; therefore, many Balanced Scorecard measures rely on internal benchmarks for setting targets and alarms.
The balanced scorecard areas that we often find using external benchmarks are customer satisfaction, quality and safety, and employee satisfaction. For customer and employee satisfactions, organizations using the same external services can include benchmarks for their selected measures. Core measures provide a starting point for quality and safety, but because every scorecard is unique, the specific satisfaction measure (based on survey questions) or core quality measure chosen will likely differ from one organization to the next. Measures must reflect what is important. For example, two hospitals may elect to include a core quality and safety measure, but one may select needle sticks and another may select falls. There is no one universal set of measures or benchmarks that should be included on a balanced scorecard.
External benchmark sources
There are a variety of options available for selecting external benchmarks. Subscription benchmarking services exist where organizations pay a yearly fee to collect a wide variety of financial, operational, satisfaction, and quality data – in return the organization can compare itself to a peer group. Figure 2 offers an example of one of the comprehensive external benchmark resources available in the market. If you are considering such a service, the following should be considered:
- Cost – Make sure the cost of the service is appropriate for the value received. Remember, external benchmarks are usually balanced with internal benchmarks.
- Ease of use – The service should be easy to use. This refers to both data collection and data access. Consider internet based tools, as this should speed both data entry and report generation.
- Relevant information – The information collected must be appropriate to your organization type and size. Services that collect and report on numerous indicators may seem impressive, but are all of the indicators necessary? Fewer targeted measures are more useful. For example, a critical access hospital will find more value from a service focused on essential measures for hospitals of similar size and situation. Note that some measures such as patient or employee satisfaction will only be relevant if all service members are using the same satisfaction tools.
- Appropriate peers – Make sure that a benchmark service has similar organizations in its database and allows for custom selection. For example, a hospital with long-term care will want to select other similar hospitals with long-term care.
- Ability to interact – Perhaps most importantly, make sure you will have the opportunity to discuss results with other participating organizations. The ability to learn from others makes the benchmarking process more valuable. This can also help to understand if the benchmark values presented are truly valid for your organization. Services connected to regional networks, systems, or groups are particularly valuable, as they allow for constant discussion and the ability to influence which measures are collected.
As an alternative to a formal benchmarking service, consider a best of breed approach. Under this approach, the organization looks for peer benchmarking sources or opportunities that are focused on a specific area of the organization (financial, quality, etc.) or similar organizations (similar geography and size).
Annual reports from national organizations can also be valuable resources – while these reports do not provide peer benchmarking, they often supply median values for common measures. These report-based measures should be used as guidelines. Figure 3 offers a starting point for exploring available benchmarking resources.
Figure 2: An example report from a benchmarking subscription service.
Conclusion
Benchmarking and performance measurement can be effective tools to aid performance improvement, but, they should be used properly and never be the sole focus of the improvement project.
In most cases, a organization will benefit from a mix “blended” approach – using external and internal benchmarks; using internal benchmarks to set targets and alarms in the initial performance measurement stages, and adding external benchmarks to add context (and adjust targets/alarms as necessary) over time.
About the Author
Bob Stephen is a Wipfli health care business professional with more than 15 years' experience in the industry. He applies cutting-edge business tools and analytical techniques to health care organizations across the nation. He can be reached at (703) 250-2578 or bstephen@wipfli.com.
Figure 3: Sources for Measure Benchmarking
The following is a partial list of measure benchmark sources. Some of these resources require subscriptions, while others are free. Many organizations will have additional resources available through parent organizations or local networks. Additional resources are available through vendor organizations (customer satisfaction, employee satisfaction, materials management are examples).
Comprehensive (Includes multiple benchmark areas)
Financial
Customer Satisfaction
Quality and Safety