By default, BIG removes outliers from the results that we report for your organization. However, you do have the option of including them if you like. The following article describes what an outlier is and the impact it may have on your results.
Definition: An outlier is a performance indicator observation / result that is either much larger or much smaller than the majority of the other performance indicators.
What is an outlier’s impact on the peer performance range?
Outliers can be of concern since a few outliers can sometimes be enough to distort the overall mean/average performance of the peer group. In other words outliers can cause the mean/average to shift either higher or lower depending on the value of the outlier(s); this is particularly true when dealing with a small number of observations/results (indicators).
The BIG performance benchmarking approach utilizes the peer range best quartile (25th percentile) and median (mid-point) performance levels of the peer group as screening targets. The peer group mean/average performance is not considered or displayed in any of the BIG performance screening methodologies and reports. This helps to minimize the impact of outliers on the peer performance ranges since quartiles are less likely to be significantly impacted by outliers as compared to the potential impact on the peer median/average.
Is there a common approach to dealing with outliers?
There is no universally accepted approach or framework to treat outliers. There is much debate about how to identify an outlier and whether or not they should be excluded from the overall results.
How does BIG’s performance benchmarking approach handle outliers?
BIG’s approach to identifying and removing potential outliers is described briefly below.
- There must be more than 8 hospitals in a performance range before any outliers are considered for removal. This provides for a least 2 hospital results per quartile and avoids removing outliers from smaller samples sizes that may or may not in-fact be outliers.
- Outliers are screened and removed for the following performance indicators:
– Net Cost based performance indicators (eg., Net Cost as a Percentage of Direct Care Costs, Net Cost per Square Meter Cleaned, etc.).
– Productivity indicators (eg., Worked Hours per Patient Day, Worked Hours per Attendance, etc.).
- Outliers are not removed for the following indicators:
– Variable Non-Labour Non-Drug Cost per Workload – These are screened at the median and as such removing outliers would have minimal impact.
– Drug Cost per Workload Indicator – Drug cost indicators are provided for information purposes and are not benchmarked. Reporting and accounting for drug cost distribution can vary significantly among hospitals.
- Outlier identification:
– The minimum peer result must be at least 65% of the Best Quartile (25th percentile) result. If the minimum value is greater than 65% below of the Best Quartile value, then the minimum value will be removed as an outlier.
– The maximum peer result cannot be more than 65% higher than the Worst Quartile (75th percentile) result. If the maximum value is greater than 65% above the Worst Quartile value, then the maximum value will be removed as an outlier.
Do I need to remove outliers?
No…the choice is entirely up to the hospital. After you establish your peer group you have the option to turn on or off the outlier screening tool. It is as simple as that. You can run one set with it on and one set with it turned off to see if there is a significant difference.