FrontPage => Stages of Implementation => Optimization and Maintenance =>
or => Building Blocks =>
Metrics
It is important to define objective, measurable goals for the project and develop metrics around these goals. These goals could relate to turn-around time, throughput, provider adoption, patient quality, provider quality, risk avoidance, etc. Once developed, these goals should be measured pre-implementation in order to establish a baseline for where the organization is in its manual environment. Once the system is implemented, the same metrics should be measured to see what improvements have been derived from use of the technology. This will help to identify the value of the investment in terms of improved patient quality, improved efficiency, etc. Some organizations have done this well while other organizations wish they had done this prior to their implementation as they had not been able to quantify benefits for their board and other key project stakeholders.
This page provides a checklist of some common CPOE metrics and gives some additional references.
Metrics for the CPOE Project
These metrics are used during the CPOE project to help evaluate and manage the progress and success.
- Project schedule (actual / planned)
- Project cost (actual / planned)
- Project scope (actual / planned)
- Project risks, risk level
- System defects, results of tests and pilots
- Participation level of physicians and other clinicians
Metrics related to adoption of the CPOE system
These metrics are useful throughout the early years of the system.
They help to manage roll out and adoption of the CPOE system.
- Physician use of CPOE
- Physician use of order sets
- Use of order sets increased from 37 percent of physician compliance (with old paper-based order sets) to a rate of 93 percent at Summa Health System, Akron, Ohio, a five-hospital, 2,060-bed health system (Healthcare Informatics 2009 September;26(9):40-44).
- Verbal order rates
- Verbal orders not signed within required time
- Order changes prompted by CDS alerts
- Number of CDS alerts implemented
- Number of CDS alerts triggered
- Number of order sets implemented
- Number of order sets used
Metrics related to clinical quality and operations
It is important to monitor and respond to metrics related to clinical quality and operations throughout the life of the system.
One of the reasons CPOE and EMR systems are implemented is to improve access to this data.
There is value in obtaining benchmarks of many of these metrics prior to system implementation.
The benchmarks can be used to identify improvements or declines in performance due to the CPOE system and workflow changes.
Productivity
- Physician ordering (time study)
- Physician rounding (time study)
- Pharmacist interventions (count of phone calls to physicians and changes in orders)
- Nursing order management activities (time study)
- Ancillary department order management activities (time study)
Turn around time
- Physician order to lab results
- Physician order to radiology results
- Physician order to first dose medication administration (STAT med, antibiotics, overall)
Financial performance - cost reduction
- Length of stay (overall, by diagnosis)
- Reduction in duplicate lab and radiology orders
- Medication substitution (Formulary, Generics, Switch to Oral from IV)
- Return on investment (ROI)
Patient safety, outcomes and clinical excellence
- Adverse medication events
- Adverse medication events fell from 0.09 events per 1,000 doses in 2003 to just over 0.04 in 2008 - in other words, to less than half the pre-CPOE level (at Children's Hospital of Pittsburgh, which has been live enterprise-wide with CPOE since October 2002) (Healthcare Informatics 2009 September;26(9):40-44).
- Medication error rates
- Adverse drug event rates
- Orders resulting in Adverse Drug Event or Medication Error where a system alert related to the error was generated
- Mortality and complication rates
- Readmission rates
- Compliance with Core Measures / Accountability Measures (CMS / The Joint Commission)
- Errors attributable to CPOE / EMR system
ARRA Meaningful Use
- Stage 1 Objectives and Measures met
- Stage 1 Clinical Quality Measures that are reportable from the system
- Stage 1 Clinical Quality Measures that are showing improvement or decline
Metrics related to IT systems and user support
Use of advanced clinical systems will result in increased demands on the IT infrastructure and the support processes.
It is important to measure the reliability and performance of the IT systems and to respond to problems quickly.
It is also important to monitor the satisfaction of all users with the system and the support processes and make improvements as needed.
The demands for a mature security program also become more important.
System availability
- Application availability (Uptime)
- Application performance (Response time)
- Modules installed / needed
- Client device availability (devices / user)
- Output device availability (Printers / faxes / label printers)
System integration
- Interfaces installed / needed
- Interfaces uptime
- HIE integration installed / needed
- HIE integration uptime
Security compliance (HIPAA, HITECH and JCAHO)
- Security standards in compliance / not in compliance
- Security training conducted
- Security reviews conducted
- Security incidents
- Security vulnerabilities addressed / not addressed (patches, risk assessment findings)
System support
- Number of support resources available
- Ratio of support resources / devices supported
- Ratio of support resources / users supported
- Help Desk tickets resolved / outstanding
- Customer satisfaction with help desk and support
- Vendor tickets resolved / outstanding
- Enhancement requests resolved / outstanding
Sources
HIMSS Topical Review on Quality Metrics
HIMSS Quality 101 Website
CPOE Lessons Learned in Community Hospitals, Massachusetts Technology Collaborative, September 2006
Saving Lives, Saving Money in Practice: Strategies for Computerized Physician Order Entry in Massachusetts Hospitals, Massachusetts Technology Collaborative, January 2009, Erica Drazen, Bethany Gilboard, Jane Metzger, Emily Welebob.
Unpublished report on CPOE metrics from David Bauer, MD, medical informatics consultant to CareTech Solutions and emergency physician at Crittenton Hospital Medical Center
The Joint Commission Core Measures
Office of the National Coordinator for Health Information Technology
Information on the ARRA Meaningful Use Objectives and Measures including the Clinical Quality Measures
A Time-Motion Study to Evaluate the Impact of EMR and CPOE Implementation on Physician Efficiency
Amen A. Amusan, PMP, MS; Scott Tongen, MD; Stuart M Speedie, PhD; and Andrew Mellin, MD, MBA, Journal of Healthcare Information Management — Vol. 22, No. 4