• If you are citizen of an European Union member nation, you may not use this service unless you are at least 16 years old.

  • You already know Dokkio is an AI-powered assistant to organize & manage your digital files & messages. Very soon, Dokkio will support Outlook as well as One Drive. Check it out today!

View
 

Metrics, Measurement and Reporting

FrontPage => Stages of Implementation => Optimization and Maintenance =>

      or => Building Blocks =>

 

Metrics


 

It is important to define objective, measurable goals for the project and develop metrics around these goals.  These goals could relate to turn-around time, throughput, provider adoption, patient quality, provider quality, risk avoidance, etc.  Once developed, these goals should be measured pre-implementation in order to establish a baseline for where the organization is in its manual environment.  Once the system is implemented, the same metrics should be measured to see what improvements have been derived from use of the technology.  This will help to identify the value of the investment in terms of improved patient quality, improved efficiency, etc.  Some organizations have done this well while other organizations wish they had done this prior to their implementation as they had not been able to quantify benefits for their board and other key project stakeholders.

 

This page provides a checklist of some common CPOE metrics and gives some additional references.

 

Metrics for the CPOE Project

These metrics are used during the CPOE project to help evaluate and manage the progress and success.

 

  • Project schedule (actual / planned)
  • Project cost (actual / planned)
  • Project scope (actual / planned)
  • Project risks, risk level
  • System defects, results of tests and pilots
  • Participation level of physicians and other clinicians

 

Metrics related to adoption of the CPOE system

These metrics are useful throughout the early years of the system. 

They help to manage roll out and adoption of the CPOE system. 

 

  • Physician use of CPOE
  • Physician use of order sets
    • Use of order sets increased from 37 percent of physician compliance (with old paper-based order sets) to a rate of 93 percent at Summa Health System, Akron, Ohio, a five-hospital, 2,060-bed health system (Healthcare Informatics 2009 September;26(9):40-44).

 

  • Verbal order rates
  • Verbal orders not signed within required time
  • Order changes prompted by CDS alerts
  • Number of CDS alerts implemented
  • Number of CDS alerts triggered 
  • Number of order sets implemented
  • Number of order sets used 

 

Metrics related to clinical quality and operations

It is important to monitor and respond to metrics related to clinical quality and operations throughout the life of the system. 

One of the reasons CPOE and EMR systems are implemented is to improve access to this data. 

There is value in obtaining benchmarks of many of these metrics prior to system implementation. 

The benchmarks can be used to identify improvements or declines in performance due to the CPOE system and workflow changes.

 

Productivity

  • Physician ordering (time study)
  • Physician rounding (time study)
  • Pharmacist interventions (count of phone calls to physicians and changes in orders)
  • Nursing order management activities (time study)
  • Ancillary department order management activities (time study)

 

Turn around time

  • Physician order to lab results
  • Physician order to radiology results
  • Physician order to first dose medication administration (STAT med, antibiotics, overall)

 

Financial performance - cost reduction

  • Length of stay (overall, by diagnosis)
  • Reduction in duplicate lab and radiology orders
  • Medication substitution (Formulary, Generics, Switch to Oral from IV)
  • Return on investment (ROI)

 

Patient safety, outcomes and clinical excellence

  • Adverse medication events
    • Adverse medication events fell from 0.09 events per 1,000 doses in 2003 to just over 0.04 in 2008 - in other words, to less than half the pre-CPOE level (at Children's Hospital of Pittsburgh, which has been live enterprise-wide with CPOE since October 2002) (Healthcare Informatics 2009 September;26(9):40-44). 
  • Medication error rates
  • Adverse drug event rates
  • Orders resulting in Adverse Drug Event or Medication Error where a system alert related to the error was generated
  • Mortality and complication rates
  • Readmission rates
  • Compliance with Core Measures / Accountability Measures  (CMS / The Joint Commission)
  • Errors attributable to CPOE / EMR system

 

ARRA Meaningful Use

  • Stage 1 Objectives and Measures met  
  • Stage 1 Clinical Quality Measures that are reportable from the system
  • Stage 1 Clinical Quality Measures that are showing improvement or decline

 

Metrics related to IT systems and user support 

Use of advanced clinical systems will result in increased demands on the IT infrastructure and the support processes. 

It is important to measure the reliability and performance of the IT systems and to respond to problems quickly. 

It is also important to monitor the satisfaction of all users with the system and the support processes and make improvements as needed. 

The demands for a mature security program also become more important.

 

System availability

  • Application availability (Uptime)
  • Application performance (Response time)
  • Modules installed / needed
  • Client device availability (devices / user)
  • Output device availability (Printers / faxes / label printers)

 

System integration

  • Interfaces installed / needed
  • Interfaces uptime
  • HIE integration installed / needed
  • HIE integration uptime

 

Security compliance (HIPAA, HITECH and JCAHO)

  • Security standards in compliance / not in compliance
  • Security training conducted
  • Security reviews conducted
  • Security incidents
  • Security vulnerabilities addressed / not addressed (patches, risk assessment findings)

 

System support

  • Number of support resources available
  • Ratio of support resources / devices supported
  • Ratio of support resources / users supported   
  • Help Desk tickets resolved / outstanding
  • Customer satisfaction with help desk and support
  • Vendor tickets resolved / outstanding
  • Enhancement requests resolved / outstanding

 

Sources

HIMSS Topical Review on Quality Metrics

HIMSS Quality 101 Website 

CPOE Lessons Learned in Community Hospitals, Massachusetts Technology Collaborative, September 2006

Saving Lives, Saving Money in Practice: Strategies for Computerized Physician Order Entry in Massachusetts Hospitals, Massachusetts Technology Collaborative, January 2009, Erica Drazen, Bethany Gilboard, Jane Metzger, Emily Welebob.

Unpublished report on CPOE metrics from David Bauer, MD, medical informatics consultant to CareTech Solutions and emergency physician at Crittenton Hospital Medical Center 

The Joint Commission Core Measures 

Office of the National Coordinator for Health Information Technology 

Information on the ARRA Meaningful Use Objectives and Measures including the Clinical Quality Measures

A Time-Motion Study to Evaluate the Impact of EMR and CPOE Implementation on Physician Efficiency  

Amen A. Amusan, PMP, MS; Scott Tongen, MD; Stuart M Speedie, PhD; and Andrew Mellin, MD, MBA, Journal of Healthcare Information Management — Vol. 22, No. 4