Qualitative Evaluation Can Help Keep Your Eye on the EHR Ball

EHR implies different things to different people. Many think about and define EHR in terms of a particular system with which they have experience. Others think about EHR in the abstract, questioning what an EHR should do. Many definitions are based on long lists of “requirements” rather than on descriptions of goals and objectives. The number of definitions probably exceeds the number of people who have given it any thought and there is no consensus about the details. There should be little to quibble about with the following common-sense, high-level definition of EHR:

An EHR is a computer system that combines a lifetime medical record with functions that improve the efficiency and effectiveness of the patient care process, the goal of which are to reduce risk to patients and improve outcome. The extent to which a computer system helps a practice achieve these goals can be evaluated either qualitatively or quantitatively. A qualitative approach avoids the ever-present urge that many have to nit-pick about the details while missing the big picture.

As an example, the following is an extensive quote from the latest drug bulletin published by our hospital pharmacy:

High Risk Medication Administration Shortcuts

When medication administration policies and procedures are circumvented to save time medication errors occur. Please avoid the following high risk behavior (shortcuts) regardless of how much time it may save.

• Administering medications before verifying that it is the right patient and right drug.
• Skipping important double-checks due to time constraints.
• Preparing IV solutions on the unit, thus bypassing pharmacy verification and specialized aseptic techniques
• Administering medications before conducting a physical assessment of the patient and/or checking vital signs, lab values, weight, and allergy status
• Removing medications from automated dispensing cabinets (PIXIS) via override without a pharmacist’s review
• Borrowing medications from one patient to administer to another patient
• Not thinking critically about drug administration while rushing through verification of new orders and actual drug administration
• Documenting administration at the scheduled time, but actually giving the drug early or late
• Altering drug administration schedules to avoid documenting late administration
• “Pre-pouring” medications ahead of time for one or more patients to accelerate the drug administration process
• Leaving medications in the room for the patient to take at the right time.

These are all worthy objectives and achieving them would be much more meaningful, in terms of patient safety and outcome, than any of the “meaningful use” criteria established by the government.

There is a simple way to decide whether an EHR can be of value here. Perform a qualitative assessment of how well an EHR meets the definition presented above. To do this, simply go though the list of goals and objectives, such as the ones in the drug bulletin, and ask: Does the EHR help, interfere with, or have no impact on achieving each goal. For those with a statistical inclination, the Sign Test (a non-parametric statistic) can be used to determine whether there are enough positive answers, relative to the negatives, to conclude that the EHR is demonstrably beneficial.

Consider two of the largest-selling EHRs, Epic and Cerner. Based on anecdotal evidence from users and the notorious YouTube video of Epic, it would be of little help in achieving any of these pharmacy-related objectives. A published report [Yong Y. Han, Joseph A. Carcillo, Shekhar T. Venkataraman, Robert S.B. Clark, R.; "Unexpected Increased Mortality After Implementation of a Commercially Sold Computerized Physician Order Entry System"] suggests that the Cerner system is difficult or impossible to configure in a way that it would be beneficial with respect to these pharmacy objectives.

The goals and objectives of a practice are not limited to pharmacy issues. Similar lists can be compiled that address preventive measures, work flow issues, disease-state management, care coordination, etc. One might go so far as to say that if a practice has not developed those lists before embarking on an EHR acquisition then there will be no objective criteria on which to base any decision; no way to assess whether a particular EHR could be of benefit prior to adoption; and no way to tell whether the benefit was obtained after implementation. If the EHR cannot be expected to provide benefit, why spend billions on it? If does not provide benefit once it is in use, why continue to sink money into it? If the product does not conform to the definition presented above, is it an EHR at all, or merely a business-oriented system dressed in clinical clothing?