Are pop-ups doing more harm than good?

Decision-support tools are designed to promote patient safety. But when here’s a pop-up for everything, the message can be diluted.

We all know the feeling. You’re in the middle of something – maybe deep into a busy shift in the pharmacy, or on the wards in the hospital, or even scrolling on your phone at home – and another pop-up appears.

You’ve seen it before, so your instinct is to dismiss it without a second thought.

While ignoring the latest alert from your fitness app might mean falling behind on your step count, dismissing a decision-support pop-up in a healthcare setting might mean missing something crucial.

Yet, with so many alerts coming through, it can be overwhelming – and almost impossible – to sort the important information from the rest.

Why do we have popups?

Pop-ups are much like passing a sign on the road. They are meant to inform your decisions, whether this be about a hidden hazard or general navigation. In health care, they have proliferated across the increasing number of digital platforms and pieces of software health professionals use to provide patient care and nowhere more so than clinical decision-support tools.

Decision-support pop-ups take many forms, from alerts about potential drug interactions to duplicate therapy warnings and allergy information. They are intended to aid health professionals in their decision-making and, in the case of dispensing pharmacists, are there to ‘remind pharmacists to be alert and to double-check that the medicine supplied is the medicine prescribed’, says PDL Senior Professional Officer Gary West FPS (he/him).

‘It might only take half a second to view them, but the difference for a patient – and for a pharmacist – can be really significant. We’ve had multiple cases where a locum or newly registered pharmacist hasn’t realised a patient is on a staged-supply arrangement because it wasn’t clear in their profile. A warning message would have prevented that patient from getting access to a larger quantity of medicines than intended.’

Mr West says both pharmacists and dispensary technicians should understand why the alerts are there – to help mitigate risk for patients and themselves.

‘We would always encourage pharmacists to use that technology, and to ensure their dispensary technicians are well trained and alert to raising certain pop-up warnings with the pharmacist associated with the dispensing event,’ says Mr West.

The pop-up paradox

But are we currently overwhelmed by pop-ups? And are some pop-ups better than others?

University of Sydney Professor of Health Research Melissa Baysari (she/her) suggests the answer to both questions is yes. A human factors expert, she has conducted extensive research into how digital health can support medicine management. Despite their good intentions, she says there is limited evidence that alerts change practice or improve patient outcomes, particularly in the hospital setting.

In a 5-year study involving patients across five hospitals in New South Wales and Queensland, Prof Baysari’s team examined whether activating drug-drug interaction alerts reduced harm. One group of hospitals had the alerts enabled, the control group did not.

Surprising results

‘We found overall that the alerts had no impact,’ she says.  ‘Having drug-drug interaction alerts did not reduce the number of clinically relevant interactions or harms experienced by patients. The study shows that if you’re going to turn all the drug interaction alerts on, they’re not going to deliver the safety benefits you expect.’

In fact, she warns that poorly designed or excessive alerts can backfire. ‘There are so many negative consequences, like wasted time, frustration for clinicians and patient safety implications, that I would not recommend turning drug-drug interaction alerts on in your systems unless you’ve got some kind of smart technology in there.’

Despite the lack of evidence about the effectiveness of alerts, organisations are often reluctant to scale them back. Prof Baysari said her team had ‘a lot of trouble’ finding hospitals that were willing to not turn on their drug interaction alerts.

‘Everyone assumed the alerts were effective and they were going to be removing an intervention that was providing a safety net for everyone,’ she says. ‘But the way I see it, if organisations just keep putting more and more alerts in, it becomes the end user’s responsibility to work out which ones are relevant and act appropriately. It’s shifting the responsibility away from the organisation, so if an incident occurs, it’s the clinician who’s to blame. I think that’s very unfair.’

Information overload

Research shows that the number of alerts a healthcare professional receives impacts how effective they are – and pharmacists see a lot, Prof Baysari says.

‘You might assume that there’s an evidence base for alerts, but we have a lot more research showing that when you implement large numbers of alerts, clinicians begin to ignore them. They’re seeing too many, and so they’re not reading the recommendations. I have a theory that this becomes a habitual response – an alert pops up, clinicians override it and move past it. It allows them to get through work quickly.’

This alert fatigue has real-world safety implications. If one alert out of 12 presented is a critical warning, and the other 11 aren’t clinically relevant for the patient, it’s likely the important alert will be missed.

‘We’re increasingly seeing incidents where alert fatigue has been identified as a contributing factor. It’s not that there wasn’t an alert in place, but that it was lost among the other alerts the clinician saw,’ Prof Baysari says.

Returning to the road-sign analogy, there comes a point where there are just too many and you just can’t process them all and they stop being useful. Or you drive past the same ones every day and stop paying attention to them.

PDL has developed its own alerts based on incidents that are frequently reported to the organisation and on medicines that are more likely to have significant consequences for patients.

‘We’re very mindful. We put in a small number of pop-ups that we think are of high importance or significance,’ Mr West says. ‘Our warnings are not theoretical. We try to prevent alert fatigue by limiting warnings to things that occur with frequency or with such consequence that they justify the pop-up. We appreciate it’s difficult when hundreds of prescriptions are dispensed every day, but we’d really encourage pharmacists to take note of them and not just tab through without any consideration.’

Making smarter systems

As pharmacists continue to expand their scope of practice, Fred IT Group Partnerships Manager Regina Cowie (she/her) says the functionality of decision-support tools will need to expand to include options like preventive care recommendations and chronic disease management tools.

‘For example, clinical decision support may be used to identify patients eligible for chronic disease screening or to assess chronic disease management,’ she says. ‘Or a vaccination prompt for eligible patients could support pharmacists to make informed decisions by dynamically changing aspects within dispensing systems to guide on healthcare opportunities, for example to improve vaccination rates.’

As well as being broader, alerts also need to become smarter. Artificial intelligence is already being used in some aspects of dispensing software, Ms Cowie says, and AI has the potential to revolutionise the way clinical decision support is used.

‘AI has the ability to analyse large amounts of patient data and provide opportunities to improve medicine safety and patient care … Fred AID (Artificial Intelligence Directions) is an optional function already in place in Fred Dispense Plus. It uses AI to predict medication directions and allows pharmacists to quickly and safely enter directions for dispensing labels, rather than typing these from scratch. 

‘In future, AI may suggest treatment plans or identify patients at risk of certain diseases, based on patient population data. In health care, where time pressures are a consistent burden on clinicians, a related opportunity from AI is its ability to provide administrative support by automating documentation such as clinical notes.’

Ultimately, there isn’t anything inherently wrong with alerts, Prof Baysari says, they just need to be better. ‘We have very basic alerts at the moment. They’re not designed well, so we’re not seeing the benefits. But we could.

‘We need better design in the backend, using AI to make them smarter, and, in the frontend, we need to think about how they look on screen,’ she says. ‘If we had good interfaces and a good backend infrastructure we’d get good safety benefits. But we have a lot of work to do.’ 

Fred IT Government and Industry Engagement Lead and former Medicines Safety Program Director of the Australian Digital Health Agency Andrew Matthews MPS says clinical decision-support tools such as alerts can increase detection of inappropriate prescribing and dispensing, improve disease management, reduce adverse patient outcomes and enhance appropriate treatment.

Fred IT, for example, has introduced an optional prompt in its dispensing software to help improve vaccination rates for the shingles vaccine in target cohorts aligned with recommendations from the National Immunisation Program (NIP).

The prompt displays only if the patient meets the NIP criteria of the patient demographics entered in the Fred Dispense Plus patient profile.

‘With the large volumes of patient data now available in electronic medical records and pharmacy systems, especially where patient disease complexity makes it difficult to quickly assess the available data,’ Mr Matthews says, ‘clinical decision-support can present relevant patient information to the clinician to facilitate clinical problem-solving, decision-making or provide additional knowledge consistent with best practice guidelines.’