Connecting simulation and quality improvement: how can healthcare simulation really improve patient care?
=========================================================================================================

* Victoria Brazil
* Eve Isabelle Purdy
* Komal Bajaj

*   simulation
*   quality improvement
*   health professions education

Simulation has an established role in the education and training of healthcare professionals, but its function as a healthcare quality improvement (QI) tool is more emergent. In this edition of the journal, Ajmi and colleagues report on a simulation-based intervention that improved door-to-needle times and patient outcomes in acute ischaemic stroke.1 This prompts reflection on the positioning of simulation-based methods within QI programmes, the role of trained simulation experts as part of QI-focused teams and the directions for future scholarly enquiry that supports integration of these fields.

The improvement report by Ajmi *et al* is a comprehensive and thoughtful example among many reports of simulation-based interventions to improve care processes and patient outcomes. Improved time-based targets in trauma,2 stroke and cardiac care are frequently cited in the literature, as are better resuscitation outcomes3 and compliance with practice guidelines.4 The identification of latent safety threats in clinical environments,5 6 including testing of new facilities prior to opening,7 is also well described. Such research is usually positioned as providing ‘proof’ that simulation ‘works’ for improving patient care. However, confounders and balancing measures may not be rigorously examined in this enthusiasm to demonstrate cause and effect.

## How, why or when simulation works for improving care is a more nuanced question

Team training using simulation can enable improvements in provider *behaviours*,8 including those described by Ajmi *et al*,1 where sequential tasks in time-critical patient journeys can be replaced by ‘parallel processing’. *Location* is also relevant, as ‘in situ simulation’—“taking place in the actual patient care setting/environment”9—affords a closer connexion to the physical environment for ‘real’, often ad hoc teams, to identify enablers or barriers for QI interventions.10 However, the effect could also lie in simulation *shaping the culture and relationships* 11 12 that underpin and support structural or process specific interventions.

Most reported simulation-based improvement efforts describe an intervention with a pre/post method of outcome assessment, without a control group, and researchers often imply that their success was a result of their approach, rather than merely associated with it.

As with most examples, it is challenging to tease out the specific aspects of the simulation programme design and facilitation that may have contributed to the impressive outcomes presented by Ajmi *et al*. Perhaps it was insightful debriefing around team behaviours and leadership for neurology registrars that made the difference? Or maybe it was increased familiarity with the real environment? It could be that getting to know each other’s names and faces through repeated, low stress practice was the key? Or maybe the true effect came from the programme acting as a strong value-signal for the organisation—a message that improving stroke care was a priority thereby facilitating the adoption of the new stroke protocol and motivating practitioners towards that collective goal? It’s probably a bit of all of the above, and more.

## What exactly do we mean when describing interventions for QI as ‘simulation-based’?

The term ‘simulation-based’ is a broad descriptor, including software-based computer modelling,13 skills training and teams practising patient care. Even when more narrowly focused on in situ simulation, the heterogeneity of simulation delivery approaches makes comparisons difficult, and may reflect simulation provider capacity and experience, rather than careful matching of method to objective. Most examples are reported in simulation journals, with simulation expert authors, rather than institutional QI teams, perhaps risking a method being ‘pushed’ at a purpose rather than ‘pulled’ into existing QI processes. The extent to which established QI methods such as healthcare failure mode and effects analysis or Plan-Do-Study-Act14 approaches are integrated within simulation-based interventions is variable, although with some exemplars.15 16 

## What do we mean by ‘works’ when evaluating the effectiveness of simulation-based interventions for QI?

Evaluation through an educational lens considers outcomes ranging from learner preference to patient outcome metrics. However, few educational interventions are measured at the latter end of this spectrum. McGaghie *et al* performed a qualitative synthesis of simulation-based medical education ‘translational science research’, reviewing whether results achieved in the educational laboratory (T1 outcomes) transfer to improved downstream patient care practices (T2 outcomes) and improved patient and public health (T3 outcomes).17 They found that these patient-oriented outcomes were “more likely when simulation-based medical education interventions are embedded in rigorous educational and health services research programs that are thematic, sustained, and cumulative”. Hence, ‘one off’ targeted simulation interventions may improve focused, narrow process goals, but might miss opportunities to cultivate other outcomes such as culture change or transferability to other care processes. McGaghie *et al* also describe cases in which “T3 health services research outcomes can be achieved without obvious educational interventions”,17 recognising that improving care is not just achieved through better training of health professionals. This is a challenge to the dominant paradigm for the use of simulation in healthcare as an educational tool.

Simulation programme leaders are well aware of the need for return on investment and demonstrable patient outcome improvements,18 but this can result in overemphasis on easily measurable time-based targets.19 Ajmi *et al* are working in a field where outcome measures are well defined and quantifiable, but this is not always the case. Interestingly, this challenge has led simulation providers to undertake scholarly work to identify relevant process, outcome and balancing measures. For example, a team attempting to improve paediatric trauma care through simulation recognised lack of an accepted definition for what constitutes a high-quality stabilisation of a traumatically injured child. This gap prompted them to shift research course and focus on deriving such quality indicators within which to situate their improvement efforts.20 

While ‘hard’ clinical outcomes are most visible, other outcomes of simulation—team performance, organisational culture and fostering relationships—that do not lend themselves as easily to common QI measures may be under-reported. For example, the simulation programme presented by Ajmi *et al*, while focused on stroke in the emergency department, may also have outfitted the neurology team with skills and confidence to more adeptly manage other neurological emergencies such as status epilepticus on the wards.

## So, how should simulation programmes and staff conceptualise their role in QI?

Historically, healthcare simulation has claimed connection between its primarily educational focus—on improving skills, knowledge and attitudes in healthcare professionals—and improved patient safety, but in many cases this assumed link is hard to demonstrate.21 

The Society for Simulation in Healthcare (SSH) describes a ‘systems integration’ role for simulation: “those simulation programs which demonstrate consistent, planned, collaborative, integrated and iterative application of simulation-based assessment and teaching activities with systems engineering and risk management principles to achieve excellent bedside clinical care, enhanced patient safety, and improved metrics across the healthcare system”.22 The SSH accreditation standards and the International Nursing Association for Clinical Simulation and Learning both emphasise governance structures and reporting relationships as evidence of translational impact. For example, “…clear evidence of participation by Simulation Program leadership in the design and processes of quality management system improvement activities at the organisational level”.22 

Words matter, and using terms like ‘translational simulation’23 may allow simulation programmes to identify which of their activities are more focused on QI and which are more educationally focused. Both aims are laudable, but clarity of purpose can lessen the risk of diluting impact or disenfranchising stakeholders through trying to be ‘all things to all people’.

In addition to a clear mission, institutional relationships and governance structures supporting integration with QI, simulation programme leadership also needs extended skills for QI roles. Practice and research methods in QI have emerged from different traditions to those in healthcare simulation, and are underdeveloped in most simulation provider faculty development. Similarly, QI practitioners could benefit from perspectives and training in simulation and debriefing. Skills development is likely to be enhanced if simulation and QI staff in the same institutions have opportunities to work closely together.

## Future directions for research and scholarship

Numerous case study examples of using simulation to improve patient care are now being synthesised in systematic reviews,10 yielding practical guidelines for using in situ simulation in QI, and building conceptual frameworks for how impacts are achieved. Sollid *et al* reported on a consensus process “to define priorities in healthcare simulation that contribute the most to improve patient safety”,24 which included two of the five identified priorities clearly at the interface with QI: effectiveness and system probing. Collaboration between practitioners, stakeholders, researchers and journals in these respective fields will enhance progress. We offer our personal reflections for individuals and teams interested in integrating simulation and QI in box 1, to complement providers primary training and experience.

Box 1 
### Suggestions for QI and simulation practitioners interested in closer integration of their fields

* **Read** *—*add articles found in quality/safety or simulation journals that integrate both fields onto your reading list.* 

* **Study**—* *seek out professional development opportunities: courses, workshops, conferences in QI methodology or simulation/debriefing*.

** *Collaborate* **—*identify individuals in your local institution and find ways to work (and research) together.* 

** *Engage—* ** *connect with the larger community of practice working on these topics via in-person meetings or platforms such as Twitter and LinkedIn*.

## So, can simulation work as a QI tool?

Undoubtedly yes, but not always. We look forward to understanding more about when it works (and doesn’t), together with why, how and in which contexts. The thoughtful integration of simulation and QI as fields of practice and research has the potential to enhance the contribution of both to improving patient care.

## Footnotes

*   Contributors All authors contributed to the writing and review of the manuscript.

*   Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

*   Competing interests VB is Professor of Emergency Medicine and is employed by Gold Coast Hospital and Health Service where she works in healthcare simulation. She is also co-producer of the Simulcast podcast (www.simulationpodcast.com). EIP is a senior resident in Emergency Medicine at Queen’s University and recently completed a fellowship in translational simulation at Gold Coast University Hospital. KB is employed as Chief Quality Officer for NYC Health+Hospitals/Jacobi, and Clinical Director, NYC Health+Hospitals Simulation Center. She is also Associate Professor at Albert Einstein College of Medicine

*   Patient consent for publication Not required.

*   Provenance and peer review Commissioned; internally peer reviewed.

## References

1.   Ajmi SC , Advani R , Fjetland L , et al . Reducing door-to-needle times in stroke thrombolysis to 13 min through protocol revision and simulation training: a quality improvement project in a Norwegian stroke centre. BMJ Qual Saf 2019;28:939–48.[doi:10.1136/bmjqs-2018-009117](http://dx.doi.org/10.1136/bmjqs-2018-009117) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjk6IjI4LzExLzkzOSI7czo0OiJhdG9tIjtzOjE5OiIvcWhjLzI4LzExLzg2Mi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.   Härgestam M , Lindkvist M , Jacobsson M , et al . Trauma teams and time to early management during in situ trauma team training. BMJ Open 2016;6:e009911.[doi:10.1136/bmjopen-2015-009911](http://dx.doi.org/10.1136/bmjopen-2015-009911) 
    
    

3.   Josey K , Smith ML , Kayani AS , et al . Hospitals with more-active participation in conducting standardized in-situ mock codes have improved survival after in-hospital cardiopulmonary arrest. Resuscitation 2018;133:47–52.[doi:10.1016/j.resuscitation.2018.09.020](http://dx.doi.org/10.1016/j.resuscitation.2018.09.020) 
    
    

4.   Landham HCT , Wood T , Hollis S , et al . G555(P) Quality improvement in endotracheal intubation in a paediatric emergency department: checklist development and implementation using simulation and action cards. Arch Dis Child 2015;100(Suppl 3):A249.1–A249.[doi:10.1136/archdischild-2015-308599.505](http://dx.doi.org/10.1136/archdischild-2015-308599.505) 
    
    

5.   Guise J-M , Mladenovic J . In situ simulation: identification of systems issues. Semin Perinatol 2013;37:161–5.[doi:10.1053/j.semperi.2013.02.007](http://dx.doi.org/10.1053/j.semperi.2013.02.007) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1053/j.semperi.2013.02.007&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=23721772&link_type=MED&atom=%2Fqhc%2F28%2F11%2F862.atom) 

6.   Patterson MD , Geis GL , Falcone RA , et al . In situ simulation: detection of safety threats and teamwork training in a high risk emergency department. BMJ Qual Saf 2013;22:468–77.[doi:10.1136/bmjqs-2012-000942](http://dx.doi.org/10.1136/bmjqs-2012-000942) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjg6IjIyLzYvNDY4IjtzOjQ6ImF0b20iO3M6MTk6Ii9xaGMvMjgvMTEvODYyLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

7.   Geis GL , Pio B , Pendergrass TL , et al . Simulation to assess the safety of new healthcare teams and new facilities. Simul Healthc 2011;6:125–33.[doi:10.1097/SIH.0b013e31820dff30](http://dx.doi.org/10.1097/SIH.0b013e31820dff30) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1097/SIH.0b013e31820dff30&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=21383646&link_type=MED&atom=%2Fqhc%2F28%2F11%2F862.atom) 

8.   Steinemann S , Berg B , Skinner A , et al . In situ, multidisciplinary, simulation-based teamwork training improves early trauma care. J Surg Educ 2011;68:472–7.[doi:10.1016/j.jsurg.2011.05.009](http://dx.doi.org/10.1016/j.jsurg.2011.05.009) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1016/j.jsurg.2011.05.009&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=22000533&link_type=MED&atom=%2Fqhc%2F28%2F11%2F862.atom) 

9.   Lopreiato JO , Downing D , Gammon W . Healthcare simulation dictionary. Society for Simulation in Healthcare, 2016. Available: [http://www.ssih.org/dictionary](http://www.ssih.org/dictionary) [Accessed 23 Jun 2019].
    
    

10.  Goldshtein D , Krensky C , Doshi S , et al . In situ simulation and its effects on patient outcomes: a systematic review. Bmj Stel 2019.[doi:10.1136/bmjstel-2018-000387](http://dx.doi.org/10.1136/bmjstel-2018-000387) 
    
    

11.  Brazil V , Purdy E , Alexander C , et al . Improving the relational aspects of trauma care through translational simulation. Adv Simul 2019;4.[doi:10.1186/s41077-019-0100-2](http://dx.doi.org/10.1186/s41077-019-0100-2) 
    
    

12.  Maestre JM , Manuel-Palazuelos JC , Moral D , et al . Clinical simulation as a tool to facilitate culture change in healthcare: practical application of advanced learning theory. Rev Colomb Anesthesiol 2014;42:124–8.
    
    

13.  Pitt M , Monks T , Crowe S , et al . Systems modelling and simulation in health service design, delivery and decision making. BMJ Qual Saf 2016;25:38–45.[doi:10.1136/bmjqs-2015-004430](http://dx.doi.org/10.1136/bmjqs-2015-004430) 
    
    [Abstract/FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoicWhjIjtzOjU6InJlc2lkIjtzOjc6IjI1LzEvMzgiO3M6NDoiYXRvbSI7czoxOToiL3FoYy8yOC8xMS84NjIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

14.  Berwick DM . A primer on leading the improvement of systems. BMJ 1996;312:619–22.[doi:10.1136/bmj.312.7031.619](http://dx.doi.org/10.1136/bmj.312.7031.619) 
    
    [FREE Full Text](http://qualitysafety.bmj.com/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTIvNzAzMS82MTkiO3M6NDoiYXRvbSI7czoxOToiL3FoYy8yOC8xMS84NjIuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

15.  Barlow M , Dickie R , Morse C , et al . Documentation framework for healthcare simulation quality improvement activities. Adv Simul 2017;2.[doi:10.1186/s41077-017-0053-2](http://dx.doi.org/10.1186/s41077-017-0053-2) 
    
    

16.  Nielsen DS , Dieckmann P , Mohr M , et al . Augmenting health care failure modes and effects analysis with simulation. Simul Healthc 2014;9:48–55.[doi:10.1097/SIH.0b013e3182a3defd](http://dx.doi.org/10.1097/SIH.0b013e3182a3defd) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1097/SIH.0b013e3182a3defd&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=24492339&link_type=MED&atom=%2Fqhc%2F28%2F11%2F862.atom) 

17.  McGaghie WC , Draycott TJ , Dunn WF , et al . Evaluating the impact of simulation on translational patient outcomes. Simul Healthc 2011;6 Suppl:S42–S47.[doi:10.1097/SIH.0b013e318222fde9](http://dx.doi.org/10.1097/SIH.0b013e318222fde9) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1097/SIH.0b013e318222fde9&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=21705966&link_type=MED&atom=%2Fqhc%2F28%2F11%2F862.atom) 

18.  Zendejas B , Brydges R , Wang AT , et al . Patient outcomes in simulation-based medical education: a systematic review. J Gen Intern Med 2013;28:1078–89.[doi:10.1007/s11606-012-2264-5](http://dx.doi.org/10.1007/s11606-012-2264-5) 
    
    [CrossRef](http://qualitysafety.bmj.com/lookup/external-ref?access_num=10.1007/s11606-012-2264-5&link_type=DOI) 
    
    [PubMed](http://qualitysafety.bmj.com/lookup/external-ref?access_num=23595919&link_type=MED&atom=%2Fqhc%2F28%2F11%2F862.atom) 

19.  Nestel D , Brazil V , Hay M . You can't put a value on that… Or can you? Economic evaluation in simulation-based medical education. Med Educ 2018;52:139–41.[doi:10.1111/medu.13505](http://dx.doi.org/10.1111/medu.13505) 
    
    

20.  MacKinnon RJ , Pukk-Härenstam K , Von Thiele Schwarz U , et al . Defining and measuring quality in acute paediatric trauma stabilisation: a phenomenographic study. Adv Simul 2019;4.[doi:10.1186/s41077-019-0091-z](http://dx.doi.org/10.1186/s41077-019-0091-z) 
    
    

21.  Kumar A , Sturrock S , Wallace EM , et al . Evaluation of learning from practical obstetric multi-professional training and its impact on patient outcomes in Australia using Kirkpatrick's framework: a mixed methods study. BMJ Open 2018;8:e017451.[doi:10.1136/bmjopen-2017-017451](http://dx.doi.org/10.1136/bmjopen-2017-017451) 
    
    

22. Society for Simulation in Healthcare. Committee for accreditation of healthcare simulation programs. systems integration—standards and measurement criteria, 2016. Available: [http://www.ssih.org/Portals/48/Accreditation/2016%20Standards%20and%20Docs/Systems%20Integration%20Standards%20and%20Criteria.pdf](http://www.ssih.org/Portals/48/Accreditation/2016%20Standards%20and%20Docs/Systems%20Integration%20Standards%20and%20Criteria.pdf) [Accessed 21 Jun 2019].
    
    

23.  Brazil V . Translational simulation: not ‘where?’ but ‘why?’ A functional view of in situ simulation. Adv Simul 2017;2.[doi:10.1186/s41077-017-0052-3](http://dx.doi.org/10.1186/s41077-017-0052-3) 
    
    

24.  Sollid SJM , Dieckman P , Aase K , et al . Five topics health care simulation can address to improve patient safety: results from a consensus process. J Patient Saf 2019;15:111–20.[doi:10.1097/PTS.0000000000000254](http://dx.doi.org/10.1097/PTS.0000000000000254)