Chapter 13 Assessing Health Care

Assessing and Improving Health Care Quality

After reading this chapter, you will be able to:

  • Describe and understand the multiple dimensions of quality in health care, i.e., what can and should be improved (MCC 78-4);
  • Define quality improvement and related terms: quality assurance, quality control, continuous quality improvement, quality management, total quality management; audit (78-4);
  • Summarize the importance of patient safety and the various forms of adverse events, including adverse drug events;
  • Discuss error mitigation strategies (e.g., effective teamwork, identifying best evidence-based practices) (78-12)
  • Describe approaches to managing care quality;
  • Describe the main methods of quality improvement.

Linking these topics to the Medical Council exam objectives, especially section 78-4 and 78-12.

Note: The colored boxes contain optional additional information; click on the box open it and to close it again.
Words in CAPITALS are defined in the Glossary

Reviewing medications

Dr. Rao visits Mrs. Richards in her long-term care home. He notices her blood pressure is high and is about to write a prescription for an antihypertensive medication. First, however, he reviews her current medications and sees that a locum physician prescribed a non-steroidal anti-inflammatory drug (NSAID) for osteoarthritis the previous summer. He suspects that the hypertension may be secondary to the NSAID, and so he suggests stopping the NSAID. He knows that daily acetaminophen can also raise blood pressure, so may not be the best option.1 Instead of starting an antihypertensive, he suggests some exercises to increase her mobility and reduce her blood pressure. He schedules another visit to check the effects of this management approach.

Patient Safety and Medical Errors

Every health care professional wants to do a good job and help patients. However, things do not always go according to plan and the safety of patients may be compromised. Practices are not always based on the best available evidence; sometimes laboratory results get lost and tests have to be repeated. Sometimes patients are given medications to which they are known to have contraindications. Practices are not always based on the best available evidence – examples of adverse events and medical errors.

Terms relating to adverse outcomes

Three concepts should be distinguished:

  1. Adverse outcome: an unsatisfactory outcome that arises as part of the natural course of the disease, for instance, a gastrointestinal bleed due to a stress ulcer in a patient with pneumonia.
  2. Adverse event: unintended injury or complication that was caused by health care management, rather than by the patient’s underlying disease, and that could lead to severe outcomes, including death, disability or prolonged hospital stays.2 Adverse events include complications that arise from treatment, inherent (and often unforeseeable) risks of interventions, and errors. For instance, a gastro-intestinal bleed may occur even with adequate monitoring of warfarin treatment.
  3. Medical error: an act of omission or commission in planning or execution that contributes (or could contribute) to an unintended result – usually but not necessarily a negative result.3 Examples would include a gastro-intestinal bleed in a patient on warfarin treatment that was not monitored adequately. Medical errors include errors of execution, such as slips (operating on the wrong side), lapses (forgetting to check for allergies), or fumbles (needlestick injuries). Such errors in automatic functioning are often due to fatigue or stress. Other errors are due to mistakes or failures of reasoning that result in a poor choice of treatment plan (treating for heartburn when the patient has classic symptoms and signs of myocardial infarction). These errors can be due to gaps in knowledge or to a failure in attitude resulting in inadequate investigation.

Because many adverse outcomes arise from inherent risks rather than from error, it is prudent to use the term “adverse event” unless error has definitively been established.

A useful resource is the glossary of medico-legal terms put out by the Canadian Medical Protective Association.

Patient safety has become a universal concern and is more likely to be improved by overall quality improvement than by piecemeal responses to adverse events. In Canada, the Romanow Commission found that responsibility for ensuring quality and safety of care is distributed among several different players and organizations in the health care system, including numerous professional and regulatory bodies.4 In such a system of divided responsibilities, lines of accountability can be unclear, so required actions may not be carried out or may not be followed up, and errors occur. Furthermore, sharing of information and learning are inhibited. Following the publication of the Canadian Patient Safety Strategy, a Patient Safety Institute was set up in 2003. Also, the Health Council of Canada was created in 2004 to provide a perspective on health care issues, including quality and safety, and their impact on the health of Canadians. Most provinces have created agencies responsible for examining the quality of care and for informing both public and government of their findings. Many hospitals have some kind of safety audit, often in the form on morbidity and mortality rounds, and physicians can be required to carry out regular assessments of their practice. However, coordinated and integrated structures to support patient safety are still lacking.5 

In most medical situations, people are protected from possible hazards by several layers of barriers. The most immediate barrier is the skill and knowledge of the people caring for them. Routines in the way procedures are carried out also protect patients. For instance, good sterile technique reduces wound infection and good management and organization help staff to apply safety procedures and make safe decisions. But no protective barrier works 100% of the time; some have transient or lasting defects. Generally, this does not result in an adverse event because other checks and barriers protect the patient. Injury results only when defects in several protective barriers align, exposing a vulnerable patient to a hazard. As Swiss cheese is known for its holes, James Reason proposed the ‘Swiss cheese model of error’ to represent the idea of multiple barriers against failure that each have limitations (Figure 13.1).6

Figure 13.1: The Swiss cheese model of error (adapted from Reason14, 15)
Figure 13.1: The Swiss cheese model of error (adapted from Reason6, 7)

Quality and safety in health care

The Canadian Patient Safety Institute provides for exchange of ideas and tools and resources relating to patient safety.

Archived material from the Health Council of Canada offers general information about health care quality and outcomes.

One method of classifying medical errors and adverse events is to relate them to the stage of medical intervention at which they occur:8

Diagnostic

  • Error or delay in diagnosis
  • Failure to employ indicated tests
  • Use of inappropriate tests
  • Use of outmoded tests or therapy
  • Failure to act on results of monitoring or testing

Preventive

  • Failure to provide prophylactic treatment
  • Inadequate monitoring or follow-up of treatment

Treatment

  • Error in the performance of an operation, procedure, or test
  • Error in administering the treatment
  • Error in the dose or method of using a drug
  • Avoidable delay in treatment or in responding to an abnormal test
  • Inappropriate (unnecessary) care

Other

  • Failure of communication
  • Equipment failure
  • Other system failure

Adverse drug events

Adverse drug events form a particular area of concern, and are significant causes of death of hospitalized patients.9 In long-term care facilities, one study estimated that two thirds of residents experienced an adverse drug event over a four year period; one in seven of these events resulted in a hospitalization.10 Elderly adults and people with complex disease are at especial risk, for whom a ‘prescribing cascade’ may occur. This happens when an adverse drug event is not recognized as such but is approached as a new medical problem that is then treated with another drug.11 The Beers Criteria and similar lists indicate drugs to avoid in older adults to avert adverse drug events.10, 12 A simple mnemonic for possible types of reaction to drugs is shown in the Nerd’s corner: the ABCDEF of adverse drug reactions.

The ABCDEF of adverse drug reactions13

Types Descriptions Examples
Augmented Dose-related- Common and predictable reaction- Related to the pharmacological mechanism of action of a drug Digoxin toxicity
Bizarre Non-dose related- Uncommon and unpredictable- Not related to the pharmacological mechanism of action of a drug Penicillin hypersensitivity
Chronic Dose-related and time-related- Uncommon- Related to the cumulative dose Chronic oral steroid use
Delayed Time-related- Uncommon- Usually dose-related- Occurs some time after the use of a drug Radiotherapy-induced cancer
End of use Withdrawal of drug- Uncommon- Occurs soon after withdrawal of drug Opiate withdrawal syndrome
Failure Unexpected failure of therapy- Common- Dose-related- Often caused by drug interaction Reduced efficacy of oral contraceptive when used with antibiotics

Monitoring medications

At a practice meeting, Dr. Rao mentions Mrs. Richards’ hypertension. Here is an excerpt from the Minutes of the meeting:

Dr. Middleton: Over-medication is a huge problem in healthcare generally.

Nurse Jennings explained that she includes a drug history in her preventive checks, but patients are often quite vague about what they are taking.

Mr. Bertoli (practice manager): If we had the money we could buy a computer application that checks recent prescriptions for drug interactions.

Dr. Middleton: Yes, but when patients get a prescription from a specialist or a walk-in clinic, our computer application wouldn’t take account of it.

On his way home, Dr. Rao drops into Mr. Roy, the local pharmacist, who says that the pharmacies in the area have a system for tracking prescriptions, although it’s not perfect. Dr. Rao asks if the local physicians could be linked into the system so they could check what drugs their patients are on before prescribing others. Mr. Roy promises to look into it.

After some more discussions in the practice, Nurse Jennings and Mrs. Austen, the receptionist, agree that Mrs. Austen will instruct patients with chronic illnesses to bring all their drugs with them to their appointment, including the over the counter medicines. Nurse Jennings will list the drugs and how the patients take them and put this list in the chart for the doctors to see.

Dr. Rao welcomes this initiative; he sees that they will be able to evaluate it using the number of patients’ charts with drug lists. They might even be able to follow how many prescriptions are changed as a result. However, he worries that it will take too much of Nurse Jennings’s time so he resolves to check with her in a couple of weeks to see how she is doing. He will also check with Mrs. Austen to see if the arrangement seems to be increasing the patients’ waiting time. Dr. Rao wonders if he can persuade the managers of the long-term care home to keep a drug list for each resident.

Patient safety issues and adverse drug reactions direct attention to the overall quality of care.

Quality in Health Care

 Defining quality

Quality of care is the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with professional knowledge.14 Quality and quality improvement involve much more than preventing adverse events; different authors include different facets in their definitions of quality. Table 13.1 lists some of these.

Table 13.1: The dimensions of quality15, 16

Dimensions of quality Health service values
Safety People should not be harmed by an accident or mistakes when they receive care.
Effectiveness People should receive care that works and is based on the best available evidence.
Patient-centred Health care providers should offer services in a way that is sensitive to an individual’s needs and preferences.
Timeliness People should get the care they need when they need it.
Appropriateness of resources Caregivers and professionals should have the appropriate skills for their practice and have access to the information, equipment, supplies, and facilities they need to look after people’s health needs.
Efficiency Supplies, equipment, time, ideas, and information should not be wasted (see Nerd’s Corner box on Using services appropriately).
Equitableness People should get the quality of care that is appropriate to their needs regardless of whom they are and where they live.
Accessibility People should be able to get the right care at the right time in the right setting from the right provider.
Continuity All parts of the system should be organized, connected, and work with one another to provide high quality care.
Population health Health care professionals should recognize their responsibility to prevent sickness and improve the health of the wider community and population.
Rewarding Caring for people who need health care should be a rewarding occupation. Some people consider this a health service value; others do not. Nevertheless, people who consider their jobs rewarding generally produce work of higher quality than those who are unhappy in their work.  Salary level is usually only a minor part of what makes a job rewarding.

Unfortunately, dimensions of quality may conflict with each other. For example, the most effective therapy may not be the most patient-centred or the most rewarding for the provider. Hence, it may be necessary to establish a hierarchy among these criteria.

Using services appropriately

One component of the quality of health services relates to how interventions are used: they may be underused, overused, or misused.14

  • Underuse: Failure to apply effective care that would improve outcomes. Examples include low rates of cancer screening among eligible groups or patients not adhering to therapy for high blood pressure.
  • Overuse: Providing a treatment even if cost exceeds benefit. Examples include unnecessary blood work that will not change a patient’s clinical management.
  • Misuse: Ordering tests or providing treatments that increase cost or the risk of complication and that are not significantly better than other options. Examples include using the latest, most expensive antibiotic when a cheaper or generic one would be equally effective, or ordering an X-ray of an ankle when the Ottawa Ankle Rule indicates that a fracture is improbable.

This simple classification of quality is useful when patient safety is the only focus, but it is one-dimensional. In fact, quality assessment encompasses all the direct and indirect factors that determine patient safety as well as those relating to patient and provider satisfaction (see Table 13.1).

Approaches to improving the quality of care

There are two ways of approaching quality improvement in health care.17 The inspection model (also known as ‘name, blame, and shame’), although inefficient and ineffective, is still in common use. This model derives from the theory of bad apples (one bad apple can spoil the whole box) and holds that once the ‘bad apples’ are removed from the system, their spoiling effect is also removed. There are two major drawbacks to this theory. The first is that most people who make mistakes are not bad apples but act erroneously because of pressures in the system surrounding them. Blaming people encourages them to hide errors and close calls, creating a ‘culture of silence’. The second drawback is that, unlike bad apples that sit individually and unmoving in a box, the health care system abounds with interacting people and complex processes, offering numerous opportunities for error. Monitoring every potential source of error is difficult, consumes resources and has not yet been shown to be effective in preventing mistakes. Furthermore, the standards imposed by an inspection model are usually no more than the bare minimum. The bad apple approach to problems of quality focuses on the symptom or its immediate cause. This is rarely sufficient to prevent recurrences.

The second and more effective way of ensuring quality is to focus on the system that delivers the health care. This ensures that multiple checks and safety nets are set in place to prevent errors, while encouraging open communication about errors and near misses, and also inviting ideas for improving quality.

The Swiss cheese model described above focuses on the system to examine all barriers that should have prevented a lapse in quality: individual and team actions, the environment or context in which the error occurred, and the organizational factors that influence the actions and their context. Thinking along these lines, the Toyota Company developed the ‘Five Whys’ as a simple way to find the root causes of an error. This is adapted to a health context in Figure 13.2.

Figure 13.2: Example of the Five Whys.
Figure 13.2: Example of the Five Whys.

In reality, there are usually several answers to each ‘Why?’ question (this was illustrated in Table 10.7). For instance, the nurse might have been inexperienced or rushed or the light might have been low. Following all the answers leads to a full description of the situation, which gives a number of options for preventing further errors.

Steps in quality improvement

Providing high-quality health care is the goal of all clinical professionals. Many professional colleges and regulatory bodies (for instance, provincial colleges of physicians and surgeons) have incorporated quality improvement practices into their mandate and operational approaches. Most have quality assurance committees and routinely conduct audits of professional practices. Whereas in the past many of these bodies merely responded to complaints, they are now studying and implementing ways of improving quality. In turn, their relationship with clinicians is shifting from a focus on discipline to one of continuing professional education and training.

The systemic approach to quality improvement can be thought of as a continuous cycle that starts by examining needs, goes on to changing practice, checking to see if the planned changes have happened, checking to see if goals have been reached, then beginning again. This should result in a continuous improvement in quality. The cycle is known as the Plan, Do, Study, Act (PDSA) cycle or the Shewart or Deming cycle, after two of the experts who developed it:

  • Plan: Plan what you want to change and make measurements to confirm the problem, using the dimensions of quality as a guide. Diagnose what can be improved and decide on the best way to make the change.
  • Do: Implement the change.
  • Study: Using the indicators in the “Plan” step of the cycle, measure the impact of the change. Look for positive, negative and unexpected impacts, because changes in complex systems can have negative side effects and unintended consequences.
  • Act: Act on the results of the study. The net result should be more modifications and changes. Depending on the results, you then re-enter the “Plan” step again (see Figure 13.3).

It is often better to do many mini-PDSA cycles (i.e., small changes) rather than one big one. Clinicians rarely have the resources to take on a major PDSA cycle. A sequence of successful small cycles is likely to achieve greater improvement in quality than a failed large one.

Figure 13.3: The Plan, Do, Study, Act cycle. (Adapted from the Institute for Health Care Improvement18)

Detailing the steps of a PDSA cycle, the Six Sigma strategy outlines steps in quality management and improvement. Originally proposed in industry, the approach is designed to identify flaws and reduce errors in a production process; it has been applied to health care. Five steps are involved in refining a procedure, program or service:

  1. In consultation with the people affected, define problems and issues within the process. Assemble a team that includes those who will be affected by the likely changes; review issues and plan a response. Establish specific improvement objectives that are feasible and, if possible, measurable.
  2. Measure: Select measures or indicators of these objectives, preferably ones that are easily available. Collect baseline data, qualitative and quantitative.
  3. Analyze the data to identify the root cause of issues or inefficiencies.
  4. Design: brainstorm ideas for improvement and implement a revised process.
  5. In a Control phase, the program is maintained and monitored; the benefits are reviewed and any shortcomings detected, leading to redesign, cycling back to step 2.19 (See, also, the Nerd’s Corner box).

Quality improvement in the context of rising costs has also promoted the concept of Lean Health Care. Lean ideas were originally developed in the Toyota company to reduce inefficiencies and waste in production.20 The principles have been applied in health care, and staff are encouraged to find ways to reduce areas of waste such as the following:

  1. Reduce idle time for staff and patients. Waiting lists, idle equipment, delayed and inefficient meetings all represent wasted resources.
  2. Minimize inventory. Storage is costly, represents tied-up capital, and risks accumulating outdated supplies. Just-in-time availability is ideal.
  3. Eradicate defects and errors. Misdiagnoses, medical errors, hospital-acquired infections all waste resources to correct, increasing costs. A pay-for-performance approach to remuneration could be used to encourage quality.
  4. Improve transportation: ferrying patients to different departments wastes time, creates crowding in corridors and increases the risk of injuries.
  5. Reduce movement. Limit the need to walk, reach or stoop to get supplies; design buildings ergonomically to reduce inefficiency.
  6. Reduce overproduction and unnecessary work: needless tests, duplicate forms to fill out or entering the same information into different systems, delayed discharge of patients are examples of waste by excess and do not improve patient outcomes.
  7. Opportunity costs: all of the above waste staff time and resources and diverts them from patient care. A lean culture reduces waste and improves quality of care, and also builds staff morale and commitment.

QI and the measurement iterative loop

The iterative PDSA cycle is similar to the measurement iterative loop proposed by Tugwell et al.21 This is a cycle in which the health needs of a population or a practice are presented in a logical sequence, from quantifying the burden of illness to identifying the likely causes, to identifying interventions likely to prevent or reduce the illness and evaluating their efficiency. Once an intervention is identified as feasible and effective, it can be applied and the result evaluated. The inputs to the cycle can be adjusted for the change achieved, and the process started again.

Widespread acceptance of the need to improve health care quality has become universal in Canada and represented a shift in perspective. Quality has broadened from an early focus on appropriateness of the process of care to include concern over the patient experience, and then to concerns over equity. But still the system faces immense challenges and the good intentions have not resolved some fundamental problems. “If, together, we can get this right, we can have the quality health system we deserve.”22  Further quality improvement is in the hands of all, and some of the guiding principles are shown in the box below:

Several principles guide actions to improve quality:23

  • The primary responsibility for quality rests with the production worker; in health care, this is the clinician.
  • Quality assurance systems aid, support and strengthen production systems; they are not simply a form of inspection.
  • Workers want to improve quality but need extra energy and skills to do so.
  • Vague exhortations, threats and blame are not effective quality improvement interventions.
  • Statistical methods should be used to measure performance and compare it with standards.
  • Quality assurance is designed to produce improvements, not to exert control.
  • Quality assurance systems should cover all aspects of an organization, including the process of management itself.
  • Quality assurance without consumer involvement is inadequate and incomplete.
  • Quality assurance focuses on systems and system failure, not on individuals and individual failure.

Some leaders in quality improvement

Walter Shewart (1891–1967) is known as the father of statistical quality control. Having a strong scientific background, he adapted the steps of observation, hypothesis generation and testing to quality management, where the steps became Plan, Do, and See what happens.

Edwards Deming (1900–1993) saw that quality depends on systems more than on individuals. In Japan, his work led to the rise of many very powerful Japanese companies, such as Sony, Panasonic, and Toyota. He generally aimed to reduce complexity, duplication, and waste. He saw variation as an opportunity for improvement, not as a problem.

Joseph M. Juran (1904–2008) trained as an electrical engineer and worked around the world as a management consultant. His particular contribution was to focus on the role of management in ensuring quality, reinforcing the idea that the system as a whole and its processes are at the base of most quality problems.

Avedis Donabedian (1919– 2000) practiced family medicine and was recruited into the School of Public Health of the University of Michigan. In 1966 he wrote the paper that introduced the concepts of structure, process and outcome, which is still the dominant paradigm for the evaluation of quality of health care and health care programme evaluation.

James T. Reason (b. 1938) started with an interest in controlling risks in the aviation industry. Reason has worked in a range of industries, including the health system. In particular, he focuses on the study of the safety and reliability of complex, well-defended hazardous systems.

Self-test Questions

1. What are the steps in the quality improvement cycle?

Plan: evaluate the current situation, prioritise what should be done, decide what can be done, decide what to do, and decide how to measure success. This step should be carried out in collaboration with the people that the action will affect.Do: carry out what was planned. Plans may need to be adjusted.

Study: assess what was done, its success, its positive and negative impacts, and obstacles and facilitating factors.

Act:  act on the results of the “study” step and proceed to the next step, which can be a reassessment of the current action or a “plan” step to reassess how the current action has affected the overall situation and whether or not to change priorities and goals.

2. What are the dimensions of quality?

Health care should be safe, effective, patient-sensitive, timely, efficient, equitable, and accessible. It should provide continuity of care. Health care professionals should have the skills and resources appropriate to their role in care. Health care should contribute to the health of the whole population. Finally, as the basis of excellent care is motivated personnel, those providing care should find their work rewarding.

Reflection Questions

1.   In your current attachment or practice, what do you think would improve the quality of care offered?
2.   Develop a plan to address it.

Institute for Healthcare Improvement

References

  1. MacIntyre IM, Turtle EJ, Graham C, Dear JW, Webb DJ. Regular acetaminophen use and blood pressure in people with hypertension: The PATH-BP Trial. Circulation. 2022;145(6):416-23.
  2. Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. Canadian Medical Association Journal. 2004;170(11):1678-86.
  3. Grober ED, Bohnen JMA. Defining medical error. Can J Surg. 2005;48(1):39-44.
  4. Romanow RJ. Building on values: The future of health care in Canada. Ottawa: Privy Council of Canada 2002 [Publication number CP32-85/2002E-IN:[Available from: http://publications.gc.ca/site/eng/237274/publication.html]
  5. Leatherman S, Sutherland K. Quality of healthcare in Canada: A chartbook. Ottawa, Ont.: Canadian Foundation for Healthcare Improvement; 2010 [Available from: http://www.cfhi-fcass.ca/SearchResultsNews/10-02-10/42054d49-16fb-4764-be05-1d03e6ff3bbb.aspx]
  6. Reason JT, Carthey J, de Leval MR. Diagnosing “vulnerable system syndrome”: an essential prerequisite to effective risk management. Quality in Health Care. 2001;10(Suppl 2):21-5.
  7. Reason JT. Managing the risks of organizational accidents. Aldershot: Ashgate; 1997.
  8. Leape LL, Lawthers AG, Brennan TA, Johnson WG. Preventing medical injury. Quality Review Bulletin. 1993;19(5):144-9.
  9. Lazarou J, Pomeranz BH, Corey PN. Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies. JAMA. 1998;279(15):1200-5.
  10. Fick DM, Cooper JW, Wade WE, Waller JL, Maclean JR, Beers MH. Updating the Beers criteria for potentially inappropriate medication use in older adults: results of a US consensus panel of experts. Archives of Internal Medicine. 2003;163(22):2716-24.
  11. Rochon PA, Gurwitz JH. Optimising drug treatment for elderly people: the prescribing cascade. BMJ. 1997;315(7115):1096-9.
  12. McLeod PJ, Huang AR, Tamblyn RM, Gayton DC. Defining inappropriate practices in prescribing for elderly people: a national consensus panel. Canadian Medical Association Journal. 1997;156(3):385-91.
  13. Edwards IR, Aronson JK. Adverse drug reactions: definitions, diagnosis, and management. Lancet. 2000;356(9237):1255-9.
  14. Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA. 1998;280(11):1000-5.
  15. Institutes of Medicine. Crossing the quality chasm: A new health system for the 21st century. Washington: National Academy Press; 2001.
  16. Ontario Health Quality Council. QMonitor: 2009 Report on Ontario’s health system. 2009.
  17. Berwick DM. Continuous improvement as an ideal in health care. New England Journal of Medicine. 1989;320(1):53-6.
  18. Institute for Healthcare Improvement. Science of Improvement: Testing Changes Cambridge, MA: Institute for Healthcare Improvement; 2010 [Available from: http://www.ihi.org/resources/Pages/HowtoImprove/ScienceofImprovementTestingChanges.aspx]
  19. Ilin M, Bohlen J. Six sigma method. Washington: StatPearls Publishing, National Library of Medicine; 2023 [Available from: https://www.ncbi.nlm.nih.gov/books/NBK589666/]
  20. NEJM Catalyst. What is lean healthcare? : NEJM Catalyst Journal; 2018 [Available from: https://catalyst.nejm.org/doi/full/10.1056/CAT.18.0193]
  21. Tugwell P, Bennett KJ, Sackett DL, Haynes RB. The measurement iterative loop: a framework for the critical appraisal of need, benefits and costs of health interventions. Journal of Chronic Diseases. 1985;38(4):339-51.
  22. System Quality Advisory Committee. Quality matters: realizing excellent care for all. Toronto: Health Quality Ontario; 2017 [Available from: http://www.hqontario.ca/Portals/0/documents/health-quality/realizing-excellent-care-for-all-1704-en.pdf]
  23. Gray JA, Austoker J. Quality assurance in screening programmes. British Medical Bulletin. 1998;54(4):983-92.

Français (French)