Academia.eduAcademia.edu
State of the art metrics for nursing: a rapid appraisal Peter Griiths with Simon Jones, Jill Maben and Trevor Murrells 2008 State of the art metrics for nursing Acknowledgements This work was commissioned and supported by the Department of Health in England as part of the Policy Research Programme which provides funding to the National Nursing Research Unit. The views expressed are those of the authors, not of the Department of Health. We thank the members of the Nursing Outcome Measures task and inish group chaired by Professor Anne Marie Raferty and convened by Professor Dame Christine Beasley, the Chief Nursing Oicer for England. These experts’ insights and experiences informed us as we undertook this review to support their work. The work presented here is our appraisal of the literature and is not intended to represent the full breadth of work undertaken by that group. We particularly thank members of the group and others who provided support and comments, and Janice Sigsworth, the Deputy Chief Nursing Oicer who was directly responsible for commissioning this report. Contact address for further information: National Nursing Research Unit King’s College London James Clerk Maxwell Building 57 Waterloo Road London SE1 8WA nnru@kcl.ac.uk http://www.kcl.ac.uk/schools/nursing/nnru i State of the art metrics for nursing ii State of the art metrics for nursing Table of contents Acknowledgments i Foreword v Overview 1 Introduction 2 Possible indicators 6 What makes a good indicator? 10 Evidence base: associations between nursing and outcomes 14 State of the art 17 Conclusion and recommendations 23 References 26 Appendix 1: Core data sources 31 Appendix 2: Indicators identiied 34 iii State of the art metrics for nursing iv State of the art metrics for nursing Foreword The search for outcome measures for nursing has a long and distinguished history tracking back to Florence Nightingale herself. Her epidemiological approach to outcomes research and measurement has attracted admiration from statisticians and epidemiologists alike. Despite this prestigious pedigree, progress on reining metrics for the outcomes of nursing care has been slow. The prioritisation of quality within the Next Stage Review of the National Health Service in England and the commitment to hold trusts accountable for and to reward quality of care promises not only a renewed but relentless focus on quality of care. This report reviews the status of the evidence base on nursing metrics and provides a road map and set of recommendations to take nursing forward. I am grateful to members of the task and inish group who helped to identify candidate metrics in key domains of care and provided examples of good practice from their organisations. As Professor Griiths and his team demonstrate, while there is much to be commended in such practice there is still much more to do. Developing metrics is only the irst step in building a robust infrastructure for implementation and fully integrating nursing into the governance and management of the NHS at all levels. Such metrics can then enable the public to make informed decisions about their care based on criteria which matter to them as well as to managers and clinicians. We have a unique and unprecedented opportunity to make the quality of nursing care count. This report is an important step in that process. Professor Anne Marie Raferty Dean and Head of the Florence Nightingale School of Nursing & Midwifery at King’s College London Chair of the Nursing Outcomes task and inish group v State of the art metrics for nursing vi State of the art metrics for nursing Overview • The Next Stage Review commits to quality measurement that relects the compassion, safety and efectiveness of nursing care. As such measurement provides both a challenge and an opportunity for the profession, this report reviews ‘state of the art’ nursing quality measurement. • Many possible indicators and existing indicator sets measure nursing’s contributions. Among the most widely used indicators are safety measures such as failure to rescue (death among patients with treatable complications), falls, healthcare associated infection and pressure ulcers. Neither efectiveness (positive contributions to well-being) nor compassion (elements of patient experience) are strongly represented in the existing measures. • We did not set out to focus speciically on acute general inpatient nursing care, but examples from this setting dominate both indicators and evidence reviewed because there has been more development in these areas. Many themes apply equally to other areas even if speciic indicators difer, and the lessons learned in acute care can assist in developing indicators for all specialties and care settings. • To be useful, indicators must be measurable with available data at reasonable cost. There must be evidence that the quality or quantity of nursing substantially contributes to changes measured by the indicator. The indicator must be recognised as important (by the public, managers and nurses) and nursing’s contribution must be recognised (by nurses and others). • Nurses must have responsibility for actions leading to outcomes in terms of legitimate authority, self-perception and sphere of practice. Measures should be chosen to minimise the risk of gaming, where improving performance on speciic indicators detracts from overall improvement. Measures focussing on the performance of care (process) rather than outcome are most vulnerable to gaming. • Not all existing indicators meet these requirements, and considerable work will be needed to develop practical, valid and useful indicators. Strong evidence supports an association between nurse staing levels and mortality, but mortality is determined by many causes and is not likely to be a useful quality measure for nursing. • This report identiies ‘best bets’ for indicator development, including measures of safety, efectiveness and compassion. Health care providers should form a quality coalition, facilitated by stakeholders includinf the Royal College of Nursing and The NHS Confederation, to share best practices and to move toward standard measurement of important nursing indicators. A programme of indicator development must include development of technical speciications for indicators, research to validate them and, crucially, patient involvement in identifying metrics for compassion. • The development of metrics and the establishment of a National Quality Board and local quality observatories present a substantial opportunity for nursing to equip itself with the tools needed to deliver excellent care, and these initiatives should be welcomed within the profession. 1 State of the art metrics for nursing Introduction This work was commissioned to inform the Nursing Outcome Measures task and inish group chaired by Professor Anne Marie Raferty and convened by the Chief Nursing Oicer for England as part of the nursing contribution to Lord Darzi’s Next Stage Review of the National Health Service. The group was tasked with identifying mechanisms for giving nurses tools, training and support to improve quality of care across the country, including: • Evidence-based metrics to measure nurse-delivered outcomes and patient experiences; • National publication of performance data to identify best-practice examples and help nurses benchmark and improve their performance; and • “Ward-to-board” accountability for the quality of nursing care. The Next Stage Review makes a commitment to develop an indicative set of metrics for nursing1 that comprises of indicators of quality relecting the issues of safety, efectiveness and compassion identiied by the task and inish group. This work will support a wider NHS initiative to establish regional quality observatories and a National Quality Board that will oversee the development of a quality measurement framework for all clinical services. Indicators serve to foster understanding of a system and how it can be improved, and to monitor performance against agreed standards or benchmarks. Crucially, indicators provide a mechanism by which care providers can be accountable for the quality of their nursing services. Accountability for nursing quality exists at many levels, from the point of care (where individual nurses are accountable to clinical managers and patients) to senior management, commissioners and beyond. While “ward–to-board” accountability for care is frequently referred to, a still wider view of accountability includes both the public and policy-makers, including health service users, the general public and funders of care2. A measuring system is needed, with a set of indicators that can: • quantify trends and characteristics; • describe performance in achieving health service goals (in this case, elements to which nursing strongly contributes); and • provide information to improve nursing care. Nursing-sensitive indicators The group was tasked with inding measures of “nurse-delivered outcomes and patient experience”. We take this to mean measures that directly relect nursing’s end results in terms of impact upon patients. A large body of work describes quality measurement systems that focus on multiple aspects of quality in the nursing process3 or on nursing outcomes in terms of activities completed4. While occasionally the relationship between a particular process and an outcome is so strongly established that a measure of the process 2 State of the art metrics for nursing may suice or even be preferable as a proxy indicator, our reference point has been patient outcome (including experience). Patient outcomes and experiences vary for many reasons and relect the work of multiple professions. Indeed, in many cases the greatest determinant of outcome is the patient, whether because of underlying health status, behaviour or aspects of the wider social environment. In considering nursing-sensitive outcomes and experience, we must identify elements of variation that can be attributed largely to nursing care quality. To do this we must seek evidence of correlation with nursing as well as evidence that this correlation is a plausible consequence of variation in nursing rather than other factors. Determining such plausibility requires professional knowledge of possible mechanisms and technical knowledge, such as research evidence that adjusts for confounding factors. These types of indicators can be used5: • to improve quality in local settings by monitoring and managing performance; • to support policy analysis and strategic decision-making, including commissioning, reimbursement systems and accreditation; and • to research the role of nursing care in determining patient safety outcomes by examining structure-outcome, process-outcome and structure-process-outcome relationships. The irst two uses signal that diferent information may be required and useful at diferent levels. The third purpose of indicators is not the immediate concern of this report, but such research is necessary to develop indicators it for the other uses. Context This initiative comes at a time of widespread public perception that nursing quality is sometimes poor and lacking in essential elements6-8, as represented in the oft-repeated accusation of nurses being ‘too posh to wash’. Recent reports on infection outbreaks in UK hospitals have highlighted situations where underlying issues of nursing care quality were given a low priority in the face of competing productivity targets9-11. Concerns about nursing care are not limited to the general public or to the UK. There is ongoing professional concern that nursing’s contribution to quality health care is underrecognised, leaving nursing services vulnerable to cost-reducing eforts12 ,13. Nursing’s contribution to quality care is not consistently recognised; an Audit Commission report on hospital staing variations14 concluded early in this decade that “Unless and until trusts that spend more [on staing] can demonstrate a clear link with the quality of care that is delivered, movement towards a more even allocation of resources seems reasonable both for patients and staf.” (p15) This presumes there is no link to quality unless otherwise demonstrated, yet a document from the same Audit Commission programme15 states: “It is diicult to … avoid the conclusion that they [variations in staing] must result in diferences in the quality of care available to patients in diferent trusts and on diferent wards.” (p3) 3 State of the art metrics for nursing While varying nursing quality is clearly a concern, tangible ways to demonstrate nursing’s contribution to quality care are less clear. In the UK, responses to healthcare acquired infections may have been instrumental in reasserting nursing’s central and fundamental role in providing a safe environment for care16. Yet this represents a narrow view of nursing’s potential contribution to patients’ experiences, health and well-being. A burgeoning number of recent studies explore the impact of variation in the quality and quantity of nursing care on a wide range of outcomes and experiences17. An increasing number of indicator sets identiied as nurse-sensitive5 are used by national and local governance or quality improvement programmes. The task and inish group examined difering systems used in a variety of NHS organisations. By bringing the nursing contribution to the fore, the Next Stage Review and the resulting commitment to measure the compassion, safety and efectiveness of nursing care provide a challenge and an opportunity for the profession. In this light it is timely to explore how nursing might demonstrate its contribution to quality outcomes and patient experiences, as well as how quality indicators can hold all service providers accountable. Developments must build on existing evidence and initiatives for consistency across settings and to ensure that best practice is used. This paper explores potential nursing-sensitive indicators identiied from published literature and indicator sets currently in use. The requirements of a good set of indicators for nursing are explored and evidence for indicators’ validity is considered through an examination of evidence linking nursing contributions and patient outcomes. The conclusion assesses the current state of the art in nursing-sensitive indicators. We did not set out to focus on acute general inpatient nursing care, but both the indicators and the evidence reviewed are dominated by examples from this setting because there has been more indicator development in these areas. While the state of the art may be more advanced in acute care, many themes apply equally to other areas even though speciic indicators may difer. Certainly the lessons learned in acute care can usefully underpin development of indicators for all specialties and care settings. 4 State of the art metrics for nursing Box 1. Introduction: summary • Evidence abounds of public and professional concerns that nursing care quality is variable and sometimes poor. • Nursing care quality has often failed to receive high level-attention in the face of competing productivity targets. • There is a demand for measures whereby nursing can demonstrate and be held to account for its contribution from point of care to the board room. Such measures also are useful to all sectors of society beneiting from and making policy for health care services, ranging from the public to politicians. • The Next Stage Review and resulting commitment to quality measurement relecting nursing care’s efectiveness, safety and compassion provides both a challenge and an opportunity for the profession. • This paper explores potential nurse-sensitive indicators identiied from published literature and sets of indicators currently in use. This evidence base is used to determine the current ‘state of the art’ of metrics for nursing. 5 State of the art metrics for nursing Possible indicators A wide range of potentially measurable indicators of nursing care quality can be indentiied from nurse-sensitive outcomes. These outcomes are aspects of patient experience, behaviour or health status (patient outcomes) that are determined in whole or part by nursing care received and variations in its quality or quantity. The precise construct is variously deined, and while most deinitions tend to focus upon outcomes as results of speciic nursing interventions18 ,19, others have emphasised system characteristics20 such as team functioning, staing levels and skill mix as important determinants of outcome. Quality indicators also can derive from known or widely presumed links between nursesensitive outcomes and nursing interventions or structural characteristics. For example, use of a nutritional risk assessment might be identiied as an indicator of quality because it is identiied as a nursing intervention leading to improved outcomes (improved nutrition). Similarly, workforce variables such as staf satisfaction or skill mix might be used as indicators because of such variables’ known or presumed relationships with important patient outcomes. Data sources used A number of sources yielded possible indicators for nursing, including Doran’s review of nurse-sensitive indicators21 and recent systematic reviews linking the ward environment, nurse staing and patient outcomes22-25. Various indicator systems were selected on the basis of their strong potential to give a high coverage of nurse-sensitive indicators, and on the basis of their prominence and advocacy within high-proile bodies such as the American Nurses Association, the UK’s NHS and the US Joint Commission for the Accreditation of Healthcare Organisations (JCAHO). This coverage was generally identiied because the indicator set was proposed as nurse–sensitive. Some relevant indicator sets were developed for care settings where nursing takes a signiicant active lead (e.g. community and home healthcare), even where the set was not proposed for nursing per se. Appendix 1 lists key sources used. To this set were added indicators identiied using database and web searches of terms such as nurse-sensitive indicators and metrics. This yielded links to additional indicator sets and locally developed ‘dashboards’ of indicators. The intention was not to achieve a comprehensive list, but to give an accurate overview. The authors explored these issues within the task and inish group and gleaned further examples from members. We stopped searching when new searches failed to identify new domains and no new individual indicators emerged that were substantively diferent from those already identiied. Few indicators were identiied solely on our general web search, suggesting that we have successfully identiied the bulk of the available content although alternative speciications of the indicators we have identiied might exist. We have not 6 State of the art metrics for nursing considered systems such as the Nursing Outcomes Classiication System26 because they intend to provide a comprehensive taxonomy for recording the outcomes of speciic interventions rather than to serve as general indicators of quality. Indicators identiied Appendix 2 lists the broad range of indicators identiied. Depending on the level of precision of deinition, a list of nursing quality indicators could run into the hundreds. The majority of sources identiied were speciic to acute general hospital care, although many indicators could apply across other settings in adapted forms. The level of deinition ofered varied considerably between sources. Reviews necessarily were less precise in specifying outcomes and data-collection protocols. Variables generally were clearly deined, although some (e.g. ICN*) deined variables in very general terms. Existing indicator sets were generally the product of an extensive development process and ofered precise speciications for data (e.g. relevant ICD codes, exclusions and riskadjustment models such as NQF). Other sources such as the AUKUH listed speciic indicators, but ofered little in terms of data speciication or operational deinitions while the ICN indicated only broad areas. Essence of Care benchmarks conversely ofered considerable conceptual detail but little detail on measurement. The broad topics most speciic to nursing are incorporated here. A range of patient outcomes was identiied including aspects of knowledge, function (including instrumental activities of daily living and continence), nutrition, experience (including communication, satisfaction and complaints), preventative care such as vaccinations, safety outcomes (e.g. failure to rescue, falls, infections, medication errors, mortality, pressure ulcers), symptoms such as pain and dyspnoea and utilisation outcomes such as hospital stay and unplanned admissions. Also identiied as possible indicators were processes directly linked to these outcomes (e.g. pain assessment, risk assessment) or relating to general aspects of quality such as planning and care coordination. Nursing workforce characteristics featured heavily as well: possible indicators included staing levels, skill mix (including qualiied nurses’ levels of educational preparation), team expertise, staf turnover rates and indicators of team functioning, such as interprofessional relations and perceptions of practice environment quality. Staf outcomes such as wellbeing and injury rates also were identiied. Table 1 lists indicators that four or more sources identiied, and the assignment given to them by the task and inish group in terms of safety, efectiveness and compassion**. Similar indicators are grouped for brevity, even though sources’ precise deinitions * See Appendix 1 for deinition of the acronyms used to refer to indicator sets ** Safety refers to adverse efects of care, efectiveness refers to positive beneits and compassion refers to aspects of patient experience such as perceived dignity, respect and quality of communication. Appendix 2 gives a fuller list of indicators identiied; these are classiied into topic areas using an earlier schema focussing on content devised before the task and inish group devised its broad classiication 7 State of the art metrics for nursing and speciications of indicators may difer. While the number of sources that identify a particular indicator hint at its level of support, this should not be taken as a literal measure because our sample of sources was somewhat arbitrary. In particular, we discuss below why number of sources does not indicate evidence strength. Table 1. Most frequently identiied indicators Indicator Area Number of sources Pressure ulcer Safety 11 Failure to rescue Safety 9 Staing levels Efectiveness 9 Falls Safety 8 Health care associated infection: pneumonia Safety 8 Staf satisfaction and well-being Efectiveness 7 Health care associated infection: urinary tract infection Safety 6 Staing, skill mix Efectiveness 6 Medication administration errors Safety 5 Mortality Safety 5 Practice environment/perceived quality Efectiveness 5 Satisfaction with (nursing) care Compassion 5 Sickness rates Efectiveness 5 Smoking advice Efectiveness 5 Staing bank or agency use Efectiveness 5 Communication Compassion 4 Staf experience, knowledge, skills and expertise Efectiveness 4 Health care associated infection: surgical wound Safety 4 Instrumental activities of daily living and self–care Efectiveness 4 Perception of adequate staing Safety 4 Use of restraints Compassion 4 Adverse events dominate the ield of nurse-sensitive indicators5. Despite our inclusion of broader-based reviews such as Doran’s21 this remains the case here and is relected by the high proportion of indicators of patient safety, particularly in relation to outcomes*. Most efectiveness indicators stem from a decision to interpret structural staing outcomes * The relative balance between the types of indicators is in part a product of the level at which they are grouped. For example, the OASIS indicators include a detailed list of instrumental activities of daily living which can be considered as several indicators or, as we did, one (IADL). 8 State of the art metrics for nursing or other measures of workforce experience as indicating an ‘efective’ workforce under this schema. Such indicators could equally well have been identiied as safety measures, since much of the available evidence relates to the link between staing and patient safety. Interpretation of items such as satisfaction under ‘compassion’ rests upon the potential of the largely patient reported measures (including satisfaction measures) to incorporate elements of patient experience that relect upon this dimension. Even allowing for their difering natures and sources, the lack of overlap Savitz5 noted remains. Indeed, agreement between sources is exaggerated by our grouping of similar indicators. However, there does seem to be a degree of consistency in identifying as indicators in acute-care settings: • failure to rescue (death among patients with treatable complications27); • falls (sometimes deined in terms of resulting injury); • health care associated infection (although the precise infections/speciication vary considerably); • pressure ulcers; and • staing levels. Nonetheless, the range of candidate indicators and lack of consistency in their content and precise deinition make it essential to identify desirable indicator characteristics. Box 2. Possible indicators: summary • Many possible indicators and indicator sets exist, mainly but not exclusively in acute care. However, little consistency exists between indicator sets. • The most widely supported outcome indicators include failure to rescue (death among patients with treatable complications), falls, healthcare associated infection and pressure ulcers. • Other commonly advocated indicators were patient experience, including communication with staf and support in activities in daily living; medication administration errors; the practice environment’s perceived quality; and workforce aspects including staing levels, skill mix and measures of well-being or satisfaction. 9 State of the art metrics for nursing What makes a good indicator? The National Quality Forum require indicators to be:important, scientiically sound useable and feasible28. Importance is deined in terms of • national goals • impact of outcomes on individual service users • the societal burden of disease • availability of strategies for improvement, • substantial variation in quality, or • quality which is consistently substandard. Scientiic criteria include • precise speciicationof the indicator • reliability and validity of measures and • adequacy of risk adjustment. Usability is the extent to which intended audiences can understand results and are likely to ind them useful in decision-making while feasibility relates to the ability to obtain quality data in a timely manner with a demand on resources that is proportionate to beneits.These criteria are generally relected in the NHS Institute for Innovation and Improvement’s Good Indicator Guide2, which speciies that indicators should describe “as much about a system as possible in as few points as possible”. (p5) So nursing indicators must be measurable with available data at reasonable cost, coding and recording must be consistent and complete and measures must be valid. If an indicator is to be used to represent the quality of nursing it must be attributable to nursing in a number of senses, including: • evidence of sensitivity to nursing • recognition of the phenomenon’s importance • recognition as a nursing contribution (owned by nurses and acknowledged by others) and • recognition as nurses’ responsibility in terms of legitimate authority, self-perception and sphere of practice Usability requires that there is suicient knowledge to inform action since identiication of a particular level of achievement is not useful unless a strategy for improvement can be identiied. Indicators must apply to many patients and types of clinical services if they are used to represent general quality across specialities. Similarly, variation in a nursingattributable outcome must be substantial if the outcome is used to indicate nursing quality as opposed to care in general. If an outcome is used to compare diverse populations and 10 State of the art metrics for nursing clinical settings, it must be possible to adjust for difering baseline risks to ensure fair comparison. The evidence behind any nursing process or structural indicator also must establish that it is linked to outcomes29,30. The choice of indicators must minimise the risk of gaming or perverse incentives. Collecting data can be seen as unwarrantedly intrusive and burdensome at any organisational level31 and proper organisational support and infrastructure for data collection and feedback are essential. To maintain commitment, attention should focus not just on the initial burden but also on the information’s perceived value. Timely and informative data must be available to provide behavioural incentives and if necessary allow remedial action to be taken. The challenge is to turn the data into “actionable information” 31 so feedback mechanisms and formats such as the increasingly popular graphics-based ‘dashboards’ and traic light systems are important31,32. It is essential that actionable information be acted upon. As one member of the task and inish group noted: “…outputs have to be used and the emerging issues addressed, otherwise staf very quickly become cynical.” Perhaps the most diicult balance to strike is between the data collection burden and the need to ensure that performance on indicators represents broad achievement of the goals of nursing or processes that deliver those goals. The more indicators that are collected the greater the data collection burden but the danger of focussing on a few narrowly deined indicators lies in the creation of perverse incentives33 where maximising performance on indicators detracts from overall performance in other aspects or changes performance in relation to the indicator that invalidates it by removing its relationship to overall quality. Numerous examples of such gaming behaviours have been noted within the English NHS34. One notable example was the 48-hour target for GP appointments that led to practices refusing to ofer appointments more than 48 hours in advance34. Indicator selection must therefore consider the potential for gaming and seek to minimise this. Essentially the aim is to identify a relatively small number of indicators that still relate clearly to the multifaceted and somewhat elusive concept of quality nursing care. All indicator sets are incomplete; the challenge is to identify indicators that are important in themselves and also strong indicators of overall quality. Bevan and Hood note that indicators must be selected from a narrowly deined subset of target areas where both data and measures of suiciently high quality are available34 (see Figure 1). The fundamental issue that indicators are primarily decided by available measures cannot be avoided. The limitations of available good measures may point towards using imperfect measures, even though this may in efect “open a can of worms”34 because such measures provide an incomplete and inaccurate picture. While pragmatic decisions may inluence the choice of indicators and imperfect measures may be used in suiciently important areas, such decisions may create additional opportunities for gaming and perverse incentives. 11 State of the art metrics for nursing Figure 1. The relationship between the system and indicators (after Bevan & Hood 2006) Total domain Subset of interest Subset with imperfect measures Subset with good measures This risk is illustrated by two examples relating to possible nursing indicators. One relates to the use of routinely collected data for indicators such pressure ulcers or HCAI. Such secondary diagnoses are notoriously poorly recorded35. While the chances of a patient developing pressure ulcers may relate to the quality of nursing care, early detection and proper documentation of pressure ulcers is also a marker of quality care which could lead to higher rates of recording in good-quality settings than in lower-quality ones. Thus a perverse incentive might be created for lower levels of surveillance and documentation among highperforming teams. It is certainly hard to see the incentive for poor-performing teams to increase their reporting. Risk adjustment for pressure ulcers is further problematic and risk assessments are not routinely conducted, centrally recorded or particularly discrmininating36 so comparisons between units (such as wards or hospitals) are diicult to interpret. In the face of such problems, measures of process seem an appealing alternative. But the apparent ease that many process measures ofer in resolving the diicult measurement issues associated with outcomes may be outweighed by the choice of a process indicator because it is measurable, but which has doubtful links to patient outcomes or experience. Process data are rarely routinely available5 (which removes a potential advantage) and it is hard to capture in routine audits the complex interpersonal care processes involved in many nursing interventions. Thus indicators such as completed pressure ulcer risk assessment tools may come to be adopted even though token compliance is possible (for example, if completion of the form takes priority above identifying need). Furthermore, the speciic process may be adopted because it is measurable rather than because it is known to be efective. In the case of formal pressure ulcer risk assessment tools, accuracy and efectiveness are both uncertain36. Resolving these issues can never be straightforward and must be a matter of informed judgement about potential beneits from deploying an imperfect indicator as opposed to no 12 State of the art metrics for nursing indicator. Evidence of links between nursing and outcome must be further examined, since without this knowledge any indicator has questionable utility. Box 3. What makes a good indicator: summary • Indicators must be measurable with available data at reasonable cost. • There must be evidence of variability associated with nursing and this variability must be substantial. • For process or structure measures, evidence must support links to important outcomes. • The indicator must be recognised as important (by the public, managers and nurses) and the contribution of nursing must also be recognised by nurses and others. • Nurses must have responsibility for actions that lead to the outcome in terms of legitimate authority, self-perception and sphere of practice. • There must be suicient knowledge to inform remedial action. • Measures should be chosen to minimise the risk of gaming, where improving performance on the indicators detracts from overall performance. • Measures, especially measures of outcome, generally need to be risk adjusted to ensure comparability across settings. 13 State of the art metrics for nursing Evidence base: associations between nursing and outcomes Kane et al.22 provide the strongest single source of evidence for a link between nursing and outcomes. This systematic review examines the impact of a general nursing variable, the quantity of nursing care available, and assesses the extent to which nursing inluences the indicators. The review included 96 studies linking nurse staing to patient outcomes. Increased RN staing was associated with lower hospital-related mortality (per additional full-time equivalent nurse per patient day) in: • intensive care units (odds ratio 0.91, 95% conidence interval 0.86–0.96) • surgical units (odds ratio 0.84; 95% conidence interval 0.80–0.89) • medical patients (odds ratio 0.94; 95% conidence interval 0.94–0.95) Table 2. variance associated with outcomes (Kane 2007) Indicator % variance associated with nurse staing Surgical wound infection (surgery) 85% Unplanned extubation (ICU) 51% Hospital acquired bloodstream infection (surgery 36% Cardiac arrest (all groups) 28% Length of stay (ICU, surgery) 24% Hospital acquired pneumonia (all groups) 19% Failure to rescue (surgery) 16% Respiratory failure (all groups) 6% 4.2% Mortality Hospital-wide, 4.2% of variation in mortality is attributable to nursing, assuming causality. Unsurprisingly, other variables show stronger association with nursing (Table 2), the strongest association being with surgical wound infection. The variation associated with nurse staing is 85%. This seems implausibly high and raises the question of the extent to which the associations observed may be a result of confounding, where there is a strong association between nurse staing and other patient or care quality variables which are wholly or partly responsible for the outcome observed. There is certainly a possibility that high nurse staing is associated with other hospital characteristics that are in turn associated with quality of care (the ‘magnet’ phenomenon37). Kane takes a cautious epidemiological approach to interpreting causation. Causality is supported by evidence of a ‘dose-response relationship’ which appears curvilinear as increased staing at the highest levels yields diminishing returns. The evidence is consistent across study designs (including the use of risk adjustment) and while diferent designs give some modiications in estimates, overall conclusions are unchanged. Evidence 14 State of the art metrics for nursing supports a temporal association as some studies demonstrate that adverse outcomes occur immediately after periods of low staing, although there is a lower estimate of efect on failure to rescue in studies assessing this temporal association. There was no consistent association between nurse staing and patient falls, pressure ulcers or urinary tract infections, outcomes among the indicators most frequently identiied. Although there is some evidence to support these21, their appearance on lists of indicators is clearly more predicated upon a convincing theoretical proposition than the strength of the evidence. Of the four most prominent outcome indicators identiied (failure to rescue, falls, HCAI [pneumonia], pressure ulcers), only failure to rescue and HCAI pneumonia are supported by Kane’s review. This creates a dilemma. Kane’s evidence of cause would be more convincing if these outcomes, which are presumed to be the most nurse-sensitive, were most closely associated with nurse staing. Limitations of available data sets, and in particular poor coding of secondary diagnoses in the administrative databases used in most of the larger studies, provide a possible explanation for this inding. However, this is a presumption; even if it is correct and the outcome is indeed linked to nursing, it appears from Kane’s study that measures of these outcomes derived from administrative databases are not necessarily valid indicators of nursing quality. Although most studies reviewed were from North America, it is unlikely that coding of secondary diagnoses is better in the UK; indeed it is almost certainly worse. Studies on UK nurse staing and patient outcomes have not utilised these outcomes38 and in general coding of complications and secondary diagnoses, although improving, is starting from a weaker baseline than in the US39. This evidence of a link between nurse staing and patient outcomes supports using staing and workforce variables as indicators. Evidence is stronger for levels of RN staing and high skill mix than for total nurse staing23,40. Other evidence tends to support a range of workforce variables such as job satisfaction and turnover21,22,24,38,41. Staf rated practice environment quality is linked to patient outcomes in a number of studies in Katz’s review and others24,42-45. Staing levels, staf satisfaction and staf perception of the quality of the practice environment were among the most frequently identiied indicators in our sources (see Table 1). Box 4. Evidence base: summary • Strong evidence associates nurse staing levels and mortality. • Evidence linking variation in nurse staing to failure to rescue and hospital acquired pneumonia suggests the potential signiicance of such indicators. • Indicators theoretically most closely associated with nursing, such as pressure ulcers, are not clearly supported by strong or consistent evidence linking variation to nurses’ work. • Evidence links other aspects of the workforce (job satisfaction, quality of the work environment including leadership) with mortality. 15 State of the art metrics for nursing Doran’s extensive review considers evidence for a wider range of possible indicators21. However, while the review process is extensive and rigorous, the synthesis is narrative and does not fully meet criteria for a systematic review46-48. The weight of evidence ofered is considered in a qualitative fashion; crucially, much of it derives from studies in which nursing interventions made a positive impact on patient outcomes. Rejecting this as a core source of evidence might seem perverse but because speciic interventions generate speciic outcomes, a focus on interventions is unlikely to yield valid overall indicators of quality that apply to clinical nursing services as a whole. The identiication and delivery of efective interventions and measurement of associated outcomes are important but indicators must be able to broadly relect quality. Thus while Doran’s review may ofer a guide to developing indicators for outcomes of speciic nursing interventions, it cannot ofer the most authoritative overall source of evidence for indicators that relect the broader care environment and characteristics. Future research might identify the variation in outcome associated with quality nursing care from systematic reviews of nursing interventions along the lines proposed by Mantz49, who assessed variation in outcome associated with quality stroke care from a systematic review of stroke units. However, searches of the Cochrane Library as part of this rapid appraisal did not reveal any equivalent reference points for the most widely advocated potential indicators, including falls, pressure ulcers and urinary tract infections. 16 State of the art metrics for nursing State of the art Best bets While indicators cannot provide a complete picture or a complete solution, they can provide a powerful mechanism to incentivise quality by making the contribution of nursing more visible within the healthcare system. However, identifying and using indicators is by no means straightforward. Table 4 illustrates the trade-ofs by considering the relative merits of two possible indicators: mortality (for which the evidence base is strong) and pressure ulcers (where there is broad consensus about the contribution of nursing). Table 4: Comparison of mortality and pressure ulcers as outcome indicators for nursing Criteria Importance Scientiic basis Useability Feasability Impact Mortality High Pressure ulcers Medium Variation in quality High Unsure Evidence of sensitivity to nursing Strong Weak Risk adjustment Feasible Problematic Speciication/deinition of the outcome Clear Problematic Reliability of data collection Good Problematic Variation attributable to nursing Low Unclear Ownership by nursing Unclear High Knowledge to inform action Unclear Clear Wide applicability Yes Yes Positive behavioural incentives High Mixed Potential for gaming Low High Timely availability of data Potentially Challenging Routinely collected data Yes No The importance of mortality as an issue is unambiguously high. There is ample evidence that some unexplained variation probably relates to the quality of health services, and evidence that some of this is attributable to variation in the quality of nursing. Generally data collection is reliable because the outcome is unambiguous and easy to deine. Risk adjustment is possible, if challenging. However, the actual variation attributable to nursing is low, and direct ownership by the profession is also likely to be low since the contribution of other professions to preventing (or causing) death is widely recognised. Similarly, speciic actions required of nursing to rectify quality problems are often unclear. However mortality is an outcome that applies to a wide range of settings, and while negative behavioural incentives are possible (for example, refusing to admit sicker patients) they are relatively unlikely at the level of a nursing service. Neither is mortality readily amenable to 17 State of the art metrics for nursing gaming, and such information is potentially available from routinely collected and timely administrative data. By contrast, the importance of pressure ulcers is harder to asses. The problem is prevalent but its social signiicance is harder to quantify, although economic impact is potentially high50. While risk adjustment models do exist51, elements of coding for both the outcome (grade of ulcer) and risk adjustment (for example mobility) are highly subjective (introducing potential bias and unreliability) and are not generally available in UK data sets. Determination of incidence is also problematic when pressure ulcer data are collected in point prevalence surveys, and the amount of variation associated with quality nursing is unclear. Clearly there is considerable ownership of the problem by the profession, as evidenced by the sources we have reviewed, but the diference in outcomes between those receiving high-quality nursing and poor-quality nursing is hard to estimate. If most pressure ulcers are preventable it could be very high, but the evidence we reviewed is neither consistent nor clear. Similarly, there is considerable scope for strategies to remedy these problems to be developed and acted upon by nursing autonomously, although the evidence base for action is not entirely clear. The outcome applies to a wide range of patients in a range of settings, although it predominantly relates to people with protracted (acute or long-term) institutional stays. Recording and coding problems potentially incentivise negative behaviours (poor reporting) and a lack of routine outcome data leaves open the potential for gaming with process measures. Because data are likely to be available only from intermittent audit, timely feedback will be challenging (but not impossible). It is hard to foresee fully valid data being derived from administrative data sets in the near future, although this area holds potential52. It is beyond the current paper’s scope to give a detailed consideration of all these issues against all the possible indicators identiied, but the preceding discussion illustrates the challenges. As part of this review we identiied a number of potential indicators and identiied the strongest evidence for an association with variation in the quality of nursing. Of the range of potential indicators, a number emerge as potential “front runners” either because of strength of evidence or strength of opinion supporting the indicator (Box 5). Failure to rescue, hospital acquired infection, pressure ulcers and falls were the most strongly advocated patient safety indicators among our sources. Of these, failure to rescue and health care associated pneumonia are most clearly supported by evidence of variation associated with nursing. Because failure to rescue is likely to be a rare outcome in most settings, its utility for local quality monitoring is likely to be low. Staing levels (generally registered nurse staing), staf satisfaction and perception of the practice environment are also strongly supported workforce quality indicators generally supported by evidence. While indicators of compassion were not strongly relected either in the evidence reviewed or the sources identiied, they remain important. We classiied satisfaction as relecting compassion but satisfaction is a complex construct. It is widely used and many measures of satisfaction do contain items relating to patient experience of compassion but general 18 State of the art metrics for nursing patient satisfaction surveys seem insensitive to important aspects of experience53. Elements of communication should be regarded as intrinsic to quality and fundamental to both compassionate and efective care, but speciic questions must be identiied to properly relect patients’ experience of ‘compassionate’ nursing. Any indicator needs to be linked to a nursing unit (for example a ward) and available in a timely fashion to provide maximum impact on staf. Box 5. ‘Front runners’ in the indicator stakes Safety • Failure to rescue • Health care associated pneumonia • Health care associated infection • Pressure ulcers • Falls Efectiveness • Staing levels and patterns • Staf satisfaction • Staf perception of the practice environment Compassion • Experience of care (patient-reported) • Communication (patient-reported) Key challenges For many of the possible outcome metrics, the extent of the contribution of nursing relative to other professions is questionable or controversial. For example, although functional status/self-care/activities of daily living (ADL) have been cited as ‘nursing’ outcomes, other professions might dispute this. In the case of mortality, despite strong evidence of an association with nursing variables it is hard to ofer it as a measure of nursing quality per se although the nursing contribution to it should not be disregarded. Of the widely advocated outcomes we identiied, the most strongly supported by evidence from Kane’s systematic review is failure to rescue. This is generally recognised as nurse-sensitive but staing levels were associated with only 16% of the variation (and only for surgical patients). Hospital acquired pneumonia (as opposed to HCAI in general) was associated with a larger proportion of variation (31%) across all (acute) settings. While these proportions may seem relatively small, this variation is related to staing levels only, not to other aspects of quality nursing care. Although both indicators show strong potential in recent reports 52, the feasibility of deriving them from UK administrative data sets is yet to be fully tested. 19 State of the art metrics for nursing Inclusion of outcome indicators such as pressure ulcers and falls would need to be tied to a strategy to incentivise surveillance, recording and coding. These speciic indicators are unlikely to be suited to use of routinely collected administrative data sets, certainly in the short term. Even US-based systems (with supposed better secondary coding) do not use routine data for pressure ulcers and instead rely on intermittent surveys54. While some outcomes may be controversial because of the relative contribution of other professions to variation, all are subject to patient-level variation. In most cases the main determinants of outcome are patients themselves, not care inputs. Adjustment for patientlevel variation in risk is likely to be a formidable challenge, particularly when comparing across institutions. Existing indicator sets such as the NQF attempt to tackle this but rely on US administrative data speciications and levels of secondary coding, so their applicability to the UK is likely to be limited. Instead, these sets may provide a relatively strong starting point for developing UK indicators to be derived from routinely collected data. Risk adjustment does not present such a substantial obstacle for local quality monitoring where the comparison is with a unit’s own past performance, provided that there is no substantial change in the patient population to alter underlying risk. It is appealing to consider care processes as useful alternatives to measuring outcomes. For nursing this has often been represented by systems focussing on documentation and recording of assessments. However, there is a potential problem of token compliance to such process measures which then have no impact upon the care delivered, so such metrics should be used with caution. In some cases process measures also need risk adjustment, although it is generally less problematic55. The burden of the audit associated with process measures will be relatively high where data are not routinely recorded in a useable format, as is generally the case in the UK. Since an audit is likely to be required, patient outcome audit may be a better investment although risk adjustment remains problematic. Any indicator set must be suiciently diverse to indicate overall quality. While isolated outcomes are poor relections of overall quality29, a purely process-based system is not well suited to nursing. Such approaches do not work well in complex systems involving high degrees of individual skill and multiple actions, and where there is great uncertainty about speciic processes that contribute to success. In these cases, individual aspects of care that are measurable do not relect the complex packages of care that represent best practice49, 55 , 56. Workforce factors emerge as strong candidates because of their association with important outcomes and because they are clearly nursing-related. However, the considerable available evidence on staing levels and skill mix does little to guide the benchmarking of staing levels for settings outside the US as contexts of care difer signiicantly. Setting benchmarks based on historical workforce characteristics and ways of working also may ‘freeze’ the current coniguration and stile innovation. Preliminary economic modelling suggests that investment in highly skilled staf may deliver better returns than investment in total staf40 ,41, indicating potential to change the current balance between qualiied and unqualiied staf, providing qualiied staf are trained to a higher 20 State of the art metrics for nursing level. Benchmarks used as measures of quality might create perverse incentives to employ more junior and less qualiied staf at the expense of fewer more highly skilled but more expensive ones and thus stile positive innovations in staing. Although not directly supported by evidence, the JCAHO indicator of staing in relation to the institution’s staing plan57 may be a more satisfactory indicator in the short term, since it requires a staing plan and some justiication of staing levels. Measures of staf satisfaction and perceived work environment quality are promising. Yet the content of the NWI’s Practice Environment Scale58, while containing items of clear face validity (relating to leadership and support, perceived standards and adequacy of care) and wide applicability, has items with questionable support or that advocate practices of questionable validity (e.g. use of nursing diagnoses). Staf-reported measures and perceptions are less amenable to gaming and perverse incentives than staing quotas and benchmarks although by no means immune. Despite these challenges (Box 6), the motivation for developing indicators is compelling. Even the harshest critics of such systems allow the potential for beneit and acknowledge the lack of clear alternatives in the context of a publically funded health system34, although there is much argument over the relative merits of process and outcome measures29,59. Desires within the nursing profession for articulation of its important contribution and among the public for improved standards point toward outcomes as a signiicant component of any indicator system, as their importance is more clearly understood and harder to contest. Finally, our proposed indicators largely lack positive articulations of nursing’s impact. Patient experience of care did not emerge prominently, although ‘satisfaction’ with nursing is among the most widely used nursing ‘outcome’ measures60 and is a major determinant of overall satisfaction scores61, 62. Nursing’s positive contributions do not easily translate into speciic objective outcomes either because nursing makes a small contribution to the overall outcome (such as functional status) or because the contribution is to the subjective experience of the patient. As patient-reported outcome measures (PROMS) provide patients opportunities to assess their treatment outcome63, they merit consideration for evaluation of speciic nursing interventions or narrowly focussed nursing services. However, PROMS’ utility as nursing indicators may be limited because the contribution of nursing services to most speciic PROMS across a patient population is still likely to be small in light of patients’ experiences with other health services personnel Some elements of care processes, such as the quality of communication as assessed by patients, are clearly on a pathway to efective care, because change in health behaviour and treatment concordance require successful communication. The task and inish group were also very clear about nurses’ contribution to patient dignity and experiences of care as humane and compassionate. Concerns about a loss of compassion and expressions of caring in the emotional sense underpin both professional and public concerns about the state of nursing noted earlier6-8. 21 State of the art metrics for nursing Therefore, in addition to key safety efectiveness and workforce indicators, any set of measures that seeks to represent the nursing system as a whole must seek to represent the experiences of people who are ‘nursed’. Our list of possible indicators touches on this in items such as dignity, communication and satisfaction, which often are measured as composite scores based on evaluations of aspects of experience. Such indicators have the potential to be intrinsically important if properly surveyed (irrespective of evidence linking them to objective outcome) and many aspects of experience, such as communication with professionals, may have important objective consequences. Certainly, a lack of evidence directly linking these indicators to other aspects of quality represents a challenge for future research but not an obstacle to the use of patient experiences as indicators of quality. While there are a number of possible candidates (such as items from the NHS patient survey) the selection of indicators should not be decided by professionals alone. Challenges include delivering results of such surveys in a timely fashion and attributing them to nursing units such as wards rather than hospitals as a whole. Yet most potential indicators face these challenges, and although diferent values and expectations complicate comparison across diferent populations, providers’ opportunity for gaming is limited. Box 6. Key challenges • Deining data and full speciication of indicators • Adjusting for risk • Improving the quality of clinical coding • Identifying indicators for nursing’s impacts in mental health, community, primary care and paediatric settings • Identifying and deining indicators that cross care pathways and boundaries • Timely reporting at the nursing unit level • Delivering action to improve quality 22 State of the art metrics for nursing Conclusion and recommendations Developing indicators for nursing is challenging, but current circumstances provide both an opportunity and an imperative for the profession to embrace tangible measures of nursing’s contributions to patient care. In this report we have identiied candidate indicators for further development. Some indicators have potential applications in a range of settings, but indicators suitable for acute hospital settings have more evidence and better speciication. We have been wary of process and structure indicators because of the uncertain link of likely process measures to outcomes or, in the case of workforce measures, the danger of ixing current workforce patterns and stiling innovation. Both processes and structures are important, and staf-reported measures of work environment and patientreported measures of their care experiences are both potentially important indicators, yet there are compelling reasons to ensure that nursing outcomes will be prominent in any indicator set. Ideally these outcomes should relect all the dimensions of efectiveness, safety and compassion. The promise of ‘no more national targets’ and the Next Stage Review64 commitment to reward quality are reassuring. We began by relecting on concerns that the quality of nursing was being neglected in the face of performance targets focusing on productivity. But performance and output of care remain important, and quality measures should directly relect elements of patient health outcome and experience that beneit from highquality nursing care, lest these elements be neglected in favor of performance targets. The Next Stage Review proposes rewarding quality to a much greater extent than at present. If the quality of nursing can be properly relected in wider sets of metrics to be developed, nursing’s contributions can be more fully recognised at all levels of NHS governance and management. The amount of work needed to fully realise this opportunity should not be underestimated, but without such metrics nursing faces increasing invisibility within a performance-managed health service. Most important, feedback on performance that is based on important measures of nursing can improve nursing care quality by providing frontline staf with information on trends, emerging problems and successes. Such feedback can help senior clinical managers by giving overviews of performance and allowing them to both troubleshoot problems and recognise success. Feedback can help managers articulate nursing’s contributions in tangible terms when seeking resources to protect and enhance quality, and it allows boards to ‘see’ these contributions and to properly support nursing. Finally, such nursing metrics can empower the public to choose between care options using indicators based on an aspect of health services which clearly matters to them as much as it matters to the profession: the quality of nursing care. 23 State of the art metrics for nursing Box 7. Conclusions • Of the range of potential indicators, a number emerge as potential “front-runners”. Failure to rescue, health care associated infection, pressure ulcers and falls were the most strongly advocated patient safety indicators. • Staing and skill mix are linked to patient outcome but their use as indicators would stile change and create perverse incentives. Use of “staing matched to planned staing” as part of a suite of indicators including outcomes has more potential. • Process indicators should be used with caution because of potential for gaming and diiculty in linking speciic processes and patient outcomes. • Patient experience of compassionate care is an important outcome in its own right and may provide the best measure of the nursing contributions to shared outcome and evaluation of complex processes that are otherwise elusive. • Despite these and other challenges, the case for developing metrics is strong. Recommendations We believe the nursing profession should embrace development of metrics and establishment of a National Quality Board and local quality observatories. These initiatives present a substantial opportunity for the profession to equip itself with tools needed to deliver excellent care into the future. Based on the evidence considered here, we recommend: The new National Quality Board should ensure that nursing’s contribution is properly represented in its programme of metrics development. In conjunction with care regulators and other stakeholders, it should develop and support a national standard set of nursing indicators including outcome measures for safety, efectiveness and compassion. As part of this programme we recommend: • Existing data sources should be used where possible, but a process to develop and adapt data collection systems to increase indicator validity also is needed. • Speciications for failure to rescue and HCA pneumonia should be developed as outcome indicators derived from routinely collected data. • These ‘shadow’ indicators should be launched and any association with known markers of quality (e.g. risk-adjusted mortality) should be tested. • Standards should be developed to collect and report data on falls and pressure ulcers, as should risk adjustment models for use in regional and national benchmarking. • Staf surveys should include speciic nursing work environment questions. • Speciications are needed for a workforce planning indicator based on the JCAHO model. • Core nursing-related indicators in patient surveys should be identiied. • Patient groups and professionals can cooperate to identify key indicators of compassion. 24 State of the art metrics for nursing The Department of Health should ensure that the Connecting for Health programme provides a suitable infrastructure to support the collection of clinical outcome and process data to provide nursing indicators and that a Minimum Dataset for Nursing (including patient dependency and staing variables) can be derived from it. The Royal College of Nursing, NHS Confederation and other key stakeholders should consider facilitating a nursing quality coalition of organisations which currently use metrics with the aim of sharing best practice, standardising data collection where this will be useful, and sharing data to facilitate research on indicator validity. The California Nursing Outcomes Coalition (CalNOC) may provide a model for this. Professional bodies and senior members of the profession should reinforce and reiterate the signiicance of nursing contribution in all settings to both nursing-sensitive and shared outcomes, such as mortality and preventable admissions. Claims need to be based on sound evidence and the nursing research community must focus more efort on exploring outcomes. Health care providers must identify mechanisms to ensure timely feedback of indicators at the nursing unit level, and to ensure that positive action results from this information. Health care commissioners and regulators should use nursing metrics as part of their assessment of quality and the commissioning process, and should move toward publication of indicators. Further work is needed to develop indicator sets outside acute settings and for patient pathways across settings. 25 State of the art metrics for nursing References 1. Department of Health. NHS Next Stage Review: A High Quality Workforce. London: Department of Health, 2008. 2. Pencheon D. The Good Indicators Guide: Understanding how to use and choose indicators. Warwick: NHS Institute for Innovation and Improvement, 2008. 3. Redfern SJ, Norman IJ. Quality assessment instruments in nursing: towards validation. Int J Nurs Stud 1995;32(2):115. 4. Higgins M, McCaughan D, Griiths M, Carr-Hill R. Assessing the outcomes of nursing care. J Adv Nurs 1992;17:561. 5. Savitz L, Jones C, Bernard S. Quality Indicators Sensitive to Nurse Staing in Acute Care Settings. Advances in Patient Safety: From Research to Implementation (AHRQ Publication No. 050021). Rockville: Agency for Healthcare Research and Quality, 2005:375-85. 6. Corbin J. Is caring a lost art in nursing? Int J Nurs Stud 2008;45(2):163. 7. Griiths P. Editorial: The art of losing...? A response to the question ‘is caring a lost art?’ Int J Nurs Stud 2008;45(2):329. 8. Raynor C. Why gossiping nurses are a symptom of an NHS in crisis. Daily Mail 2008 online http://www.dailymail.co.uk/pages/live/articles/health/healthmain. html?in_article_id=561164&in_page_id=1774 (accessed April 20th 2008) 9. Healthcare Commission. Investigation into outbreaks of Clostridium diicile at Stoke Mandeville Hospital, Buckinghamshire Hospitals NHS Trust. London: Commission for Healthcare Audit and Inspection, 2006. 10. Healthcare Commission. Investigation into outbreaks of Clostridium diicile at Maidstone and Tunbridge Wells NHS Trust. London: Commission for Healthcare Audit and Inspection, 2007. 11. Healthcare Commission. Learning from investigations. 2008. 12. Maben J. The art of caring: Invisible and subordinated: A response to Juliet Corbin: ‘Is caring a lost art in nursing’. Int J Nurs Stud 2008;45(3):335. 13. Clarke SJ. Research on Nurse Staing and its Outcomes. In: Nelson S, Gordon S, editors. The Complexities of Care: Nursing Reconsidered. New York: Cornell University Press, 2006. 14. Audit Commission. Review of National Finding: Ward Staing. Acute Hospital Portfolio. London: Audit Commission, 2001:15. 15. Audit Commission. Ward Staing Management Tools: Practical Tips for Budget Setting. London: Audit Commission, 2001:3. 16. Black N. Rise and demise of the hospital: a reappraisal of nursing. BMJ 2005;331(7529):1394. 26 State of the art metrics for nursing 17. Clarke SPP. Registered Nurse Staing and Patient Outcomes in Acute Care: Looking Back, Pushing Forward. Med Care 2007;45(12):1126. 18. Maas ML, Johnson M, Moorhead S. Classifying Nursing-Sensitive Patient Outcomes. J. Nurs. Sch. 1996;28(4):295. 19. Doran D. Preface. In: Doran D, editor. Nursing-sensitive outcomes: state of the science. Sudbury / London: Jones and Bartlett Pub., 2003:vii-ix. 20. Mitchell PH, Ferketich S, Jennings BM. Quality Health Outcomes Model. J. Nurs. Sch. 1998;30(1):43. 21. Doran D. Nursing-sensitive outcomes: state of the science. Sudbury / London: Jones and Bartlett Pub., 2003. 22. Kane R, Shamliyan T, Mueller C, Duval S, Wilt T. The Association of Registered Nurse Staing Levels and Patient Outcomes: Systematic Review and MetaAnalysis. Med Care 2007;45(12):1195. 23. Lankshear AJ, Sheldon TA, Maynard A. Nurse Staing and Healthcare Outcomes: A Systematic Review of the International Research Evidence. Advances in Nursing Science 2005;28(2):163. 24. Kazanjian A, Green C, Wong J, Reid R. Efect of the hospital nursing environment on patient mortality: a systematic review. J Health Serv Res Policy 2005;10:111. 25. Lang TA, Hodge M, Olson V, Romano PS, Kravitz RL. Nurse-patient ratios: a systematic review on the efects of nurse staing on patient, nurse employee, and hospital outcomes. J Nurs Adm 2004;34(7-8):326. 26. Johnson M, Maas M. The Nursing Outcomes Classiication. J Nurs Care Qual 1998;12(5):9. 27. Silber JH, Romano PS, Rosen AK, Wang Y, Even-Shoshan O, Volpp KG. Failure-to-Rescue: Comparing Deinitions to Measure Quality of Care. Med Care 2007;45(10):918. 28. National Quality Forum. A Comprehensive Framework for Hospital Care Performance Evaluation: A consensus report. Washington: National Quality Forum, 2003. 29. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet 2004;363(9415):1147. 30. Mainz J. Deining and classifying clinical indicators for quality improvement. Int J Qual Health Care 2003;15(6):523. 31. Riehle AI, Hanold LS, Sprenger SL, Loeb JM. Specifying and Standardizing Performance Measures for Use at a National Level: Implications for Nursingsensitive Care Performance Measures. Med Care Res Rev 2007;64(2_suppl):64S. 32. Donaldson N, Brown D, Aydin C, Bolton M, Rutledge D. Leveraging Nurse-related Dashboard Benchmarks to Expedite Performance Improvement and Document Excellence. J Nurs Adm 2005;35(4):163. 27 State of the art metrics for nursing 33. Boyne GA, Law J. Setting Public Service Outcome Targets: Lessons from Local Public Service Agreements. Pub Money and Man 2005;25(4):253. 34. Bevan G, Hood C. What’s Measured Is What Matters: Targets and Gaming in the English Public Health Care System. Pub Admin 2006;84(3):517. 35. Quan H, Parsons GA, Ghali WA. Assessing accuracy of diagnosis-type indicators for lagging complications in administrative data. J Clin Epidemiol 2004;57(4):366. 36. Pancorbo-Hidalgo PL, Garcia-Fernandez FP, Lopez-Medina IM, Alvarez-Nieto C. Risk assessment scales for pressure ulcer prevention: a systematic review. J Adv Nurs 2006;54(1):94. 37. Havens DS, Aiken LH. Shaping Systems to Promote Desired Outcomes: The Magnet Hospital Model. JONA 1999;29(2):14. 38. Raferty AM, Clarke SP, Coles J, Ball J, James P, McKee M, et al. Outcomes of variation in hospital nurse staing in English hospitals: Cross-sectional analysis of survey data and discharge records. Int J Nurs Stud 2006. 39. McKee M, Coles J, James P. ‘Failure to rescue’ as a measure of quality of hospital care: the limitations of secondary diagnosis coding in English hospital data. J Pub Health 1999;21(4):453. 40. Needleman J, Buerhaus PI, Stewart M, Zelevinsky K, Mattke S. Nurse Staing In Hospitals: Is there a Business Case for Quality? Health Af (Millwood) 2006;25(1):204. 41. Newbold D. The production economics of nursing: A discussion paper. Int J Nurs Stud 2008;45(1):120. 42. Aiken LH, Clarke SP, Sloane DM, Sochalski J, Silber JH. Hospital Nurse Staing and Patient Mortality, Nurse Burnout, and Job Dissatisfaction. JAMA 2002;288(16):1987. 43. West E, Maben J, Raferty A. Nursing and Patient Outcomes: How Can Employers Provide the Right Environment for Nurses to Deliver High-Quality Care? Harvard Health Policy Review 2006;7(1):64. 44. Stone PW, Mooney-Kane CM, Larson EL, Horan TM, Glance LG, Zwanziger J, et al. Nurse Working Conditions and Patient Safety Outcomes. Med Care 2007;45(6):571. 45. Estabrooks CA, Midodzi WK, Cummings GG, Ricker KL, Giovannetti P. The impact of hospital nursing characteristics on 30-day mortality. Nurs Res 2005;54(2):74. 46. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Metaanalysis of Observational Studies in Epidemiology: A Proposal for Reporting. JAMA 2000;283(15):2008. 28 State of the art metrics for nursing 47. Moher D, Cook D, Eastwood S, Olkin I, Rennie D, Stroup D. Improving the quality of reports of meta-analyses and randomised controlled trials: the QUOROM statement. Lancet 1999;354:1896 48. Center for Reviews and Dissemination. What are the criteria for the inclusion of reviews on DARE? York: Center for Reviews and Dissemination online http://www. york.ac.uk/inst/crd/darefaq.htm (accessed 01/05/2008) 49. Mant J. Process versus outcome indicators in the assessment of quality of health care. Int J Qual Health Care 2001;13(6):475. 50. Bennett G, Dealey C, Posnett J. The cost of pressure ulcers in the UK. Age Ageing 2004;33(3):230. 51. Berlowitz DR, Brandeis GH, Morris JN, Ash AS, Anderson JJ, Kader B, et al. Deriving a Risk-Adjustment Model for Pressure Ulcer Development Using the Minimum Data Set. J Am Geriatr Soc 2001;49(7):866. 52. Bottle A, Aylin P. How NHS trusts could use patient safety indicators to help improve care. Health Care Risk Report 2008;May 2008:12. 53. Jenkinson C, Coulter A, Bruster S, Richards N, Chandola T. Patients’ experiences and satisfaction with health care: results of a questionnaire study of speciic aspects of care. Qual Saf Health Care 2002;11(4):335. 54. National Quality Forum. National Voluntary Consensus Standards for NursingSensitive Care: An Initial Performance Measure Set. Washington: National Quality Forum, 2004. 55. Rubin HR, Pronovost P, Diette GB. The advantages and disadvantages of processbased measures of health care quality. Int J Qual Health Care 2001;13(6):469. 56. Kendrick S. Using all the evidence: towards a truly intelligent National Health Service. Health Bulletin 2001;59(2). 57. JCAHO. Staing Efectiveness in Hospitals. Oak Brook: Joint Commission Resources, 2003. 58. Lake ET. Development of the practice environment scale of the nursing work index. Res Nurs Health 2002;25(3):176. 59. Donabedian A. Evaluating the Quality of Medical Care. Milbank Quarterly 2005;83(4):691. 60. Avis M, Bond M, Arthur A. Satisfying solutions? A review of some unresolved issues in the measurement of patient satisfaction. J Adv Nurs 1995;22(2):316. 61. Yellen E. The Inluence of Nurse-Sensitive Variables on Patient Satisfaction. AORN 2003;78(5):783. 62. Abramowitz S, Coté A, Berry E. Analyzing patient satisfaction: a multianalytic approach. Quality Review Bulletin 1987;13(4):122. 63. Smith S, Cano S, Lamping D, Staniszewska S, Browne J, Lewsey J, et al. Patient-Reported Outcome Measures (PROMs) for routine use in Treatment Centres: recommendations based on a review of the scientiic evidence. London: London School of Hygiene and Tropical Medicine, 2005. 29 State of the art metrics for nursing 64. Department of Health. High Quality Care for All: NHS Next Stage Review Final Report. Norwich: Stationary Oice, 2008. 65. Healthcare Commission. Acute Hospital Portfolio Review: ward staing topic guide. London: Healthcare Commission, 2004. 66. International Council of Nurses. Nursing Sensitive Outcome Indicators. Geneva: International Council of Nurses online http://www.icn.ch/matters_indicators.htm (accessed 01/05/2008) 67. Naylor MD. Advancing the Science in the Measurement of Health Care Quality Inluenced by Nurses. Med Care Res Rev 2007;64(2_suppl):144S. 68. Van den Heede K, Clarke SP, Sermeus W, Vleugels A, Aiken LH. International Experts’ Perspectives on the State of the Nurse Staing and Patient Outcomes Literature. J Nurs Schol 2007;39(4):290. 30 State of the art metrics for nursing Appendix 1: Core data sources Name Agency for Healthcare Quality and Research Patient safety Indicators (nursing subset) American Nurses Association Association of UK University Hospitals nurse-sensitive indicators Doran 2007 (review) Short ref Description Setting/ clinical group Ref/link AHRQ Patient safety indicators Acute care* http://www. qualityindicators.ahrq.gov/ psi_overview.htm 5 ANA Nursesensitive indicator set Acute, paediatric, long term care and psychiatric http://www. mnursingworld.org/ ainMenuCategories/ professionalNursing/ PatientSafetyQuality/ NDNQI/NDNQI_1/ ngSensitiveIndicators.aspx AUKUH Nursesensitive indicator set Acute care http://www.aukuh.org.uk/ members/PCP.htm Doran Review of Acute care nurse-sensitive outcomes 21 Essence of care EoC Benchmark statements for the quality of nursing care Not speciied http://www.dh.gov.uk/en/ Publicationsandstatistics/ Publications/ publicationsPolicy AndGuidance/ DH_4005475 Healthcare Commission HC Factors assessed in ward staing review Acute care http://www. healthcarecommission. org.uk/erviceproviderin formation/reviewsand studies/servicereviews/ ahpmethodology/ wardstaing.cfm 65 31 State of the art metrics for nursing Name Short ref Description Setting/ clinical group Ref/link International Council of Nurses Brieing ICN Review of Not nurse-sensitive speciied outcome indicators http://www.icn.ch/ matters_indicators.htm 66 Joint Commission staing efectiveness Indicators JCA Staing Acute care efectiveness indicators including NQF indicators 31 Kane 2007 (review) Kane Review of evidence for staing outcome relationship Acute care 22 Kazanjian 2005 (review) Kaz Review of Acute care evidence linking ward environment to mortality 24 Lang 2004 (review) Lang Review of evidence for staing outcome relationship Acute care 25 Lankshear 2005 (review) Lank Review of evidence for staing outcome relationship Acute care 23 Nursing Home Quality Initiative Medicare/ Medicaid Quality Compare NHQI Long-term care/post– acute care indicator set Long-term care http://www.medicare.gov/ NHCompare/ 32 State of the art metrics for nursing Name Short ref Description Setting/ clinical group Ref/link NQF Nursesensitive indicator set Acute care http://www.qualityforum. org/nursing/54 ,67 OASIS Home health care indicator set Home/ community health care http://www.cms.hhs.gov/ OASIS/ Other Oth Other clinical indicator sets /dashboards identiied on web searches Generally acute N/A Quality Outcomes Framework (nursing items) QOF General Primary practice care quality indicators with speciic items relating to nursing http://www.dh.gov.uk/en/ Healthcare/Primarycare/ Primarycarecontracting/ QOF/index.htm Van den Heede 2007 VdH Expert Not consensus speciied regarding nurse-sensitive outcomes 68 National Quality Forum Consensus Standards for Nursingsensitive Care OASIS (Medicare reporting subset) 33 Appendix 2: Indicators identiied ANA AUKUH Doran EoC HC ICN JCA Safety √ √ √ √ √ √ √ √ Failure to rescue Safety √ √ √ √ √ √ Staing levels (nurses/hours per patient) Workforce √ √ √ √ Falls Safety √ √ √ HCAI (pneumonia) Safety √ √ Staf satisfaction and well-being Workforce HCAI (UTI) Safety Area Pressure ulcer 34 Staing, skill mix Workforce Medication administration errors Safety Mortality Safety Practice environment/perceived quality Workforce Satisfaction with (nursing) care Experience Sickness rates Workforce Smoking advice Preventative Staing bank/agency Workforce Communication Experience Staf experience, knowledge, skills, expertise HCAI (surgical wound) √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ Use of restraints Function Safety Experience √ √ √ √ √ 11 √ 9 √ √ 9 √ √ 8 √ √ 8 √ √ 7 √ √ 6 √ 6 √ 5 √ 5 √ √ √ √ √ Oth QOF √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ √ 5 √ 5 √ 5 √ 5 4 √ 4 4 √ √ √ √ √ √ √ √ 5 √ √ √ √ √ Count √ √ √ √ VdH OASIS √ √ Workforce √ NQF √ √ Perception of adequate staing NHQI √ √ √ √ Lank √ Safety Arrest/shock √ √ Instrumental activities of daily living and self-care Function Complaints √ √ Workforce Lang √ √ √ Kaz √ √ √ √ Kane √ √ 4 √ 4 √ 4 √ 3 √ 3 State of the art metrics for nursing AHRQ Indicator Indicator Area Conidence and trust AHRQ AUKUH EoC HC ICN Experience √ √ √ Continence Function √ Emergency equipement/drugs Safety Extubation, unplanned Safety Safety √ Injuries to staf Safety (staf) Interprofessional relations Workforce Utilisation 35 Safety NHQI NQF OASIS Oth QOF √ √ √ 3 √ √ RN vacancies Workforce Symptom control (other, e.g. nausea) Symptom √ Ability to talk to nurse Experience √ Care planning/assessment processes Planning √ Cleanliness Experience √ Confusion, delirium Safety Dignity/respect Experience √ 3 √ 3 √ 3 √ 3 √ 3 3 √ √ √ √ √ √ √ 3 3 √ √ 3 3 √ √ Safety 3 √ √ Safety 3 √ 3 √ √ √ √ √ √ 3 √ √ √ Count 3 √ √ √ VdH √ √ √ Respiratory failure Utilisation Lank 3 √ Record keeping/reporting systems Emergency care Lang √ √ Pain assessment/assessment intervention cycles Symptom PE/DVT Kaz √ Experience Nutrition Kane √ √ √ 2 2 2 √ √ √ √ √ 2 2 2 State of the art metrics for nursing Safety HCAI (bloodstream) Length of stay JCA √ HCAI (any) Nutritional assessment/screening Doran √ √ Knowledge of condition and treatment ANA Indicator Area HCAI (c. dif.) Safety HCAI (central line) Safety HCAI (MRSA) Safety Utilisation Workforce Pain/pain control Symptom Patient involvement Experience ANA AUKUH Doran EoC HC JCA Kane Kaz √ 36 RN turnover Workforce √ Staf intent to leave Workforce √ Symptom control (dyspnea) Safety √ Time spent in CPD Workforce √ Understaing (compared to staing plan) Workforce √ Appraisal Workforce Cannula iniltration Safety √ 2 2 2 √ 2 √ 2 2 √ √ √ √ Count 2 √ √ 2 2 √ 2 √ √ 2 √ √ 2 2 √ 2 √ √ 2 2 √ Preventative Safety VdH √ Preventative Allergies recorded QOF √ Risk assessments/other health promotion Safety Oth √ Nutrition Accidents/incidents OASIS √ √ Protected meal times policy implemented Safety NQF √ Safety Workforce NHQI √ Safety Vascular access device ‘incidents’ Lank √ Post-operative complications Workload Lang √ Pressure ulcer risk assessment/planning Vaccination rates ICN √ √ 2 √ 2 √ √ √ 2 1 √ 1 √ 1 √ 1 State of the art metrics for nursing Hospital admissions/readmissions Leadership AHRQ Indicator Area Discharge from caseload Utilisation Discharge planning/case management processes Planning Safety Induction Workforce Mortality in low–mortality groups Safety Patients absconding or lost Safety Personal learning plan Workforce 37 Pressure ulcer monitoring systems Safety Psychiatric physical/sexual assault Safety Psychological well-being Experience Record of training and updating Workforce Restraint documentation Self-harm risk assessment ANA AUKUH Doran EoC HC ICN JCA Kane Kaz Lang Lank NHQI NQF OASIS Safety Use of long-term catheter Function Violence and aggression toward staf Safety (staf) Weight loss Nutrition VdH Count 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ 1 √ Timely assessment/intervention for pneumonia Safety Upper GI bleed QOF √ Function Safety Oth √ 1 1 √ 1 √ √ A=acute care, C=home/community health care, L=long-term care, N=not speciied, P=paediatric, Pc=primary care, Ps=psychiatric 1 1 State of the art metrics for nursing HCAI surveillance system AHRQ State of the art metrics for nursing 38 To order further copies Please contact: The National Nursing Research Unit at King’s College London Waterloo Campus, 57 Waterloo Road London SE1 8WA Telephone 020 7848 3057 Fax 020 7848 3069 http://www.kcl.ac.uk/schools/nursing/nnru PDF download: free Hard copy: administration charge £15 (including UK p&p) Cover picture: The Diagram [based on the “coxcomb” diagram used by Florence Nightingale to depict rate and cause of mortaliry among British soldiers during the Crimean war] Laura Potter 2007 Silver, red gold, yellow gold Approx. 600cm chain Photo: Tomás Valenzuela Part of the Culture & Care Project ISBN: 978-0-9555991-2-5 © King’s College London 2008