EUROPEAN  TISSUE  REPAIR  SOCIETY

WHAT HAPPENS WHEN WE DISAGREE ABOUT THE QUALITY OF THE 'EVIDENCE'
IN EVIDENCE BASED PRACTICE?
Dr Michael Clark, Cardiff

The philosophy and politics of health care have become increasingly biased towards ‘evidence based practice’. While such a concept originally marked the best marriage between clinical experience and the available scientific evidence, ‘evidence’ has increasingly come to be equated with the conclusions of randomised controlled trials (RCT’s). While RCT’s do represent an elegant approach to defining the efficacy of health care technologies, many questions regarding their use and interpretation remain. In his recent paper in Advances in Wound Care, Kloth¹ ably demonstrated both the confusion and the potential consequences that can arise when different groups interpret RCT evidence in radically different ways.

Electrical stimulation, according to Kloth, is one intervention that may assist the healing of chronic wounds. This intervention has been used, perhaps primarily by physiotherapists, since the early 1960s, and has been investigated in at least fifteen controlled or randomised controlled investigations. The potential value of electrical stimulation in pressure sore healing appeared to be accepted without question in the early 1990s. Within the United States Agency for Health Care Policy and Research (AHCPR) guidelines on pressure sore treatment, electrical stimulation was considered to be ‘the only adjunctive therapy with sufficient supporting evidence to warrant recommendation … and that clinicians (should) consider a course of treatment with electrical stimulation for Stage III and Stage IV pressure ulcers that have proved unresponsive to conventional therapy’. This statement was one of only thirteen specific recommendations made in the US pressure sore treatment guidelines that appeared to be based upon sound research evidence, with five RCT’s cited to support this recommendation. This early support for the use of electrical stimulation was reinforced by a systematic review reported in 1998, where, according to Kloth, fifteen randomised controlled and controlled studies also suggested that wounds treated with electrical stimulation healed faster than control wounds.

However, a second review of the clinical studies that reported the use of electrical stimulation in wound healing suggested that there was no significant difference between the healing of wounds exposed to electrical stimulation and controls. Based upon the conclusions of this second, negative, report the US Health Care Financing Administration (HCFA) proposed to deny reimbursing practitioners for the use of electrical stimulation in wound healing! This regulatory decision prompted litigation both from patients and from the American Physical Therapy Association. Kloth describes the course of this litigation that finally prompted the HCFA to delay the reimbursement changes indefinitely. Superficially it is easy to dismiss Kloth’s account of the struggle for reimbursement for the use of electrical stimulation of chronic wounds as something that could only occur in America with its peculiar health care system. However, this case raises fundamental questions for all evaluation of health care technologies.

The negative report on the value of electrical stimulation appeared to focus upon the perceived deficiencies in the design of the studies undertaken to evaluate the intervention. So were the earlier reviews of electrical stimulation too generous to inadequate study designs, or was the later review too harsh? It is generally recognised that the (few) randomised controlled trials that give support to wound care interventions are methodologically weak. If we were to dismiss weak study designs then little hard evidence of the efficacy of wound care would remain. But if we do accept weak designs, at what point do such weaknesses prove fatal to the appropriate interpretation of study results? This is an important matter and deserving of significant debate, a debate that is currently lacking.

One key point within the controversy surrounding the use of electrical stimulation was the interpretation of the control regimes. The negative report initially considered that electrical stimulation had not been compared against alternative, appropriate interventions but had been studied in situations where the control arm received minimal, or no intervention. The more favourable reviews considered that comparisons had been made against appropriate interventions. Obviously both conclusions cannot be correct, and the legal ruling fell in favour of the view that electrical stimulation had been compared against appropriate interventions. But what were these apparently appropriate interventions? Kloth commented that often electrically stimulated wounds were compared with others treated using moist gauze dressings. Was this a fair comparison, and does it represent the best alternative care those wounds could have received? This entire controversy may have been avoided if there had been a general acceptance of the ‘best’ or ‘most appropriate’ control regime. Perhaps one central challenge for wound care research is to identify a control regime that represents current best practice and against which new interventions can be compared?

Perhaps the most surprising theme within Kloth’s report is the confusion over what was, and what was not a randomised controlled trial. The US pressure ulcer guidelines recommended electrical stimulation based upon five RCT’s. However, during the period of litigation, the AHCPR initially considered these five studies to be observational designs and not RCT’s! Subsequent comments from the AHCPR then focused upon the weaknesses of the design of these five investigations. Ignoring the political and financial dimensions of this controversy how can evidence be interpreted when there is a lack of agreement over the design of the studies under review? Furthermore, the reviews that appeared favourable to the use of electrical stimulation included non-randomised controlled studies. Can we combine different study designs, particularly where bias may be suspected through the non-random allocation of patients to different interventions?

While the saga of whether or not US health care would reimburse a wound care intervention may appear irrelevant to other health care systems, the confusion apparent between the various reviewers highlights key problems for evidence based health care. When is an RCT not an RCT? When do study flaws become fatal, what are appropriate control regimes and the synthesis of different study designs into a single meta-analysis are all key questions that must be addressed if evidence based practice is to thrive. There would appear to be a fundamental need to debate and agree upon what constitutes ‘evidence’ without such debate other technologies will face similar challenges to their reimbursement or general diffusion within health care systems.

Reference

1. Kloth, LC. The APTA Electrical Stimulation Lawsuit and Its Aftermath. Advances in Wound Care 1999, 12(9): 472–475

Dr Michael Clark
Senior Research Fellow
University of Wales, College of Medicine
Wound Healing Research Unit
Cardiff, Wales