From a Senior Clinician Down Under: Anecdotes and Medicine, We are Actually Talking About Two Different Things

A poster who wishes to remain anonymous, a Senior Clinician in the state of Victoria, Australia, added this comment to my March 2011 post on 'anecdotes.' (That post was entitled "Australian ED EHR Study: An End to the Line "Your Evidence Is Anecdotal, Thus Worthless?".)

He makes a critical point I think has gotten lost in the HIT domain (emphases mine):

Anonymous August 15, 2011 9:26:00 PM EDT said...

Anecdote and Medicine.

We are actually talking about two different things here.

1. Anecdotal reporting of a new and potentially exciting finding in Medicine is NEVER a reason to widely implement a new treatment or procedure. It represents the lowest category of evidence in any systematic review In any orthodox system of medicine in the developed world a new intervention would not be ratified or re-imbursed without EXTENSIVE study in Randomised trials - ie the mandatory three phase trial arrangement for new drugs.

2. Anecdotal reporting of side effects/failure for an implemented treatment is a crucial part of any risk management strategy within a healthcare setting. Individual incident reporting of harmful events AND near misses is crucial to help organisations (and regulatory agencies) understand where risks to patients and staff are to be found. A root cause analysis can then be undertaken and corrective measures introduced and their subsequent impact assessed. The process of 'closing the audit loop' is required or no reduction in risk can be verified. This is separate from the regular audit cycle which each Department should apply to aspects of its work, usually reviewing a particular intervention in rotation.

In the above debate the pro e-Health lobby find themselves mis-interpreting both definitions.

They support the positive anecdotes for the adoption of Electronic Health Records WITHOUT proper randomised evidence being available and they decry the anecdotes of negative experiences of implemented systems that in reality represent episodes of incident reporting.

It appears the over-exuberant proponents of e-Health in general and for everything, need to attend revision courses in research methodology and risk management... [a fascinating observation - ed.]

nb Irony of irony;

Victoria now has State-wide implementation of risk management software which for all healthcare staff is the obligatory reporting mechanism for all incidents and near-misses.

The system is so user unfriendly and time demanding virtually none of the busy hospital doctors I have spoken to access the system, even though 'training' has been undertaken. The system has been widely condemned at our Medical Staff Committee.

Victoria has recently congratulated itself for a fall in annual number of critical incidents occurring in public hospitals!

I will leave it to you to work out how such a positive risk management statistic could be generated in a healthcare system working near capacity with increasing year on year demand for its services...

Per these observations:

A critical distinction that seems to have become lost, even among the Medical Informatics academic elite (see, for instance, my Sept. 2010 post "The Dangers of Critical Thinking in A Politicized, Irrational Culture"), is the distinction between research observations on the one hand, and risk management-relevant incident reports on the other.

It seems a form of erroneous thinking or logical fallacy.

The lost distinction between research methods and risk management methods, that require very essential, very different consideration of "anecdotes", and the conflation of the two types of "anecdotes", are brilliant observations.

Finally, the loss of consideration of the distinctions between the two different types of "anecdotal reporting" is part of what I have termed the lack of the rigor of medicine itself in HIT.

-- SS