Smoke and Mirrors

Good basic research in EMS is terribly hard to get done right. The numbers of patients required to validate any findings is much higher than in most other sample driven research, because effective study controls are much more challenging. Let’s say for example, you want to study the performance of a particular ventilator or resuscitator. An in hospital study would use each patient as their own control. They would try two different methods on the same patient and compare the results in a quantifiable fashion, like by comparing blood gases. Since you have two measurements on the same person, after 20-100 people any emerging pattern will likely be valid.

EMS researchers will rarely if ever have that opportunity. Most EMS studies must compare one patient to another. Since patients vary so much in size, age, weight, condition, etc., you need an awful lot of patients for any study to be considered valid. For example, instead of just 20 patients; you may need 200 to get to the same place. You also need a protective oversight authority to continuously monitor the study and control for the many variables in the patient population. We don’t have this in EMS either. Most of our Medical Control Physicians are just part time, and they also pull full time hours in an ER that would make a dairy farmer wince. They just don’t have much time left over to do research. 

Another huge problem is the issue of informed consent. It is real tough to get a legal consent from a cardiac arrest or critical trauma patient. This makes the most serious patient conditions and therapies also the most difficult ones to study. How do you even ask relatives in crises to make an informed choice about their family members desire to participate in a research study?

Hospitals are the primary acute care research sites, in part because they have a process for evaluating proposed clinical trials, and carefully monitoring them while underway. They are called IRB’s, or Investigational Research Boards. These select groups of physicians and other clinicians at each hospital evaluate any study proposal, and approve or reject the application. Then if it is approved, they continue to monitor the study while it is in progress. They can even terminate the study at any time, should early data warn of specific dangers. This gives your research an effective built in administrative review process and lends the resulting data great credibility when combined with a pre-publication peer review process. The majority of published studies on patients from outside the hospital are pharmaceutical trials. Just like in EMS they require very large patient populations to control for the many variables. But the big difference is that they have the last required element of good pre-hospital research, money. 

The massive dollars available from big drug companies provide funds for full time investigators, large-scale trials, and in-house clinical review. This all happens long before peer-reviewed publication, and it costs a fortune. If you don’t have the money to pay for staff time, clinical oversight and review resources, it is real tough to get good research done. EMS Agencies simply do not have such resources. Because of these realities, EMS research will always be tough to do right. This brings me to my subject title, Smoke & Mirrors. 

Since it is very difficult to do good research in pre-hospital care, all of us need to be highly vigilant when reading EMS related studies. My father was an investigational research professor with a PhD in Biochemistry. He spent his entire career doing and publishing research. He really knew how to read studies. This is by itself is an important skill that everyone in EMS needs to get better at. He would often scan a published report and in only a few seconds, tell me if the study was garbage and why. I’ll never forget him laughing at a major study on a sugar substitute, that had concluded there was a cancer risk associated with using the product. He pointed out that the amount of the substance used in the study was a huge percentage of the study animals total body mass. It was the statistical equivalent to eating the stuff all day long every day. What I learned from him is to critically examine for myself the conclusions of any clinical study. Learn to read all the lines, and then in between the lines, to carefully examine any clinical study before you accept its findings as fact. 

I recently became aware of a manufacturer making claims about the benefits of its product to enhance certain biological functions during cardiac arrest. They claimed conclusive findings validated by clinical trial. This was widely accepted without challenge, and even got some supporting press published in one of our EMS magazines. But when I actually read their principally cited study, I found out it was one performed on patients that had a pulse. They had taken the leap from pulsed patients to patients without a pulse, and applied their studies conclusions from one patient population to another very different patient population.  I call this Smoke & Mirrors science.   

If you have read an entire published study, and then still have questions about the results, ask them. If nobody can explain to your satisfaction how the conclusions were reached and substantiated, you should probably question the conclusions. Never accept the suggestion that you do not have the clinical background or enough education to assess the study for yourself. Any decent EMT or Paramedic has more practical problem solving skill than most members of Congress. Just because somebody has more letters after their name then you do does not give them more “horse sense”. Use your own mind and beware of the Smoke and Mirrors, for they are all around us. Because of the many challenges in getting good research done in EMS, they will likely stay around too.

Last, please support those rare few who do perform real EMS research. Set aside a half hour a week to do nothing but read studies, in journals or even on the web. The best way you can support good research is to become a good regular reader of research. Make it a part of your job, and your professional life. Good doctors do it, so should you. Also take a minute to review those lonely research poster presentations at Conferences, and thank the authors. What they are trying to do is very difficult, yet very important to our future.


About Dan White

I'm a retired Paramedic and EMS Instructor with 35 years EMS and emergency medical product experience. I love canoes, cars and EMS. I have written a lot about EMS Technology on the Paramedic Blog, the Insights on Innovation column for, on and I can be reached directly at 573-240-0002.
This entry was posted in EMS Research. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s