Studies into medication often mislead
By William Smith
A new study warns that you should be careful what you read, even in peer-reviewed influential medical journals. Research findings are often presumed to be impartial but this study shakes that presumption.
In the review by the UCLA-Harvard, researchers stated that studies and articles about medications are often designed in a way that yields misleading and confusing results.
Investigators looked at all the randomised medication trials published in the six highest-impact general medicine journals between 1st June 2008 and 30th September 2010. They wanted to determine if the outcomes were reported in a way that makes data interpretation difficult.
They also reviewed each study’s abstract to determine the percentage that reported results using relative rather than absolute numbers, as this can also mislead.
The findings of this study can be found online in the Journal of General Internal medicine.
The six journals which were examined by the researchers were :
- The new England Journal of Medicine
- The Journal of the American Medical Association
- The Lancet
- The annals of Internal medicine
- The British Medical Journal
- The Archives of Internal Medicine
Each of these journals included studies that used outcome measures that are frequently criticised by scientific experts.
Some examples of outcome measures that may mislead or confuse include:
- Surrogate outcomes (37 percent of studies), or the reporting of outcomes on intermediate measures, such as a heart medication’s ability to lower blood pressure, but which may not be a good indicator of the medication’s impact on more important clinical outcomes, like heart attacks;
- Composite outcomes (34 percent), which consist of multiple individual outcomes of unequal importance, lumped together — such as hospitalizations and mortality — making it difficult to understand the effects on each outcome individually;
- Disease-specific mortality (27 percent), which measures deaths from a specific cause rather than from any cause; this may be a misleading measure because, even if a given treatment reduces one type of death, it could increase the risk of dying from another cause, to an equal or greater extent.
‘Patients and doctors care less about whether a medication lowers blood pressure than they do about whether it prevents heart attacks and strokes or decreases the risk of premature death,’ said the study’s lead author, Dr. Michael Hochman.
‘Knowing the effects of a medication on blood pressure does not always tell you what the effect will be on the things that are really important, like heart attacks or strokes,’ Hochman added. ‘Similarly, patients don’t care if a medication prevents deaths from heart disease if it leads to an equivalent increase in deaths from cancer.’
‘Patients also want to know, in as much detail as possible, what the effects of a treatment are, and this can be difficult when multiple outcomes of unequal importance are lumped together,’ said Dr. Danny McCormick, the study’s senior author and a physician at the Cambridge Health Alliance and Harvard Medical School.
The authors also found that trials or studies that used surrogate outcomes and disease-specific mortality were more likely to be exclusively commercially funded, for instance, by a pharmaceutical company.
Commercial sponsors of research may encourage the use of outcomes that ate more likely to indicate favourable results for their products, the researchers suggest.
The pharmaceutical companies have millions of dollars at stake in research and even more in potential sales. This rests on whether a medication is found to be effective or not.
‘For example, it may be easier to show that a commercial product has a beneficial effect on a surrogate marker like blood pressure than on a hard outcome like heart attacks,’ Hochman said. ‘In fact, studies in our analysis using surrogate outcomes were more likely to report positive results than those using hard outcomes like heart attacks.’
The new study also shows that 44% of study abstracts reported study results exclusively in relative, rather than absolute numbers, which can be misleading.
‘The way in which study results are presented is critical,’ McCormick said.
‘It’s one thing to say a medication lowers your risk of heart attacks from two-in-a-million to one-in-a-million, and something completely different to say a medication lowers your risk of heart attacks by 50%. Both ways of presenting the data are technically correct, but the second way, using relative numbers, could be misleading,’ McCormick added.
The authors acknowledge that the use of surrogate and composite outcomes and disease-specific mortality is appropriate in some cases.
These measures may be indicated early in the study of a medication (early-phase), in which researchers hope to quickly determine whether a new treatment has the potential to help patients.
The reporting of medication outcomes can be improved if study methodology committees overseeing research studies closely scrutinise study outcomes to make sure that lower-quality outcomes are only used in appropriate circumstances, the study authors say. Results should also be reported as absolute numbers, either instead of or in addition to relative numbers.
‘Finally, medical journals should ensure that authors clearly indicate the limitations of lower-quality endpoints when they are used, something that does not always occur,’ McCormick said.
Source: UCLA
