More replication study needs to be carried out on mental health research
By Liz Lockhart
A new study suggests that additional scrutiny and replication need to be done before we accept findings as true. Specifically this applies to research which suggests a genetic-environmental linkage for the development of some psychiatric illnesses.
A comprehensive review of a decade of mental health research linking genetics and environmental variables was undertaken by McLean Hospital and Harvard Medical School investigator Laramie Duncan, Ph.D., and co-author Matthew Keller, Ph.D., at the University of Colorado.
To date genetic-environmental interactions have been proposed for psychiatric disorders which include depression, attention-deficit hyperactivity disorder (ADHD), suicidal behaviours and alcohol abuse.
‘Based on our calculations and the date from related fields, we estimate that many of the positive findings in this particular area of research may, unintentionally, be incorrect,’ said Duncan.
There is a strong requirement for scientific research to be replicated. Studies need to be redone to see if the same results occur again and again. Only after research is replicated on several occasions can the findings be trusted and introduced into clinical practice.
‘What we suggest, to be certain about such correlations, is a focus on the cornerstone of scientific investigation—which is replication. The more we can replicate original findings in follow-up studies, the more we can be sure the results are accurate,’ says Duncan.
Duncan emphasised that her paper is intended to shed light on the fact that consistent, replicable results deserve more attention than novel findings and indirect replications. It is in no way meant to call for scepticism about the existence of gen-by-environment interactions, or psychiatric research in general.
‘Genetic research is like trying to identify a needle in a haystack and statistically, it is predicable that investigators will find and report false positives,’ Duncan said. ‘To separate the wheat from the chaff, we need to do follow up studies and replicate the outcomes. That’s the only way we can differentiate between accurate findings and the inevitable false positives.’
Duncan and Keller categorised each one of the 103 research studies conducted in the field of gene-by-environment interaction research in psychiatry between 200 and 2009. This included both novel-representing the first reports of particular interactions and replication studies.
From this analysis, Duncan and her team found that the rate of published significant noel or first-time results far outnumbered the rate of replicated results of the same studies.
Past reviews have found that novel findings have a greater likelihood to be published but are also more likely to be false positives.
‘Upon comparing novel with replication studies, we realise that the many discrepancies are likely a result of publication bias toward positive findings,’ said Duncan.
The tendency to favour publishing significant results over non-significant results is common in research, due to both the desire of publications to showcase groundbreaking findings and to the decision of authors not to submit null findings, according to Duncan and Keller.
Duncan warns that this bias can be misleading, especially if one is unaware of it and does not take into account take this into account when interpreting the validity of published findings.
‘Publication bias is problematic because it produces a distorted representation of findings in an area of study,’ said Duncan.
‘Through our research we found that 96% of novel studies were significant compared to just 27% of replication attempts, suggesting tht novel findings appear much stronger than they actually are,’ Duncan added.
Source: McLean Hospital/Harvard Medical School
