posted at big think
“The field of psychology has been shaken by a massive replication effort, which has found that out of 98 papers published in the top three psychology journals only 39 could be replicated.
(…) Whenever you hear the words “new study,” alarm bells should ring. It isn’t new studies that you should base your opinions on; it is old studies that have been replicated again and again, and the results reported in meta-analyses and systematic reviews.
(…) we have every indication to believe that new research is similarly difficult to replicate across many areas of science. For example, 30 percent of the most widely cited randomized controlled trials in the world’s highest-quality medical journals have later been found to be wrong or exaggerated and that number rises to five out of six for non-randomized trials — a number that is in fact worse than the rate found by the psychology reproducibility project.
(…) The solution is to take a skeptical approach to the world around us, to treat every new claim not as a problem solved, but as an open question.
(…) Sexy findings sell newspapers for the same reasons that they get published in top journals. In the increasingly cutthroat age of digital journalism, few journalists outside specialist publications have the time or inclination to critically assess new findings or place them in appropriate context.
So what are you to do? The first step is to learn the basics of how to evaluate research critically for yourself. The next step is to seek out informed, critical voices who can present balanced and evidence-based analysis of new research. If the publication you are reading reports a study as fact without offering any criticisms or bothering to reference any other studies, seek out an alternative.
The simplest way to avoid being misled is to stop getting your psychology and science news from, well, the news.(…)
When it comes to psychology and science, the news is simply very often the single worst place to get informed. There are far better places where you can read informed perspectives, where claims are placed in the context of the literature rather than treated as standalone nuggets of truth. At the top of any list should be specialist publications that employ experts to evaluate new findings against what we already know.(…)” read full article