Doubts about Johns Hopkins research have gone unanswered, scientist says
The numbers didn't add up.
Over and over, Daniel Yuan, a medical doctor and statistician, couldn't understand the results coming out of the lab, a prestigious facility at Johns Hopkins Medical School funded by millions from the National Institutes of Health.
He raised questions with the lab's director. He reran the calculations on his own. He looked askance at the articles arising from the research, which were published in distinguished journals. He told his colleagues: This doesn't make sense.
"At first, it was like, ‘Okay — but I don't really see it,' " Yuan recalled. "Then it started to smell bad."
His suspicions arose as reports of scientific misconduct have become more frequent and critics have questioned the willingness of universities, academic journals and the federal government, which pays for much of the work, to confront the problem.
Eventually, the Hopkins research, which focused on detecting interactions between genes, would win wide acclaim and, in a coup for the researchers, space in the pages of Nature, arguably the field's most prestigious journal. The medical school even issued a news release when the article appeared last year: "Studies Linked To Better Understanding of Cancer Drugs."
What very few readers of the Nature paper could know, however, was that behind the scenes, Yuan's doubts seemed to be having profound effects.
In August, Yu-yi Lin, the lead author of the paper, was found dead in his new lab in Taiwan, a puncture mark in his left arm and empty vials of sedatives and muscle relaxants around him, according to local news accounts — an apparent suicide.
And within hours of this discovery, a note was sent from Lin's e-mail account to Yuan. The e-mail, which Yuan saved, essentially blamed him for driving Lin to suicide. Yuan had written to Nature's editors, saying that the paper's results were overstated and that he found no evidence that the analyses described had actually been conducted. On the day of his death, Lin, 38, the father of three young daughters, was supposed to have finished writing a response to Yuan's criticisms.
The subject line of the e-mail to Yuan, sent by an unknown person, said "your happy ending."
"Yu-yi passed away this morning. Now you must be very satisfied with your success," the e-mail said.
Yuan said he was shocked by the note, so much so that he began to shake.
But in the seven months since, he has wondered why no one — not the other investigators on the project, not the esteemed journal, not the federal government — has responded publicly to the problems he raised about the research.
The passions of scientific debate are probably not much different from those that drive achievement in other fields, so a tragic, even deadly dispute might not be surprising.
But science, creeping ahead experiment by experiment, paper by paper, depends also on institutions investigating errors and correcting them if need be, especially if they are made in its most respected journals.
If the apparent suicide and Yuan's detailed complaints provoked second thoughts about the Nature paper, though, there were scant signs of it.
The journal initially showed interest in publishing Yuan's criticism and told him that a correction was "probably" going to be written, according to e-mail rec*ords. That was almost six months ago. The paper has not been corrected.
The university had already fired Yuan in December 2011, after 10 years at the lab. He had been raising questions about the research for years. He was escorted from his desk by two security guards.
More recently, a few weeks after a Washington Post reporter began asking questions, a university spokeswoman said that a correction had been submitted to Nature and that it was under review.
"Your questions will be addressed with that publication," a spokeswoman for the Hopkins medical school, Kim Hoppe, wrote in an e-mail.
Neither the journal nor the university would disclose the nature of the correction.
Hoppe declined an opportunity to have university personnel sit for interviews.
In the meantime, the paper has been cited 11 times by other published papers building on the findings.
Read Part Two of ‘Big Pharma': Medicare overspending on anemia drug
It may be impossible for anyone from outside to know the extent of the problems in the Nature paper. But the incident comes amid a phenomenon that some call a "retraction epidemic."
Last year, research published in the Proceedings of the National Academy of Sciences found that the percentage of scientific articles retracted because of fraud had increased tenfold since 1975.
The same analysis reviewed more than 2,000 retracted biomedical papers and found that 67 percent of the retractions were attributable to misconduct, mainly fraud or suspected fraud.
"You have a lot of people who want to do the right thing, but they get in a position where their job is on the line or their funding will get cut, and they need to get a paper published," said Ferric C. Fang, one of the authors of the analysis and a medical professor at the University of Washington. "Then they have this tempting thought: If only the data points would line up . . . "
Fang said retractions may be rising because it is simply easier to cheat in an era of digital images, which can be easily manipulated. But he said the increase is caused at least in part by the growing competition for publication and for NIH grant money.
He noted that in the 1960s, about two out of three NIH grant requests were funded; today, the success rate for applicants for research funding is about one in five. At the same time, getting work published in the most esteemed journals, such as Nature, has become a "fetish" for some scientists, Fang said.
In one sense, the rise in retractions may mean that the scientific enterprise is working — bad work is being discovered and tossed out. But many observers note that universities and journals, while sometimes agreeable to admitting small mistakes, are at times loath to reveal that the essence of published work was simply wrong.
"The reader of scientific information is at the mercy of the scientific institution to investigate or not," said Adam Marcus, who with Ivan Oransky founded the blog Retraction Watch in 2010. In this case, Marcus said, "if Hopkins doesn't want to move, we may not find out what is happening for two or three years."
The trouble is that a delayed response — or none at all — leaves other scientists to build upon shaky work. Fang said he has talked to researchers who have lost months by relying on results that proved impossible to reproduce.
Moreover, as Marcus and Oransky have noted, much of the research is funded by taxpayers. Yet when retractions are done, they are done quietly and "live in obscurity," meaning taxpayers are unlikely to find out that their money may have been wasted.
Johns Hopkins University typically receives more than $600 million a year from NIH, according to NIH figures.
For someone who has taken on a battle with Johns Hopkins and Nature, Yuan is strikingly soft-spoken.
He grew up in Gainesville, Fla., and attended MIT and then medical school at Johns Hopkins. He worked briefly as a pediatrician and an assistant professor of pediatrics before deciding that he preferred pure research. He has a wife and two kids and is an accomplished violinist.
In 2001, he joined the lab of Jef Boeke, a Hopkins professor of molecular biology and genetics. Boeke's work on the yeast genome is, as academics put it, "highly cited" — that is, other papers have used some of his articles numerous times for support. Last year, he was named a member of the prestigious American Academy of Arts and Sci*ences.
The lab's research focused on developing a methodology for finding evidence of genes interacting, primarily in the yeast genome and then in the human genome. Genetic interactions are prized because they yield insights into the traits of the genes involved.
During Yuan's time there, the lab received millions in NIH funding, and according to internal e-mails, the people in the lab were under pressure to show results. Yuan felt the pressure, too, he says, but as the point person for analyzing the statistical data emerging from the experiments, he felt compelled to raise his concerns.
As far back as 2007, as the group was developing the methodology that would eventually form the basis of the Nature paper, Yuan wrote an anguished e-mail to another senior member of the lab, Pamela Meluh.
"I continue to be in a state of chronic alarm," he wrote in August 2007. "The denial that I am hearing from almost everyone in the group as a consensus is troubling to me."
Meluh quickly wrote back: "I have the same level of concern as you in terms of data quality, but I have less basis to think it can be better. . . . I'm always torn between addressing your and my own concerns and being ‘productive.' "
Then Boeke weighed in, telling Yuan that if he could improve the data analysis, he should, but that "the clock is ticking."
"NIH has already given us way more time than we thought we needed and at some point we've got to suck it up and run with what we have," Boeke wrote to Meluh and Yuan.
A few years later, another deadline was looming, and Elise Feingold, an NIH administrator, wanted to know what the lab had accomplished.
"I do need some kind of progress report on what you have been doing the past two years . . . and what you think you can accomplish with these funds," she wrote to Boeke.
Citing Feingold's message, *Meluh wrote to Yuan, asking for help in explaining what the lab had produced. Its members had worked diligently, Yuan says, but hadn't arrived at the kind of significant findings that generally produce scientific papers.
"I want to make it look like we've been busy despite lack of publications," Meluh wrote.
Meluh did not respond to a request for an interview. Boeke referred questions to the university's public relations team, which declined to comment further. An NIH official declined to comment.
While Yuan was growing increasingly skeptical of the lab's methodology, Yu-yi Lin, who was also working at the lab, was trying to extend it. In the past, it had been applied to the yeast genome; Lin would extend it to the human genome — and this would become the basis of the Nature paper.
Lin, who was from Taiwan, was an up-and-comer. As a graduate student at Johns Hopkins just a few years before, he'd won an award for his work in cell metabolism and aging. He was also arranging for a prestigious spot at National Taiwan University.
At one point, when he was still at the Boeke lab at Hopkins, Lin asked Yuan to help analyze the data that would become the basis for the Nature paper, Yuan says. Yuan said he declined to get involved because he thought the methodology still had deep flaws.
Interactions between Lin and Yuan at the lab were few, Yuan said, and at any rate, Yuan had other things to worry about. He was slowly being forced out. He was demoted in 2011 from research associate to an entry-level position. A disagreement over whether Yuan should have asked Boeke if he wanted a byline on a paper erupted into further trouble, e-mail and other records show.
The Johns Hopkins spokeswoman, Hoppe, declined to discuss Yuan's job termination.
On Dec. 15, 2011, Yuan was forced to leave the lab. He wasn't allowed to make copies of his cell collection. He spent the next month trying to keep his mind busy. He read books about JavaScript and Photoshop, which he thought would enrich his research abilities. As he looked for other research jobs, he sensed that he had been blackballed.
Then, in February 2012, the Nature paper was published.
The research was a "profound achievement" that would "definitely be a great help to solve and to treat many severe diseases," according to a news release from National Taiwan University, where Lin was now working.
Upon reading it, Yuan said, he was astonished that Lin had used what he considered a flawed method for finding genetic interactions. It had proved troublesome in the yeast genome, he thought. Could it have possibly been more reliable as it was extended to the human genome?
Lin, Boeke and their co-authors reported discovering 878 genetic interactions, or "hits."
But Yuan, who was familiar with the data and the statistics, reanalyzed the data in the paper and concluded that there was essentially no evidence for any more than a handful of the 878 genetic interactions.
One of the key problems, Yuan wrote to the Nature editors, was that the numerical threshold the investigators used for determining when a hit had arisen was too low. This meant they would report far more hits than there actually were.
Yuan also calculated that, given the wide variability in the data and the relative precision required to find a true hit, it would have been impossible to arrive at any conclusions at all. By analogy, it would be like a pollster declaring a winner in an election when the margin of error was larger than the difference in the polling results.
"The overwhelming noise in the . . . data and the overstated strength of the genetic interactions together make it difficult to reconstruct any scientific process by which the authors could have inferred valid results from these data," Yuan wrote to the editors of Nature in July.
His analysis attacks only the first portion of the paper; even if he is correct, the second part of the paper could be true.
Nevertheless, Yuan wanted Nature to publish his criticism, and following instructions from the journal, he forwarded his letter to Boeke and Lin, giving them two weeks to respond.
Just as the two weeks were to elapse, Boeke wrote to Nature asking for an extension of time — "a couple weeks or more" — to address Yuan's criticism. Boeke explained that end-of-summer schedules and the multiple co-authors made it difficult to respond on time.
A day later, Lin was discovered dead in his office at National Taiwan University.
"Renowned scientist found dead, next to drug bottles," the headline in the Taipei Times said.
Even in his death, the Nature paper was a kind of shorthand for Lin's scientific success.
"A research team [Lin] led was featured in the scientific journal Nature in February for their discovery of the key mechanism for maintaining cell energy balance — believed to be linked to cellular aging and cancer," the newspaper said.
If there was a suicide note, it has not been made public, and it is difficult to know what went through Lin's mind at the end of his life. The apparent suicide and the e-mail to Yuan suggest only that Lin may have been distraught over the dispute; they do not prove that he acted improperly.
Shortly after the Nature paper appeared, Yuan hired lawyer Lynne Bernabei to challenge the way he was terminated at Hopkins.
In late August, Yuan asked the Nature editors again whether they would publish his criticism. Lin was dead, but Boeke and the others had had a month to respond, and Yuan hadn't heard a thing.
On Sept. 28, a Nature editor informed Yuan by e-mail that the journal was still waiting on a fuller response from Boeke and that "experiments are being done and probably a Correction written."
Such a correction has not appeared.
So as a last attempt, he figured he'd try the federal government, which paid for much of the research. But the government suggested that the threat to the federal research, if there was any, ended with Lin's death.
"It is our understanding that these allegations are being investigated by Johns Hopkins University," said the letter from the Office of Research Integrity.
Besides, it noted, the person responsible for the paper was Lin.
"Deceased respondents no longer pose a risk," the letter said.
http://www.washingtonpost.com/busin...c84-11e2-82e8-61a46c2cde3d_story.html?hpid=z3
