The Experimental Deficit of Academia

Creativity is hard to assess. When someone proposes a fanciful idea they might be a creative genius or a weirdo, it’s hard to know for sure. If you’re faced with a fanciful idea and have to decide whether to fund it or not, you have to come up with a system. For example, you could ask other creative people you trust — maybe because they have succeeded in the past — for their honest assessment.

This is how peer evaluation works in academia. Researchers write papers and grant proposals, then other researchers in their field vote “yea” or “nay”.

The problem is that, over time, almost everyone in a given research community starts considering the same ideas as promising or not. Those who disagree end up being voted off the island. They see their funding reduced.

Often, academics voted off the island are real weirdos whose research would never lead to anything. Sometimes, however, they are creative geniuses.

It was the case of Geoff Hinton, Yann LeCun and Yoshua Bengio, 20 years ago. Back then, most machine learning scientist dismissed research into neural networks as uninteresting. After all, biomimicry — using the brain as inspiration to build learning systems — was a far cry from the serious mathematics other researchers were working on.

As a result, research into neural networks struggled to get funded. We must thank CIFAR, a relatively small Canadian research organization, for supporting the trio of scientists who eventually gave us the deep learning revolution.

Another creative genius that was ignored by peer-moderated funding is Kati Kariko, whose research into messenger RNA laid the foundation for the development of the COVID vaccine.

For her entire career, Dr. Kariko has focused on messenger RNA, or mRNA — the genetic script that carries DNA instructions to each cell’s protein-making machinery. She was convinced mRNA could be used to instruct cells to make their own medicines, including vaccines.

But for many years her career at the University of Pennsylvania was fragile. She migrated from lab to lab, relying on one senior scientist after another to take her in. She never made more than $60,000 a year.

Dr. Kariko’s struggles to stay afloat in academia have a familiar ring to scientists. She needed grants to pursue ideas that seemed wild and fanciful. She did not get them, even as more mundane research was rewarded.

“When your idea is against the conventional wisdom that makes sense to the star chamber, it is very hard to break out,” said Dr. David Langer, a neurosurgeon who has worked with Dr. Kariko.

“We both started writing grants,” Dr. Weissman said. “We didn’t get most of them. People were not interested in mRNA. The people who reviewed the grants said mRNA will not be a good therapeutic, so don’t bother.”

By Gina Kolata, New York Times

The hero of the day struggled to get her research funded. Let that sink in.

Then ask yourself how many similar creative geniuses were not able to pursue their research. How many potentially world-changing innovations did we miss on?

This is what I call our experimental deficit. The value we lose from the chances we dont take. I believe it’s huge. I believe it’s getting bigger. I believe we should try to measure it.

Fighting that deficit requires more diversity of ideas. It requires embracing a bit of randomness when we choose what to fund or not. It requires us to spend that funding over more researchers rather than betting on a few superstars.

Most importantly, it requires us to be OK with being wrong every now and then.