← Back to Writing

In Defense of the Half-Baked Idea

A couple years ago, a colleague of mine submitted a paper on an alternative explanation for dark matter. He had noticed that, when you squint at the math, you could explain a few properties we attribute to dark matter without needing any additional mass beyond what we see in stars. I won’t get into the details here, but there was one slight catch with his idea: it required negative mass. Not a lot of negative mass – essentially, it was just an artifact of the math – but negative mass nonetheless.

This sort of situation isn’t really anything particularly unique; someone probably posts a paper on the arXiv theorizing about a new solution to dark matter every single day. Each of these new solutions usually requires some new physics, or at least irregular applications of the usual physics (because if they used the regular methods, we would have discovered those solutions by now). While some of these theories are posed by crackpots – folks, typically retired male engineers, who become obsessed with theoretical physics and produce nonsensical theories of everything – my colleague wasn’t included in their ranks.

He had shown me his theory ahead of time, worked through the math with me. I had agreed that, while maybe a little unorthodox, there didn’t seem to be anything immediately wrong with the math. Actually, I was surprised by how well it seemed to work for what it was. It was a fun idea, if nothing else.

The community hated it. My colleague had been reluctant to even submit the paper, knowing that this was going to be the likely reaction. I had done my best to reassure him that publication was the right move. Even knowing the physics community and what it was, I was still shocked by the responses that he got. “Negative mass?” they cried, “What’s next, you’ll violate the conservation of weak isospin?” (This is a scathing insult for a physicist. We’re not the most socialized bunch.) He got hateful and accusatory emails for weeks. The editor of the journal went back and forth with him several times, saying that although there didn’t seem to be anything wrong with the paper, surely it can’t be published in this state – after all, it talks about negative mass!

Eventually, my colleague published the paper. I cheered. Well, I gave it a sincere smile while reading publications online that morning. I think I might have been the only one to celebrate.

To fully appreciate this, I need to give you a little background first. Take a walk with me.

What physicists call dark matter isn’t really a “thing,” so to speak. It was first indirectly noticed in the 1970s by Vera Rubin and Kent Ford, who saw that galaxies appeared to be rotating more quickly than they should have been based on how many stars they had. Similar effects were then observed in other areas of astronomy, like the velocities of large clusters of galaxies, and radio data of the first light from the beginning of the Universe.

(At the risk of going off on a tangent, people will often insist that Fritz Zwicky was the first to discover dark matter. However – not to diminish the incredible contributions of Zwicky – his paper reads more like a suggestion that there is a lot more non-luminous gas than expected, rather than what we would consider a modern understanding of dark matter. It’s sort of like saying that we’ve known about climate change since the 1820’s just because around that time one editorial in a newspaper pointed out that maybe all this coal dust could potentially do something funky to the atmosphere, without any real data. As a result, the insistence that Zwicky discovered dark matter often comes across as being upset that a woman discovered dark matter. Usually I would give the benefit of the doubt in a scenario like this, as astronomers are typically an accepting and inclusive bunch, but I’ve had too many interactions with senior professors who specifically insist that Zwicky discovered dark matter while making a nasty comment about a woman graduate student’s body in the same breath.)

The important thing to realize is that dark matter isn’t something we’ve been able to go out and touch, or collect. It’s not really, truly something at all, at least not yet. What we call dark matter is a collection of observations about the Universe that cannot be explained by the physics and the types and amount of stuff that we’re used to seeing every day. The most popular theory (by far) which explains these observations is that dark matter is a specific type of particle that doesn’t interact with light (making it “dark”), but is still affected by gravity. This particle theory of dark matter has been around nearly as long as we’ve known about dark matter, and it’s an appealing theory. It’s able to explain a lot of the things we observe when we look out into space.

The main problem with the particle theory of dark matter is that, as of right now, we haven’t ever been able to directly detect a dark matter particle. Not a single one. This is a big problem, because if dark matter is really made up of particles, there are expected to be millions and millions of them streaming through our bodies every second. It’s not for lack of trying: we’ve spent a massive sum of money on various experiments to detect the elusive dark matter particle, and while each one has gotten a little more precise than the last, we have yet to get our hands on a bona fide dark matter particle.

Now, if we haven’t been able to find the dark matter particle, then why are we still searching for particles at all? What if dark matter is really something different? The most common alternative to particle dark matter is that gravity actually works differently on large scales than we think it does. But the basic theories of modified gravity have their own problems: namely, that they can’t describe all of our observations of dark matter’s effects, which is a much bigger problem than not being able to find a dark matter particle. This isn’t to say that we’ve ruled out all theories of modified gravity – there are a lot of them out there, including some that are still potentially promising. But, astrophysicists tend to scoff when you bring up theories of modified gravity. There are both scientific and historical reasons for this (the people who supported the “original” theory of modified gravity were insular as a community, behaved poorly to their peers, and broadly participated in what can be considered non-scientific practices), but the key takeaway here is that particle dark matter is taken seriously, and alternative theories are generally not.

This has led to a cultural understanding that if you’re a “real physicist” – whatever that means – you believe in particle dark matter. In turn, this leads to strange conversations in dark corners of bars or private Zoom rooms where someone will ask you, “So, what do you think dark matter actually is?” with the same curious yearning for the naughty and forbidden as someone trying weed for the first time, or a teenager who’s worried that their parents will find out they snuck out last Tuesday night. This essay isn’t really about my personal beliefs on individual theories of dark matter, but I can give you my go-to answer when I’m asked this question: I don’t know, and I think anyone who says that they do know is trying to sell you something.

The reason this has led to such an awkward pretension in the field is that you really can damage your reputation by publishing a serious paper on something other than particle dark matter. I’m really not trying to vilify the people who study particle theories of dark matter. Truly. But when an otherwise reputable physicist puts out a paper on modified gravity or some strange new idea that looks interesting, they’re relegated to the bin of kooks in the corner who had their time in the sun, but aren’t to be taken seriously anymore.

I’m worried about this particular part of physics culture. People often raise concerns that modern physics hasn’t contributed anything in the last 50 years. This is of course incorrect; physics has contributed an immense amount to the development of many technologies in the last several decades, including things like better semiconductors, viable superconductors, quantum computers, medical imaging technology, and so on, which have materially and substantially improved our quality of life. What people mean when they say this is that physics hasn’t contributed any big things in the last half-century. General relativity and quantum mechanics were enormous breakthroughs from the early 20th century – if we’re employing so many physicists, why haven’t we come up with Quantum Mechanics II: Son of Quantum Mechanics or General Relativity 2: Age of Ultron yet?

I’m far from the only person thinking and worrying about this, but I’m happy to give my two cents. I think the answer to this question lies somewhere in how modern physics treats proposals for new theories. Think back to 7th grade science class, when you learned about the Scientific Method: First, one develops a hypothesis (in other words, they guess); next, you figure out what predictions are made by this hypothesis, and then you go and test them to see if they match your ideas. If they match, then great – your theory has some promise. What’s even better is if the old theory predicts something different than your new one, and you can show how your new theory blows the old one out of the water. This is how scientific progress is made. Granted, this new science usually looks like incremental improvements in niche fields, but this is where the real work is done.

So, based on this understanding of the scientific method, we can go one step further. If you want to propose some new theory and unseat the old one, you can’t just prove that your new method performs better than before by testing one single case – you have to test ALL the possible cases that are explained by the old theory, and you better hope that your theory performs as well or better on each test. If it doesn’t, then it’s to Davy Jones' Locker for your theory.

This is how physicists judge a theory. If you want to propose a theory to replace quantum mechanics, then your new theory better be able to reproduce ALL phenomena we can currently explain with quantum mechanics, and then something extra. Only then will physicists take your theory seriously. This arises partially out of preference and partially out of necessity. Physicists universally get a constant barrage of emails from crackpots asking the physicist to take a look at their pet theory of everything and how they’ve solved physics. These emails are generally misinformed at best, and AI-generated hallucinatory psychosis at worst. I’ve only heard of a couple of (legendary) cases where the message had any sort of real, useful content. If physicists took each one of these ideas seriously, and spent the time to disprove them, the physicist wouldn’t have any time for their own work. So, consequently, the physicist has to build a fairly high barrier to entry in order to protect their mental effort. If you can’t show that you’ve reproduced all the phenomena from a previous theory plus something new, then you won’t be taken seriously.

This makes sense. I get this, because I get these exhausting emails too. In my mind, the broader issue arises when we begin to treat people who are clearly informed about physics with the same disdain and disregard (not that I hate crackpots – I find them fascinating, which is its own problem – but many of my colleagues look down on them, which is, again, its own problem). When a physicist asks a new question, they don’t expect it to revolutionize physics. They’re just being curious. It’s natural to see how far these ideas go, and when they perform fairly well, it makes sense to see how far the rabbit hole goes. This is the sort of thing that happened to my colleague: he had an interesting idea, it caught his attention, and because he couldn’t easily and quickly prove it wrong, he wanted to share it with the community.

Now, ideas like this aren’t uncommon. I agree that the onus is on the author or interested parties to show that their idea should be taken seriously, and where it may be valuable. Again, if physicists paid serious attention to each one of these ideas, there wouldn’t be any time left for their own work. When my colleague posted his paper, what was said out loud was “Okay, you’ve shown that this new theory works for 3-5 major things. But it’s not a real theory until it works for every single thing! I won’t pay attention until you’ve shown this.”

This appears consistent with how we understand the scientific method. I would like to argue that it isn’t.

In my mind, what the community actually, secretly, was saying was “Oh, another crackpot idea for dark matter. Go away! We know what dark matter is: it’s particles! You’re crazy for thinking otherwise.” I don’t really think I’m being overly sensitive or dramatic here. While the requirement that you prove that your new theory can do everything the old theory can and more may seem scientifically sound on the surface, it’s not actually a reasonable ask. Asking for this is just a way to politely get these theories out of the room so that you don't have to engage with it in good faith. You can’t realistically ask one person or even one team to go and solve each one of these problems before you take their theory seriously – and I think this is supported by the historical events that led to the foundations of our most successful modern theories.

It’s well known that general relativity was incorrect in the way that Einstein initially formalized it. There were multiple errors in the math, including serious errors that led to incorrect predictions of some phenomena. When Einstein initially wrote it down, the theory wasn’t able to reproduce all of the natural experiments people had explained using classical physics. Why then, should they have taken it seriously? Even more importantly, general relativity was endorsed and studied by many physicists before the theory had progressed to the point where all of these features had been explained. General relativity was initially “half-baked”, and it took an entire field a whole generation to hoist the theory up into its modern pedestal. If they had rejected general relativity at the first instance of it not working, instead of saying "huh, okay, let's see if we can't figure out what's going wrong here...", we wouldn't have our modern understanding of the Universe.

For this reason, I think there’s virtue in the half-baked ideas.

Understand that I don’t mean half-baked as an insult; rather, I mean that the idea just hasn’t had every single thing sorted out yet. Maybe no one has looked at how this idea performs at very high energies, or under extreme pressure. Maybe no one has thought about what this idea means for the very early Universe. That’s okay. We can get to those things in time, like the pioneers of quantum mechanics and general relativity did in the early 20th century. They were able to recognize a theory for its own merit regardless of complete success. I fear that we’ve lost the ability to do that among the inundation of AI slop and crackpot emails that we have to wade through these days. And, with this in mind, perhaps it’s no wonder that we haven’t been able to make any big advancements in physics for the last 50 years.

It’s clear that there’s more to physics than we currently understand. Where is the dark matter? Can the Standard Model actually predict the measured magnetic moment of muons? Why can’t we properly measure the Hubble parameter?

I dream that we’ll figure out what dark matter is within my lifetime. I desperately want General Relativity 2: Age of Ultron. But, we won’t get to the bottom of these questions unless people are willing to explore the half-baked theories which don’t quite meet muster, but almost do. These ideas are where insight is to be found.