A Physics Outsider Says NASA Asteroid Scientists Are All Wrong. Is He Right?
(Spoiler: No)

People love a good “David versus Goliath” story, especially when David is an outsider, a lone voice against a big government agency that he’s accusing of being incompetent and wasting money.

The problem with that narrative is that sometimes David is wrong.

Nathan Myhrvold is a billionaire, the former chief technology officer at Microsoft, and holds a Ph.D. in physics. He has a scientific background, obviously, and recently became interested in the search for near-Earth asteroids, the kind that get close enough to our home planet to be a threat.

After doing some of his own research, he thinks that a team of scientists at NASA/JPL and Caltech have made serious errors in measuring the properties of asteroids. And he’s gone farther than that: He’s accused them of making “colossal error[s],” and made hints of fraud.

That’s an extremely serious accusation. It’s also, in my opinion, grossly wrong (NASA also issued a response backing up the scientists). After considering his research, I have concluded that the team is not in error, Myhrvold is.

The team of scientists is working on a potential NASA mission called NEOCam—the Near-Earth Object Camera—to map out the locations, orbits, and physical characteristics of near-Earth asteroids as a way to systematically determine what threat they pose to our fair planet. To do this, the scientists need to understand how their telescope operates; that is, if it sees an asteroid, how do you convert that brightness measured to a diameter? The size of the asteroid is a critical factor; without that it’s very difficult to understand much else about it. Over time, the team has published several papers showing how they make this calculation, using methods used before for earlier satellites, including NEOWISE, the predecessor of NEOCam.

Myhrvold, wanting to explore this topic himself, set about trying to replicate their results on his own. What he found, though, was that his numbers for asteroid diameters disagreed with the results from the team of scientists, sometimes by a huge margin.

One of them was wrong. Which one?

Myhrvold contacted the team, including Amy Mainzer, the NEOCam principal investigator, who has a long history of publishing scientific papers in the asteroid discipline, in order to discuss his ideas. (Full disclosure: I have known Amy for many years and consider her a friend. I talked with her about Myhrvold’s claims, and she was very helpful in pointing out references and explaining some of the physics and mathematics involved.)

At this point, the accounts diverge. Myhrvold says the team was not cooperative about their work and gave him “cryptic” answers to his questions.

Mainzer told me a very different story. She said she worked with Myhrvold multiple times, trying to show him where some of his ideas were either incorrect or not applicable to the work they were doing, but he remained defiant. She pointed out specific errors, but despite that the errors remain in his work.

The errors she mentioned are various, including his confusing diameter with radius in his calculations and using a model that incorrectly determines diameters. For his part he says their model doesn’t include some basic physics, and that some of their numbers are suspicious.

So what’s going on? Other articles cover some of the basic claims (see "further reading" at the end of this article), but I want to take on some of the science claims.

The Light and the Heat

Myhrvold claims that the physical model of asteroids the team uses doesn’t work and that they ignore “basic physics.” Now, setting aside the rather bizarre idea that an entire team of professional scientists at JPL and Caltech might not understand undergraduate-level physics, his claim that their model doesn’t work is simply wrong. Worse, it’s his own model that falls short.

The basic goal here is to determine the diameters of asteroids. How do you do that? The idea behind NEOCam is that it will use very sensitive detectors to measure the amount of infrared light the asteroids emit. Anything warm emits infrared light, and the amount of light an asteroid gives off depends in part on its size. By measuring different colors of infrared, complicating factors (like reflectivity) can be accounted for, and the diameter found.

To make sure they’re doing it right, the NEOCam team looked at asteroids for which the diameters were previously known (that part’s important). There are quite a few with accurately measured diameters; for example, some have been pinged with radar, which yields very good results for asteroids that get near the Earth. Others have had their sizes measured using occultations: Sometimes an asteroid will pass directly in front of a star, blocking its light as seen from Earth. Knowing the velocity at which the asteroid moves around the Sun together with the length of time it blocks the star gives you its size.

NEOWISE (the earlier satellite) observed these asteroids. The team then used the known diameters to create a model (a mathematical equation) of how the light is emitted. That way, they can then observe other asteroids with unknown diameters and use their brightness to measure their sizes. This is the model they’re using for NEOCam.

This is the model Myhrvold claims is wrong. However, the asteroid diameters found by the NEOWISE team agree very well with previous satellite measurements. NEOWISE looked at many of the same asteroids as an earlier mission called IRAS—a couple of thousand of the same asteroids—and found that the diameters calculated for those asteroids matched the measurements using IRAS to about 10 percent. Not only that, measurements using a Japanese satellite called Akari also yielded similar results, and all three agree well with the radar and occultation measurements.

That’s a very good indication the NEOCam team is doing things right.

Despite this, Myhrvold disagrees. He says the team is ignoring basic physics, and the diameters they are getting are wrong.

Myhrvold has written a paper with his results, and when I read it, it became clear to me that this accusation is based on a false premise. He is trying to calculate asteroid diameters starting with basic physics (from the ground up, so to speak), while the NEOCam team is doing it empirically, based on observations (so, from the top down). In the latter case the physics is built in to the way the model is generated.

Here’s an analogy: Imagine a quarterback trying to throw a football into a receiver’s hands. An accomplished quarterback has thrown the ball thousands of times, and already knows by feel and muscle memory how to throw the ball, giving it the right speed, direction, and spin to complete the pass.

What Myhrvold is trying to do is start with the physics of motion, trigonometry, momentum transfer, air drag, and so on and then telling the quarterback how to throw the ball.

But the physics is built in to the quarterback’s experience. The passer doesn’t need to calculate all the physics to complete the pass. If we humans had to do the calculus every time we wanted to move, we’d be frozen stiff. So Myhrvold’s accusation that the NEOCam’s team ignores basic physics is incorrect.

Worse, Myhrvold’s model is wrong. He published his results in his paper, and the numbers he gets are way off. For example, in Figure 21, he uses his model to calculate the diameter of the asteroid 295 Theresia. The known diameter of that asteroid is about 28 kilometers. Myhrvold gets a diameter of 660 kilometers, more than 20 times too big (if he were correct, it would be the second largest asteroid known)! Other examples with similar erroneous diameters can be found. It’s worth noting that other groups have used the NEOWISE data to compute their own models and have had no problems.

Let me be clear here: If you are accusing scientists of messing up the single most important thing they’re trying to measure—an asteroid’s diameter—you’d better get that right yourself. Myhrvold didn’t.

There’s more. For example, Mainzer told me that he confused radius for diameter in several places in his work—which she pointed out—but those errors remained even as he updated his paper. She also told me he based some of his numbers on an old paper about IRAS that had a systematic error in it, one that overestimates the sizes of asteroids.

Despite his own errors, he is accusing the NEOCam team of being wrong.

But it gets worse.

Cut and Paste

Myhrvold wrote what’s called a “white paper,” an overview of the situation as he sees it using simpler language. In it, he notes that some of the diameters for asteroids the NEOCam team calculated are exactly the same as found by an earlier mission. Like, exactly, to several decimal places. Because the chances of this are essentially zero, Myhrvold speculates that the NEOCam team may have had a bug in their code that copied the diameters from the earlier results and propagated it to subsequent results.

Or, he goes further: He speculates that maybe this is due to fraud.

Yes, fraud, as in knowingly faking these numbers. This is a stunningly serious accusation. But there’s a much simpler explanation about the duplicate numbers. I asked Mainzer about this specifically. She told me the numbers are the same because they are copied, but it’s not due to fraud. The mathematical model they use does indeed calculate the diameter, as well as several other important variables that are initially unknown (for example, how reflective the asteroid is in visible light and infrared light). However, if the diameter has already been accurately measured in other ways (radar, occultations or from previous satellite measurements), they can use that number to better calculate the other variables.

This technique is discussed in a paper published in 2011. Myhrvold acknowledges that but then says this is never mentioned in subsequent papers. If true, that would make it seem like the team, in those subsequent papers, is claiming the diameters were calculated using their model, and were not actually from previous measurements. Hence his suspicion of fraud; in fact in his white paper he goes on at length about this. However, as just one example, in this subsequent paper by the team (in Section 3, first paragraph), they clearly reference the first paper describing the method they use, including using measured diameters if available to better nail down the other variables.

In other words, Myhrvold is wrong. They do in fact discuss how they got the diameters in later papers. He may have simply missed this in the later paper. But either way he should have been more diligent in discussing this with the NEOCam team, especially given the seriousness of his accusations.

If you mention fraud, even as a possibility, you’d better have solid data to back it up. From what I’ve seen, Myhrvold’s claims don’t even come close.

There’s one final thing, and it’s important: Myhrvold’s paper has not yet been peer reviewed. This is a critical step in the scientific process: Other scientists look over the paper and make sure it has the proper methodology, it doesn’t have errors, it’s relevant, and more. They make suggestions to the author, who then rewrites the paper (or redoes the research if necessary), and resubmits it. If the reviewers approve, it gets published in a journal.

The peer review process is absolutely necessary to vet work from start to finish, but Myhrvold’s paper has not yet gone through this process. It may not be foolproof, but it’s at least likely to catch errors that are hard for the author to see (like his own erroneous asteroid diameter calculations). An outsider’s opinion can sometimes be valuable.

Despite that, he issued a press release, which is unconventional to say the least. This is sometimes done when a team wants the general community to contribute and comment, but that’s not what Myhrvold is doing. He’s using it as a platform to make accusations that have not been thoroughly checked. It lends a bad smell to what he’s done.

I know this is a complicated issue, but it’s an important one. Questioning results is fine, if it’s done in good faith and by someone who is well-versed in the subject. Myhrvold has cast himself as the outsider with a unique perspective, as David. But in this case, there’s a reason Goliath is, well, Goliath. The NEOCam team has the data, they’ve been open about their methods, and everything they’ve done is online and available for scrutiny.

I’ll be very curious to know what happens once Myhrvold’s paper is peer reviewed. I hope he’ll be as public about it then as he has been up to now.

Further Reading

Because this topic is complex, here are few links to other articles about this to help you parse it all:

• The New York Times article (which is sympathetic to Myhrvold’s claims)
• An article critical of Myhrvold in the Washington Post
• The NASA response
• A neutral synopsis at Inverse
• A neutral synopsis at Science (the comments are interesting too)
• An occultation astronomer opinion critical of Myhrvold on the Minor Planet Mailing List
• A member of the NEOCam team also posted to MPML with a critical look at the claims