Night vision AI that can see color in the dark might have just beaten out Superman

For anyone who has ever wanted to see in infrared like Superman, humans still aren’t there yet, but robots are getting closer and closer.

Our species doesn’t really have super vision. We can only see in a narrow part of the spectrum, which is called visible light for a reason. But outside of that, other wavelengths are either too long (such as infrared and radio) or too short (such as UV). What colors we see depend on the wavelengths of light the eye picks up when they bounce off something. We also have trouble making out colors when there is little or no visible light around. Though a Kryptonian or Daxamite would laugh at our terrible night vision, a new AI can now see in the dark — in color.

Infrared night vision goggles have given us a broader range of vision. The problem is that they still see in black and white, but now researchers at UC Irvine, who recently published a study in PLOS ONE, have developed an AI that can automatically figure out what colors are supposed to be in a human face, even when that face is in the shadows. They were able to use visible light and several different wavelengths of infrared light to teach the AI the colors seen in human faces and how to recognize them in the absence of light. Kal-el probably couldn’t do that.

“We sought to develop an imaging algorithm powered by optimized deep learning architectures whereby infrared spectral illumination of a scene could be used to predict a visible spectrum rendering of the scene as if it were perceived by a human with visible spectrum light,” they said.

So what the AI was able to see is what we would see if our eyes had evolved color vision for the dark. Most night vision technologies we are familiar with use infrared light to figure out what they are seeing, and the images they come up with are then transformed into monochrome images in visible light. There is some advanced night vision tech that can pick up on and amplify the visible light in an area to produce an image. While earlier versions of AI night vision exist, and were able to detect objects in the infrared, there were always inaccuracies when it attempted to translate them into color images as they would appear in the visible part of the spectrum.

What this AI did differently was learn to take what it saw with infrared and near-infrared vision, then put it back together again in the visible part of the spectrum so we can actually see it. The researchers tested its accuracy by using red, green and blue (RGB) images illuminated by infrared light. That means that any colors could be mixed in the images so long as they were mixed from only those three. The researchers figured out how to translate each of these colors from how they appeared in the infrared to how they appeared in the visible part of the spectrum, so the AI would recognize them in the dark.

The bot needed to be taught to recognize the colors involved in human faces if it was going to recognize them. For this, images of actual faces were taken with a monochrome camera that could respond to both visible and infrared light. The same camera was also used to take photos in the dark. Now that it knew what colors ought to be in a human face, it was able to use what it learned to guess which colors were there and correctly generate images that looked as if they had been taken in visible light, maybe slightly darker but still eerily similar.

"This work suggests that predicting high-resolution images is more dependent upon training context than on spectroscopic signatures,” said the researchers.

Someday, maybe this tech could be used to create the ultimate night vision goggles that see in color even when the lights go out. Bye, Superman.