How these see-through fish swimming in virtual seas can (sort of) predict the future

Transparent fish living in a VR simulation sounds more like a minor subplot in a ‘90s-era straight to VHS cyberpunk movie than real life. But for real — it’s the setting of actual research at the RIKEN Center for Brain Science, where Dr. Hitoshi Okamoto and colleagues are studying decision making in fish. Their findings were published in the journal Nature.

Decision making in animals is driven by maximizing reward or avoiding surprise. The team set out to quantify these behaviors in fish through the use of a simulated environment. Zebrafish were chosen, in part, because of a pigment-deficient mutation which makes them partly transparent, allowing for convenient observation of brain activity.

The fish were tethered in place using a custom harness. This was necessary in order to capture the desired neural activity. “Currently, we do not have a camera which can be attached to the head of a swimming fish,” Okamoto told SYFY WIRE. “In the case of mice, there is an endoscopic camera which can image the neural activity of a limited region through an inserted columnar GRIN lens. But it is too big for the small head of a zebrafish.”

The tank was then surrounded on all four sides by LCD TVs which displayed a color landscape, either white, blue, or red. In initial tests, blue areas were coded as dangerous while red areas were safe. White areas were neutral and used as the interstitial landscape between tests. Because the fish were harnessed in place, movement throughout the virtual environment had to be simulated. A camera placed above the fish monitored tail movements and shifted the landscape toward the tail to simulate forward swimming.

Okamoto did initially try using projected images of predators, but that method did not provide the expected results. “We have tried to project the image of the zebrafish predator, nandus nandus, on the screen to see if the fish try to escape from the image. They did not behave as we expected,” Okamoto said.

Instead, fish were trained to avoid blue areas through electric shock delivered via electrodes placed on either side of the body. They were then tested through either a GO or NOGO scenario to measure active or passive avoidance behaviors. In GO scenarios, the fish was placed in a blue environment. They were given ten seconds to escape to a red zone by simulated forward swimming, in order to avoid a shock. NOGO scenarios presented a red environment wherein the aim was to remain in the safe zone.

“At the initial stage of training, the fish learns that they will receive a shock, a negative reward, upon presentation of a blue color on the screen. Then, they experience that they can avoid receiving shock if they enter into the red arena. Avoidance of expected negative reward means for the fish the same as a positive reward,” Okamoto said.

They found that fish are capable of assigning rules to their environment, comparing those rules to what they experience, and modifying their behavior. Essentially, they were able to build predictive models which stated either “blue is dangerous” or “red is safe.” In some cases, individual fish were able to build both mental models. Those fish performed better than their peers.

In a subset of test subjects, the colors were switched after initial testing. In this second phase, red became dangerous and blue became safe. Experiments showed that in at least some individuals they were able to overwrite the previous rules to learn the newly favorable conditions.

Interestingly, of 129 total fish used in the study, only one-third of them were capable of meeting the learning criteria. What may have contributed to the inability of the remainder to successfully build a predictive model is unknown. This may be related to the limitations of measuring brain activity.

The team used a two-photon microscope to collect signals near the surface of the telencephalon—this area of a fish brain is analogous to the isocortex in mammals, which is crucial for decision making—future experiments aim to measure other parts of the brain simultaneously. Those experiments could further unlock clues to what’s going on inside the brain of a fish when it’s making a decision.

The predictive prowess of fish is sufficient for survival and malleable enough to accommodate changing conditions, but they probably won’t be winning the powerball or preventing pre-crime anytime soon.