Saturday, March 3, 2018

Computers can guess what you're seeing

In previous posts, we've seen how artificial-intelligence systems can generate captions for images, and can even create a photo-real image from a written description.

A deep-learning algorithm can tell what you're looking at by scanning your brain. Photo CNN
Now, scientists in Japan report that the same kind of technology can guess what you're looking at from your brain activity alone. The process uses fMRI brain technology, which scans blood flow in the brain in real time.

After a period of training, the system generates a short natural-language caption based on what it supposes you're looking at. Sometimes the captions are right on. For example when a person was looking at the photo to the left, the computer correctly hypothesized that the image showed: A man is playing tennis on the court with his racket.

When the guess was a little wrong, it was wrong in an interesting way. Once, when the subject being scanned was looking at a man kayaking on a river, it concluded: A man is surfing in the ocean on his surf board.

Right now the technology is limited by practical issues (the subject has to lie down in a big expensive machine). However, it could become part of an efficient brain-computer interface (BCI), which requires close two-way communication between the user and the machine. 

It also portends a variety of creep-factor sci-fi implications. For example, conceivably authority figures would be able to monitor "thought crimes" based on what's going on in your brain.
Scientific article: Describing Semantic Representations of Brain Activity Evoked by Visual Stimuli
Popular article on Futurism: What Are YOU Looking At? Mind-Reading AI Knows
Text to Image Synthesis
Computers are Learning to Caption Photos
Style Transfer by Deep Learning


Celia said...

Big creep factor.

Unknown said...

I used to think that technology was going to free up our time to spend pursuing the Arts and restoring our eco-systems- that we all would be able to truly realize our full potential. I could almost see the future where humanity would pursue new knowledge that would be used in such a way it would reflect on what a noble species we had become. The new world would be a garden and we its' artful caretakers.

I understand this new technology, if used responsibly, could be used to help us better understand the brain and how it works. Unfortunately, tech seems to be focused more on control and manipulation. We use our computers to compile data that enable us to do those very tasks in politics and business. We have enough advances in technology now to truly change this world for the better. So, why haven't we? Is it just business as usual?

Corey said...

It would be fascinating to allow this to record descriptions of your dreams. Imagine being able to know what you dreamed the previous night even if you don’t remember it!
Or if it could even create a video approximating what the dream was like?
Very cool stuff!