Sunday, October 21, 2018

Creating Sculptures from 2D Videos

Animators, athletes, and dancers need to study and understand complex motions. But even when you review video footage, it can be hard to see exactly what's going on with all the moving parts traveling in three dimensions.

MIT’s Computer Science and Artificial Intelligence Laboratory (CSAIL) has developed an algorithm called MoSculpt that creates a detailed 3D model of a broad movement.

The input can be a simple 2D video of a person in motion. The subject doesn't have to be fitted with mo-cap markers and they don't have to be shot in front of blank backgrounds. (Link to YouTube)

The software lets you choose various input and output parameters. You can either rotate the 3D form virtually on the computer or print them out as an actual sculpture. 

The results resemble the 19th century stroboscopic photos and sculptures by √Čtienne-Jules Marey, which I discussed recently in the post on Chronophography.
Read more:
MoSculp: MIT CSAIL uses AI to create 3D printed 'motion sculptures' from 2D videos
Creating 3-D-printed “motion sculptures” from 2-D videos
Previously: Chronophography


Anonymous said...

Reading about Marey got me to thinking of how many things are fascinating to artists vs how short our lives are. How do you decide what to focus your attention on knowing that you could never study everything?

James Gurney said...

Vanessa, You're right that life is short and the world is full of wonders, more than we can ever embrace in a lifetime. So we shine our flashlights in the dark forest. Writing a blog post each morning is a kind of exercise for my mind, a form of clarifying my understanding of something I've just encountered.

Jim Douglas said...

Fascinating! The MoSculpt output offers an intriguing mix of the figurative and the abstract. It resembles a neoclassical sculpture by Antonio Canova walking through an abstract sculpture by Charles O. Perry. However, it doesn't feel like two sculptures carelessly mashed together. Rather, the relationship is intimate and by no means arbitrary. The Yin informs and shapes the Yang and vice versa. Both are equally represented and influential. The only missing piece is the lack of an artistic eye shaping and editing the output. How might a thoughtful artist manipulate the output to express something human?

Lorin Wood said...

I worked on a feature pitch at Disney back in 2007 with David Goyer. The movie eventually got shelved because we were designing these exact visuals. The premise was about characters who could jump between universes (David had just written the movie 'Jumper' which was a very loose version of our film) through a membrane-like dimension that was the embodiment of time. When they pass through (we were trying out some really cool stereo 3D effects work as well on the lot) you could see every possible position in space that they would or are passing through so it creates this semi-transparent, gelatinous mass very similar to these shapes.

Cool find. Thanks for posting this James!