Turning that frown upside down with creative coding

18 November 2016

Turning that frown upside down with creative coding

A Victoria University of Wellington researcher is using an internet robot to investigate what it means to be creative in an increasingly computerised world.

Tom White, senior lecturer in Victoria’s School of Design, has createdSmilevector—a bot that examines images of people, then adds or removes smiles to their faces.

“It has examined hundreds of thousands of faces to learn the difference between images, by finding relations and reapplying them,” says Mr White.

“When the computer finds an image it looks to identify if the person is smiling or not. If there isn't a smile, it adds one, but if there is a smile then it takes it away.

“It represents these changes as an animation, which moves parts of the face around, including crinkling and widening the eyes.”

The bot can be used as a form of puppetry, says Mr White.

“These systems are domain independent, meaning you can do it with anything—from manipulating images of faces to shoes to chairs. It's really fun and interesting to work in this space. There are lots of ideas to play around with.”

The creation of the bot was sparked by Mr White’s research into creative intelligence.

“Machine learning and artificial intelligence are starting to have implications for people in creative industries. Some of these implications have to do with the computer's capabilities, like completing mundane tasks so that people can complete higher level tasks,” says Mr White.

“I'm interested in exploring what these systems are capable of doing but also how it changes what we think of as being creative is in the first place. Once you have a system that can automate processes, is that still a creative act? If you can make something a completely push of the button operation, does its meaning change?”

Mr White says people have traditionally used creative tools by giving commands.

“However, I think we're moving toward more of a collaboration with computers—where there's an intelligent system that's making suggestions and helping steer the process.

“A lot will happen in this space in the next five to ten years, and now is the right time to progress. I also hope these techniques influence teaching over the long term as they become more mainstream. It is something that students could work with me on at Victoria University as part of our Master of Design Innovation or our new Master of Fine Arts (Creative Practice).”

The paper Sampling Generative Networks describing this research is available as an arXiv preprint. The research will also be presented as part of the Neural Information Processing Systems conference in Spain and Generative Art conference in Italy in December.