Chinese Student Creates AI That Can Turn Photos into Anime

An undergraduate student from China’s Fudan University showcased a new artificial intelligence (AI) software that can turn regular human photos into an anime masterpiece using the Generative Adversarial Network (GAN) and Deep Learning method.

Yanghua Jin is attempting to create a computer program that can learn from its own mistakes the longer it works. This is all done using GAN’s two networks: the generator and the discriminator, according to SoraNews24.

The generator is in charge of producing the anime picture that runs through the software. It uses attributes taken from anime images, such as hair and eye color, whether the hair is long or short, and whether the mouth is open or not, and studies them.

The network also recognizes “noise,” such as the proportion of the eyes to the rest of the face or the angle at which the figure in the photo is posing. It will then generate an anime version of the photo and try to “fool” the other network.

The discriminator network would compare the images that the generator produced to the images in the library to determine if the photo is either a synthetic or a genuine image.

During a discussion at the Deep Learning workshop in Tokyo, Japan, in March, Jin showed how the generator becomes better at being able to produce an image and the discriminator gradually becomes better at judging whether the image is a success or not.

However, there seem to be some challenges that CycleGAN, a form of the GAN software that’s often successful at applying different textures to images or footage, encountered. Jin explained that the software has problems when it comes to producing moe anime artwork primarily due to it having larger eyes and a smaller mouth and nose.

Through its use, the software becomes more and more successful when producing images after learning from its mistakes, as seen in the slides below.

Jin also showcased CycleGAN’s ability to apply certain texture on videos such as when the software added stripes onto a horse to make it look like a zebra.

The pictures were initially horrible with all the deformed anime drawings, but later on, CycleGAN showed some improvements. Some anime lovers were not exactly thrilled, while others had different priorities in mind.

“The progression is so quick, at first they looked like monsters, but now…”

“If everything becomes computer-made that would be boring, but with this kind of genre it’s all right.”

“So computers can make images. How long until they’re writing and drawing manga?”

“Whoa, those computer-generated moe girls are really cute.”

“How long before the computer gets a taste for it and starts producing hentai?”

With this AI software, it is not hard to imagine a future in which industries won’t require human hands to draw manga — computers will be the ones in charge of creating this form of entertainment.

Featured Image via pxhere

Share this Article