Chinese Student Creates AI That Can Turn Photos into Anime

Yanghua Jin is attempting to create a computer program that can learn from its own mistakes the longer it works. This is all done using GAN’s two networks: the generator and the discriminator, according to SoraNews24.

The generator is in charge of producing the anime picture that runs through the software. It uses attributes taken from anime images, such as hair and eye color, whether the hair is long or short, and whether the mouth is open or not, and studies them.

The network also recognizes “noise,” such as the proportion of the eyes to the rest of the face or the angle at which the figure in the photo is posing. It will then generate an anime version of the photo and try to “fool” the other network.

The discriminator network would compare the images that the generator produced to the images in the library to determine if the photo is either a synthetic or a genuine image.

During a discussion at the Deep Learning workshop in Tokyo, Japan, in March, Jin showed how the generator becomes better at being able to produce an image and the discriminator gradually becomes better at judging whether the image is a success or not.

However, there seem to be some challenges that CycleGAN, a form of the GAN software that’s often successful at applying different textures to images or footage, encountered. Jin explained that the software has problems when it comes to producing moe anime artwork primarily due to it having larger eyes and a smaller mouth and nose.

Through its use, the software becomes more and more successful when producing images after learning from its mistakes, as seen in the slides below.

Jin also showcased CycleGAN’s ability to apply certain texture on videos such as when the software added stripes onto a horse to make it look like a zebra.

The pictures were initially horrible with all the deformed anime drawings, but later on, CycleGAN showed some improvements. Some anime lovers were not exactly thrilled, while others had different priorities in mind.

With this AI software, it is not hard to imagine a future in which industries won’t require human hands to draw manga — computers will be the ones in charge of creating this form of entertainment.

Featured Image via pxhere