» Waifu Synthesis- real time generative anime
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Waifu Synthesis- real time generative anime

Watch on Vimeo

http://everyoneishappy.com
https://www.instagram.com/everyoneishappy/
Bit of a playful project investigating real-time generation of singing anime characters, a neural mashup if you will.

All of the animation is made in real-time using a StyleGan neural network trained on the Danbooru2018 dataset, a large scale anime image database with 3.33m+ images annotated with 99.7m+ tags.

Lyrics were produced with GPT-2, a large scale language model trained on 40GB of internet text. I used the recently released 345 million parameter version- the full model has 1.5 billion parameters, and has currently not been released due to concerns about malicious use (think fake news).

Music was made in part using models from Magenta, a research project exploring the role of machine learning in the process of creating art and music.

Setup is using vvvv, Python and Ableton Live.

StyleGan, Danbooru2018, GPT-2 and Magenta were developed by Nvidia, gwern.net/Danbooru2018, OpenAI and Google respectively.

Kyle McLean, Wednesday, Aug 7th 2019 Digg | Tweet | Delicious comments on vimeo  
  • 1

anonymous user login

Shoutbox

~24h ago

joreg: Release Candidate 3 for #vvvv beta39 is out: beta39-release-candidate Please test and report your findings!

~1d ago

young47: Will visit Köln and Düsseldorf this week. I'm an artist from Moscow, would be happy to meet someone! If you will be there, reply :)

~1d ago

microdee: vpm got a clean-up update. Download here: https://github.com/vvvvpm/vpm/releases/tag/1.3

~3d ago

joreg: @udo2013 please note that bass is only available with x86 builds!

~3d ago

aeckheim: Can't find the visual c++ runtime 2008. setup.exe link leads to nowhere. try to setup newest beta version

~4d ago

joreg: More #vvvv workshop dates for the coming months are announced: upcoming-full-day-vvvv-gamma-workshops-in-berlin

~6d ago

mburk: @synth can confirm crashes with instance noodles. studio drivers fixed it.

~7d ago

synth: @nissidis: So i am not crazy! Thanks for confirmation :) These are the things i do have problems with as well.

~7d ago

nissidis: @synth I do with Instance Noodles and FieldTrip (RTX 2080) I rolled back to Studio Drivers (instead of GameReady ones)