» Waifu Synthesis
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Waifu Synthesis

credits Kyle McLean

Waifu Synthesis- real time generative anime

Bit of a playful project investigating real-time generation of singing anime characters, a neural mashup if you will.

All of the animation is made in real-time using a StyleGan neural network trained on the Danbooru2018 dataset, a large scale anime image database with 3.33m+ images annotated with 99.7m+ tags.

Lyrics were produced with GPT-2, a large scale language model trained on 40GB of internet text. I used the recently released 345 million parameter version- the full model has 1.5 billion parameters, and has currently not been released due to concerns about malicious use (think fake news).

Music was made in part using models from Magenta, a research project exploring the role of machine learning in the process of creating art and music.

Setup is using vvvv, Python and Ableton Live.
everyoneishappy.cominstagram.com/everyoneishappy/

StyleGan, Danbooru2018, GPT-2 and Magenta were developed by Nvidia,gwern.net/Danbooru2018, OpenAI and Google respectively.

Models
GPT-2 Text
everyoneishappy, Monday, Jul 1st 2019 Digg | Tweet | Delicious 1 comments  
sinao 01/07/2019 - 15:35

Awesome. I’ve been following your project since you created an Instagram post and try to recreate it a bit.

  • 1

anonymous user login

Shoutbox

~19h ago

joreg: Two #vvvv workshops this week in #berlin: Friday: Physical Computing, Saturday: Computer Vision. Tickets: upcoming-full-day-vvvv-gamma-workshops-in-berlin

~1d ago

joreg: @cznickesz also feel free to join our chat for such questions: chat

~1d ago

cznickesz: @joreg: Yeah, I´ll give it a try! Thx

~1d ago

joreg: @cznickesz: we're hoping this week. but depends on feedback. please test your project and let us know if it works.

~2d ago

cznickesz: @joreg: I should stop asking my questions so cryptic ;-) So right away: is final Beta39 hours, days or weeks away?

~2d ago

joreg: @cznickesz: the idea of an RC is exactly to make sure your projects will work with it. your last chance to report problems with b39.

~2d ago

cznickesz: So how "RC" is the "RC3"? I have to update an old project soon and I´m wondering if I should wait and directly update to Beta39

~2d ago

joreg: @takuma, the "one frame" issue was fixed in RC3 for beta39, see: beta39-release-candidate

~2d ago

Takuma: @joreg thank!! Ill try this. Are there a way to send video texture from vvvv to vl? fromimage node in vl somehow only sends 1 frame

~2d ago

cznickesz: @ravasquez thx, so I need to stop avoiding nuget in the end ;-)