» Waifu Synthesis- real time generative anime
This site relies heavily on Javascript. You should enable it if you want the full experience. Learn more.

Waifu Synthesis- real time generative anime

Watch on Vimeo

http://everyoneishappy.com
https://www.instagram.com/everyoneishappy/
Bit of a playful project investigating real-time generation of singing anime characters, a neural mashup if you will.

All of the animation is made in real-time using a StyleGan neural network trained on the Danbooru2018 dataset, a large scale anime image database with 3.33m+ images annotated with 99.7m+ tags.

Lyrics were produced with GPT-2, a large scale language model trained on 40GB of internet text. I used the recently released 345 million parameter version- the full model has 1.5 billion parameters, and has currently not been released due to concerns about malicious use (think fake news).

Music was made in part using models from Magenta, a research project exploring the role of machine learning in the process of creating art and music.

Setup is using vvvv, Python and Ableton Live.

StyleGan, Danbooru2018, GPT-2 and Magenta were developed by Nvidia, gwern.net/Danbooru2018, OpenAI and Google respectively.

Kyle McLean, Wednesday, Aug 7th 2019 Digg | Tweet | Delicious comments on vimeo  
  • 1

anonymous user login

Shoutbox

~6d ago

domj: Midweek Patch Therapy later today at 17:30 ⌚ Come discuss sequencers, the VL language and your patches. https://discourse.vvvv.org/t/come-join-midweek-patch-therapy/18430/17?u=domj

~7d ago

joreg: Looking for support for your #AI project? Try this: https://www.link-niedersachsen.de/link-masters

~14d ago

blausand: Ill be in #Warszawa next weekend. Any #vvvv people around to meet?

~16d ago

sebescudie: Last call, c'est ce soir : webinaire en français pour débutants sur #vvvv gamma! https://thenodeinstitute.org/event/introduction-au-creative-coding-avec-vvvv-gamma-francais/

~16d ago

domj: Ultra high definition volumetric visualization using GPUDirect storage https://youtu.be/GAZP1NcdWMo

~19d ago

CeeYaa: thx for V4-Meetup - some days ago 40th_RetuneStudioVisit from old friends RefikAnadol and Quadrature https://vimeo.com/429510091

~21d ago

david: 4. worldwide vvvv meetup will be here...https://www.youtube.com/watch?v=i0zd68tDUVE

~24d ago

mediadog: @metrowave Magnetic repulsion has always been pure magic to me, proof we only sense a small part of the universe - thanks!