Generating GAN Images from Minecraft Video with StyleGAN2 ADA
ฝัง
- เผยแพร่เมื่อ 25 ก.ย. 2024
- Frames from a video cam be a great source of material for a GAN. I recorded myself playing Minecraft in a new world and extracted the individual frames with ffmpeg to produce training data for NVIDIA StyleGAN2 ADA. In this video I demonstrate how I did this and provide a pretrained model that you can run from CoLab to generate your own images.
Code:
github.com/jef...
Lenovo ThinkStation:
www.lenovo.com...
NVIDIA Quadro:
www.nvidia.com...
** Follow Me on Social Media!
GitHub: github.com/jef...
Twitter: / jeffheaton
Instagram: / jeffheatondotcom
Discord: / discord
Patreon: / jeffheaton
Fantastic video Jeff I'll definitely try this out myself!
This was awesome Jeff. Love the applied approach you always take and actually showing things working and being executed. Great work and thanks for the code also.
Thanks! Glad you like the approach.
Great video, Hickory snoring is priceless too! 😊
Thanks for the great videos Jeff! Is it possible to render videos with a zoom in transition effect between seeds with this script? If not, what would you recommend to achieve this?
Hi Jeff, love the video, is there a way for a beginner to start doing machine learning without classes? What would you recommend for self learning? Or like tutorial series you recommend for easy intro projects.
So cool
ReLU is a switch🤔 f(x)=x connect, f(x)=0 disconnect. A light switch in your house is binary on off yet it connects and disconnects a continuously variable AC voltage signal. The dot product of a number of dot products is still a dot product. Then when all the switch states in a ReLU net become known it collapses to a simple matrix. Which you can examine with various metrics. For example you can look at one output value in the output vector and see what the associated dot product is looking at in the input vector to support that value.
@Hoa Xuan O'Connor These YT NN algorithms are telling you what to expect from the future. There will be a high amount of random noise in your life as you are allowed and not allowed, admitted and not admitted, given and not given, on a seemingly random basis.
Thank-you Jeff, other interested video! Regards!
will do!
Amazing jeff ...
Thanks, Shivam.
Im trying to replace the URL pointing to the .pkl file in your notebook with a URL pointing to my own .pkl file but I get "UnpicklingError: invalid load key, '\x0a'." Any ideas?
Never trust a person with 22,272 unread emails
I need to delete a couple hundred.
Great vid! btw. would you recommend recording in low framerate (like 1 fps)? you'll get a lot of similar images with eg. 30fps Does the GAN care about having similar Images?
Not Jeff, but the answer depends on what you want the GAN to learn. If you want it to learn a wide range of biomes, then a lower fps over a longer period of time would make for a better dataset. If instead you only want one biome, then a higher fps could be okay.
I did have to discard some of the images because at 30fps capture often two images were the same. However, generally, I was going for max training data, so I kept the FPS as high as I could and discarded duplicates.
@@MaxMitchell Great points, I did do other experimentation where I stuck to a single biome and time of day. Accuracy increased. In general, a GAN trained to generate cats OR dogs, will always be outperformed by a GAN that specializes on cats or dogs.
I wonder if the fixed snowflakes and raindrops are due to them being independent of the rest of the image. Since they more or less could be placed anywhere and still be valid, the generator might consider adjusting them a dead-end because they don't impact the discriminator score.
I like that line of thought, it could be.
Okay, let's try : Fast Transform fixed filter bank neural networks.
Stick or not stick?
fucking epic