It depends significantly on the model type you are utilizing. In the drawing example they specifically have a layer in-between the encoder and decoder layers, so if you knew the name of that layer you could just ask the tensorflow session to provide you the output of that layer for some given input. If you are utilizing a pre-trained classification model, you could consider the logits layer output, right before the softmax operation for the class probabilities, as a latent vector. The best current latent vector models are Variational Auto-Encoders (shown in the drawing & music presentations) and Generative Adversarial Networks (shown in the cosmology presentation). If you already have a pre-trained version of one of these types of models then you would typically generate a random latent vector, run that vector through the network to see what it produces, and modify the latent vector variables and see how those variable changes correspond to changes in the sample output (like how they showed the transition of facial drawings at the beginning). If you had some prior knowledge about what kinds of images come from what latent vectors (or vice versa) then you could start with a more bounded random vector within some subsection of the latent space (such as the subspace within you get smiling faces).
That's much more exciting than Game of Thrones last season
Mirror/imitation is extremely powerful. Think about when going into empathy in the future on this basis.
Thank you for amazing work and inspiration!
How do you create a latent vector using a pre-trained model? Does Google Cloud Vision API have a solution?
It depends significantly on the model type you are utilizing. In the drawing example they specifically have a layer in-between the encoder and decoder layers, so if you knew the name of that layer you could just ask the tensorflow session to provide you the output of that layer for some given input. If you are utilizing a pre-trained classification model, you could consider the logits layer output, right before the softmax operation for the class probabilities, as a latent vector. The best current latent vector models are Variational Auto-Encoders (shown in the drawing & music presentations) and Generative Adversarial Networks (shown in the cosmology presentation). If you already have a pre-trained version of one of these types of models then you would typically generate a random latent vector, run that vector through the network to see what it produces, and modify the latent vector variables and see how those variable changes correspond to changes in the sample output (like how they showed the transition of facial drawings at the beginning). If you had some prior knowledge about what kinds of images come from what latent vectors (or vice versa) then you could start with a more bounded random vector within some subsection of the latent space (such as the subspace within you get smiling faces).
Are any of the projects open sourced?
Roberto Chavez
Tensor flow is open source machine learning library for data flow programming
Sorry but that doesn't answer my question. Yes? No? I know Tensorflow is open sourced
magenta is open sourced
Magenta: magenta.tensorflow.org/
NSynth Super on GitHub: github.com/googlecreativelab/open-nsynth-super