Pytorch Geometric tutorial: Graph attention networks (GAT) implementation
ฝัง
- เผยแพร่เมื่อ 5 มี.ค. 2021
- In this video we will see the math behind GAT and a simple implementation in Pytorch geometric.
Outcome:
- Recap
- Introduction
- GAT
- Message Passing pytroch layer
- Simple GCNlayer implementation
- GAT implementation
- GAT Usage
Download the material from the official web site:
antoniolonga.github.io/Pytorc... - วิทยาศาสตร์และเทคโนโลยี
Thanks for putting together these PyTorch Geometric tutorials, really valuable material 👏
Great series so far. The explanations are extremely clean and balanced.
I did encounter several TH-cam videos explaining the paper, but I am glad I bumped into your video with the actual implementation code explained so elegantly. Thanks for taking the time to share it.
guy was bored out of his mind 15:13 deep breath LOL. Thank you though I appreciate this work helps me.
That was so nicely explained, thank you ❣
Very nice tutorial!
Great Tutorial！
Thanks for these nice tuotials!
Thanks :)
Great Tutorial
Great Tutorial. Thanks Antonio. One quick question.Could you please explain how the normalization step is working in your code? it's the only ambiguous part for me. I would really appreciate your answers.
Hi it is really nice tutorial, but can I know specifically about masked attention?
I think it is related to adjacency matrix, if a position of an adjacency matrix is 1 then it goes to e(what we calculated)
if not, very little number(-9e+15).... Why we should use it?
In pytorch geometric, when should I use a tuple as the in_channel? what exactly is the source and destination dimension in in_channels that the pytorch documents refer to?
good video，know many details about how to training GAT thk u
I have a question! Does GAT class provide an argument or function to turn skip-connection on for PPI dataset?
23:49 I don't get the normalization step, I checked the "degree" function that computes the degree of the edge_index and btw on the line "row, cow = edge_degree" we cannot get the row or the column, because this is the COO representation of the graph. Another thing that is strange is that after (somehow) you get deg_inv_sqrt, how it works for both source and target after you specify the [row] and [col] index in deg_inv_sqrt.??? Can you please explain the step 3 in more detailed way?
@ 21:57, may be the 'int' method should be 'init'? Since the initialization should be defined at the __init__ function.
@Antonio Longa Have you checked out the DropEdge mechanism to make the GNN's go deep? If yes, could you please make a tutorial or explain the mechanism behind it.
Thanks, for the suggestion. We will consider it :)
These are really great! Is there any way that the typing sound could be removed in upcoming videos, though? I find it a little distracting.
Thanks so much for these videos! :)
Hello Jacan,
I'm really happy you like our project.
Unfortunately, we were at the beginning, and we did some mistakes in recording videos.
However, if you proceed with the tutorials, they became a little bit better (in terms of audio)
:)
@11:26 , could you please explain what's "a", how did we get the 2Fx1? Is it randomly initialised weight vector?
Yes, "a" is randomly initialised, then its weight is learned (in the backpropagation process)
Hi, Great video. However everybody talks about cora dataset, wherein nodes and edges can be easily defined.
My query is how can we perform node classification for data comprising of ECG signal,or tabular data with M observations and N features or say Images. How to begin with construction of graph for such datasets , defining nodes, then construction of graph's?
Hello,
I think it is a really good idea!
We will consider to do a talk regarding what you are talking about!
Thanks
@@94longa2112 looking forward to hearing from you. Thanks ✌️🙏
@@94longa2112 Yes please do make a video or at least provide some help on how to convert tabular data into graph format. Thank you so much for providing such a high value contents.
Hi hope you're doing well
Is there any graph neural network architecture that receives multivariate dataset instead of graph-structured data as an input?
I'll be very thankful if you answer me i really nead it
Thanks in advanced
The idea of GNNs is to receive Graphs as inputs. If you will not use graphs as input, you do not need GNNs.
Hi Antonio, I have an error: cannot import name 'container_abcs' from 'torch._six' when I run " USE IT" in "tutorial3" code. Do you know how to solve it?
I'll load a codelab so that you can execute notebooks directly online
Hello, thank you for your speech
I'm new to this. I have a question
What is the purpose of this method?
Hi, the basic idea is to apply attention mecanism in the framework of GNN. It is a consolidated technique in the usual CNN, and this is a work that do the attention mechanism to GNN
34:53 I think it's a bit difficult to understand why you use -9e15 as mask value, if you don't explain that the attention matrix has to be softmaxed, so a such BIG number will be squished to zero. Don't you think it's better to mask the attention matrix AFTER the softmax, with the simple element-wise multiplication with adj?
Anyway, very interesting presentation!
-9e15 is a small number. -9000000000000000 (if I counted correctly 15 zeroes). That is different from 9e-15 that is very close to zero 0.000000000000009 (also 15 zeros) and still different than 9e15 9000000000000000 (15 zeros too) that is, in this case, a very large number. The idea of -9e15 is to express something like -infinity. The idea here is, since the number is very negative, when applying the softmax the probabilities will crunch to 0, therefore "ignoring" that fields. Masking after will make the softmax compute them in the normalization and the event probabilities after will not sum to 1. That is not what we want when talking about probabilities. In fact I think it should call `minus_infinity_vec`, or `minfty_vec` to short.
How can I apply the GNN to a custom dataset? Can you give me any link for that?
Hello @sudiptapaul2825 I need this too. Did you find it?
@@onewhoflutters4866 I too need this how do we go about doing this ?
@@kaviyarasanpr Hello, Incorporating Edge Features into Graph Neural Networks for Country GDP Predictions (When I add the link it is deleted. a work in medium) I used here. data from world bank
next time ask the person who presses the buttons to leave - it makes it very difficult to listen
-9e15 is a very large negative number
There is lot of background noise, which is really distracting.
this is a terrible methodology and a more terrible explanation