Got to be the best pytorch tutorial series on youtube. And I really like the outro of the videos, putting some philosophy and depth as to why we need collective intelligence and to question ourselves why we are doing it. Great work.
Thank you for you guys great work. Look forward the next part. Before this video: 1、I know tensorboard.I used it both tensorflow and pytorch 2、I just monitor the loss and acc in tensorboard 3、I expect to know how to use it and what to watch in tensorboard What I learned: 1、pytorch can easily access the writer. (Before this , must use third-party package) 2、How easy the api is (add). Add_scalar,add_image,add_historgram 3、add_what_ever_you_want(‘name’,data,epoch) will help me track the data. I don’t know why tensorflow isn’t use the same logic…. 4、dot will check all the image 5、Histgram timeline from back to front see the frequency.I didn’t get this part before. Now I got it.Maybe we should attend max and min. Don’t get very clear part: 1、add_image . The grid part . Why we use grid . Grid for what. If we have something else data.What will change
At the video moment about 9:00, when coming to Tensorboard, I can see the images but when switching to the graph section, I cant see any graph? Thank you. :)
If you only see 2 blank rectangles on TensorBoard, you need to uninstall pytorch and reinstall is as nightly version The problem can be seen from: github.com/pytorch/pytorch/issues/24157
get_num_correct returns undefined, which makes sense as we haven't defined it and it looks like a function ... did I miss anything ? did we define it anywhere ?
Running all Tensorboard commands in Anaconda, I get the "TensorFlow installation not found - running with reduced feature set" error. If you get this error, do not panic. If you do not have TensorFlow installed (like me), this is expected. Everything runs fine with PyTorch.
Hi, deeplizar, I had exactly the same code as yours, but my "scalar" page shows only one single point instead of a curve. Also, my distribution plot shows nothing. Your histogram looks like 3d, mine looks like 2d. Any idea what is causing the problem? thanks
Hey Matthew - I like the question. A couple of things that come to mind: 1) Detecting vanishing and exploding gradients. Reference: deeplizard.com/learn/video/qO_NLVjD6zE 2) Seeing the impact of regularization. Reference: deeplizard.com/learn/video/iuJgyiS7BKM 3) Seeing the impact of dropout. Reference: deeplizard.com/learn/video/DEMmkFC6IGM Another thing to throw into the mix here is the layer activations. I'll refer you to this part of a fast.ai lecture: Analyzing CNN activations: th-cam.com/video/HR0lt1hlR6U/w-d-xo.html Essentially, we want our layer outputs to have a mean of 0 and standard deviation of 1. This has been shown to produce good results. Paper here: arxiv.org/abs/1511.06422 I hope this helps! Chris
@@deeplizard what a great response. thank you! I can see how a histogram of gradients can help find vanishing/exploding gradients. However, question #1 I'm not sure what these histograms would look like if I was overfitting, and needing regularization/dropout. Also, for question #2 we're printing out the weights and not the outputted values. is it possible from the weights or bias to tell if we have a layer output mean of 0 and std or 1?
#1 I'm not aware of a signal that would allow this to be detected 🤔 viewing the histogram before and after regularization/dropout should allow the effect to be visualized. #2 No. You'd need to get the values directly. This can be done inside the network's forward method (not preferred) or using PyTorch hooks, which allow us to hook into the forward method using a callback. Hooks have not been covered in the course. However, hooks are on the list as potential topics to be added. 😃
hiiii i need help .. When i run this command in anaconda powershell prompt "tensorboard --logdir=logs" It gives me this output: c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint8 = np.dtype([("qint8", np.int8, 1)]) c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint8 = np.dtype([("quint8", np.uint8, 1)]) c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint16 = np.dtype([("qint16", np.int16, 1)]) c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_quint16 = np.dtype([("quint16", np.uint16, 1)]) c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. _np_qint32 = np.dtype([("qint32", np.int32, 1)]) c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'. np_resource = np.dtype([("resource", np.ubyte, 1)]) Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all TensorBoard 2.2.1 at localhost:6006/ (Press CTRL+C to quit) And when i try to see see the graphs and images like in the video but I'm not able to :( Can someone help me?
Hi Deeplizard. Great fan of your videos. Probably the best tutorial for deep learning using pytorch. My issue is after pip installing tensorflow in my windows machine ( I have Anaconda Python 3.8 btw) , my powershell is not recognizing tensorboard command. But I do have tensorboard which I checked in Jupyter notebook. Kindly help me how I can replicate what you have shown in this video.
I got "ModuleNotFoundError " when I tried to run this code( torch.utils.tensorboard import SummaryWriter). I have a torch version of 1.0.1 and tensorboard version of 2.0.0. Please help me to get rid of this error.
Hi, iam having an error with the accuracy scaler: 32 tb.add_scalar("loss", loss, epoch) ---> 33 tb.add_scalar("Accuracy", accuracy(preds, labels)) 34 35 tb.add_histogram('conv1.bias', network.conv1.bias, epoch) ~\Anaconda3\lib\site-packages\torch\utils\tensorboard\writer.py in add_scalar(self, tag, scalar_value, global_step, walltime) 261 """ 262 if self._check_caffe2_blob(scalar_value): --> 263 scalar_value = workspace.FetchBlob(scalar_value) 264 self._get_file_writer().add_summary( 265 scalar(tag, scalar_value), global_step, walltime) NameError: name 'workspace' is not defined
Hello dear all, Thanks for your great idea of Collective Intelligence. When I try tensorboard --logdir=runs, I get the localhost:6006 but when I want to open it in the browser it says "localhost refused to connect." what you think is causing this problem? Thanks in advance
Hey Hamed - You are welcome! This is a pretty general error. You may find various ideas by googling for the error. This one has a few possible solutions: stackoverflow.com/questions/40106949/unable-to-open-tensorboard-in-browser
@@deeplizard I ran into the same thing. For some reason the dependency doesnt get handled by their yaml. Here's the stack trace running pytorch 1.2.0 and torchvision 0.4.0a0+6b959ee. Installing future fixed it --------------------------------------------------------------------------- ModuleNotFoundError Traceback (most recent call last) in ----> 1 import torch.utils.tensorboard /opt/anaconda3/envs/dev/lib/python3.7/site-packages/torch/utils/tensorboard/__init__.py in 4 raise ImportError('TensorBoard logging requires TensorBoard with Python summary writer installed. ' 5 'This should be available in 1.14 or above.') ----> 6 from .writer import FileWriter, SummaryWriter # noqa F401 /opt/anaconda3/envs/dev/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py in 16 from tensorboard.summary.writer.event_file_writer import EventFileWriter 17 ---> 18 from ._convert_np import make_np 19 from ._embedding import make_mat, make_sprite, make_tsv, append_pbtxt 20 from ._onnx_graph import load_onnx_graph /opt/anaconda3/envs/dev/lib/python3.7/site-packages/torch/utils/tensorboard/_convert_np.py in 10 import six 11 ---> 12 from caffe2.python import workspace 13 14 /opt/anaconda3/envs/dev/lib/python3.7/site-packages/caffe2/python/workspace.py in 13 import logging 14 import numpy as np ---> 15 from past.builtins import basestring 16 import shutil 17 import socket
I couldn't get the tf-nightly or tensorboard to work with pytorch in a conda environment but there is another way of doing this using tensorboardX. Install the following on your local instance of python. pip install tensorboardX from tensorboardX import SummaryWriter tb = SummaryWriter(cwd+'/run/') # continue like normal ... tensorboardX will not run tensorboard web app on its own. For this install tensorboard to your default instance of python in a cmd. pip install tensorboard tensorboard --logdir=
Hey Barry - Thank you for providing these details. One small note is that I installed tb-nightly (not tf-nightly) to get it working: pip install tb-nightly
Let me know if you get the up and running!
👉 Check out the blog post and other resources for this video:
🔗 deeplizard.com/learn/video/pSexXMdruFM
Got to be the best pytorch tutorial series on youtube. And I really like the outro of the videos, putting some philosophy and depth as to why we need collective intelligence and to question ourselves why we are doing it. Great work.
Hey Subhankar - Thank you! You are one of the first people to comment on the endings. Awesome! 🧠
Why only 1.5k views? All of your videos deserve million views and millions likes.
Hey Edward - These videos aren't too popular at the moment. Thank you for your encouragement!
maybe if he shared the code he would have more views.
Amazing content. So much quality and depth in one single series...
Thank you for you guys great work. Look forward the next part.
Before this video:
1、I know tensorboard.I used it both tensorflow and pytorch
2、I just monitor the loss and acc in tensorboard
3、I expect to know how to use it and what to watch in tensorboard
What I learned:
1、pytorch can easily access the writer. (Before this , must use third-party package)
2、How easy the api is (add). Add_scalar,add_image,add_historgram
3、add_what_ever_you_want(‘name’,data,epoch) will help me track the data. I don’t know why tensorflow isn’t use the same logic….
4、dot will check all the image
5、Histgram timeline from back to front see the frequency.I didn’t get this part before. Now I got it.Maybe we should attend max and min.
Don’t get very clear part:
1、add_image . The grid part . Why we use grid . Grid for what. If we have something else data.What will change
The grid is a grid of images from the batch. It's just a way to quickly see a batch of images in a small grid to verify the look as expected.
Great content, looking forward to seeing more!
This is insane. Wasn't expecting this video to be this good! thank you!
Thanks!
How we can add validation accuracy as well ?
Amazing video
Was waiting for it
If possible can u make videos on RNNs and their applications like character generation, sentiment analysis
Thanks Sarthak! Will consider.
Hi could you point me in the direction of info regarding how to visualize boxes and labels in training detection in tensorboard with pytorch please?
At the video moment about 9:00, when coming to Tensorboard, I can see the images but when switching to the graph section, I cant see any graph? Thank you. :)
If you only see 2 blank rectangles on TensorBoard, you need to uninstall pytorch and reinstall is as nightly version
The problem can be seen from: github.com/pytorch/pytorch/issues/24157
get_num_correct returns undefined, which makes sense as we haven't defined it and it looks like a function ... did I miss anything ? did we define it anywhere ?
Hey Chen - This video is part of a series. The function was defined earlier. Check here: deeplizard.com/learn/video/p1xZ2yWU1eo
keep it going 😍
Running all Tensorboard commands in Anaconda, I get the "TensorFlow installation not found - running with reduced feature set" error. If you get this error, do not panic. If you do not have TensorFlow installed (like me), this is expected. Everything runs fine with PyTorch.
Hi, deeplizar, I had exactly the same code as yours, but my "scalar" page shows only one single point instead of a curve. Also, my distribution plot shows nothing. Your histogram looks like 3d, mine looks like 2d.
Any idea what is causing the problem? thanks
Can you explain how you would use the histogram data with the weights, bias, gradients? I can't think of how that would be actionable.
Hey Matthew - I like the question.
A couple of things that come to mind:
1) Detecting vanishing and exploding gradients. Reference: deeplizard.com/learn/video/qO_NLVjD6zE
2) Seeing the impact of regularization. Reference: deeplizard.com/learn/video/iuJgyiS7BKM
3) Seeing the impact of dropout. Reference: deeplizard.com/learn/video/DEMmkFC6IGM
Another thing to throw into the mix here is the layer activations. I'll refer you to this part of a fast.ai lecture:
Analyzing CNN activations: th-cam.com/video/HR0lt1hlR6U/w-d-xo.html
Essentially, we want our layer outputs to have a mean of 0 and standard deviation of 1. This has been shown to produce good results. Paper here: arxiv.org/abs/1511.06422
I hope this helps!
Chris
@@deeplizard what a great response. thank you! I can see how a histogram of gradients can help find vanishing/exploding gradients. However, question #1 I'm not sure what these histograms would look like if I was overfitting, and needing regularization/dropout. Also, for question #2 we're printing out the weights and not the outputted values. is it possible from the weights or bias to tell if we have a layer output mean of 0 and std or 1?
#1 I'm not aware of a signal that would allow this to be detected 🤔 viewing the histogram before and after regularization/dropout should allow the effect to be visualized.
#2 No. You'd need to get the values directly. This can be done inside the network's forward method (not preferred) or using PyTorch hooks, which allow us to hook into the forward method using a callback. Hooks have not been covered in the course. However, hooks are on the list as potential topics to be added. 😃
@@deeplizard Thank you.
@deeplizard there is a typo in the quiz (question 3). The answer is front-end not font end
Thank you for your help reporting that! We fixed it! 🦎 ❤️
@@deeplizard You rock!
hiiii i need help ..
When i run this command in anaconda powershell prompt "tensorboard --logdir=logs"
It gives me this output:
c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:516: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint8 = np.dtype([("qint8", np.int8, 1)])
c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:517: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint8 = np.dtype([("quint8", np.uint8, 1)])
c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:518: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint16 = np.dtype([("qint16", np.int16, 1)])
c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:519: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_quint16 = np.dtype([("quint16", np.uint16, 1)])
c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:520: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
_np_qint32 = np.dtype([("qint32", np.int32, 1)])
c:\users\utilizador\anaconda3\lib\site-packages\tensorflow\python\framework\dtypes.py:525: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.
np_resource = np.dtype([("resource", np.ubyte, 1)])
Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
TensorBoard 2.2.1 at localhost:6006/ (Press CTRL+C to quit)
And when i try to see see the graphs and images like in the video but I'm not able to :(
Can someone help me?
Hi Deeplizard. Great fan of your videos. Probably the best tutorial for deep learning using pytorch. My issue is after pip installing tensorflow in my windows machine ( I have Anaconda Python 3.8 btw) , my powershell is not recognizing tensorboard command. But I do have tensorboard which I checked in Jupyter notebook. Kindly help me how I can replicate what you have shown in this video.
I got "ModuleNotFoundError " when I tried to run this code( torch.utils.tensorboard import SummaryWriter). I have a torch version of 1.0.1 and tensorboard version of 2.0.0. Please help me to get rid of this error.
Hey Sriharsha Buddi - Your torch version needs to be updated. Version 1.1.0 is required. Reference here: deeplizard.com/learn/video/pSexXMdruFM
@@deeplizard thank you so much
Great Video as always !
One question tho, How would you go about doing tensor boards if you're using a Fastai learner ?
You'd need to create a callback. Have a look here: forums.fast.ai/t/tensorboard-callback-for-fastai/19048/2
Hi,
iam having an error with the accuracy scaler:
32 tb.add_scalar("loss", loss, epoch)
---> 33 tb.add_scalar("Accuracy", accuracy(preds, labels))
34
35 tb.add_histogram('conv1.bias', network.conv1.bias, epoch)
~\Anaconda3\lib\site-packages\torch\utils\tensorboard\writer.py in add_scalar(self, tag, scalar_value, global_step, walltime)
261 """
262 if self._check_caffe2_blob(scalar_value):
--> 263 scalar_value = workspace.FetchBlob(scalar_value)
264 self._get_file_writer().add_summary(
265 scalar(tag, scalar_value), global_step, walltime)
NameError: name 'workspace' is not defined
I haven't seen that one. You'll probably want to check on Google. Let us know if you find a solution.
Hello dear all,
Thanks for your great idea of Collective Intelligence.
When I try tensorboard --logdir=runs, I get the localhost:6006 but when I want to open it in the browser it says "localhost refused to connect." what you think is causing this problem?
Thanks in advance
Hey Hamed - You are welcome! This is a pretty general error. You may find various ideas by googling for the error.
This one has a few possible solutions: stackoverflow.com/questions/40106949/unable-to-open-tensorboard-in-browser
thanks!
how to run on gpu?
For anyone having issues with getting tensorboard working, make sure you pip install future. Only installing tb 1.15 and pytorch1.1.0 will call issues
Hey Pablo - Thanks for the intel. What is future and what issue did it help you solve? Thanks for any additional details.
@@deeplizard I ran into the same thing. For some reason the dependency doesnt get handled by their yaml. Here's the stack trace running pytorch 1.2.0 and torchvision 0.4.0a0+6b959ee. Installing future fixed it
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
in
----> 1 import torch.utils.tensorboard
/opt/anaconda3/envs/dev/lib/python3.7/site-packages/torch/utils/tensorboard/__init__.py in
4 raise ImportError('TensorBoard logging requires TensorBoard with Python summary writer installed. '
5 'This should be available in 1.14 or above.')
----> 6 from .writer import FileWriter, SummaryWriter # noqa F401
/opt/anaconda3/envs/dev/lib/python3.7/site-packages/torch/utils/tensorboard/writer.py in
16 from tensorboard.summary.writer.event_file_writer import EventFileWriter
17
---> 18 from ._convert_np import make_np
19 from ._embedding import make_mat, make_sprite, make_tsv, append_pbtxt
20 from ._onnx_graph import load_onnx_graph
/opt/anaconda3/envs/dev/lib/python3.7/site-packages/torch/utils/tensorboard/_convert_np.py in
10 import six
11
---> 12 from caffe2.python import workspace
13
14
/opt/anaconda3/envs/dev/lib/python3.7/site-packages/caffe2/python/workspace.py in
13 import logging
14 import numpy as np
---> 15 from past.builtins import basestring
16 import shutil
17 import socket
I am so bummed that the "next tensorboard video" isn't on youtube :(
What "next tensorboard video"?
I like that women robot voice, made me feel like a cyberpunk who's about to conquer the world using my lstm model, ALL SHALL BOW BEFORE ME🙂
😆🤖
Please make a video on pytorch based gan model deployment on mobile Android/iPhone.much needed.
I couldn't get the tf-nightly or tensorboard to work with pytorch in a conda environment but there is another way of doing this using tensorboardX.
Install the following on your local instance of python.
pip install tensorboardX
from tensorboardX import SummaryWriter
tb = SummaryWriter(cwd+'/run/') # continue like normal
...
tensorboardX will not run tensorboard web app on its own. For this install tensorboard to your default instance of python in a cmd.
pip install tensorboard
tensorboard --logdir=
Hey Barry - Thank you for providing these details. One small note is that I installed tb-nightly (not tf-nightly) to get it working:
pip install tb-nightly