วีดีโอ

Great clarification question! Thank you for asking. Yes. Behaviors spanned multiple frames. Therefore, each frame that showed the mouse doing the behavior-of-interest was pulled into a separate labeled folder. For example, frames showing rearing behaviors were copy-pasted into a folder called "rears."

@@NeuroGuides Thanks for clarifying! I have a few more questions: i) What should I enter for "sample rate of your annotation files". If for one-hot encoding I split a 5 minute video to split into 1000 frames (each frame 300/1000 = 0.33), it would be 0.33(1/0.33)? ii) Can I upload multiple DLC pose estimation .csv files and annotation files corresponding to their videos to get better result? iii) In predict tab, is there an option to export the output? Where does the 'Create labeled videos' being saved, in the same output directory?

@@atanu_giri i). Sample Rate of Annotation Files: If you split a 5-minute video into 1000 frames, each frame would be 0.33 seconds apart (since 300 seconds / 1000 frames = 0.33 seconds per frame). For one-hot encoding, you should, therefore, enter 0.33 as the sample rate. ii). Yes, you can indeed upload multiple DLC pose estimation .csv files and their corresponding annotation files. This can help improve the model's performance by providing more comprehensive training data. iii). There should be an option to export the output in the predict tab. The 'Create labeled videos' option will save the labeled videos in the same output directory where your input files are located.

@@NeuroGuides Thank you!

Hi @ZhudiTang, I was planning to make the second part, but my Matlab license expired. I will make the part 2 video once I am able to renew my license. Meanwhile, if you are looking for anything specific about the BehaviorDEPOT, you can reach out via GitHub and I can try to help you more.

@schaefermuellermusic, Hi, Sorry for the late reply. Please tag me in you comments so youtube will notify me. The error message suggests that you are missing a column in a MultiIndex DataFrame. Are you running a multianimal model?

Hey @@schaefermuellermusic, please reach out to me via Github if you can. I would be happy to help if I can, and will be better able to track the issues you are facing with the code. If you can, please open a new discussion in the Github discussions section (github.com/farhanaugustine/DeepLabCut-Analysis-Jupyter-Scripts/discussions). Please describe in detail at which step in your analysis are you getting stuck and if possible share as much detail as possible so I can understand where the problem might be. The jupyter script should work for novel object test.

Thank you so much for letting me know this. It really means a lot. I’m glad this video helped you with your analysis. I have a few more videos planned around 3D mitochondrial morphology analysis. Stay tuned for those in the future.

Hi @samyou6552, the frame drop issue is with the "DetectPose" node. Bonsai-DLC community is aware of this issue, and DLC seems to be the bottle neck. Here is what I do, which seems to work on my hardware. It may or may not work for you, but it is worth trying. I use scale factor in Bonsai interface (see time stamp 2:55) and downsample my videos. I personally downsample my videos from 60fps (1080p) to 20fps (720p and some even to 360p for long videos) so that my computer can run DLC without bottlenecking frames. Downsampling my videos tends to halt the frame loss in my analysis. I also scale the incoming frames for Bonsai-rx. In default settings of your DLC pose_config.yaml, you can see the lowest and highest global scale and image input size that your model can use for inferences. The default is 0.5 to 1.25 for input size and 1 for global scale, meaning that you can scale your input images down to 0.5 times the input image size for faster prediction with minor-to-mild decrease in precision. I know is might not be the answer you were hoping for, but it's what works for me. I will let you know if I can figure out a better work around in the future. This issue is known in the Bonsai-DLC community. Unfortunately, there is no real fix to this issue for now and DLC DetectionPose seems to be the bottlenesk. Addressing your last question: lol, please don't dragging the progress bar manually, it will cause a massive loss of frames. I did it in my video because I wanted to finish the video analysis quickly. 😅

Thanks！😊😊I am also wondering that if bonsai can use the DLC nodes to analyze the 3d model or multiple animals model? Have you ever tried? If so, i look forward to see more tutorial videos 😊

Thank you for your question! I will look into it. I have tried using ma_dlc models with Bonsai-rx before (in 2022), but at that time, the results were not satisfactory. The key points generated by the models often didn’t make sense. Officially, ma_dlc models can work in Bonsai-rx multi-animal tracking. However, with the release of DLC 3.0+, which supports Pytorch, I anticipate that integrating these models with Bonsai-rx will need to be reworked entirely. Given these considerations, I wouldn’t recommend using ma_dlc models in Bonsai-rx at the moment unless it’s vital. Once I have some time, I can revisit this and see if there have been improvements.

@@NeuroGuides Thank you very much for your patient and detailed answer. I appreciate it.❤️

@@NeuroGuides Hello, I apologize for bothering you again. I would like to ask you more about the issue of implementing multi-animal tracking in Bonsai. I'm using the bonsai.sleap module for multi-animal tracking and can track multiple animal body parts in real time, but I'm encountering some issues when exporting the data for each mouse ID. I was wondering if you have looked into the application of the SLEAP open-source package in Bonsai's real-time prediction models? If you have some time, could you please try it out and perhaps create a tutorial video on it? I would greatly appreciate it!

😄 glad it helped!

Hi @cesarcaballero4780, I'm just following up to see if you were successful in applying Yolo towards your behavior analysis. I hope your behavior analysis is going well.

@@NeuroGuides Thank you so much for your interest. Unfortunately, my lab and I were in the final part of the semester and needed to train a new generation of bachelor students to do lab work during the summer holidays. Now, we are on holiday, and I hope to apply YOLO. It’s a bit difficult for me because I have learned with DLC. If I have an issue, I will write to this channel.

I personally use Yolo for open field tests. I also find Yolo to be easier and faster to train. In addition, Yolo workflow is easier to expand upon using Colab and Python.

Yes, I meant analyze the videos. Thank you for brining it to my attention.

You should you be able to convert the DLC export model. However, you should try the DLC-live with Autopilot first. You can always reach out to DLC-Live team on their GitHub page. There are various ways to get the DLC to work with Arduino and Raspbarry Pi controllers. Kane et al., 2020 has a nice paper highlighting real-time closed-loop feedback for markerless posture tracking. Autopilot integrates DLC-live and you can use your pretrained models via Ras Pi. Autopilot Website: docs.auto-pi-lot.com/en/latest/guide/quickstart.html DLC-Live SDK: github.com/DeepLabCut/DeepLabCut-live DLC-Live team's Github page: github.com/DeepLabCut/DeepLabCut-live/issues/50

Thanks for the reply! I was able to get all the libraries installed on a raspberry pi 4 and when running inference using a normal model export I was getting less than 1 fpsand 100% cpu usage lol. I ordered a jetson nano which is a bit beefier. I will try again on that device@@NeuroGuides

@@thegtlab I would love to know if you jetson nano works for you. Are you using Mobilenet or Resnet model?

@cesarcaballero4780 I am glad that you find these videos useful, and I appreciate your question. A lot of people use Google Colab, including myself. I will try to incorporate it into some of my other videos.

Hi @sheshendra7, I will need more information about any errors you are receiving. If you are not receiving any errors and the code is executing without issues, then I suspect that your paths in pose_config or config.yaml file have been altered in some way. I have recently uploaded a video where I walk you through how to initiate a DLC project, extract and label frames, train your model, and export your model for use with Bonsai. Please use the information provided in that video to help you out. th-cam.com/video/pTFPo14dIYQ/w-d-xo.html

I use model export function. It is part of the Helperfunctions.md and simple to execute. Are you using Colab or using local anaconda DLC installation to extract your trained model? You can execute the following command from terminal (either from anaconda or Docker) using the following command. deeplabcut.export_model(cfg_path) You can find more information about it here: github.com/DeepLabCut/DeepLabCut/blob/main/docs/HelperFunctions.md

When i was executing the above command,This is the error" deeplabcut.export_model("C:/Users/jdeora/Desktop/DLCdlc-models/iteration-0/DLCMar6-trainset95shuffle1/train/pose_cfg.yaml") Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\ProgramData\anaconda3\envs\DEEPLABCUT\lib\site-packages\deeplabcut\pose_estimation_tensorflow\export.py", line 312, in export_model sess, input, output, dlc_cfg = load_model( File "C:\ProgramData\anaconda3\envs\DEEPLABCUT\lib\site-packages\deeplabcut\pose_estimation_tensorflow\export.py", line 113, in load_model train_fraction = cfg["TrainingFraction"][trainingsetindex] TypeError: 'NoneType' object is not subscriptable" , i was encountered in terminal and jupyter Notebook. @@NeuroGuides ,kindly help me with this.

@@sheshendra7 The error message "TypeError: 'NoneType' object is not subscriptable" is raised when you try to access an item from a None object. In your case, it seems like the cfg["TrainingFraction"] is None. This could be due to several reasons: 1. The configuration file 'pose_cfg.yaml' might not be correctly formatted or might be missing some key-value pairs. Or could have become corrupted. Please check the file and ensure it contains the Training Fraction. 2. The 'pose_cfg.yaml' file might not be correctly loaded, resulting in cfg being None. You could can open the pose config file and see if the file path is correct and accessible. Also, according to GitHub issues related to DeepLabCut, this type of error can occur if an image defined in your training dataset is not found at its original location (most likely because it got deleted, or the “project_path” in your config.yaml file is incorrect). I had this happen to me once when I downloaded the project from my Google Drive after training my model in Colab. I was able to fix the issue after correcting project_path in config.yaml file and paths defined in the pose config file. This issue can also sometimes come up when deeplabcut fails to update project paths when additional training frames are added. In your case: train_fraction = cfg["TrainingFraction"][trainingsetindex] TypeError: 'NoneType' --- So I would first look at the 'pose_cfg.yaml' and 'config.yaml' to make sure all the paths listed are correct.

Hi @meenakshimaurya8233 - my apologies for this late reply. I did not have my notifications turned on. Regarding your question, I have used MiNA in the past. Personally, I find it very useful for 2D images. I have not tried it for 3D and 4D microscopy data.