วีดีโอ

Sadly agree. iX killed the ability to run docker in some update and will introduce again in the next version, and in the mean time can't use True charts either. No easy way until this fall - wanted a custom image but can't without some jailmaker which I think I read the developer is no longer going to work on? Need to read more but really unfortunate.

@alexsoupir yeah dev is done. I'm sure what's next. im running docker rn, but its not the same true charts was great!

It's going to be much better when they roll out the docker and docker compose native update.

Going to have to look and see if there is an easy way. Current version of TrueNAS doesn't work with custom docker images or docker compose - they are going to add it in a version closer to the end of this year. At that time I could try. Found this: github.com/dockur/windows which, if familiar with setting environmental variables in TrueNAS I think could be worked through.

I hope so, because I'm almost throwing my laptop through the window I really hate having to try setting things again and again because windows can't even handle path variables

Were you able to get it to work? Laptops work great as frisbees, but I would urge using something else to throw.

Of course. Something to keep in mind though is that the number of classes, type of dataset you have, and number of samples with greatly impact how much you can shrink dimensions with your latent space. More data, smaller the middle layer. Less data, less you'll be able to compress the data.

Are you installing in the correct order? I've noticed that is you install pytorch before other GPU dependencies like the cudatoolkit, it won't pick up the GPU. Might be worth a shot.

I don't have the most updated version of TrueNAS because, for me, they broke Docker in v23 the way I was using it in v22. The version I had that worked I think was 22.12.2, and have seen others on Reddit say in recent versions things have been breaking everywhere (most recently there's a RAM leak/release failure??). I haven't updated to v23 or even v24 because of the constant issues I see others have online in combination with the docker containers no longer working. Even tried to use docker in the terminal on TrueNAS v23 and it still threw out errors. Very frustrating. In other posts too, it looks like they're trying to move away from docker in favor of docker-compose, but i'm not familiar with docker compose (another reason i'm sticking with the old version of the OS). Unpopular opinion, but seems like a constantly changing, using users as the test case, operating system that is only friendly to people who like to solve problems constantly. They also seem to be at ends with TrueCharts for whatever reason. If you want, you can try to install the old v22 here: www.truenas.com/docs/softwarereleases/ see if that allows you to install containers and start apps. If so, then it's definitely something iX has been doing with TrueNAS. I have nothing super security focused so staying a few versions behind until they figure out their issues is no problem to me. Don't like the idea of having to constantly fix things once they are working. (sorry for the minor rant)

@@alexsoupir thank you very much for the detailed explanation! I started all this 1 week ago, so Im totaly new to all of this networking stuff and yes its a veeeerrry wide and often frustrating field to me so far... I even thought about switching to unraid, since a friend of mine is constantly talking about how perfekt it is...but 250 bucks for the unlimited update version is a lot! Even more than my hardware :D

@@Pyrocraker haha I'm with you on the unraid thing. If it was free I'd probably switch too. Been thinking about playing with Ubuntu server since docker is really the thing I currently rely on the most but it's hard to choose one way or another. Once you find one that works for you, then can stick with it. Hard to initially find it though. Like I mentioned maybe an older version of truenas is the way to go. Find your fit and the ride it along.

Hey @alexsoupir, I made a video on using docker compose on TrueNAS with Sandboxes and Jailmaker. Jailmaker <2.0.0 works on TrueNAS Bluefin and Cobia iirc, and also on the latest releases. It’s a good way to run custom docker apps, and may unblock you. th-cam.com/video/S0nTRvAHAP8/w-d-xo.html

They moved the interface from "Launch Docker Image" to "Discover Apps" > "Custom App".

nvidia cuda GPU is a requirement, yes.

Haha a lot of times in informatics/data science, if it works don't try to fix it. It installed and runs/recognizes cuda in the opposing versions. BUT, since it works and uses the GPU, I consider that a win. This question has come up other times though and the real answer for why it works is "I don't know". Assuming Pytorch is backwards compatible, as long as cuda and drivers aren't ahead of pytorch, perhaps that's why it doesn't complain.

Interesting. Don't think I had it installed before because other environments when not installed in the right order, pytorch doesn't know cuda is available so defaults to the CPU version. I did have the *driver* installed, but not the toolkit.

This sounds like a driver problem. Do you have the latest Nvidia GPU driver installed?

There's a few possible reasons. Usually when "NaN" is produced over "NA" that is due to spatstat.explore::Gest not liking the input data since it passed our checks (e.g. needing so many cells to calculate). I would plot the sample that is producing these "NaN" values and see if there are any reasons visible. Maybe no cells within the search radius and maybe not an appropriate edge correction method (just guessing). Another thing might be that the cells of interest are too close to the edge of the sample and too far away from each other that the edge correction is dominating the weight of the nearest neighbor measurement and spatstat just kicks it out. Can extract that sample, create a window with convexhull.xy, create a ppp object with ppp(data$xlocation, data$ylocation, window, ...), then run Gest on that ppp object to see what happens. Would just need to explore your data a bit more.

@@alexsoupir What confused me that I followed your tutorial step by step with the same dataset but the output was as I typed previously

@@abdelrahmanmahany133 That is strange unless it's a sample thing as I mentioned. Did the same sample/cell type that gave NaN for use give values for me? There are instances that NaN is expected.

@@alexsoupir Yes it is the same sample/cell type. The only different thing in my trial that I installed the most recent version of spatialTIME as I noticed that you said that the version in the video is not the most recent.

@@abdelrahmanmahany133 Open an issue on GitHub and if you can remove all other spatial data frames from the mIF to upload on GitHub. Then I can take a look

No worries! When creating a window for the calculations, it is done with `convexhull.xy` using the cells x and y locations (as specified with `xloc` and `yloc`). Neither `polygon` or `mask` are options in this function and I have never used those options when calculating spatial summary functions. I *have* used a mask to mask off parts of a window with *simulating* point patterns with functions like `rpoispp` to create a Poisson point pattern from the Poisson process, but this is irrelevant here. The edge corrections work correctly with the `owin` object created with the `convexhull.xy` as the cells are creating the boundary of the observation window which is not always the case when using masks. If you are really interested in diving into some of the methodology, play around with some of the subfunctions from the spatstat such as `edge.Trans` to convince yourself of the impact the cells distance to the window has on the amount it's being adjusted (closer = greater). Lots of functions that are within `Kest` and `Gest` may help with understanding these. But to answer your question, we are essentially creating a single polygon that is created by drawing as round of a circle as possible around all cells in a sample with `convexthull.xy` so the edge corrections are working as they should.

@@alexsoupir Dear Sir, thank you so much for your response !

This is interesting. Wonder if the drivers have it and happened to already be on mine? There are some discussions online about now having a cuda specific library. NVCC must have been part of some other package install like a driver install or something. From reading, it doesn't get installed specifically without the cuda library (see: stackoverflow.com/questions/56470424/nvcc-missing-when-installing-cudatoolkit). While not having cuda installed on my system before installing here, it must be carried over from another software. Even made sure that outside of this cuda environment that cuda is NOT available. Great issue you happened across. Not always fun, but good find. Hopefully someone else can benefit from this.

@@alexsoupir Got it. Thanks once again for your insightful video.

The posterior probabilities (the likelihood a cell belongs to a cluster) should all be set and not change after running InSituType. When plotting, there must be some randomness used when setting the *locations* of the cluster centers. As you can imagine, the X and Y locations in the flightpath plot are arbitrary as long as the other clusters are placed at relative distances. Have to remember that this is just visualization of the data and we aren't going to do any analyses downstream with the point locations on the plot. To play around a little bit, can try setting a seed (using `set.seed(123)`) for the plot. A UMAP also requires a seed because of randomness which is why within the Seurat `RunUMAP` function they use the default seed of '42'. Agree for reproducibility they should have included a seed parameter. But, as I said, this plotting is just for visualization purposes and the exact locations aren't entirely meaningful - point at (2,2) isn't half whatever as a point at (4,4).

Wasn't sure what to make videos for anymore but then gained some new experience with new methods and technologies and thought useful to share. Arduino project was put on hold because plants died..tried again recently but the power died again. Not sure what the issue is but don't have a yard or anything so stuck with pots. Not ideal..

Hey Zhiyuan. I had a clinical fellow run into this issue as well but can't remember off the top of my head what the problem was. Sounds like some sample is missing enough cells (points) to properly create a window. Are any of your spatial data frames containing less than 2 rows? Alternatively, are you specifying the `xloc` and `yloc`? We'll get the bottom of this because have come up before. Pretty sure it's the not enough cells/points problem.

@@alexsoupir Hi,Thanks for all your attention. You were right. It is because that shortage of data point on my file for trial Thank u so much

Good question to which I don't have a good answer. Might be any number of things like drivers for the GPU, some other system things, something Pytorch did to allow backwards compatibility, etc? I agree would think we would want to match them but as long as it's working, and we can document versions, that's the important part.

That does sound strange, huh? It's okay to use VS code for R and I think there are people that like it because of the flexibility between different programming languages. The only down side is that VS code doesn't support the R Projects (or at least last time I look - may have changed since). RStudio is pretty bad with memory (and the R language as a whole) so a really large Seurat object may be hitting the limit of RStudio while not VS code. That's my guess.

@@alexsoupir Thanks . I will try to keep up with you using vs code or I will try to increase the ram size for my device 😅

@@abdelrahmanmahany133 totally possible! Only think that will change is the .RProj part of automatically setting a working directory. Rest would work fine. Good luck!

@@alexsoupirThanks for your help

@@alexsoupirI created a swap folder and it solved the problem

If you have Nvidia card and didn't work, can always check to see if the drivers for the card are updated and then making sure the order of things installed in the conda environment is followed. If things are installed in different order (say installing pytorch before the cudatoolkit) then pytorch will be installed with the CPU version because at the time of install the tool kit isn't available. Usually easier to remove the conda environment and try again from scratch vs trying to remove the correct packages. Can also try changing the cudatoolkit version to something like the nvidia-smi version.

I made a model actually with my data and it seemed to work fairly well and am trying to work it into an ITCR grant or R01. There are other methods out there that do similar things but, in my opinion, they're way over complicated. Will see about making a video on this since I do think it's a valid method especially with the slightly more noisy spatial gene expression data. Maybe write up a paper quick to put on rxiv first.

Hey. Thanks much. I don't have experience in that yet but we have spatial protein from orthogonal sections of our spatial tramscriptomic data and I just need time to learn how to do integration. We also have bulk DNA and RNA from them so all the data is there. I'm working with a clinical fellow right now to try and classify cells based on transcript interaction at the cell level (looking at interaction of different transcripts within a cell). Idea is that location plus counts can help refine cell types. Data is very sparse though.

So this could depend on the process, pipeline, core that you work in. If you are in a bioinformatics core then one could argue to have a folder of different versions of the genome/genomes and use whatever a PI wants you to use. Alternatively, if you have a "1 and done" sample run or perhaps a couple, then incorporating the genome building into the workflow *could* save key strokes. Really what you think is best. Also, could have a check in a job to where if there is an already built genome then you can just use it but otherwise build the genome. This would be beautiful if working in a core and maybe someone has a mouse, maybe someone wants hg38 for a comparison, maybe another hg19 because thats what theyve always used. Hopefully this makes sense. Ultimately comes down to the situation for the project and future project possibilities. If not needing to be flexible then no doesn't make much sense to include but for adaptation to unique projects could be valuable.

@@alexsoupirThanks for the clearance

That could be a reason. One hopes that as time goes on, there is more and more refinement of tools. If the same version and everything was used, it might come down internal random seeds of where to begin building contigs that it arrives at a different final 'optimal' final product. Can't think of a different example that's easy to understand outside of K-Means clustering - a random location is generated at the beginning of the fit. If the final product differs from run to run, there might actually be a way to leverage this to increase the build of our genome and decrease contig number. Lets say we run it 10 times and get slightly different contig numbers and sizes each time, we can then align all of our contigs and see if any contig from different runs are able to bridge gaps. Can start thinking one step further and see what pops up. What happens if you align the longest contig from different builds? Are they identical? Are there differences? If there are differences, can we produce a consensus but building the genome over and over and seeing what changes? Great thing with some of this stuff is that if/when there's a deviation from 'typical', that leaves room to get creative!

@@alexsoupir Great. I got it now. Thanks a lot.

Can I contact with you Dr abdelrahman ​@@abdelrahmanmahany133