Finding NVIDIA-card compatible NIM models and running their docker containers locally

NVIDIA NIMs are ready to run pre-packaged containerized models. The NIMs and their included models are available in a variety of profiles supporting different compute hardware configurations. You can run the NIMs in an interrogatory mode that will tell you which models are compatible with your GPU hardware. You can then run the NIM with the associated profile.
joe.blog.freem...

[@FreemanSoft]

I made a mistake when I recorded the video and corrected it with some annotations. TP1, TP2, TP4, TP8 are measures of Tensor Parallelism and not tensor core generations. I have one video card so in general I look at the TP1 models.