Really great videos Jim as nobody is doing these in-depth videos of explaining security as well as you ( I know it's quite difficult to explain these ). Just one caveat on some videos you've missed some commands out visually by not showing on the video, including this one at the end when removing the added IP was not shown. This was easily figured out, but for some visual newbies it would leave them stranded a bit. Anyway great job keep up the good work as they have really helped me with my homelab journey.
Great chanel, Jim!! One thing you need to explain is log rotation of your docker containers. You cannot keep log files for a long time because the size of them will become huge!!
Great video Jim, towards the end your screen capture was showing next video to watch instead of what you are trying to demo. I would love to see a video on crowdsec with nginx proxy manager if that is an option. Also if you have a diagram like the one at the beginning of this video that shows all containers and data flow logic that has numbers of how it flows that would great. Just a suggestion. I'm a fan.
For anyone having issues with the acquis.yaml file. The solution i have found is to just make the config folder inside of the crowdsec folder and make the acquis.yaml file in that folder. Then you dont need to bind it as a volume by itself anymore, because youre already binding that config folder which is where its going to be anyway.
Exclente contenido. He visto muchos canale de este tipo. Pero tienes un don para explicar sigue asi. Me suscribo. No me puedo perder este contenido tan valioso. 🎉
Thank you for the amazing video, one last thing that has been on my mind. For example if I got 2 docker hosts each on different vlans, do I need two treafik instances?. Like one for local and the other one for external access. Or is it better to have a dedicated nginx rproxy for external access and treafik for internal use.
You raise a good question and it's something I'm planning on doing a video about. No, you don't need two instances you can route Traefik to external services (I do it for Proxmox gui). You can use two proxies, one for internal and one for external if you like. Or, you could use a single Traefik instance with multiple entrypoints (some of which are exposed).
It looks really cool, but unfortunately when I try to get it up and running, I'm just getting lots of failed to run filter : invalid character 'i' in literal true (expecting 'r') (1:1) | UnmarshalJSON and UnmarshalJSON : invalid character 'i' in literal true (expecting 'r'). I opened a thread on the Crowdsec forum since I couldn't Google *any* issue with similar error message. Has that happened to you as well? I'm using Podman not Docker, but it should behave in the same way ...
@@Jims-Garage already tried several times both for crowdsec, the bouncer and traefik. Didn't help unfortunately. Possibly some issue with cloudflare DNS proxy? On a separate issue I think to issue the let's encrypt certificate the first time I have to turn off DNS proxy. But everything was working correctly before introducing the bouncer 🤔
@@Jims-Garage the apikey part? Yes created and put in the compose.yml file, the did a podman-compose up - d. Restarted traefik and crowdsec several times without avail. For now I disabled crowdsec in traefik because nothing is working anymore 😔
If you put your container which has Traefik and Crowdsec through a Cloudflare proxy, is it possible to see the external IP coming in or ban external IPs? Currently, I am only seeing local IPs in the logs.
Ciao Jim, to me it is not clear how bouncer works. Do I have to add one for each service i want to expose (example: plex, home assistant, nextcloud)? or do I only have to add the one for traefik (if everything is managed by it) like you did?
Hi Jim, how to add the entry in the config.yml when I am using authelia as middleware? I am confised about it because after adding crowdsec-bounser under middleware I am not abble to open any https site any more, I guess because it is not routed to authelia any more? I think/hope it is a small change in the config.yml. Thanks a lot.
@@Glatze603 under each entrypoint in your Traefik config add the crowdsec middleware. Did you remember to register the bouncer? It might be worth getting crowdsec working first and then adding Authelia back
Hi Jim, Great video .. I have some problems after generating the token via command line .. (14:29). ERROR = msg="while fetching bouncer info: select bouncer: ent: bouncer not found: unable to query"... to solve it i am using this command : docker exec crowdsec cscli bouncers add docker-crowdsec-npm-bouncer . This works for me .. Thanks for the great tutorial.
Hi Jims, nice tutorial. i have error on log crowdsec: "failed to yaml decode /etc/crowdsec/acquis.yaml: yaml: input error: read /etc/crowdsec/acquis.yaml: is a directory". why? can you help ?
You have likely not created the file before deploying the container. When this happens Docker creates a folder with the file name. You need to delete the folder acquis.yaml, then create a file called acquis.yaml, and populate it with the example variables. Then when you next deploy it'll work.
ok. I try again (I delete VM and make a new VM). Than it work with crowdsec and bouncer. but traefik doesn't work, after install crowdsec. I put e.g. 192.168.x.y:8080. The site is not accessible. Why? is because, traefik has the same port 8080 like crowdsec ? please help. Thanks.
Hi Jim, great content - thank you! @7:33 you mentioned crowdsec requires 4 volume mounts, but you've specified the acquis.yaml file which (having followed this walk through) creates a directory called acquis.yaml Am I missing something? Cheers!
Thanks @@Jims-Garage. Yep, I tried that (& many other variations ) but no luck. Been searching, stackoverflow, etc. & it should work as you've stated but it's just not. I'll sort it eventually & post back with my eureka moment - cheers!
Hey Jim followed the video but at tue end of it can no longer access the traefik dash oord or gue nginx web page receding a 404 error page not found. Did tue traefik setup then crowdsec and skipped the pihole video assuming that is not a requirement?
Hi, unfortunately crowdsec breaks the Traefik dashboard, but both should still be working. I'm not sure why, and I have reached out to crowdsec for support but they weren't much help. I think it might be due to port conflicts on 8080. Nginx should be reachable though, not sure why that isn't working. Let me double check on my end using my configs.
@Jims-Garage ok noted,. If I put the port 8080 at the end of the docker it I can reach the ngonx page. Had to open the port for the Jellyfin one in order to reach its page as well as not to reaching it withe the name.
@@Jims-Garage I'm also having an issue with not being able to access Traefik page. I followed everything in your video. I noticed by removing the middleware crowdsec-bouncerfile added on traefik.yml, I was able to access my traefik page again. but then I think crowdsec does nothing doing this lol.
@@Jims-Garage i got the issue now, that traefik does not handle http/https anymore: ERR error="middleware \"crowdsec-bouncer@file\" does not exist" entryPointName=http routerName=http-to-https@internal i'm still in investigation why this is happening at the http to https right now
hi thanks for this traefik has always been 2difficult" for me, unitl now :) i updated my traefik config to include crowdsec, now traefik dashboard is not loading, just a blank page, realised i'm running unifi contorller on the docker host which uses port 8080, how do i change the crowdsec config file to utilize a different port please?
I recommend you leave crowdsec and Traefik as is, and simply run the unifi controller through Traefik (add the labels). This is the whole point of having the proxy. If that is not possible, change the port on unifi or crowdsec to accommodate.
k thanks, i only open the Inform port (8080) and stun (3478) to my unifi controller, just followed an article on how to change the crowdsec ports, but realised i have to also change the bouncer-traefic listening port too. PS is discord the best comm's, simply refuse to use discord (company background) and they also want your telephone number to post messages?
@@myhometvaccount9365 Discord is the most popular, I added phone number requirements to prevent bots. I do have a matrix server for anonymous conversation (check out my video, no installation required).
@@Jims-Garage You are correct and that is my issue. I usually sync a lot of files through nextcloud but I believe crowdsec seems to see it as brute force so I keep getting forbidden errors. I was hoping for a way to whitelist the cloudflare IP addresses so crowdsec doesn't block it. I cant seem to find how to do that in docker.
Really enjoying the content mate, keep them coming 👍🏻
Thanks so much for the feedback.
Really great videos Jim as nobody is doing these in-depth videos of explaining security as well as you ( I know it's quite difficult to explain these ). Just one caveat on some videos you've missed some commands out visually by not showing on the video, including this one at the end when removing the added IP was not shown. This was easily figured out, but for some visual newbies it would leave them stranded a bit. Anyway great job keep up the good work as they have really helped me with my homelab journey.
Man, you really got top-notch Guides/Tutorials. Thank you!
@@Chiren much appreciated 👍
Great chanel, Jim!! One thing you need to explain is log rotation of your docker containers. You cannot keep log files for a long time because the size of them will become huge!!
Thanks 👍 yes, I might put a short out on how to do it.
@@Jims-Garage This would be great to know how you are doing it!
Great video Jim, towards the end your screen capture was showing next video to watch instead of what you are trying to demo. I would love to see a video on crowdsec with nginx proxy manager if that is an option. Also if you have a diagram like the one at the beginning of this video that shows all containers and data flow logic that has numbers of how it flows that would great. Just a suggestion. I'm a fan.
Thanks for your support and suggestions, I'll fix that.
Certainly something I'll keep in mind as I know nginx is very popular.
For anyone having issues with the acquis.yaml file. The solution i have found is to just make the config folder inside of the crowdsec folder and make the acquis.yaml file in that folder. Then you dont need to bind it as a volume by itself anymore, because youre already binding that config folder which is where its going to be anyway.
Exclente contenido. He visto muchos canale de este tipo. Pero tienes un don para explicar sigue asi. Me suscribo. No me puedo perder este contenido tan valioso. 🎉
Thanks 😊
Thanks for the demo and info, have a great day
Thanks, you too!
I can't help but close my eyes and hear David Bombal in your videos😂
All good, although difficult to see the dark blue in the console windows.
Thanks, noted. Will try to avoid that in future videos.
Thank you for the amazing video, one last thing that has been on my mind.
For example if I got 2 docker hosts each on different vlans, do I need two treafik instances?.
Like one for local and the other one for external access.
Or is it better to have a dedicated nginx rproxy for external access and treafik for internal use.
You raise a good question and it's something I'm planning on doing a video about.
No, you don't need two instances you can route Traefik to external services (I do it for Proxmox gui). You can use two proxies, one for internal and one for external if you like. Or, you could use a single Traefik instance with multiple entrypoints (some of which are exposed).
@@Jims-Garage Is there a video on this curious myself?
@@MacJFitness no, but use an external service within Traefik. I do this for Proxmox UI
It looks really cool, but unfortunately when I try to get it up and running, I'm just getting lots of failed to run filter : invalid character 'i' in literal true (expecting 'r') (1:1)
| UnmarshalJSON and UnmarshalJSON : invalid character 'i' in literal true (expecting 'r'). I opened a thread on the Crowdsec forum since I couldn't Google *any* issue with similar error message. Has that happened to you as well? I'm using Podman not Docker, but it should behave in the same way ...
I've witnessed that before, restart the containers.
@@Jims-Garage already tried several times both for crowdsec, the bouncer and traefik. Didn't help unfortunately. Possibly some issue with cloudflare DNS proxy? On a separate issue I think to issue the let's encrypt certificate the first time I have to turn off DNS proxy. But everything was working correctly before introducing the bouncer 🤔
@@silverstone7778 did you register the bouncer?
@@Jims-Garage the apikey part? Yes created and put in the compose.yml file, the did a podman-compose up - d. Restarted traefik and crowdsec several times without avail. For now I disabled crowdsec in traefik because nothing is working anymore 😔
If you put your container which has Traefik and Crowdsec through a Cloudflare proxy, is it possible to see the external IP coming in or ban external IPs? Currently, I am only seeing local IPs in the logs.
X-forwarded header should show the original IP
@@Jims-Garage How would I set that up?
Ciao Jim, to me it is not clear how bouncer works.
Do I have to add one for each service i want to expose (example: plex, home assistant, nextcloud)? or do I only have to add the one for traefik (if everything is managed by it) like you did?
Just add to Traefik once, everything is then passed through it.
@@Jims-Garage Thank you very much! and thanks for the quick reply, I recently discover your channel and I love it! cheers from Italy! ciaoo
@@crc-error-7968 appreciate the feedback, have a good one 👍
Hi Jim, how to add the entry in the config.yml when I am using authelia as middleware? I am confised about it because after adding crowdsec-bounser under middleware I am not abble to open any https site any more, I guess because it is not routed to authelia any more? I think/hope it is a small change in the config.yml. Thanks a lot.
Have you added the middlewares to both entrypoints?
@@Jims-Garage Could you give me an example of what and how you mean?
@@Glatze603 under each entrypoint in your Traefik config add the crowdsec middleware. Did you remember to register the bouncer? It might be worth getting crowdsec working first and then adding Authelia back
Hi Jim, Great video .. I have some problems after generating the token via command line .. (14:29). ERROR = msg="while fetching bouncer info: select bouncer: ent: bouncer not found: unable to query"... to solve it i am using this command : docker exec crowdsec cscli bouncers add docker-crowdsec-npm-bouncer . This works for me .. Thanks for the great tutorial.
Thanks, glad you figured it out.
Hi Jims, nice tutorial.
i have error on log crowdsec: "failed to yaml decode /etc/crowdsec/acquis.yaml: yaml: input error: read /etc/crowdsec/acquis.yaml: is a directory". why? can you help ?
You have likely not created the file before deploying the container. When this happens Docker creates a folder with the file name. You need to delete the folder acquis.yaml, then create a file called acquis.yaml, and populate it with the example variables. Then when you next deploy it'll work.
ok. I try again (I delete VM and make a new VM). Than it work with crowdsec and bouncer.
but traefik doesn't work, after install crowdsec.
I put e.g. 192.168.x.y:8080. The site is not accessible. Why? is because, traefik has the same port 8080 like crowdsec ?
please help. Thanks.
Hi Jim, great content - thank you!
@7:33 you mentioned crowdsec requires 4 volume mounts, but you've specified the acquis.yaml file which (having followed this walk through) creates a directory called acquis.yaml
Am I missing something?
Cheers!
@@derrickrobinson3333 create the file otherwise it'll create a folder instead.
Thanks @@Jims-Garage. Yep, I tried that (& many other variations ) but no luck. Been searching, stackoverflow, etc. & it should work as you've stated but it's just not. I'll sort it eventually & post back with my eureka moment - cheers!
Hey Jim followed the video but at tue end of it can no longer access the traefik dash oord or gue nginx web page receding a 404 error page not found. Did tue traefik setup then crowdsec and skipped the pihole video assuming that is not a requirement?
Hi, unfortunately crowdsec breaks the Traefik dashboard, but both should still be working. I'm not sure why, and I have reached out to crowdsec for support but they weren't much help. I think it might be due to port conflicts on 8080.
Nginx should be reachable though, not sure why that isn't working. Let me double check on my end using my configs.
@Jims-Garage ok noted,. If I put the port 8080 at the end of the docker it I can reach the ngonx page. Had to open the port for the Jellyfin one in order to reach its page as well as not to reaching it withe the name.
@@Jims-Garage I'm also having an issue with not being able to access Traefik page. I followed everything in your video. I noticed by removing the middleware crowdsec-bouncerfile added on traefik.yml, I was able to access my traefik page again. but then I think crowdsec does nothing doing this lol.
@@Jims-Garage i got the issue now, that traefik does not handle http/https anymore:
ERR error="middleware \"crowdsec-bouncer@file\" does not exist" entryPointName=http routerName=http-to-https@internal
i'm still in investigation why this is happening at the http to https right now
@@marcussteck3782 Did you find out why? I have the same issue, this and the 404 error reported above.
hi thanks for this traefik has always been 2difficult" for me, unitl now :) i updated my traefik config to include crowdsec, now traefik dashboard is not loading, just a blank page, realised i'm running unifi contorller on the docker host which uses port 8080, how do i change the crowdsec config file to utilize a different port please?
I recommend you leave crowdsec and Traefik as is, and simply run the unifi controller through Traefik (add the labels). This is the whole point of having the proxy.
If that is not possible, change the port on unifi or crowdsec to accommodate.
k thanks, i only open the Inform port (8080) and stun (3478) to my unifi controller, just followed an article on how to change the crowdsec ports, but realised i have to also change the bouncer-traefic listening port too. PS is discord the best comm's, simply refuse to use discord (company background) and they also want your telephone number to post messages?
@@myhometvaccount9365 Discord is the most popular, I added phone number requirements to prevent bots.
I do have a matrix server for anonymous conversation (check out my video, no installation required).
Thanks for this. Does anyone know how to whitelist IP addresses for crowdsec in docker?
What do you mean? Crowdsec applies to all traffic hitting Traefik (AFAIK).
@@Jims-Garage You are correct and that is my issue. I usually sync a lot of files through nextcloud but I believe crowdsec seems to see it as brute force so I keep getting forbidden errors. I was hoping for a way to whitelist the cloudflare IP addresses so crowdsec doesn't block it. I cant seem to find how to do that in docker.