find HIDDEN urls!! (subdomain enumeration hacking) // ft. HakLuke
ฝัง
- เผยแพร่เมื่อ 4 ส.ค. 2024
- Learning how to hack is fun, but having people spy on your internet traffic is not. Get 3 months of PIA for free when you sign up for a 2 year plan!(ps. That’s only $2.11 a month) ntck.co/PIA
In this video Chuck is going to talk to you about Hakrawler, an awesome Subdomain Enumeration tool created by Hakluke! Chuck is going to walk you through how to install Hawkrawler and another Subdomain Enumeration tool called Gau on a Kali Linux machine using Docker.
🔥🔥Join the NetworkChuck Academy!: ntck.co/NCAcademy
**Sponsored by Private Internet Access
Walkthrough ⏩ ntck.co/3baEOpz
Interview with HakLuke ⏩ ntck.co/HakLuke
Follow HakLuke Here:
- TH-cam: @hakluke
- Twitter: / hakluke
- GitHub: github.com/hakluke
- Medium: / hakluke
SUPPORT NETWORKCHUCK
---------------------------------------------------
➡️NetworkChuck membership: ntck.co/NCAcademy
☕☕ COFFEE and MERCH: ntck.co/coffee
Check out my new channel: ntck.co/ncclips
🆘🆘NEED HELP?? Join the Discord Server: / discord
STUDY WITH ME on Twitch: bit.ly/nc_twitch
READY TO LEARN??
---------------------------------------------------
-Learn Python: bit.ly/3rzZjzz
-Get your CCNA: bit.ly/nc-ccna
FOLLOW ME EVERYWHERE
---------------------------------------------------
Instagram: / networkchuck
Twitter: / networkchuck
Facebook: / networkchuck
Join the Discord server: bit.ly/nc-discord
0:00 ⏩ Intro
1:23 ⏩ What are we doing here and why are we doing it?
3:13 ⏩ What is Hakrawler?
4:44 ⏩ Getting Hakrawler installed
5:51 ⏩ How do we use Hakrawler?
7:54 ⏩ How is Hakrawler working?
9:15 ⏩ Don’t be afraid and to tinker!
10:52 ⏩ How is your privacy?
12:30 ⏩ Two tools in one video!?
13:00 ⏩ Installing and using Gau
14:28 ⏩ Outro
AFFILIATES & REFERRALS
---------------------------------------------------
(GEAR I USE...STUFF I RECOMMEND)
My network gear: geni.us/L6wyIUj
Amazon Affiliate Store: www.amazon.com/shop/networkchuck
Buy a Raspberry Pi: geni.us/aBeqAL
#Hakrawler #Kalilinux #Docker - วิทยาศาสตร์และเทคโนโลยี
Learning how to hack is fun, but having people spy on your internet traffic is not. Get 3 months of PIA for free when you sign up for a 2 year plan!(ps. That’s only $2.11 a month) ntck.co/PIA
In this video Chuck is going to talk to you about Hakrawler, an awesome Subdomain Enumeration tool created by Hakluke! Chuck is going to walk you through how to install Hawkrawler and another Subdomain Enumeration tool called Gau on a Kali Linux machine using Docker.
🔥🔥Join the NetworkChuck Academy!: ntck.co/NCAcademy
**Sponsored by Private Internet Access
Walkthrough ⏩ ntck.co/Hakrawler
Interview with HakLuke ⏩ ntck.co/HakLuke
0:00 ⏩ Intro
1:23 ⏩ What are we doing here and why are we doing it?
3:13 ⏩ What is Hakrawler?
4:44 ⏩ Getting Hakrawler installed
5:51 ⏩ How do we use Hakrawler?
7:54 ⏩ How is Hakrawler working?
9:15 ⏩ Don’t be afraid and to tinker!
10:52 ⏩ How is your privacy?
12:30 ⏩ Two tools in one video!?
13:00 ⏩ Installing and using Gau
14:28 ⏩ Outro
You should put that into description
Mullvad on top
Can u do a review some software that I made
netsniff-ng
PIA is in the five eyes jurisdiction, Need a provider such as surfshark or express VPN for full protection.
This is one of my favorite parts of web enumeration. That one website you've been going to for years actually has several other subdomains that you've never accessed before
Says you have a reply but I can’t see it. TH-cam be on weird Shit
Exactly.
Chuck: i'm *not* giving you promission
Me: i'm going to pretend I didn't hear that
Ive been posting my terrible code publicly for a few years now. My favorite thing about it is being to watch it improve over time. It's also nice to have something to link to if I find someone needing help with an issue Ive already solved. And there have been some folks that have found my code, or my docker images and found them useful. That's a win to me.
Luke is a really nice guy. The part about creating and not worry about bad critics was great advice man;)
Great video as always. Keep doing great job Chuck!!
Great content and tutorials Chuck! Please keep them coming!
Hey Chuck, I love your content and hope you keep it up for a long time! I had a quick question though...I'm a remote teacher for cybersec, and I've been trying to find a tool that works with teams so I can use a wacom to highlight things when I'm teaching like you do. Could you let me know what tool you use for that? I'm a primary Linux user, since MS just isn't great for hacking tools. ;)
Thanks Chuck, great content as usual.. I've recommended your videos to all my fellow Admin team members, to get them up to speed quickly on Linux..
Thank you! 🐧
Man, was literally just about to research this topic. Thank you!!
Hey chuck. Love your videos, have learned a lot. I am not sure if you consider video requests, but I’ve used ALOT of what you’ve taught, and mixed it with what I know about Linux to build and hack on macOS.
I’m having to dig and find a lot of info on some of the qwerks of Mac, but it’s similarities to Unix and the stability of macOS makes it a fantastic computer.
I would love to see some videos on a security researchers feed, like yourself, about how to do some of these things, compile code, hack etc on Mac instead of Linux. There’s not much, or any content on hacking on a mac, building, compiling, or anything seriously deep on Mac.
Thanks for having me on your channel @NetworkChuck!
Thanks for the video. I was actually working on a webscraping project with Python and Beautiful Soup that did something similar. However, it’s a work project looking for keywords on a website, nothing hacking related. Cool to see it written in Go.
This is one that's been on my mind for a while. Keep it up, Chuck
Literally been studying information gathering active/passive with INE & TCM recently. This is an awesome addition!
What are INE and TCM? Hearing them first time
Since I started watching you I’ve gotten back it doing this/ what you so GLAD THAT I FOUND YOUR CHANNEL AND NOW IM LEARNING MORE THEN BEFORE SP THANK YOU
Super video Chuck your videos are awesome And informative
I need to go binge watch your channel
Chuck you are the best teacher in the world
Cool video as always! DNS Enumeration/brute forcing could be an interesting topic to cover too!
I prefer DNS encapsulation
Literally needed something like that a few hours ago, perfect timing 🙂
Why whatcha doing?
@@deaditeboimler4141 nunya
@@Nonflushableturd nunya
Chuck, with CloudFlare you should be able to put all your subdomains under Access/ZeroTrust, only allowing access to the sub domains/IP's from within your ZeroTrust network/list of whitelisted IP's
Add RDP to the Access/ZT tool and you should have a decently secure network with secure remote access.
LOL that Meme about the truck at 11:59 xD
Love your videos man, but quick pro tip: did you know you can do 'sudo !!' To run the last typed command as root? Keep 'em coming 😉
I recently found subfinder and Amass and I believe after 2 years of hacking and failing. I think I now know where to go and look beyond the main site because the obvious bugs have already been found.
What I learned in ethical hacking in order to find those sweet high or critical bugs you either need to dig deep or search farther as legally possible.
wow first time I managed to see you video as one of the first 100:D
I've been to a couple of Halloween Horror Nights and the one on your shirt was the one I missed and was upset that I didn't get to experience it!
i am computer science student and i am learning alot from your videos and i was going to watch your videos from the very first one and make a full playlist and get as many skills as i can thanks in advance for everything
Phew... finally!! I've been waiting for this video!
Your content is amazing. Big thanks 😊
Welcome back chuck ❤️
That is one thing that I've noticed that the IT and Blacksmithing Communities have in Common... If you need a tool... you make a tool... I mean... Blacksmiths and Chip Manufacturers are essentially doing the same thing... Shaping Metal With Fire
Great content!!! It doesn't top dressing up like nosferatu but it was quality. Thank you!💜💙🧡❤️💚
I needed this!
Who needed this too?
i did the same as u mentioned about making ur own scripts to automate things and i did one for wireguard vpn that connects to the vpn and check if its connected also connects on start up
NC, can you (unless you already ahve) go over how how you have customized your terminal screen. I really like the prompt and the behavior of your terminal screen.
4:25😳🕷️
Da😱n it Chuck! Give me a heads up next time.😬
I could've thrown my device and broke it.😓
Probably should have seen it coming with you having that Halloween shirt on.
Since it's too late 4 it 2 be an April Fools' joking😅 (8:35🤔).
Did that rhyme? I don't know (5:51).
Love your shirt Chuck.
Hey I recently started learning ethical hacking as a side hobby because it seems fun and it would be cool to say I know how to hack but I was wondering about the line between legal and illegal. like what level of permission do I need from the "target" specifically is it illegal to hack my friend if they tell me they are cool with it or do I need some like write up of them saying its ok?
Do you think cmd is enough for run hacking method for good purposes?
love this channel 💙
Your video always help us great, thank u .
Love your videos!
Will this install on Kali Linux 2022. 2?
I receive 443 error when trying to install
I subscribed. Now I'm addicted to coffee.
Chuck, I'm assuming Hakrawler completely ignores any =nofollow directive in the robots.txt file, correct?
My “coffee” is a bottle of Hennessy lol
Honest question, for subdomains, it would be enough to check the DNS servers right? I mean every subdomain must be defined in a DNS record, if you want to know the different servers available. Wildcard doesn’t count, because a wildcard always go to one location, it’s the non-wildcard subdomains which are interesting and can be pointed to different servers.
Subdomains don't need to be defined in DNS records, though they often are. They also don't have to have their A/AAAA records public, unless the service the server is providing is meant to be accessible to the public. Misconfigured firewalls/DMZs/DNS servers are a problem though, so querying the DNS servers (even better: requesting a zone transfer) is a good tactic, but not the only one. There's also more reasons than just subdomain enumeration to run a crawler on a website...hidden pages or admin portals can also be found through tools like dirb and nikto.
@@CyphDragon Thank you for following up my question.
But for the subdomain part, how can you find it with enumeration when it’s not publicly advertised and doesn’t have DNS records to actually point a subdomain to the corresponding server? I often work with a non-advertised subdomain, which the webserver does now about, and I can only access it by adding it to my host file. (Is that what you meant with not having any A/AAAA record? Hidden like that?
How will the subdomain then be found by enumeration or even crawling when there isn’t any public record of the existence of any subdomain?
Once again, a honest question which I cannot grab my head around, maybe I don’t understand the goal exactly to that the video is titled. (It wasn’t really in-depth in the video about how it technically works)
Good point
@@xoniq-vr Just reviewed the code for hakrawler, and it's even more simple than I described - it's going through and parsing (aka scraping) the website for any href tags, and then visiting the URLs referenced in those href tags, and repeat for the configured depth...so in this case, the subdomains would need to have public DNS records, and pulling a zone transfer or brute forcing DNS queries would both likely work as well. This is, of course, the joy of hacking - there's almost always more than one way to pet a cat, depending on what you want to see!
Hi, I really appreciate your work. Your video is amazing. I have a question- I purchased a domain and I am managing it on cloudflare. I want to create a subdomain and I want to give sub domain ownership to another cloudflare account that manage buy my friend. Can I do that ? And if possible then suggest How I can achieve this. Thank you for your attention.
Yup. Nice topic Chuck
There's an ancient tool called HTTP copier (something like that) that has GUI version for windows and it would download whole websites for you to review them offline also by specifying depth of the links.... But this is fun stuff doing it from a command line. Chuck became third persone btw... :)
is there a way to do this that might be less thorough and not require permission? And if so, not requiring a tool that needs to be installed? Perhaps using a "passive" enumeration?
Hey dude! How are you?
Please continue your Bash Script series.
if the tool uses the wayback machine then you should be safe because you are not hitting the real site.
you could use passive mode instead so it just downloads the html file and generates the urls from the html files.
you could also use a proxy or vpn to furtherly protect your isp
Your voice is verging on ASMR, idk if you’re going for that, or if you’re naturally more soft-spoken, but it’s definitely interesting (In a good way) your voice, despite certain inflections, tends to stay around the same volume without major dips or rises which I find very interesting, maybe because I’m naturally very loud
Benefits of having your own website.
You can just attack yourself.
I created a website last week , just for fun , and I didn't even know my website still had a wordpress sample page attached to it.
Came to know about it from this tool
Sir thank you so much for this educational video, I learned so much. and sir watched u since my career and sir u everytime bring video which is always leans to knowledge. And sir your coffe break which is most lovely thing we did , if u dont mind sir i can get a cup of yours sir? and once again thank you sir.....................
@@GroupFacade9264 yea bcz uk i cant wait to just mssg him and i thought he would reply bcz it was just 3 min it uploaded so and again no offense ☺☺
He's got a nice keyboard! Hehe. Fellow musician.
This is dangerous for the Django's Admin page too much. Interesting how you can secure it's URL ...
Incredible insights Chuck!!! Thank you Very much.
Vpn on its own does not making you private. There is always DNS mark if you wont spoof it.
Would this be a good way to find urls to black list for pi-hole?
hi , is there any chance to find hidden download links that websites make them show only to Vip users ?
Chuck! Waiting for python episodes !!
I can only say..... thank you network chuck
May we use this for good purpose, Amen!
What if I wanted to look into a server side application to see live data results.
Let's say this app runs code to determine a result that it will act upon and I want to see what that result is
How would I go about that?
Can we show which files are saved in my website. that's helpful to make more secure my own WebApp.
I want to find all assets or multi-media files.
HHN Shirt! Awesome!
I only have a potato pc soo can i start learning hacking in it?? Can someone help me?
would it be the same with webcopier?
Just yesterday I was looking for a feature and there was no tool available that had the specific feature I wanted so I wrote a tool to get this feature and started using it.
Is there a way to find private domains owners? I want to see who owns a certain domain
Can you teach how to build and set up our own cloud storage (as cost-free as possible)?
Lol, videos like this make me nervous the same way TCM opening their PNPT Live course with OSINT makes my teeth itch. I get that in the hacking process, this makes sense early on, but in as far as you can get yourself in trouble with this information, it's always a little nerve inducing.
ironically, i started working on a web crawler in python just last week! 🎉
What are the best sources to learn hacking ?
Just add your user to the docker group. I cry every time people type sudo with docker
Do you do videos on learning how to code like python
what about DirBuster. it's a tool brute forces directories. useful when the robots.txt is blocked
Ngl feels better to have your own tools, made by you
I use Sublist3r
What do you think about this tool ?
Wait where is the ip subnet thing you said was in the next video??
That's cool. I wanted to learn some basic Python while doing something useful, so I made a program that webscrapes all the images associated with a web comic on various hentai web pages 😁
I love your videos!!
just curious did reddit give you permission?
Please make one video for web application testing
from this demo video,
username and password : kali ,
your website using wordpress..
anyway, good work mate..
cmiiw
Correct me if i'm wrong but why would I need permission of the domain owner if hakrawler is spidering PUBLIC information via the wayback machine?
Google is the largest thief/crawler and nobody gave them permission to scrape the world. At the same time Google cries like a baby if someone “steals their info”, that they actually stole first. 😂🤔
The permission part is about server performance/costs. Companies likes to serve real visitors, not stealing bots. Some companies have up to 40% of their traffic from bots! Bots ignore ads and lower the performance for real humans.
It also do active crawling
You don't. All search engines do this every second. That's how they index their search results. And I don't think Google asks permission whenever they index a website. If your URL is public, it can be accessible by anyone.
@@smith4591 website owner can tell search engines to not crawl some pages by defining what's called robots.txt file
Thats what I need, thx bro.
Would you do a video about vpn? Is it really useful?
I have question where can I start hacking video game
Can you help me in next video can you do how hacker acces webcam in metasploit it can hack but the victim need to visit the website firt that like ip but if victim dont want to visit how we are going to hack webcam
Thank you so much
Thank you, good sir.🙂🙃
Thannks man frfr
So quick question, do these tools also pick up urls that are not indexed by search engines like Google?
A lot of websites use what's called a "Robot.txt" file which tells search engines not to index their webpage or certain parts of their webpage. So yes, you could definitely find URLs that google is not allowed to show you by using a program that manually crawls a webpage and bruteforces common terms to find sub-domains and directories.
Also, you can view these robot.txt files (if the website has one) to see what they are telling search engines not to index. But be careful. Some web dev's have bait robot.txt files that trick rogue spiders to a fake part of their website so that they can block them. This got me once and I wasn't even mad, I was really impressed.
@@TheHonorlord thanks for the advice.
If I want to secure my android phone what books resources should I use?
I have big privacy issues with my phone.
use a iPhone
@@smith4591 Thanks, Smith
But currently I don't have enough budget for that solution.
🔥🔥🔥
will you continue the "learning python series"?