Another lovely presentation Andreas, thank you. I was considering, as you spoke, how wonderful and powerful the open source world is. There are some great assets here. What a view from the shoulders of giants! I can't help thinking that your evenings and nights are now no longer available for balloon hunting. All of your time is now going to be spent answering questions from your fans.
Do not forget my new IC-7300 sitting in my newly created "radio room". I already had contacts to more than 50 countries all over the world :-) A lot of side-projects...
Unbelievable Andreas, you're so damn good at this. No one else makes videos that are even half as helpful as yours. You have no competition out there, IMO. You are well on your way to million+ subscribers and you will deserve every one of them. Thank you again for sharing all of your hard work and unique talent. I find your videos immensely helpful. I feel like I need to watch five to ten videos from other content producers to get the information you deliver in a single video. Always love hearing that accent too.
Finally got this to work after 3 attempts (probably because I missed JoFie 1's comments from 1 month ago), I started with the new menu which did not succeed but using the old menu then adding Wireguard with the docker-compose settings it works fine now (even on a pi3 with just a 16Gb SD card). This is absolutely brilliant. I had several holes in my router before (for my external access) but now just one and with Wireguard encryption it is well worth the effort to set it up. Thank you for showing us how to do this, I love it. Happy New Year Arthur
I watch this video about 6 times over. Cause iam learning about dmr , VoIP, zoiper and echo link for ham radio, this is along the same Avenue as ham radio and device communication, as to weather or not you want your content to be accessible by the big guys like google and microsoft or if you want a secure acess tunnel aka portal. That only select devices or peers can acess. Agin iam no expert, I really enjoy watching you videos. This is right in line with ham radio and internet platform radio to radio connections via the internet and routing software such as pi star or braidmeister. I probabley got some things wrong. But I really enjoy these videos. It gives me a better understanding as to how devices communicate. Cause most of the world uses computers and sdr phones aka androids and iPhones. But most people have no idea what is going on underneath there touch screens, anyways I really enjoy your videos.
@@AndreasSpiess cool. I use to talk on the 147.435 repeater on Santiago peak in california witch was a naughty repeater. Now that I live in kingman az. Iam waiting for my first dmr radio from bridge comm, even though I have been a ham sence 2006. Cause there was a plethora of repeaters in california. Iam learning as much as I can about sdr/rtl and other radio platforms and I have a long long way to go. Just so I can get back on the winsystem or the papa system. Or link up to one of the many dmr talk groups , I also want to build an allstar VoIP witch i know is older analog but i like the clear sound of analog. Iam taking notes on hex codes decoded through frame rates and etc and learning about emmerse- satsatellites. Agin i have a lot of note taking to go, memorizing and learning to go before I feel confident about my capabilities.
@@AndreasSpiess iam waiting on my new callsign to show up in the fcc url. So I can get a dmr I'd. My old callsign was ki6gvx. I let my licence elapse. So I had to retake the element 2.
@@AndreasSpiess Yeah, and me! I like the sense of community, hearing TH-camrs mention each other; yourself, BigClive, Dave, Jonathan Oxer from Superhouse (and some I've forgotten to mention) - you've all been my mentors and helped me with a lot of Home Automation (and other) stuff. Thanks guys!!
Thanks for sharing details of this project. Although I won't use a Raspberry Pi for this, I do have an old PC I can re-use. This was a very inspiring video that will help me a lot share files between my work and home computers, store photos I don't want to send to google etc.!
For syncronizing between rpi's and other devices, syncthing is a great piece of software. Then, a question: are there any chance of getting an implementation of wireguard on ESP32? It is supposed to be quite small compared to other VPN's, so why not?
Thanks for the link to syncthing! Concerning the ESP32: I prefer to secure the net where my ESP32s are connected to. I do not think the whole key handling and stuff is simple for a IOT device.
@@AndreasSpiess -i just like simplicity, not associated with them- regarding getting a useable public connection its best to use an *unlimited data* ,fixed speed vpn with their generic debian10 +docker instance like ovhcloud vps '1 vCore,2 GB ,40 GB SSD NVMe ,250 Mbps *unmetered* for cheap rather than any of those American limited fixed data @ variable speeds corps. you can docker your remote WireGuard server there & just connect to/from your local WireGuard as you please + other dockers over there too. [yt keep deleting this post]
Nice. With a raid 0 the solution would be complete - maybe the external enclosure used provides hardware raid 0 ? Also what would have been interesting would be some figures regarding transfer rate to see how it compares with much pricier options.
Very nice video again, as I'm used to from you, Andreas :-) From a security perspective wireguard should be running on a dedicated pi with a hardened OS. This can be a pi2 or pi3, even a pi0. Having more softwares than wireguard on the pi increases the attack surface for attackers and it increases the risk of a breach by zero day vulnerabilities. This is increased by the fact that many ISP provided routers don't have the option to forward just one port, but only an option for a DMZ.
You are for sure right about the number of software running on the VPN server. Of course you can run the Wireguard machine in the DMZ if you know how to set it up. If I believe some stories (cameras, etc.) what people do, the solution presented here is already quite safe compared to just opening ports. I also ask myself: Why would somebody attach an ordinary household with a zero-day vulnerability. I would assume there are more rewarding targets. My standard router seems to have the possibility to only forward one port. I also read "DMZ" somewhere. Maybe I investigate a little in this direction.
@@AndreasSpiess I agree that this solution is relatively safe. The threat is real though. Cyber criminals are just looking for resources. If your devices are compromised they can become part of a botnet which are often used to hold companies for ransom with DDOS attacks. Also, it is not someone targeting you. Cyber criminals use automation to find vulnerabilities around the world, for various purposes. Think of it as scouts, software bots that go looking for openings. Depending on what is found and reported back, specific actions will be taken. No human is directly involved. Zero days simply mean it is not a matter if you will be compromised, but when.
I have been running Nextcloud on Pi for 4 years now. First on 3 and now on 4. To make it visible to Internet I bought a domain name and DynDNS-service from No-IP , which costs me around 30$ a year. From Let's Encrypt I got the SSL certificate for free. 2016 there were no free DynDNS services that supported also Let's Encrypt. At least I didn't find any. The data is stored on external usb double dock with 2 drives and raid1. System has been working without problems. With Pi 3 the max. speed through internet was around 10-12 MB/s, with Pi 4 it is double that.
@@AndreasSpiess To run a internet server at home your ISP and your router must support port forwarding. Especially ports 80 and 443. To run your own server at home with your own domain name, you'll need a dynamic DNS service. Your ISP changes your home IP address from time to time so you need dynamic dns service to keep track of your home IP address. DynDNS service provider will direct the internet trafic intended to your server to your current home IP address. No-IP is one of them but there are others too. They offer also a service to turn your domain name from your current web hotel to the dynamic dns providers servers so you can run your own server with your own domain name at home. Your server must be running the dynamic dns service provider's client program to keep their servers up to date with your current IP address at home. To get the Let's Encrypt SSL sertificate you must registrate your domain name to them, then load certbot program to your linux machine and run it. Of course before you can install the cerficate you must have a working http server running on your computer. I really recommend using Apache but there are instruction for other servers also. So the order of things to run your own Nextcloud server at home but also visible to internet 1. get a domain name or prepare to transfer your current domain name from web hotel to dyndns provider's servers. Check your current contracts about the web hotel and domain name. So that there are no nasty or expensive suprises from the web hotel when transferring the domain name. 2. choose a dynamic dns provider . No-IP as mine dyndns provider was more or less a random process. Today I would prefer a European provider. But I'm too lazy to change mine while it's working. 3. check that your chosen dyndns provider supports Let's Ecrypt. And vice versa. That Let's Encrypt supports chosen dyndns. 4. choose another dyndns provider until the terms are satisfied... 5. make a fresh linux installation 6. make your router settings. Port forward 80 and 443 to your newly installed linux computer. Set in the router a static local IP address to your linux computer's MAC address 7. install apache, php with needed additions and dyndns client program. Do not install Nextcloud at this point 8. registrate to Let's Encrypt 9. install certbot and run it with Let's Encrypt credentials to get a valid SSL certificate 10. backup your SSL certificate files to a safe place and media 11. install Nextcloud 12. start Nextcloud for the first time and make the settings. And fix the things it is complaining about. When you have everything running properly, for extra safety you can change your web servers ports to 5 number ports. If you do that, certbot won't be able to renew itself because it uses the defaults port numbers, so you must change the ports back to normal numbers in every second month to renew the certificate manually. For that I have two batch files. First to return the normal port numbers and reboot computer. Second to renew the certificate and change the ports back to long numbers and reboot. Not really a method for production environment but for me its working good enough. Unwanted visits to my http server ended. You must also make port forwards in the router for those long number ports. In Nextcloud client programs you give your server address followed with :portnumber (for example www.myownnextcloud.com:54321) For maintenance I have also a ssh connection to the Nextcloud server. Ssh is configured so that the port number is changed to a long number and you can login only with ssh keys. Also that port is forwarded in the router. But don't do that if you don't have to do maintenance over internet. PS. Four years ago it was not possible to get a free dyndns domain name that also works with Let's Encrypt. Maybe the situation is changed now. Someone else will know it better than me. If that is possible you can have a new domain name for your nextcloud server and keep the old one where it is.
@@AndreasSpiess Getting a Let's Encrypt's SSL certificate to work with a free dyndns domain name started to intrique me and I started digging the matter. Nowadays there are several free dyndns providers which domain names works with Let's Encrypt. So I decided to test it with a domain name from duckdns.org. And it worked. Easily. If you have your http server running properly and it is installed to use the domain name you have got from duckdns.org, you just load certbot and run it. It asks your e-mail address and your domain name and installs the certificate. More instructions of installing certbot from certbot.eff.org You have to run certbot once in every second month to renew the certificate. For installing and renewing the certificate you must have port 80 forwarded from the router to your server. For normal https trafic you must have port 443 forwarded. More instructions how to install duckdns update client from here www.duckdns.org/install.jsp
Thanks for the clarification. After your and a few other comments I decided to not open my home network. It seems too dangerous for me because of my hobby. I fear somebody would want to prove he is better than me ;-)
On NC, there is an icon of a link, so you can share certain files (and maybe folders) with others. I think the Collabora function may work in a larger scale way.
If you own a domain and can port it to NameCheap.com as your registrar they provide scripts that you can run to update your A record. So anytime your ISP changes the IP you get it updated so that your domain always comes to your home.
Thank you for this video. I have installed the containers for NodeRed, Grafana, InfluxDB ( after that the wifi doesn't work at all except when I restart the dhcpcd service and it crashes due to errors ). Also I had a backup from the containers on my google drive ( when the Pi's Wifi crashed, I tried to reinstall the OS and then restore all the containers and it doesn't work for me ). Could you help me ?
Just use WebDAV on the Pis for connecting them to the Nextcloud Instance. For external access, you can forward Port 80 and 433 of the router to the Nextcloud-Pi and when you got external Access you can just create file sharing links for each file ^^
Andreas, I am a 68 year old nube. I have been trying to teach myself electronic since last winter.. I do know a little about computers having played with them since the 70's with the tandy's and later the commodore.. But have not played with then since the late 90's.. all the kids got better then me so I didn't need to.. But now I am trying to get back into it all and find that even in computers I am a lost nube.. I have saved this in my project file and wish to do it this winter.. I do not have the new pi 4 but have several pi 3 b's Will this all work with them or is it a pi 4 I need.. I have an ssd set up on one of my pi's with movies on it for travel.. But things have gotten so far from me it is like learning all over.. thank my brother.. you channel is always fun to watch and a great teaching tool for me.. Carry on!
The Pi3 should work, too. But it has only the slow USB2. So an SSD does not help a lot. And the Pi4 also has more memory which also helps with speed. I am also 64, so no big difference ;-)
@@AndreasSpiess Well one big difference.. I never owned a soldering station until i was 67 and never ever tried to build something electronic in my life until this year.. But I have played with computers..
Every time when I did a docker-compose down my NextCloud ended up with an internal server error. Seems that in the compose-override for me a - ./volumes/nextcloud_db/config:/config folder was missing at the nextcloud_db service. After adding this line NextCloud kept his config after down the servers and bringing them up again. I was wondering if more people experienced this problem or that I did something wrong at 1st setup?
@@AndreasSpiess Indeed that. Fortunately the open source community has gotten much better about it in the last few years. Chalk it up to corporate sponsors, perhaps, but I'll take it.
you can use an SD card but they run a lot slower than an SSD. I just upgraded my pi4 to an SSD and got a 7x benchmark improvement over the SD as the system disk. it is much more responsive now.
Hi Andreas, thank you for the video. After installing all the container once I start the nextcloud and add the required password it says "Error while trying to create admin user :failed to connect to the database" can you please help
Around 2004, when a 128MByte (Mega, not Giga) USAB-Stick was a huge one, I started my VPN experience using "Unslung"-FW and OpenVPN. Some years later, I was happy to migrate to the stock VPN of my router. Today the most routers provide VPN out of the box. No port forwarding, no additional raspi, no hazzle. And I have access to my complete home network incl. my ioBroker data set, Synology NAS und much more. Appropriate for a private person. Do the Swiss routers not provide VPN? Many iot systems as e.g. ioBroker also offer cloud access and provide "European" cloud solutions. But up to now I prefer my home network with VPN access and try to avoid cloud solutions.
My router provided by the ISP does not offer VPN. But I cannot speak of all other ISPs here. Using the router is of course also a good way to get secure access. For sure it is easier. As you mention, I also wanted to avoid the cloud. But for the moment it seems it is easier to keep it for file sharing.
You rock Andreas! Super interesting and useful. Did you know that there is a duck dns container. Maybe more elegant then crontab. Also nice to add swag container for reverse proxy
couple of things: 1. webdav sync 2. accessibility through the internet 3. nextcloud performance 4. big files: You don't really need a raspberry pi client, you can use curl to do file operations or even create a linux webdav mount (docs.nextcloud.com/server/16/user_manual/files/access_webdav.html). Locally, you can (php occ mainentance:mode --on) manipulate files directly into nextclouds data//files directory and run php occ files:scan to scan for these changes (php occ mainentance:mode --off). The analytics plugin creates RESt api endpoints for you to upload data from, for example, iot devices like weather stations, detailed instructions on uploading via api are here: github.com/rello/analytics/wiki/API. I imagine you have already tried this but: to connect your pi to the internet you just need one change: port-forward the appropriate port on your router and then access it using http(s)://: on your browser or sync client. I personally run nextcloud directly on linux with nginx + php-fpm and only have a SD card and HDD. To increase performance, I mounted nextcloud directly onto the HDD using /etc/fstab. To get more performance, add php opcache if not already done. On my instance website loading speeds went from multiple seconds down to a second or less. For more info on server tuning, see docs.nextcloud.com/server/latest/admin_manual/installation/server_tuning.html. Additionally, check how nextcloud is running it's cron.php (settings -> basic settings). If it is using ajax, loading times might be affected depending on whether or not the cron needs to do something. I'd switch to crontab where possible. To be able to upload bigger files (>100's MB), I needed to increase php's max execution time. The nextcloud pc sync client will upload the file in chunks of approx. 10M and then re-combine the file in one command. The recombining step was taking so long on my pi that php was timing out.
Thank you for your tips! A few of the tips are probably more for the builder of the NextCloud container. At least I would expect that they optimize the setup. The speed with my Pi4 and an SSD was ok for me. I can imagine that the USB2 of the older models can create a bottleneck.
Waw, a very amazing and interesting video. Some of the parts like D.DNS I was already familiar with. But everything else is very nice to start tinkering !
3:52 Do I really need 'SSD Boot enabled' on RPi? Can I just use a good SD Card for boot/OS - and separate to this - a large SSD over USB 3.0 for storage??
Adieu! (in swiss french) For internet access of your nextcloud, you should user jlesage/nginx-proxy-manager. Then in next cloud, you should able to share folder like dropbox. Hope it's help
I got an issue with USB booting from an external ssd: every time I reboot using ssh, the pi doesn‘t come online again and I have to reboot it manually. During this time, the lan leds don‘t blink. Any suggestions?
Very good video as always, Andreas. I've similar setup, but I run on DockSTARTer. Better support and easy to maintain in my opinion. Also Wireguard is blazing fast and secure. Love QR code feature. Take care.
If you don't want to expose your RPi to the whole internet there's another possibility: TailScale. This is a free service that manages Wireguard for you and also traverses NAT/router boundaries transparently. I have it installed on my RPi as well as my phone and laptop, so I have a multi-device VPN that works everywhere. You can also configure IP routing on the RPi so that you expose all other devices in your home network to the VPN. This is how I can access my home NAS from my devices. Quite nice and easy to setup.
Looks interesting! It probably solves problems I do not have with my home network. But for companies, this seems to be cool. But you do not have to go through any hassle. One thing I probably would not like is the single user in the free plan, even if you can use several devices. But maybe this limitation does not apply if my wife also uses the same user.
Using Tailscale for everything - I've got a dozen computers, phones, raspberries etc. Works very well and allows not to expose a single port to the internet! Even my hosted machine doesn't have a single port open, even for ssh - Tailscale allows it.
Hey everyone, any ideas on why I am not getting the peer folders in the Wireguard volumes folder? Same override config for wireguard as Andreas. I do have an older version of the IOTstack from a few months ago and will try updating tomorrow. Cheers from Aus.
Excellent tutorial Andreas! I followed it through almost to the end, but fell at the last post when installing Wireguard on my Android mobile! At 'Connection request' selecting OK does nothing and when cancelled results in "Error bringing up tunnel: VPN service not authorized by user", I've downloaded the log, but it doesn't mean much to me at the moment. Jim
The problem turned out to be the particular 'phone. I was using an Xioami Redmi Note 4. I then dug out an old Motorola Model G from 2014 and Wireguard works fine. Many thanks for the tutorial Andreas!
Hi i have a doubt i did the wire guard setup, the containers are running and i did port forward also but when i check with port checker its showing the port is closed. and i could'nt able to connect to internet via vpn So after stopping the container, I tried running a simple hellow world server using node on the same port and its working from duckdns and port checker also showing the same port(51920) is open. so what am i missing over there in the docker wiregaurd config? or is it somethingelse. please do respond, thanks.
Sharing with others, speaking outside of your home network is a difficult task. Especially if the whole purpose was in first place, to keep your storage private. I guess google drive or dropbox is a welcome solution to that on a file per file basis. My QNAP NAS offers a good mix of both: Local, private storage with offering the option of sharing a file or a whole folder with others like a dropbox. It’s a very good solution but properitary to QNAP NAS’ses (i guess Synology has that as well)
How do I gain access to the other computers in my Lan through wireguard? I read there is a setting to route traffic, but i am not sure where to set that.
You don't need to "install a Raspberry Pi client", it's usually already supported. The keyword is WebDAV and all you need is to connect the RPi to the NextCloud server using WebDAV. It's also supported on Windows.
@Michael: Does webdav synchronize my Raspberry directories with Nextcloud? @Tim: So I would use rsync to synchronize my "Nextcloud" directories on my Pi with Nextcloud?
WebDAV has been really hit or miss for me in the past. Reading a directory with lots of images will time out on the client, and trying to upload large files will also fail. Then you get into weeds trying to configure the Apache proxy if that's the NC image you used, or the PHP.ini files. Then the problem might be the WebDAV client is not fully compatible. I really wanted all devices on my home network to use WebDAV with NC but in the end, used SMB shares to a mounted HDD. I do have NC but use it for syncing images from all smartphones. Using the iOS and Android apps. The windows client app is annoying as it tries to sync a local PC folder with NC so if you assign it to your images folder, it will back then all up to an images folder on NC - great - but then start deleting those images on windows and the client software will delete them on NC too. Which is fine if there was some options to change the behaviour, but there isn't. I use duck DNS to port forward to my NC docker image so I can access NC from anywhere for the images to upload. Using let's encrypt to manage SSL automatically
Hi Andreas Is there an easy way to install 'Certbot'? It looks like it can be installed via docker, but I know very little about docker and portainer and am frightened of breaking my system. Jim
Not sure if this has been answered, you can easily share files like dropbox or google drive with others by just providing link, you can even set password and expiration date for it, the user doesnt need an account to download the files, similarly with next cloud you can create an upload link (secure drop) where user can upload files. Please also try out syncthing its much easier to setup than NC
@@AndreasSpiess sorry I think I didnt make it clear in my previous reply, I meant just like Dropbox and Google drive you can share files in Next cloud. You can do it from the webinterface and through the next cloud client as well. I switched from Google drive and Dropbox a long time ago to Next Cloud.
@@AndreasSpiess I have gigbit internet on VPS side and 300 MBPS upload and download speed at home. Still when i test the tranfer speed with iperf3 it shows max 110 mbits/second. Do you know what might be the problem to this?
mariadb doesn't support ARM V7, which means the mariadb cannot be executed properly in the docker of a raspberry pi. How did you make the nextcloud work?
Dear Andreas, I watched your perfect videos every now and then for a year now, and you are definitely the best that I could find during many many queries. Regarding pi4 and nextcloud, I would have a desire: run a raid10 (2 disks + 2 disks, as you may know much better than me) so to take advantage of the 4 usb ports and, well, so to keep all the data in my hands only. I am keen to use also 4 flash drives, in case that this would help for the electric current draw. Would you be able to "teach us" how to do that, as an expansion and in conjunction to this perfect project of yours?
@@AndreasSpiess Thank you. I have read that there is a software called "mdadm" that helps in defining the disks, but I would find myself today unable to "add" that part into your project. It steps into "defining labels" so to recognise the logical disks and the physical disks. I don't even know how to use dockers, for the moment. For the speed I would love to imagine that even having usb 2.0 would make me satisfied for a very modest home user only. Most importantly, thank you again.
I've added the wireguard lines to my compose-override.yml file but when I start menu.sh I can't see Wireguard in the stack list and it's not installing. Am I missing something?
@@AndreasSpiess I found the issue. I foolishly tried to use a prior Docker install instead of installing it via IOTStack as mentioned in the official documentation. I reinstalled docker by using IOTStack's menu.sh and everything works now.
Hi andreas, i use another container called swag (linuxserver/swag) in my stack for accessing from the outside. it works as cert endpoint with letsencrypt and reverse proxy. dns cnames (dyn IP @home) do the rest. let me know if you need more information! the configuration is verry simple
Thanks Andreas for this video that shows better how to combine iostack, nextcloud and show the details how to make it working. I begun last week the configuration of my raspberry 4 based on your videos 295 and 352 but I got problem with usb boot when installing wire guard . 3 questions 1) Do you use the last stable firmware ( November) for USB boot or critical version of September ? 2)Is wireguard running in a container or is it installed on the host from PI VPN project? 3) what about power supply regarding RPI when connecting one SSD and one USB HDD together? Do you use external power supply for HDD?
They seem to have changed the EEPROM quite a lot and recently created some problems. I used the version which was actual during the making of the video and used the script to install Nextcloud. Maybe you try to get a stable boot situation without any IOTstack first to separate the two problems. And go to the Discord channel if you need additional support
some internet providers will assign cg nat ips (the same public ip is shared between several customers), making port forwarding not possible :( in such cases you can try to contact them to assign your connection to a regular ipv4 ip schema.
exactly. I wished the Author would have mentioned this In the video aswell. At least my provider in Switzerland does exactly that. Or if you are trying to setup a VPN on a RiP that is connected via 4G to the Internet.
2:45 small correction - "This is why we need a service called *dynamic* DNS", where Duck DNS is just one provider (very useful, make no mistake!). Also note that port forwarding will not work if your (cheap) ISP uses a "carrier-grade NAT" and your "external" IP is not actually reachable from internet.
@fabio foltran There are ways to make it work, but they are not completely free. I use a cheap VPS with a public IP which acts as the main wireguard "hub" connecting my services and devices. Not super complicated, but as Andeas pointed out, not super easy either.
I'm new to Linux and Raspberry Pi. What's the point of using an SSD to store files instead of a usual microSD card? apart from a higher speed? are the system files too large for the microSD card?
@@AndreasSpiess Makes sense, I often hear corrupted microSD card is a problem for a small single board system like this. How much minimum SSD storage would you recommend for this project?
I think perhaps it is a mistake to use the latest of a bunch of images because it seems to have bit-rotted already. Any chance you could list version tags for the images that worked with the compose-override.yml published?
Maybe you ask for help on the Discord channel. We want that the project also runs with the newest versions. But keep in mind. The project changed a lot recently.
@@AndreasSpiess I just want it to work as it did in the video. I followed it but I get: Error while trying to create admin user: Failed to connect to the database: An exception occurred in driver: SQLSTATE[HY000] [1045] Access denied for user 'nextcloud'@'nextcloud.IOTstack_NextCloud' (using password: YES) I see lots of people get the same, so I think something must have changed. I didn't change any of the passwords in compose-override.yml. I see it is has been merged correctly into docker-compose.yml and I give the mySQL_password to Nextcloud but it can't connect to the database. I have tried Discord but the reply wasn't helpful, the suggestion was I need to configure mariadb to accept nextcloud or change nextcloud to match mariadb but compose-override.yml configures them both to match, so I have no idea why it doesn't work for me but it worked for you.
I am having the same problem. Either something is wrong with the database or something has changed significantly. @nophead were you able to find a solution?
@@kafadek825 I did, but it was 2 years ago on a friends RPI and I can't remember exactly how. In my notes from the time I wrote "Correct the MYSQL_PASSWORD in docker-compose.yml" after the Build Stack and before the starting the stack.
If you want to share outside your network you can use Hamachi, i have the same configuration but with owncloud, and I can share my files with other people outside my network
Andreas thanks a lot from Mexico, but I have a question!, Could I use this setup installation as a NAS and server phone system? Any suggestions o options! I appreciate your comments!
Hi Andreas. I really like your videos as they present real-life solution. For me, however, the instructions in this video do not work. The resulting compose file fails to create the NextCloud docker and shows errors. I only was able to create the portainer-ce container. Not sure how to share the yml files resulting from the script. I will try with the developer. Thanks again for all your work.
Hey mate, I followed your guide for wireguard setup. It worked flawlessly but one day later it said could not establish handshake. The only change I made to the IOTstack was to add pihole and then I changed my wifi dns on my laptop to connect to pihole. As soon as i did that wireguard stopped responding and even after changing my dns back to normal it still is not running. Do you know what may be causing this ?
Im recently installed NextCloudPi on a raspberry pi 3, and although its slow due to the slow ethernet and usb interface, its awesome. I did not use docker or wireguard im wondering if theres an advantage to using docker over a normal install. Also with NextCloudPi im able to share files with people outside the network. Im going to get a raspberry pi 4 and deploy it like you did since it seems more secured with wireguard
I like docker because the install of the software including dependencies in the containers is done by knowledgeable people and you easy can get the newest version. In addition you can restart and shutdown each container in seconds. And if you delete it it is gone with all its dependencies.
Fantastic! Thank you. One question. If I understood this correctly, there is no client for Raspberry Pi. I have a Pi 3 running Node Red which runs my home automation with a mix of Sonoffs and Shellys all flashed with Tasmota. Does this mean, I cannot use my existing User Interface on my phone to tunnel into my Home Automation Raspberry Pi?
I found that duck.sh got "502 bad gateway" more often than not from duckdns.org when run from crontab but when I ran it from the command line it always returned OK. I realised that everybody is configuring their RPI to hit duckdns every 5 minutes, on the 5 minute boundary, so all at the same time. Changing the crontab setting to every 5 minutes starting at 1 solved it for me. Seems like their server is being overloaded on the 5 minute boundaries by all the IOT devices. We need a crontab option for random jitter!
@Andreas Spiess Definately check it out. Wireguard is awesome but if all you want is to connect your devices and never think about hole-punching again, ZeroTier is way faster to set up. Each device (pi, pc, android, etc) gets a virtual adapter. Join. Authorise through their web site and assign a static IP to make life simple. Done. Direct connect from any connected device. Down side is that the free tier limited to 50 clients. Pretty fair.
With many ISP's now using CGNAT (double nat) poking holes through the router won't work for external access in those cases. Zerotier (free) is one solution there are others. I think all require a reverse proxy type approach.
For the question how to share files with people outside your network: There are 2 obvious ways. One is to bring the guests into your network using Wireguard. I find this very inconvinient, since I don't wanna bring those guys into my private network. The other one is to open a tcp port on your router, to share the html of nextcloud to the outside. This is a security risk, but a small one. However, from an IT point of view, You would have to create kind of a safer network environment, with several stages like a DMZ, and I would never ever run Wireguard, Maria DB and Nextcloud on the same machine. Conclusion: tough decisions to make. For private installations, one has to check convinience, security and effort against each other. Nice video, I like the output provided by the project around IOT stack
That is what I thought, too. I hoped somebody has a "silver bullet"... Solution 1 is obviously a no-go. And if I have to have a Pi in my DMZ I probably stay with Google or Dropbox. So it seem to be a good thing for a family or a small company.
A good possibility. I hoped to be able to use it for community projects because the free google service does not allow for permission schemes. If I have to pay for NextCloud hosting I probalyy can also pay for a Google plan. What was your reasoning for Nextcloud on a paid server instead of cloud solution?
@@AndreasSpiess That private server runs several other stuff I need, so the server costs for nextcloud are very minor. And, it was an advantage in terms of privacy. I try to avoid cloud services when possible. Yes, me writing on TH-cam, I know.
Understood. I am very pragmatic with cloud services. Many problems can be easily solved with such services. I try to optimize my time. It gets more precious every year because I get closer to my death day...
@@AndreasSpiess likely, or something with it being 32 bit. I bought a used office pc with a 6th gen intel chip, doing much better progress on the installation, docker already up.
Hi Andreas, thank you for this video and the instructions. I can mount an external HD in my PI4 and enable the external storage app in nextclound but fail to add the external HD with /Disk1 in nextcloud (get a red dot in front of the entry). I have installed smbclient eventhough I get this warning in nextcloud. Any idea what the problem could be? thank you in advance.
Thanks this is really very informative video, can we use a SSD for Pi OS and a HDD for nextcloud contents at the same time, does RPI4 gives enough power to operate both without issues?
Another lovely presentation Andreas, thank you.
I was considering, as you spoke, how wonderful and powerful the open source world is. There are some great assets here. What a view from the shoulders of giants!
I can't help thinking that your evenings and nights are now no longer available for balloon hunting. All of your time is now going to be spent answering questions from your fans.
Do not forget my new IC-7300 sitting in my newly created "radio room". I already had contacts to more than 50 countries all over the world :-) A lot of side-projects...
He probably has hidden the docker container with the answering module ;-)
What band(s) do you focus on Andreas?
@@AndreasSpiess Where do you find the time?
@Jim: Mainly FT8. All "traditional bands (80/40/20/15)
@Richard: Good question...
Unbelievable Andreas, you're so damn good at this. No one else makes videos that are even half as helpful as yours. You have no competition out there, IMO.
You are well on your way to million+ subscribers and you will deserve every one of them.
Thank you again for sharing all of your hard work and unique talent. I find your videos immensely helpful.
I feel like I need to watch five to ten videos from other content producers to get the information you deliver in a single video.
Always love hearing that accent too.
Thank you for your nice words!
Thanks for being to the point, and thanks for not adding background music. Very helpful video.
You are welcome! I create my videos as if they were made for me ;-)
Thank you guy with the Swiss accent. This exactly what I’m in the middle of trying to set up
Now it should become easier...
Andreas, don't ever stop making these videos. They are brilliant.
Thank you. So far I have no plans to stop...
Thanks Andreas, I appreciate your brevity and links to work that’s already documented.
My pleasure!
Wow! Such powerful information in just 12 Min. GREAT!
Glad you liked it!
Will come in very handy when Google starts counting the data from the upload of photos on the 1st of June 2021.
This is new to me. Thanks for the alert.
Is it real?
@@FikiFirmansyah Multiple sources and I got an email. Google .. google photos 1 June 2021
@@FikiFirmansyah Yes, asli
this is also new to me, its getting a crazy world of greed
Finally got this to work after 3 attempts (probably because I missed JoFie
1's comments from 1 month ago), I started with the new menu which did not succeed but using the old menu then adding Wireguard with the docker-compose settings it works fine now (even on a pi3 with just a 16Gb SD card). This is absolutely brilliant. I had several holes in my router before (for my external access) but now just one and with Wireguard encryption it is well worth the effort to set it up. Thank you for showing us how to do this, I love it.
Happy New Year
Arthur
Good to know that it worked. I think the new menu still has some issues and will take some time. Happy new Year to you, too!
I watch this video about 6 times over. Cause iam learning about dmr , VoIP, zoiper and echo link for ham radio, this is along the same Avenue as ham radio and device communication, as to weather or not you want your content to be accessible by the big guys like google and microsoft or if you want a secure acess tunnel aka portal. That only select devices or peers can acess. Agin iam no expert, I really enjoy watching you videos. This is right in line with ham radio and internet platform radio to radio connections via the internet and routing software such as pi star or braidmeister. I probabley got some things wrong. But I really enjoy these videos. It gives me a better understanding as to how devices communicate. Cause most of the world uses computers and sdr phones aka androids and iPhones. But most people have no idea what is going on underneath there touch screens, anyways I really enjoy your videos.
I often listen on the Redit DMR Talkgroup (HB9BLA)
@@AndreasSpiess cool. I use to talk on the 147.435 repeater on Santiago peak in california witch was a naughty repeater. Now that I live in kingman az. Iam waiting for my first dmr radio from bridge comm, even though I have been a ham sence 2006. Cause there was a plethora of repeaters in california. Iam learning as much as I can about sdr/rtl and other radio platforms and I have a long long way to go. Just so I can get back on the winsystem or the papa system. Or link up to one of the many dmr talk groups , I also want to build an allstar VoIP witch i know is older analog but i like the clear sound of analog. Iam taking notes on hex codes decoded through frame rates and etc and learning about emmerse- satsatellites. Agin i have a lot of note taking to go, memorizing and learning to go before I feel confident about my capabilities.
@@AndreasSpiess I just realized that was your callsign. I looked you up on qrz.com. iam such a dunce.
@@AndreasSpiess iam waiting on my new callsign to show up in the fcc url. So I can get a dmr I'd. My old callsign was ki6gvx. I let my licence elapse. So I had to retake the element 2.
DMR and Brandmeister currently are a good way to go. With an Anytone radio or so. I use a Hotspot in my basement.
Danke!
Thank you for your support and welcome to the channel!
'...in like Flynn'. Just for a second there I thought I was listening to Dave. Thanks Andreas, educational as usual!
Dave was one of the first TH-camrs I watched, you are right!
@@AndreasSpiess Yeah, and me! I like the sense of community, hearing TH-camrs mention each other; yourself, BigClive, Dave, Jonathan Oxer from Superhouse (and some I've forgotten to mention) - you've all been my mentors and helped me with a lot of Home Automation (and other) stuff. Thanks guys!!
I'm from Venezuela and I understand you very well. Good English and good videos.
And I don't know English very well
It even should have Spanish subtitles.
@@AndreasSpiess yes. I did not know
Thanks for sharing details of this project. Although I won't use a Raspberry Pi for this, I do have an old PC I can re-use. This was a very inspiring video that will help me a lot share files between my work and home computers, store photos I don't want to send to google etc.!
It also should work for a PC.
Thank you very much, but I think there is an error in compose-override.yml the volumes: entry in nextcloud should be /dev/Disk1:/Disk1, no?
I am a Linux noob. It worked for me. But feel free to use a better command.
Oh it will work like this, but the data is not saved on the external drive. At least when I tried it like that, this was not the case.
Cheers
Oliver
Hi Andreas,
the excellent content as usual.
One security hint, I hope you already discarded your QR codes presented on You Tube.
You are right I deleted the whole installation on the Pi ;-) And the DuckDNS domain is also free again for somebody else.
Hi Andreas, very good content as usual, I followed your instructions and everything went well. Thank you for this wonderful video...
Glad it worked. Thanks for the feedback!
For syncronizing between rpi's and other devices, syncthing is a great piece of software. Then, a question: are there any chance of getting an implementation of wireguard on ESP32? It is supposed to be quite small compared to other VPN's, so why not?
Thanks for the link to syncthing! Concerning the ESP32: I prefer to secure the net where my ESP32s are connected to. I do not think the whole key handling and stuff is simple for a IOT device.
2021 WireGuard-ESP32
Communication
WireGuard implementation for Arduino ESP32
Author: Kenta Ida
Maintainer: Kenta Ida
@@AndreasSpiess -i just like simplicity, not associated with them- regarding getting a useable public connection its best to use an *unlimited data* ,fixed speed vpn with their generic debian10 +docker instance like ovhcloud vps
'1 vCore,2 GB ,40 GB SSD NVMe ,250 Mbps *unmetered* for cheap rather than any of those American limited fixed data @ variable speeds corps.
you can docker your remote WireGuard server there & just connect to/from your local WireGuard as you please + other dockers over there too. [yt keep deleting this post]
Nice. With a raid 0 the solution would be complete - maybe the external enclosure used provides hardware raid 0 ? Also what would have been interesting would be some figures regarding transfer rate to see how it compares with much pricier options.
I had the transfer rate test in an earlier video about SSD boot.
Exciting stuff Andreas, well done!
Glad you think so!
Very nice video again, as I'm used to from you, Andreas :-) From a security perspective wireguard should be running on a dedicated pi with a hardened OS. This can be a pi2 or pi3, even a pi0. Having more softwares than wireguard on the pi increases the attack surface for attackers and it increases the risk of a breach by zero day vulnerabilities. This is increased by the fact that many ISP provided routers don't have the option to forward just one port, but only an option for a DMZ.
You are for sure right about the number of software running on the VPN server. Of course you can run the Wireguard machine in the DMZ if you know how to set it up. If I believe some stories (cameras, etc.) what people do, the solution presented here is already quite safe compared to just opening ports.
I also ask myself: Why would somebody attach an ordinary household with a zero-day vulnerability. I would assume there are more rewarding targets.
My standard router seems to have the possibility to only forward one port. I also read "DMZ" somewhere. Maybe I investigate a little in this direction.
@@AndreasSpiess I agree that this solution is relatively safe. The threat is real though. Cyber criminals are just looking for resources. If your devices are compromised they can become part of a botnet which are often used to hold companies for ransom with DDOS attacks. Also, it is not someone targeting you. Cyber criminals use automation to find vulnerabilities around the world, for various purposes. Think of it as scouts, software bots that go looking for openings. Depending on what is found and reported back, specific actions will be taken. No human is directly involved. Zero days simply mean it is not a matter if you will be compromised, but when.
I have been running Nextcloud on Pi for 4 years now. First on 3 and now on 4. To make it visible to Internet I bought a domain name and DynDNS-service from No-IP , which costs me around 30$ a year. From Let's Encrypt I got the SSL certificate for free. 2016 there were no free DynDNS services that supported also Let's Encrypt. At least I didn't find any. The data is stored on external usb double dock with 2 drives and raid1. System has been working without problems. With Pi 3 the max. speed through internet was around 10-12 MB/s, with Pi 4 it is double that.
Interesting. I have domain name with a webhoster. How do I have to go from there?
@@AndreasSpiess To run a internet server at home your ISP and your router must support port forwarding. Especially ports 80 and 443.
To run your own server at home with your own domain name, you'll need a dynamic DNS service. Your ISP changes your home IP address from time to time so you need dynamic dns service to keep track of your home IP address. DynDNS service provider will direct the internet trafic intended to your server to your current home IP address.
No-IP is one of them but there are others too. They offer also a service to turn your domain name from your current web hotel to the dynamic dns providers servers so you can run your own server with your own domain name at home. Your server must be running the dynamic dns service provider's client program to keep their servers up to date with your current IP address at home.
To get the Let's Encrypt SSL sertificate you must registrate your domain name to them, then load certbot program to your linux machine and run it. Of course before you can install the cerficate you must have a working http server running on your computer. I really recommend using Apache but there are instruction for other servers also.
So the order of things to run your own Nextcloud server at home but also visible to internet
1. get a domain name or prepare to transfer your current domain name from web hotel to dyndns provider's servers. Check your current contracts about the web hotel and domain name.
So that there are no nasty or expensive suprises from the web hotel when transferring the domain name.
2. choose a dynamic dns provider
. No-IP as mine dyndns provider was more or less a random process. Today I would prefer a European provider. But I'm too lazy to change mine while it's working.
3. check that your chosen dyndns provider supports Let's Ecrypt. And vice versa. That Let's Encrypt supports chosen dyndns.
4. choose another dyndns provider until the terms are satisfied...
5. make a fresh linux installation
6. make your router settings. Port forward 80 and 443 to your newly installed linux computer. Set in the router a static local IP address to your linux computer's MAC address
7. install apache, php with needed additions and dyndns client program. Do not install Nextcloud at this point
8. registrate to Let's Encrypt
9. install certbot and run it with Let's Encrypt credentials to get a valid SSL certificate
10. backup your SSL certificate files to a safe place and media
11. install Nextcloud
12. start Nextcloud for the first time and make the settings. And fix the things it is complaining about.
When you have everything running properly, for extra safety you can change your web servers ports to 5 number ports. If you do that, certbot won't be able to renew itself because it uses the defaults port numbers, so you must change the ports back to normal numbers in every second month to renew the certificate manually. For that I have two batch files. First to return the normal port numbers and reboot computer. Second to renew the certificate and change the ports back to long numbers and reboot. Not really a method for production environment but for me its working good enough. Unwanted visits to my http server ended. You must also make port forwards in the router for those long number ports. In Nextcloud client programs you give your server address followed with :portnumber (for example www.myownnextcloud.com:54321)
For maintenance I have also a ssh connection to the Nextcloud server. Ssh is configured so that the port number is changed to a long number and you can login only with ssh keys. Also that port is forwarded in the router. But don't do that if you don't have to do maintenance over internet.
PS. Four years ago it was not possible to get a free dyndns domain name that also works with Let's Encrypt. Maybe the situation is changed now. Someone else will know it better than me.
If that is possible you can have a new domain name for your nextcloud server and keep the old one where it is.
Thank you very much for your writeup. It seems to be not easy, but possible...
@@AndreasSpiess Getting a Let's Encrypt's SSL certificate to work with a free dyndns domain name started to intrique me and I started digging the matter.
Nowadays there are several free dyndns providers which domain names works with Let's Encrypt. So I decided to test it with a domain name from duckdns.org.
And it worked. Easily. If you have your http server running properly and it is installed to use the domain name you have got from duckdns.org, you just load certbot and run it. It asks your e-mail address and your domain name and installs the certificate. More instructions of installing certbot from certbot.eff.org
You have to run certbot once in every second month to renew the certificate. For installing and renewing the certificate you must have port 80 forwarded from the router to your server. For normal https trafic you must have port 443 forwarded.
More instructions how to install duckdns update client from here www.duckdns.org/install.jsp
Thanks for the clarification. After your and a few other comments I decided to not open my home network. It seems too dangerous for me because of my hobby. I fear somebody would want to prove he is better than me ;-)
On NC, there is an icon of a link, so you can share certain files (and maybe folders) with others. I think the Collabora function may work in a larger scale way.
Maybe. I decided to stay in the cloud with my collaborations because of security.
Sadly I am unable to connect to the VPN. I dont know what is the problem their. I also weren't ask to configure the DB type like in 7:56
Maybe you go to the discord channel of the project. There are the specialists
If you own a domain and can port it to NameCheap.com as your registrar they provide scripts that you can run to update your A record. So anytime your ISP changes the IP you get it updated so that your domain always comes to your home.
Thank you for the tip!
Very nice bundle of useful information. Thanks.
You are welcome!
Thank you for this video. I have installed the containers for NodeRed, Grafana, InfluxDB ( after that the wifi doesn't work at all except when I restart the dhcpcd service and it crashes due to errors ). Also I had a backup from the containers on my google drive ( when the Pi's Wifi crashed, I tried to reinstall the OS and then restore all the containers and it doesn't work for me ). Could you help me ?
Maybe you go to discord for support?
@@AndreasSpiess Support for IOTStack ? or your discord channel ?
The discord of IOTstack
Just use WebDAV on the Pis for connecting them to the Nextcloud Instance. For external access, you can forward Port 80 and 433 of the router to the Nextcloud-Pi and when you got external Access you can just create file sharing links for each file ^^
Thank you for the info!
Andreas, I am a 68 year old nube. I have been trying to teach myself electronic since last winter.. I do know a little about computers having played with them since the 70's with the tandy's and later the commodore.. But have not played with then since the late 90's.. all the kids got better then me so I didn't need to.. But now I am trying to get back into it all and find that even in computers I am a lost nube.. I have saved this in my project file and wish to do it this winter.. I do not have the new pi 4 but have several pi 3 b's Will this all work with them or is it a pi 4 I need.. I have an ssd set up on one of my pi's with movies on it for travel.. But things have gotten so far from me it is like learning all over.. thank my brother.. you channel is always fun to watch and a great teaching tool for me.. Carry on!
The Pi3 should work, too. But it has only the slow USB2. So an SSD does not help a lot. And the Pi4 also has more memory which also helps with speed.
I am also 64, so no big difference ;-)
@@AndreasSpiess Well one big difference.. I never owned a soldering station until i was 67 and never ever tried to build something electronic in my life until this year.. But I have played with computers..
This is just what ive been looking for
:-)
Every time when I did a docker-compose down my NextCloud ended up with an internal server error. Seems that in the compose-override for me a - ./volumes/nextcloud_db/config:/config folder was missing at the nextcloud_db service. After adding this line NextCloud kept his config after down the servers and bringing them up again. I was wondering if more people experienced this problem or that I did something wrong at 1st setup?
Maybe you go to their discord channel for support?
Constantly amazed how well Docker streamlines things. I tried it shortly after it came out but it was too finicky for me. Perhaps time to revisit!
Docker is only the infrastructure. What people prepare for others is also very important.
@@AndreasSpiess Indeed that. Fortunately the open source community has gotten much better about it in the last few years. Chalk it up to corporate sponsors, perhaps, but I'll take it.
How large does the ‘system’ SSD needs to be? And is it possible to use an 32gb micro SDHC for that purpose?
you can use an SD card but they run a lot slower than an SSD. I just upgraded my pi4 to an SSD and got a 7x benchmark improvement over the SD as the system disk. it is much more responsive now.
@@gngdunn thanks for the answer, that does make the decision easy
Hi Andreas, thank you for the video.
After installing all the container once I start the nextcloud and add the required password it says "Error while trying to create admin user :failed to connect to the database" can you please help
Maybe you go to their discord channel for support. The link is in the description
Around 2004, when a 128MByte (Mega, not Giga) USAB-Stick was a huge one, I started my VPN experience using "Unslung"-FW and OpenVPN. Some years later, I was happy to migrate to the stock VPN of my router. Today the most routers provide VPN out of the box. No port forwarding, no additional raspi, no hazzle. And I have access to my complete home network incl. my ioBroker data set, Synology NAS und much more. Appropriate for a private person. Do the Swiss routers not provide VPN?
Many iot systems as e.g. ioBroker also offer cloud access and provide "European" cloud solutions. But up to now I prefer my home network with VPN access and try to avoid cloud solutions.
My router provided by the ISP does not offer VPN. But I cannot speak of all other ISPs here. Using the router is of course also a good way to get secure access. For sure it is easier.
As you mention, I also wanted to avoid the cloud. But for the moment it seems it is easier to keep it for file sharing.
You rock Andreas! Super interesting and useful. Did you know that there is a duck dns container. Maybe more elegant then crontab. Also nice to add swag container for reverse proxy
Thanks for the tip. The team on discord always do some improvements. Maybe they will add also the duckdns container.
Is there a list of alternate time zones for compose-override? Is it the standard wpa supplicant ones?
I do not know. Please ask the developers on discord.
couple of things: 1. webdav sync 2. accessibility through the internet 3. nextcloud performance 4. big files:
You don't really need a raspberry pi client, you can use curl to do file operations or even create a linux webdav mount (docs.nextcloud.com/server/16/user_manual/files/access_webdav.html). Locally, you can (php occ mainentance:mode --on) manipulate files directly into nextclouds data//files directory and run php occ files:scan to scan for these changes (php occ mainentance:mode --off). The analytics plugin creates RESt api endpoints for you to upload data from, for example, iot devices like weather stations, detailed instructions on uploading via api are here: github.com/rello/analytics/wiki/API.
I imagine you have already tried this but: to connect your pi to the internet you just need one change: port-forward the appropriate port on your router and then access it using http(s)://: on your browser or sync client.
I personally run nextcloud directly on linux with nginx + php-fpm and only have a SD card and HDD. To increase performance, I mounted nextcloud directly onto the HDD using /etc/fstab. To get more performance, add php opcache if not already done. On my instance website loading speeds went from multiple seconds down to a second or less. For more info on server tuning, see docs.nextcloud.com/server/latest/admin_manual/installation/server_tuning.html. Additionally, check how nextcloud is running it's cron.php (settings -> basic settings). If it is using ajax, loading times might be affected depending on whether or not the cron needs to do something. I'd switch to crontab where possible.
To be able to upload bigger files (>100's MB), I needed to increase php's max execution time. The nextcloud pc sync client will upload the file in chunks of approx. 10M and then re-combine the file in one command. The recombining step was taking so long on my pi that php was timing out.
Thank you for your tips! A few of the tips are probably more for the builder of the NextCloud container. At least I would expect that they optimize the setup.
The speed with my Pi4 and an SSD was ok for me. I can imagine that the USB2 of the older models can create a bottleneck.
Another Great job you have helped me out so much. Many many thanks
Glad it helped!
hi, in 11:28, as I understand, you ask for a somethig to syncronize several Rasp... I think tha "syncthing" may do the trick...
Thank you for the tip!
Waw, a very amazing and interesting video. Some of the parts like D.DNS I was already familiar with. But everything else is very nice to start tinkering !
It is really interesting what can be built with this cheap technology!
3:52 Do I really need 'SSD Boot enabled' on RPi? Can I just use a good SD Card for boot/OS - and separate to this - a large SSD over USB 3.0 for storage??
You can do that of course if you intend to use a second disk. No problem. I just would not use it for data storage.
Thank you very much ! Have a good day !
You are welcome!
Adieu! (in swiss french) For internet access of your nextcloud, you should user jlesage/nginx-proxy-manager. Then in next cloud, you should able to share folder like dropbox. Hope it's help
Is there an online guide to which you could point us for that?
@@PilotAtInception th-cam.com/video/bQdqf5xAyUk/w-d-xo.html
@@dgeordgy21 thanks! I missed that one!
I got an issue with USB booting from an external ssd: every time I reboot using ssh, the pi doesn‘t come online again and I have to reboot it manually. During this time, the lan leds don‘t blink. Any suggestions?
There is a link to a page with a disk compatibility list in one of my SSD boot videos.
Thanks for the helpful video and special thanks for the subtitles.
My pleasure!
Merci vielmals! Das Video ist sehr gut verständlich und hat mich echt weitergebracht!
Sehr gut. Danke für das Feedback.
I have noticed wiregaured is no longer available with IOTstack - what alternative option is there
I was not aware of this fact. Maybe you ask on their discord channel?
Very good video as always, Andreas. I've similar setup, but I run on DockSTARTer. Better support and easy to maintain in my opinion. Also Wireguard is blazing fast and secure. Love QR code feature. Take care.
Never heard of dockstar. Thanks for the link. Looks very similar, you are right. It looks like it is more flexible and less predefined.
There is a container for duckdns, everything is inside docker and you do not need the cron job
Good to know. Thanks!
Fantastic video
Thank you!
I can't find wireguard in the IOTstack menu anymore to create the container. I am stuck
They have a discord channel for support.
Super verständlichi clips! Wiiter so!
Danke!
If you don't want to expose your RPi to the whole internet there's another possibility: TailScale. This is a free service that manages Wireguard for you and also traverses NAT/router boundaries transparently. I have it installed on my RPi as well as my phone and laptop, so I have a multi-device VPN that works everywhere. You can also configure IP routing on the RPi so that you expose all other devices in your home network to the VPN. This is how I can access my home NAS from my devices. Quite nice and easy to setup.
Looks interesting! It probably solves problems I do not have with my home network. But for companies, this seems to be cool. But you do not have to go through any hassle.
One thing I probably would not like is the single user in the free plan, even if you can use several devices. But maybe this limitation does not apply if my wife also uses the same user.
@@AndreasSpiess Yes the single user requirement is a bit cumbersome. I use a dedicated Google account that is shared across devices.
Using Tailscale for everything - I've got a dozen computers, phones, raspberries etc. Works very well and allows not to expose a single port to the internet! Even my hosted machine doesn't have a single port open, even for ssh - Tailscale allows it.
Hi is this also possible with a IPv6 adress? if so is there a written guide or video for this?
- I do not know if IP V6 will work
- Maybe you look at the links in the description?
Hey everyone, any ideas on why I am not getting the peer folders in the Wireguard volumes folder? Same override config for wireguard as Andreas. I do have an older version of the IOTstack from a few months ago and will try updating tomorrow. Cheers from Aus.
I used the newest version from Github.
Thanks Andreas... how about using syncthing to synchronize your files across RPi? I have it running on containers on all my machines....
I never tried it, but quite a few viewers suggested the same. So I think, I have to try it once...
How do I get wireguard to generate a new QR code for mobile app if I've put this pi on a different network
I do not remember. The project has a discord channel for support. The link is in the description.
Very interesting topic. Thanks for video
You are welcome!
Las week I was thinking about it ... maybe google drive new terms and conditions have helped us to think the same.
Good video in the best channel ....
Thank you for your nice words!
Excellent tutorial Andreas! I followed it through almost to the end, but fell at the last post when installing Wireguard on my Android mobile! At 'Connection request' selecting OK does nothing and when cancelled results in "Error bringing up tunnel: VPN service not authorized by user", I've downloaded the log, but it doesn't mean much to me at the moment. Jim
Maybe you go to the Discord channel of the project for support.
The problem turned out to be the particular 'phone. I was using an Xioami Redmi Note 4. I then dug out an old Motorola Model G from 2014 and Wireguard works fine. Many thanks for the tutorial Andreas!
Hi. I am using nginx proxy manager with cloudflare. Could I just open that one port?
Hi i have a doubt i did the wire guard setup, the containers are running and i did port forward also but when i check with port checker its showing the port is closed. and i could'nt able to connect to internet via vpn
So after stopping the container, I tried running a simple hellow world server using node on the same port and its working from duckdns and port checker also showing the same port(51920) is open.
so what am i missing over there in the docker wiregaurd config? or is it somethingelse.
please do respond, thanks.
Maybe you go to their discord channel for support?
Sharing with others, speaking outside of your home network is a difficult task. Especially if the whole purpose was in first place, to keep your storage private. I guess google drive or dropbox is a welcome solution to that on a file per file basis.
My QNAP NAS offers a good mix of both: Local, private storage with offering the option of sharing a file or a whole folder with others like a dropbox. It’s a very good solution but properitary to QNAP NAS’ses (i guess Synology has that as well)
You are right. I decided to continue to use dropbox and google for that purpose. As a TH-camr I feel a little less secure :-(
How do I gain access to the other computers in my Lan through wireguard? I read there is a setting to route traffic, but i am not sure where to set that.
In my case I had access to other computers without doing anything.
You don't need to "install a Raspberry Pi client", it's usually already supported. The keyword is WebDAV and all you need is to connect the RPi to the NextCloud server using WebDAV. It's also supported on Windows.
rsync
@Michael: Does webdav synchronize my Raspberry directories with Nextcloud?
@Tim: So I would use rsync to synchronize my "Nextcloud" directories on my Pi with Nextcloud?
@@AndreasSpiess think webdav as something like smb, except that it also works over the net
@@AndreasSpiess have a look at Syncthing
WebDAV has been really hit or miss for me in the past. Reading a directory with lots of images will time out on the client, and trying to upload large files will also fail. Then you get into weeds trying to configure the Apache proxy if that's the NC image you used, or the PHP.ini files. Then the problem might be the WebDAV client is not fully compatible. I really wanted all devices on my home network to use WebDAV with NC but in the end, used SMB shares to a mounted HDD. I do have NC but use it for syncing images from all smartphones. Using the iOS and Android apps. The windows client app is annoying as it tries to sync a local PC folder with NC so if you assign it to your images folder, it will back then all up to an images folder on NC - great - but then start deleting those images on windows and the client software will delete them on NC too. Which is fine if there was some options to change the behaviour, but there isn't.
I use duck DNS to port forward to my NC docker image so I can access NC from anywhere for the images to upload.
Using let's encrypt to manage SSL automatically
Hi Andreas
Is there an easy way to install 'Certbot'? It looks like it can be installed via docker, but I know very little about docker and portainer and am frightened of breaking my system.
Jim
Maybe you ask the question in the Discord channel. There are the specialist.
Thank you very much! Greetings from Austria! :)
You are welcome!
Hi, I cant seem to find wireguard in the menu. was it removed?
I do not know. Maybe you ask in their discord channel.
No QR Codes in my install. Everything else was spot on. Now trying to figure out how to get Wireguard to create them.
There is a Discord channel for support.
So what if my SSD breaks? Can I access the encrypted data on my external storage (Disk1) by somehow setting up the system again on a new SSD?
Of course one could always back up the system-image but maybe there is a more elegant way
Maybe you watch my other video about the topic where I explain the backup strategy of the project?
Not sure if this has been answered, you can easily share files like dropbox or google drive with others by just providing link, you can even set password and expiration date for it, the user doesnt need an account to download the files, similarly with next cloud you can create an upload link (secure drop) where user can upload files. Please also try out syncthing its much easier to setup than NC
I use sharing on dropbox and google and, as you describe, it works fine. I will have a look at Syncthing
@@AndreasSpiess sorry I think I didnt make it clear in my previous reply, I meant just like Dropbox and Google drive you can share files in Next cloud. You can do it from the webinterface and through the next cloud client as well. I switched from Google drive and Dropbox a long time ago to Next Cloud.
Hi,
Can you also upload the data transfer rate from outside your local network to your local network?
This depends on the contract you have with your internet provider.
@@AndreasSpiess I have gigbit internet on VPS side and 300 MBPS upload and download speed at home. Still when i test the tranfer speed with iperf3 it shows max 110 mbits/second. Do you know what might be the problem to this?
mariadb doesn't support ARM V7, which means the mariadb cannot be executed properly in the docker of a raspberry pi. How did you make the nextcloud work?
It worked with me and AFAIK it still is part of the IOTstack.org for the Pi
Why is it that, when I start my containers, it kills my network interface on the RP4 and how can I prevent this?
Dear Andreas,
I watched your perfect videos every now and then for a year now, and you are definitely the best that I could find during many many queries.
Regarding pi4 and nextcloud, I would have a desire:
run a raid10 (2 disks + 2 disks, as you may know much better than me) so to take advantage of the 4 usb ports and, well, so to keep all the data in my hands only.
I am keen to use also 4 flash drives, in case that this would help for the electric current draw.
Would you be able to "teach us" how to do that, as an expansion and in conjunction to this perfect project of yours?
I am no specialist on raid systems. So somebody else has to do that. Pls. keep in mind that the Pi4 only has 2 USB3 connectors.
@@AndreasSpiess Thank you.
I have read that there is a software called "mdadm" that helps in defining the disks, but I would find myself today unable to "add" that part into your project. It steps into "defining labels" so to recognise the logical disks and the physical disks.
I don't even know how to use dockers, for the moment. For the speed I would love to imagine that even having usb 2.0 would make me satisfied for a very modest home user only.
Most importantly, thank you again.
Webdav can be used to share files from nextcloud
Thanks for the info!
I've added the wireguard lines to my compose-override.yml file but when I start menu.sh I can't see Wireguard in the stack list and it's not installing. Am I missing something?
You have to select it in menu. Last line.
@@AndreasSpiess I found the issue. I foolishly tried to use a prior Docker install instead of installing it via IOTStack as mentioned in the official documentation. I reinstalled docker by using IOTStack's menu.sh and everything works now.
Hi andreas,
i use another container called swag (linuxserver/swag) in my stack for accessing from the outside. it works as cert endpoint with letsencrypt and reverse proxy. dns cnames (dyn IP @home) do the rest. let me know if you need more information! the configuration is verry simple
Maybe it would be interesting to add it to the IOTstack project?
Thanks Andreas for this video that shows better how to combine iostack, nextcloud and show the details how to make it working.
I begun last week the configuration of my raspberry 4 based on your videos 295 and 352 but I got problem with usb boot when installing wire guard .
3 questions
1) Do you use the last stable firmware ( November) for USB boot or critical version of September ?
2)Is wireguard running in a container or is it installed on the host from PI VPN project?
3) what about power supply regarding RPI when connecting one SSD and one USB HDD together? Do you use external power supply for HDD?
They seem to have changed the EEPROM quite a lot and recently created some problems. I used the version which was actual during the making of the video and used the script to install Nextcloud. Maybe you try to get a stable boot situation without any IOTstack first to separate the two problems. And go to the Discord channel if you need additional support
Superb video! I have an Older version of portainer installed. Can I select the new one or should I remove the old one first?
You do not need to remove containers. If you talk about Portainer and portainer-CE: They use different ports. Usually, you only have to select one.
@@AndreasSpiess I removed and freed port 9000 so i could use it for portainer CE; works! Thanks!!
some internet providers will assign cg nat ips (the same public ip is shared between several customers), making port forwarding not possible :( in such cases you can try to contact them to assign your connection to a regular ipv4 ip schema.
exactly. I wished the Author would have mentioned this In the video aswell. At least my provider in Switzerland does exactly that. Or if you are trying to setup a VPN on a RiP that is connected via 4G to the Internet.
2:45 small correction - "This is why we need a service called *dynamic* DNS", where Duck DNS is just one provider (very useful, make no mistake!). Also note that port forwarding will not work if your (cheap) ISP uses a "carrier-grade NAT" and your "external" IP is not actually reachable from internet.
Swisscom also does CGNAT. And it is not a cheap ISP
You are right for DuckDNS. But my videos have to be simple. The guys with more knowledge like you anyway do what they want ;-)
So essentially if my mobile operator does CGNAT this thing won't work? That's a real bummer
@fabio foltran There are ways to make it work, but they are not completely free. I use a cheap VPS with a public IP which acts as the main wireguard "hub" connecting my services and devices. Not super complicated, but as Andeas pointed out, not super easy either.
@@MichalKottman yeah in fact it's already out of what I know ...I'm not really good in networking XD
Thank you for your videos!
Glad you like them!
I'm new to Linux and Raspberry Pi. What's the point of using an SSD to store files instead of a usual microSD card? apart from a higher speed? are the system files too large for the microSD card?
SD cards sometimes get corrupted because most of them are not made for this application.
@@AndreasSpiess Makes sense, I often hear corrupted microSD card is a problem for a small single board system like this. How much minimum SSD storage would you recommend for this project?
@@syvanonhboualaphanh1019 Any SSD will go. This project only needs MB, not GB ;-)
@@AndreasSpiess Thanks a lot, will try this out, as well as your other projects.
I think perhaps it is a mistake to use the latest of a bunch of images because it seems to have bit-rotted already. Any chance you could list version tags for the images that worked with the compose-override.yml published?
Maybe you ask for help on the Discord channel. We want that the project also runs with the newest versions. But keep in mind. The project changed a lot recently.
@@AndreasSpiess I just want it to work as it did in the video. I followed it but I get:
Error while trying to create admin user: Failed to connect to the database: An exception occurred in driver: SQLSTATE[HY000] [1045] Access denied for user 'nextcloud'@'nextcloud.IOTstack_NextCloud' (using password: YES)
I see lots of people get the same, so I think something must have changed. I didn't change any of the passwords in compose-override.yml. I see it is has been merged correctly into docker-compose.yml and I give the mySQL_password to Nextcloud but it can't connect to the database.
I have tried Discord but the reply wasn't helpful, the suggestion was I need to configure mariadb to accept nextcloud or change nextcloud to match mariadb but compose-override.yml configures them both to match, so I have no idea why it doesn't work for me but it worked for you.
I am having the same problem. Either something is wrong with the database or something has changed significantly. @nophead were you able to find a solution?
@@kafadek825 I did, but it was 2 years ago on a friends RPI and I can't remember exactly how. In my notes from the time I wrote "Correct the MYSQL_PASSWORD in docker-compose.yml" after the Build Stack and before the starting the stack.
@@nophead thanks so much. I will give that a try. Appreciate your going back to check for me
If you want to share outside your network you can use Hamachi, i have the same configuration but with owncloud, and I can share my files with other people outside my network
They do not need a VPN tunnel? Just a link as with google or dropbox?
@@AndreasSpiess yes, they only need the link like dropbox
Andreas thanks a lot from Mexico, but I have a question!, Could I use this setup installation as a NAS and server phone system? Any suggestions o options! I appreciate your comments!
Raspberry Pi is not very fast. But they are Linux systems and you can do what you can do with other Linux systems.
Hi Andreas. I really like your videos as they present real-life solution. For me, however, the instructions in this video do not work. The resulting compose file fails to create the NextCloud docker and shows errors. I only was able to create the portainer-ce container. Not sure how to share the yml files resulting from the script. I will try with the developer. Thanks again for all your work.
There is a link in the description to a Discord server where you should get help.
Hey mate, I followed your guide for wireguard setup. It worked flawlessly but one day later it said could not establish handshake. The only change I made to the IOTstack was to add pihole and then I changed my wifi dns on my laptop to connect to pihole. As soon as i did that wireguard stopped responding and even after changing my dns back to normal it still is not running. Do you know what may be causing this ?
Maybe you go to their discord channel for support.
Im recently installed NextCloudPi on a raspberry pi 3, and although its slow due to the slow ethernet and usb interface, its awesome. I did not use docker or wireguard im wondering if theres an advantage to using docker over a normal install. Also with NextCloudPi im able to share files with people outside the network. Im going to get a raspberry pi 4 and deploy it like you did since it seems more secured with wireguard
I like docker because the install of the software including dependencies in the containers is done by knowledgeable people and you easy can get the newest version. In addition you can restart and shutdown each container in seconds. And if you delete it it is gone with all its dependencies.
@@AndreasSpiess docker seems awesome, im just very new to the concept of containers
Fantastic! Thank you. One question. If I understood this correctly, there is no client for Raspberry Pi. I have a Pi 3 running Node Red which runs my home automation with a mix of Sonoffs and Shellys all flashed with Tasmota. Does this mean, I cannot use my existing User Interface on my phone to tunnel into my Home Automation Raspberry Pi?
You should see your node-red page on your mobile if you created a Wireguard tunnel.
@@AndreasSpiess Thank you very much for the prompt reply.
Is it absolutely necessary to attach an SSD drive? This setup cannot run off a regular SD card that Pi's take? Hm
It works with an SD card, but much slower.
@@AndreasSpiess Ok. Good to know.
I found that duck.sh got "502 bad gateway" more often than not from duckdns.org when run from crontab but when I ran it from the command line it always returned OK. I realised that everybody is configuring their RPI to hit duckdns every 5 minutes, on the 5 minute boundary, so all at the same time. Changing the crontab setting to every 5 minutes starting at 1 solved it for me. Seems like their server is being overloaded on the 5 minute boundaries by all the IOT devices.
We need a crontab option for random jitter!
Good tip. Thank you!
@@AndreasSpiess Adding sleep $[ ( $RANDOM % 60 ) + 1 ]s to the front of duck.sh should reduce collisions as well.
Haven't you considered odroid HC-4 for such kind of project?
I am a Linux noob and therefore I only use Raspberries...
Hi, nice video, thanks for sharing. Do you know Zerotier? See you
No, I never used it.
@Andreas Spiess Definately check it out. Wireguard is awesome but if all you want is to connect your devices and never think about hole-punching again, ZeroTier is way faster to set up. Each device (pi, pc, android, etc) gets a virtual adapter. Join. Authorise through their web site and assign a static IP to make life simple. Done. Direct connect from any connected device. Down side is that the free tier limited to 50 clients. Pretty fair.
With many ISP's now using CGNAT (double nat) poking holes through the router won't work for external access in those cases. Zerotier (free) is one solution there are others. I think all require a reverse proxy type approach.
@@nittygritty-q7t I have CGNAT ISP connection, how free is ZeroTier?
For the question how to share files with people outside your network: There are 2 obvious ways. One is to bring the guests into your network using Wireguard. I find this very inconvinient, since I don't wanna bring those guys into my private network. The other one is to open a tcp port on your router, to share the html of nextcloud to the outside. This is a security risk, but a small one. However, from an IT point of view, You would have to create kind of a safer network environment, with several stages like a DMZ, and I would never ever run Wireguard, Maria DB and Nextcloud on the same machine. Conclusion: tough decisions to make. For private installations, one has to check convinience, security and effort against each other. Nice video, I like the output provided by the project around IOT stack
That is what I thought, too. I hoped somebody has a "silver bullet"...
Solution 1 is obviously a no-go. And if I have to have a Pi in my DMZ I probably stay with Google or Dropbox. So it seem to be a good thing for a family or a small company.
@@AndreasSpiess I rent a server to run my own nextcloud. This puts my local network to low risk, but costs some bucks...
A good possibility. I hoped to be able to use it for community projects because the free google service does not allow for permission schemes. If I have to pay for NextCloud hosting I probalyy can also pay for a Google plan. What was your reasoning for Nextcloud on a paid server instead of cloud solution?
@@AndreasSpiess That private server runs several other stuff I need, so the server costs for nextcloud are very minor. And, it was an advantage in terms of privacy. I try to avoid cloud services when possible. Yes, me writing on TH-cam, I know.
Understood. I am very pragmatic with cloud services. Many problems can be easily solved with such services. I try to optimize my time. It gets more precious every year because I get closer to my death day...
Doesn't work, unfortunately( on raspberry pi2 docker install results on 2021-02-12 in a corrupted dounloaded package(
Strange. I never used a Pi2 for such complex work. Maybe you hit some memory or swap size limits?
@@AndreasSpiess likely, or something with it being 32 bit. I bought a used office pc with a 6th gen intel chip, doing much better progress on the installation, docker already up.
Hi Andreas, thank you for this video and the instructions.
I can mount an external HD in my PI4 and enable the external storage app in nextclound but fail to add the external HD with /Disk1 in nextcloud (get a red dot in front of the entry). I have installed smbclient eventhough I get this warning in nextcloud.
Any idea what the problem could be?
thank you in advance.
There is a discord server for support of this project. You find the link in the video description.
Thanks this is really very informative video, can we use a SSD for Pi OS and a HDD for nextcloud contents at the same time, does RPI4 gives enough power to operate both without issues?
HDDs often need too much power (and sometimes even 12 volts. Maybe you will need an additional adapter for the HDD