How to upload 10k NFTs to IPFS in about 1 hour
ฝัง
- เผยแพร่เมื่อ 5 ก.พ. 2022
- In this video I will show how to upload 10.000 NFT images (5.7GB of data) to IPFS in about 1 hour from an average home internet connection.
Referral link for DigitalOcean: m.do.co/c/bcc172152095
Twitter
/ marco_lipparini
Website
www.liarco.net - วิทยาศาสตร์และเทคโนโลยี
Thanks for the video Liarco, I’m getting an error when trying to install the nodejs npm, it starts off well then it gives me a bunch of errors and “failed to fetch”. Any reason this could be happening?
Hi Zishan, "failed to fetch" errors should be related to network connection issues. This might be just a temporary network issue on DigitalOcean's side (if it's happening during the package installation). Did you try multiple times or maybe waiting a bit between retries?
@@LiarcoDevTips Hey, I’m coming across the same issue right now. I guess I’ll just try again in an hour or so if it’s just a network issue? But is there a work around or any other thoughts you can share to get through this step?
Just figured it out. When it fails, at the end it says run “apt-get update”. Then run “apt install -y nodejs npm” again.
Hi Denni, thank you for taking time to follow up on this. It helps a lot!
Probably the default installation of the droplet now has outdated apt cache and that's why it fails fetching the data... so running "apt-get update" is required before installing the software. It's always a best-practice to run it once the machine is ready I'm sorry if I missed it in the video, my fault.
Thank you very much for sharing the solution!
yay!!! first video and first comment. I am so proud. Lot of love to you Marco. you are a genius
Hi Santhosh, happy to see you here and thank you very much for your support!
Thank you so much for your hard work. lots of love from srilanka
Thanks Liarco, please keep uploading coding tutorial content!
Hi Douglas, I will do my best. Thank you for your feedback ;)
Thank you very much Liarco, hope to see more videos from you soon. I was wondering, Pinata will also charge us if we pass the 1GB free option right? Also could you please make a video about provenance building and how to go about it like the BAYC? thank again man!
Hi Ricardo, thank you for your feedback!
First of all: you are right about Pinata, you will be charged $0.15 per GB of storage (price at current time) which to me is a fair amount for the given service. So depending on your collection size (and how much you can optimize it) you can have different costs.
As far as the "provenance hash" solution is concerned, I like it! I have to think a lot about it because, while it's not too hard build per se, it still requires changes in the whole workflow (collection and metadata build, contract code, etc.). And also, collections follow many new concepts nowadays compared to when BAYC launched so I think that kind of solution should be properly actualized a bit before we use it. (e.g. many collections now feel the need to reveal before selling out, that's something that you can't do with the BAYC contract: if you want to reveal earlier, you have to burn the remaining tokens).
Stay tuned, I might come up with something about this topic in future videos! ;)
Hi Marco, great video ! Let me know if you come to Milan for a coffee ;)
Hello @Liarco DevTips and thanks for this video. I'm a bit late to the party but am using your video to help me upload my first every NFT collection, but I'm having the same 'failed to fetch' issues as the guys below. I have can the "apt-get update" as terminal has prompted me, but then I get a purple 'Package configuration' screen that asks me which services should be restarted - this is way above my coding knowledge so any help would be greatly appreciated. Also, if I already have nodejs and npm installed from using the Hashlips Art Engine, do I actually need to do this step anyway?
OK - I've got past that bit OK now, but now I'm getting a "HTTPError: Response code 400 (Bad Request)" when I try to upload my folder - please help!
Ottimo 👍 Grazie mille 🙏
È un piacere, grazie per il feedback! :)
First of all, great video and amazing support! I have one issue with the last step, where I updated the file folder and had to run the command: pinata-cli -u foldername. It gives me an error and says: it doesn't exist even though the folder is visible and has all the images in it thats needed. Can you support on what to do? Many thanks!
Hi! It seems like folder is not in the current path in which you are running the command. You can use the "pwd" command in order to check the exact path in which your terminal is and "ls" in order to list all the files and folders available in the current path.
I think this might help you troubleshooting the issue. ;)
I have the same issue, did you solve it?
Thanks for the tutorial. All was going well until the end.... I had the 'Killed' message. So I resized the droplet. Even that would not work. Turns out Pinata has a 25GB upload limit, my images folder was 30.5Gb so i've contacted them for a custom solution. Fingers crossed. If anyone has any ideas how to overcome this 413 error please let me know. Tried uploading CID from IPFS desktop same problem.
it is good to upload a collection without metadata? just images?
Hi Echoboyart, you always have to upload both your assets (images) and metadata in order to have a complete collection. In my video I'm showing the upload process for the images since they are the ones that usually take a lot of time to upload because of their size (an average collection has 5-7 GB of images and just a few MB of metadata).
You can repeat the same process for your metadata or even use the web interface depending on how fast your internet connection is. I hope this answers your question.
Thank you! How i can get json of all CID’s inside folder?
Hi! The CID is just one for the whole folder, then you can append the filename (e.g. "ipfs://YOUR_FOLDER_CID/subfolder/filename.extension").
@@LiarcoDevTips thank you for response! thats why i am asking. this is what i am trying to get. i need each cid of the file in the folder.
for example folder/files
i have cidFolder/1.png
and person can remove 1.png ang get access to the whole folder and i dont need that. i am trying to find the way to get CIF of each file. for example to move them in json like 1 = cid of 1.png
2 = cid pf 2.png
@@LiarcoDevTips i m just found the way! thank you... ohh i needed to use IPFS LS cid of folder. thank you
@@VELIXYZ As far as I know that's not possible, by uploading the whole folder as a single resource you get just one CID. You have two options:
1) upload each file separately (I would not recommend this)
2) replace the filenames with something that cannot be guessed (like a hash)
If you are trying to hide your tokens from snipers, there is a tool that I developed that addresses that: github.com/hashlips-lab/safe-nft-metadata-provider
I'm also working on a new version of this tool that will be much easier and lightweight. My plan is to release it very soon (days 🤞).
@@VELIXYZ Thank you for letting me know, this is really interesting!
I'm still not 100% sure the original folder won't be exposed in any way by using CIDs of files in it, but I have to do some research on this. ;)
Hey I don't have meta data I created the 500 art from Google colab how can I uplode. I have any option to create the Mata data
Hi, I'm sorry but I don't know Google Colab, but if you already have your assets, then you can upload them in the exact same way.
If you need a way to build the metadata, then I would suggest watching some of the great tutorials by HashLips: th-cam.com/users/HashLipsNFTvideos
do i have to pay pinata monthly basic after the upload. Kindly reply
Hi, yes Pinata is a paid service, but it's extremely affordable and anyone can pin the CID of a specific resource and pay for that. This is a very cool concept because members of a community can decide to maintain the IPFS resources even if the project dies completely and nobody needs to have access to the original files, accounts or anything like that. The CID is enough to pin the resource and keep it alive.
Then bro how to transfer from binata to open sea or rarible ?
Hi, in order to publish a collection on an NFT marketplace you need to create the NFTs first, publishing to Pinata is not enough.
For example, you can take a look at this playlist which shows the complete process to publish an NFT collection using a smart-contract on the blockchain: th-cam.com/video/zjlg-0622PU/w-d-xo.html
Brother can we use command prompt to run the program
Hi! You can use any terminal you like: the built-in one, ora a third-party one (e.g. iTerm which I’m using in the video or even the one available on visual studio code)
Hi, when I write the command: pinata-cli -u (my Folder Name) it generates the error: it doesn't exist! what did I wrong?
Hi, you might be in the wrong directory. Please make sure you are in the right path (you can use “pwd” to get the current path you are in, or “ls” in order to list the files and folders that are available in the current path).
If you are inside the wrong directory you can use “cd INSERT_NEW_PATH_HERE” to move to another folder.
First off thank you, but you said do the same thing with your metadata, do the same what?
Hi Mark,
thank you for your feedback! When you generate your NFTs you generally end up having a folder for your assets (images, videos, etc.) and a folder containing the metadata files. Doing the same for the metadata means uploading all of your metadata files using the same exact steps as the images.
The whole process will be covered in a free workshop next week: twitter.com/Hashlipsnft/status/1499441531027087362?s=20&t=FvJL1j5EWioqqiD-mg5Jdw
After finishing the process trying to upload to pinata it says Killed can u help
Hi, that means that your Droplet doesn’t have enough RAM to process your collection, you should increase the plan.
Every time I put the last command it waits a minute then says killed ? What am I doing wrong ?
Hi, did you follow my suggestion about RAM? That error happens when your droplet doesn’t have enough RAM to process the whole folder. Please try upgrading the machine plan.
I hope this helps 😉
@@LiarcoDevTips where to upgrade machine plan? I can't do on my own
@@AnjarMoslem please take a look at this documentation page from DigitalOcean: docs.digitalocean.com/products/droplets/how-to/resize/
Let me know if it works! ;)
Just did it, need to deposit $30 to upgrade plan for 16GB RAM. Thank you sir you helped me a lot, just uploaded my 9GB files to pinata
great one but how i can upload to smart contract after that
Hi! Once you have the CIDs for your file you can follow this playlist to learn how to do create a smart contract and a dapp easily: ERC721 Collection 101
th-cam.com/play/PL5G6EQziKpIpMo9ZEgAG3GHBpfPeaF5Hr.html ;)
@@LiarcoDevTips i want to upload mint and list on Ethereum blockchain i want to know around how much i should pay fees?
@@yznzi7403 Using my tools the average fee for a collection deployment is between 0.1-0.15 ETH, mint fees are around 3-10 USD per transaction. These values may vary depending on the gas price and also depending on how many tokens you are minting at once.
How about metadata & baseUri ???
Hi what do you mean exactly? The video is simply showing an alternative way to upload your data. If you need to create or update your metadata you should follow some dedicated videos like the ones from my friend Daniel: th-cam.com/users/HashLipsNFTvideos
Hello, tell me please, I need a smart contract, where revealed NFT after purchase? Many thanks
Hi, I'm not sure I understand your question, but if you are asking how to reveal a collection after purchase, here are some tips:
1) if the collection is sold-out: set the revealed URI prefix to point to your IPFS folder ("setUriPrefix(...)" function on the smart contract that I released together with HashLips), then run the "setRevealed(...)" function passing "true" as value
2) if you want to reveal just the minted tokens and protect the unminted ones, then you can use my "Safe NFT metadata provider", here is a video on how to set it up: th-cam.com/video/fO5nT-TCIZs/w-d-xo.html
@@LiarcoDevTips yes, I need to open each nft separately, that is, if you bought nft, it will open, and if not bought, it will also be closed. I don't know how to do this, I watched a lot of videos, but there is no information about this anywhere. help me please
Well, option number 2 should be exactly what you are looking for. It’s a brand new solution we released a couple of days ago and it can be used free of charge 😉
If you need help with it, you can join HashLips’ Discord server, you will find a lot of people that can help you.
@@LiarcoDevTips Thank you
Is it possible that this process is completely free?
Hi, the cost related to Pinata cannot be avoided (or at least, I wouldn't suggest using cheaper options because you might encounter issues later), as far as DigitalOcean is concerned you can use my referral link in order to get 100$ free credit for 60 days: m.do.co/c/bcc172152095
That would allow to do this (and much more) for free.
1h ? It take like 5 min
Hi NFT2626, would you mind explaining what you mean? It really depends on the collection size, with high resolution collections (5-10GB) it's hard to uploaded everything in 5 minutes with an average internet connection. Do you know any faster method?
Thank you
@@LiarcoDevTips I think it's better by simply run a IPFS node locally ?
I see your point, it really depends on your internet connection, I had terrible experiences with local IPFS nodes, also depending on network performances from other nodes. Uploading to an S3 endpoint is much faster compared to spreading the data to IPFS nodes on the net. This is also very convenient if you want to use something like the Safe NFT Metadata Provider (github.com/hashlips-lab/safe-nft-metadata-provider) because in this way you are uploading just once from your local network. ;)
@@LiarcoDevTips ipfs + nftstorage 🔑 it's a good option for me then !
Thank you, NFT.Storage sounds interesting, I will take a look at it. ;)
sub 🤟
Ubuntu? get lost dude....
@Liarco DevTips Hi, I am trying to get the file uloaded. but keep getting the message ''killed'' what am i doing wrong?
Hi, did you follow my suggestion about the RAM? It seems like your droplet doesn’t have enough RAM to process your files…
@@carmenaustin6194 if your data is 9GB, then 8GB of RAM are not enough. Your droplet must be able to load all of your files to RAM in order to perform the operation, so you need at least 1GB + the collection size. DigitalOcean supports droplets with up to 256GB of RAM, but in order to have access to higher tiers you will have to send a direct request to them from the dashboard (you should find the option to contact them inside the plan selection widget).
@@carmenaustin6194 sometimes I had to upload folders with more than 20GB of data in them, everything worked fine, but of course you must have a proper Pinata plan as well, the basic free one won't accept more than 1GB
@@carmenaustin6194 yep, the PICNIC plan seems to be a perfect fit for your situation: www.pinata.cloud/pricing
Unfortunately the old 5$ plan is not available anymore