HarvestingClouds
HarvestingClouds
  • 35
  • 496 794
Getting started with Big Data in Azure - Azure Data Lake Storage (ADLS) and Azure Data Factory (ADF)
Getting started with Big Data in Azure with Azure Data Lake Storage and Azure Data Factory. This session was presented at the Global Azure 2022 event. This session included detailed discussion and demos around getting started with Big Data in Azure. The session focused on Azure Data Lake Storage Gen 2 (ADLS Gen2) and Azure Data Factory (ADF).
The slide deck can be downloaded here: github.com/HarvestingClouds/SessionsAndDecks/tree/main/Global%20Azure%202022
Session content includes:
00:00 Overview and Agenda
00:43 Introduction
01:30 Data Storage Concepts
13:13 Demo - Azure Data Lake Storage (ADLS) - Gen 2
17:39 Azure Data Factory Concepts
26:46 Demo - Azure Data Factory
42:14 Certification Path - Azure Data Engineer Associate
43:06 Getting setup with free Azure account
43:39 Key Takeaways and Next Steps
44:41 References, Q&A and thanks
มุมมอง: 1 559

วีดีโอ

Building a Strategy for Periodic Optimization of Azure Environments - with Eric Bogenschuetz
มุมมอง 1.8K2 ปีที่แล้ว
Building a strategy for periodic optimization of Azure environments with Eric Bogenschuetz. This session was presented at the Global Azure 2022 event. This session includes practical strategies for optimizing 4 key focus areas: 1. Security 2. Cost 3. Performance 4. Supportability/Management The slide deck can be found here: github.com/HarvestingClouds/SessionsAndDecks/tree/main/Global Azure 202...
Connecting Virtual Networks in Microsoft Azure via Peering
มุมมอง 37K3 ปีที่แล้ว
In this video, we take a practical look at Virtual Network peering (or commonly referred to as vNet peering). We look at all the caveats and also break down various myths. Virtual Network peering is the way you connect two different Virtual Networks. It is one of the most common things that you will require while setting up networking in any environment. It enables you to seamlessly connect two...
Showdown - Service Endpoints vs Private Endpoints in Microsoft Azure
มุมมอง 49K4 ปีที่แล้ว
In this video, we compare Service Endpoints and Private Endpoints in Microsoft Azure. Both the features allow you to easily connect to Azure Services from your Virtual Networks. We look at six different ways these two services are different from each other. We look at the architecture and how each of the services works before jumping into the comparison. Before you watch this video, I highly re...
How to create Service Endpoints for Virtual Networks in Azure
มุมมอง 31K4 ปีที่แล้ว
In this video, we explore Service Endpoints in Microsoft Azure. This is a feature of Virtual Networks that provides you better connectivity to various Azure resources. In the video, we look at connecting to Azure Storage Accounts from Virtual Machines from a subnet within a Virtual Network. The concepts are equally applicable to other supported Azure services like Microsoft Azure SQL Databases,...
Locking Azure Resources - Ultimate protection for your resources
มุมมอง 2K4 ปีที่แล้ว
In this video, we are checking how you can protect your resources against any unintended deletions or updates. We explore the Azure Resource Manager Locks with practical hands-on examples in this video. We see how to secure our critical resources so that even administrators and owners can't delete or make unnecessary changes to the most important resources within the cloud. We look at best prac...
How to Create Virtual Networks in Microsoft Azure
มุมมอง 41K4 ปีที่แล้ว
In this video, we switch gears to practical aspects of networking. We are seeing the details related to creating a new virtual network or vNet in the Microsoft Azure portal. We create one virtual network with 3 subnets. We look at various restrictions when creating virtual networks. If you need to modify anything, then how to do that after the creation of the virtual network is also explored in...
Designing Networks and performing Networking related calculations - with examples
มุมมอง 4.5K4 ปีที่แล้ว
This video builds on the previous video where we saw the fundamentals of IP addresses. That video can be found at this link: th-cam.com/video/bfV6kRi0U98/w-d-xo.html In this video, we use the knowledge from the previous video and start designing the Networks. These Networks could be Virtual Networks (vNets) in Microsoft Azure or Virtual Private Clouds (VPCs) in Amazon AWS. We look at 3 examples...
How IP addresses work - Networking Fundamentals
มุมมอง 6K4 ปีที่แล้ว
This video is all about building the right foundation for all the upcoming Networking related videos. In this video, we talk about the basics of IP Addresses. This information may seem simple but is very powerful in understanding the complex concepts later on. We will be dissecting the IP addresses and understanding each component. We also learn about Subnet masking and about different classes ...
Creating an Azure Private Endpoint Connection with Azure Storage Accounts
มุมมอง 62K4 ปีที่แล้ว
In this video, we are creating an Azure Private Endpoint connection with Azure Storage Account. We begin by discussing the scenario that we are building in this video and discussing what is it that Azure Private Endpoint Connection is providing us in Azure. We will attempt the connection prior to creating the Private Endpoint connection. And then we will set up the Private Endpoint. After that,...
Understanding Private Endpoints - Azure Services Simplified
มุมมอง 66K4 ปีที่แล้ว
In this video, we are exploring what are Azure Private Endpoints. We look at the problem first that Microsoft is solving with Azure Private Endpoints. And then we look at how this works to solve that problem. We take Azure Storage Account as an example and look at how you connect to it without Private Endpoint. And then we look at how Private Endpoint works to provide you better and more secure...
The Ultimate guide to working with Azure Storage Explorer
มุมมอง 26K4 ปีที่แล้ว
Azure Storage Explorer is a small GUI based utility for working with Azure Storage Accounts. This utility is very versatile. It provides various ways to connect and work with storage accounts. You can customize the experience with lots of settings available in this utility. It uses the AzCopy command-line utility under the hood, to upload and download the data to and from the cloud. In this vid...
Disaster Recovery of Azure Storage Accounts
มุมมอง 9K4 ปีที่แล้ว
In this video, we talk about the disaster recovery of the Azure Storage Accounts. We discuss how you can configure the replication strategy of a storage account and how can you make it more resilient. This helps you if there are any service disruptions within the cloud. This provides you with a guarantee that your data will be available in case of any disastrous event. We configure the Globaly ...
Configuring Firewall and Virtual Network access on Azure Storage Accounts
มุมมอง 16K4 ปีที่แล้ว
In the previous video, we saw how to securely accessing Azure Storage accounts using Shared Access Signatures. You can check that video here: th-cam.com/video/pXsVR67gQ7s/w-d-xo.html In this video, we are taking the security one step further by limiting the access to storage accounts completely by configuring Firewalls on them. We can limit the access to either particular subnets within specifi...
How to securely access Azure Storage Accounts - enhancing security
มุมมอง 16K4 ปีที่แล้ว
Azure Storage Accounts provide powerful ways to store your data in the cloud. It also means that you need to do everything you can to ensure that these storage accounts are accessed securely. You have various ways to access the storage accounts. You can leverage access keys to access storage accounts as administrators. Or you can use Shared Access Signature to enhance security. In this video, w...
10 Ways to move your data to Azure Storage - Part 2
มุมมอง 3.3K4 ปีที่แล้ว
10 Ways to move your data to Azure Storage - Part 2
10 Ways to move your data to Azure Storage - Part 1
มุมมอง 8K4 ปีที่แล้ว
10 Ways to move your data to Azure Storage - Part 1
Azure Storage Accounts - How to Create new Storage Accounts
มุมมอง 20K4 ปีที่แล้ว
Azure Storage Accounts - How to Create new Storage Accounts
How to work with AWS Auto Scaling Groups and AWS Auto Scaling service
มุมมอง 4.2K4 ปีที่แล้ว
How to work with AWS Auto Scaling Groups and AWS Auto Scaling service
How to Create AWS Launch Configurations and Launch Templates
มุมมอง 15K4 ปีที่แล้ว
How to Create AWS Launch Configurations and Launch Templates
How to Configure Autoscaling on Microsoft Azure Virtual Machine Scale Sets (VMSS)
มุมมอง 20K4 ปีที่แล้ว
How to Configure Autoscaling on Microsoft Azure Virtual Machine Scale Sets (VMSS)
Creating Microsoft Azure Virtual Machine Scale Sets (VMSS)
มุมมอง 24K4 ปีที่แล้ว
Creating Microsoft Azure Virtual Machine Scale Sets (VMSS)
How to protect Azure Virtual Machines (VMs) against any disasters
มุมมอง 7624 ปีที่แล้ว
How to protect Azure Virtual Machines (VMs) against any disasters
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 6 - Monitoring EC2 Instances
มุมมอง 1754 ปีที่แล้ว
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 6 - Monitoring EC2 Instances
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 5 - Monitoring Azure VMs
มุมมอง 2594 ปีที่แล้ว
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 5 - Monitoring Azure VMs
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 4 - Backing up EC2 Instances
มุมมอง 1654 ปีที่แล้ว
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 4 - Backing up EC2 Instances
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 3 - Backing up Azure VMs
มุมมอง 4294 ปีที่แล้ว
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 3 - Backing up Azure VMs
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 2 - Creating EC2 Instances
มุมมอง 9244 ปีที่แล้ว
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 2 - Creating EC2 Instances
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 1 - Creating Azure VMs
มุมมอง 3.5K4 ปีที่แล้ว
Microsoft Azure VMs vs Amazon AWS EC2 Instances - Part 1 - Creating Azure VMs
Azure Automation - Hybrid Workers
มุมมอง 6K6 ปีที่แล้ว
Azure Automation - Hybrid Workers

ความคิดเห็น

  • @sanjaygopal627
    @sanjaygopal627 23 วันที่ผ่านมา

    Thanks for a great video. Can a service endpoint in Tenant A access resources in Tenant B? Not sure if service endpoints is the approach to take when resources in one tenant need to be made availabe to consumers in another tenant.

  • @vijayalakshmiu4337
    @vijayalakshmiu4337 หลายเดือนก่อน

    We have few appservices in 2 subnets of single vnet. Now the communication between webapps from subnet 1 to webapps of subnet2 is configured via private end point. But it is not working and giving IP forbidden error. Please suggest somw solutions bro

  • @melsoliano1937
    @melsoliano1937 2 หลายเดือนก่อน

    well explained, keep up the good work mate!

  • @shoumou9504
    @shoumou9504 2 หลายเดือนก่อน

    Thank u.

  • @AkshayGupta108
    @AkshayGupta108 2 หลายเดือนก่อน

    True to title "Simplified.. " Thanks...

  • @juancarlos8552
    @juancarlos8552 2 หลายเดือนก่อน

    Thank you very much for the clarity of your explanation. Could I use this type of connection to connect my power automate with my storage account located in two different regions? What else should I configure apart from what you show in the video? What should I add to the NSG? Thank you for your support.

  • @upulwaruna.abeysinghe
    @upulwaruna.abeysinghe 2 หลายเดือนก่อน

    a best explanation thank you

  • @kasperskyns
    @kasperskyns 2 หลายเดือนก่อน

    Best explanation so far. Good work

  • @hanumantshinde5652
    @hanumantshinde5652 3 หลายเดือนก่อน

    Thanks for the video. I understood GRS completely and I would like to know in a similar way how ZRS and LRS works

  • @gerardbaste5265
    @gerardbaste5265 3 หลายเดือนก่อน

    This is by far the best explanation I have seen on this topic, you did a fantastic job here !

  • @LuisOlias
    @LuisOlias 3 หลายเดือนก่อน

    Thanks for sharing your amazing knowledge! , greetings from Spain.

  • @NareshKumar-qm3em
    @NareshKumar-qm3em 3 หลายเดือนก่อน

    What is case if we have both private end point and service endpoint storage resource

  • @mukeshkane
    @mukeshkane 3 หลายเดือนก่อน

    Great video A help needed, my client is not able to connect to my storage account because the ingress IP for them is coming as a private IP while we have whitelisted their public IP. Any suggestions how can we force their servers to use only public IP. PS: client's servers are also in the azure and possibly same region

  • @shaikmeeravali
    @shaikmeeravali 4 หลายเดือนก่อน

    Excellent explanation

  • @venugopalreddy
    @venugopalreddy 4 หลายเดือนก่อน

    This great. I was trying to do this Service endpoint for Azure DB for PostgreSQL and I am not able to do it. Can you please guide me/Make a video for the same?

  • @jarecheiroso7460
    @jarecheiroso7460 4 หลายเดือนก่อน

    thanks! : )

  • @complexity8851
    @complexity8851 4 หลายเดือนก่อน

    Just had one doubt, if I enable a private endpoint for one of my storage accounts, will it disable all access via public internet?

  • @user-zm4kv4ww9x
    @user-zm4kv4ww9x 4 หลายเดือนก่อน

    Hello @HarvestingClouds sir, Thank you for the video. We have an Azure SQL server and a SQL private endpoint, with no NSG or route table attached. We have already established VPN connectivity between the on-premises server and Azure using Azure site-to-site VPN. We have an Azure firewall and an on-premises firewall. We want to connect from the on-premises server to the Azure SQL private endpoint. Can you please guide us on how to do that? Do we have to open a port in both the Azure firewall and the on-premises firewall, and also add the on-premises firewall public IP addresses to the Azure SQL database firewall configuration in networking? Or is any one option enough?

  • @InstituteForCloudComputing
    @InstituteForCloudComputing 4 หลายเดือนก่อน

    This playlist is just wow. But since we are comparing AWS and Azure, i wish you present how internet gateway, rout table and NAT gateway functionalities are handled in Azure. Also may be a presentation on how any database is used in Azure. May God continue to bless you

  • @rs-tarxvfz
    @rs-tarxvfz 4 หลายเดือนก่อน

    Nice try, but you are just parroting the the things without explaining.

  • @anthonyp3961
    @anthonyp3961 5 หลายเดือนก่อน

    How would you access the storage account using a web browser? This doesn't seem to work?

  • @boseashish
    @boseashish 5 หลายเดือนก่อน

    thanks a lot

  • @shubhamkalra-th4lp
    @shubhamkalra-th4lp 5 หลายเดือนก่อน

    Crisp and Clear 😀

  • @shubhamkalra-th4lp
    @shubhamkalra-th4lp 5 หลายเดือนก่อน

    Crisp and Clear 😎

  • @barwalgayatri4655
    @barwalgayatri4655 5 หลายเดือนก่อน

    Hi when new machine spins in, how ecommerce application will be loaded in it ? It will be a completely new vm right?? I am trying to search this answer but not getting it anywhere

  • @g.lingak4562
    @g.lingak4562 5 หลายเดือนก่อน

    ❤ it, thanks for this video

  • @cyrilfacey8534
    @cyrilfacey8534 5 หลายเดือนก่อน

    One of the best tutorials I've come across. I hope that you will continue to introduce more TH-cam video content.

  • @mihaneman3129
    @mihaneman3129 5 หลายเดือนก่อน

    thank you so much

  • @mihaneman3129
    @mihaneman3129 5 หลายเดือนก่อน

    muchas gracias

  • @helloharshad
    @helloharshad 6 หลายเดือนก่อน

    Wow! I came across this video after 3 Years, and its explained so well and in a very simple way with example. I understood it for good, you presented it so well, thank you.

  • @padmanathanramasamy8354
    @padmanathanramasamy8354 6 หลายเดือนก่อน

    During the failback process, when I change the data replication back to GRS. I understand WestUS will become primary and eastUS will become secondary. How do we change back to original state. I mean Eastus become primary and westus as secondary. Do I need to repeat the same steps?

  • @AllenOlayiwola
    @AllenOlayiwola 6 หลายเดือนก่อน

    Thanks, great video

  • @srilatha3643
    @srilatha3643 6 หลายเดือนก่อน

    videos are really great! please do more videos on AKS

  • @user-em3gw8on5i
    @user-em3gw8on5i 6 หลายเดือนก่อน

    hi, can anybody please help? Is that possible to set the specific secondary region (geo-replication) with own choice? ****Very important****, I need this help on priority, so I will really appreciate it if you could help. TIA

  • @chefe417
    @chefe417 7 หลายเดือนก่อน

    When trying to access the storage account from the VM,... at 2:34 you're saying that it doesnt leave the MS backbone. Also you say it goes over the internet. I am new to networking so maybe I just not firm with definitions, but I would have thought that the MS backone is NOT the internet. And therefore, accessing the storage endpoint over (e.g. a service endpoint) is private. Can you please explain my error in thinking?

  • @allenbythesea
    @allenbythesea 7 หลายเดือนก่อน

    what about point to site vpn?

  • @ramin9842
    @ramin9842 7 หลายเดือนก่อน

    Never came across such a precise, concise and to the point explanation on the topic so far. keep it up

  • @ahzidmahmood6904
    @ahzidmahmood6904 7 หลายเดือนก่อน

    i love you so much, after 3 days of finding an accurate and good example of learning this content today I can that you are the best teacher vs all the cloud gurus have

  • @user-zw6rv6jh5d
    @user-zw6rv6jh5d 7 หลายเดือนก่อน

    Hi HarvestingClould Team, Your classes are really super and, am looking for one particular topic is, How to give the storage account access to external user on particular time for every day

  • @safkaify7875
    @safkaify7875 7 หลายเดือนก่อน

    Nicely explained. Helped me a lot. Please keep up your good work.

  • @many214
    @many214 8 หลายเดือนก่อน

    Thank you for the explanation great video!

  • @Oomtet
    @Oomtet 8 หลายเดือนก่อน

    thank you you make it sound very simple.

  • @SuperAnildesai
    @SuperAnildesai 8 หลายเดือนก่อน

    Hi, instead of using a virtual machine host-level metric, can we use a custom apache2 guest-level metric in VMSS, to scaleup the instance based on request per minute.? any suggestion for this.

  • @mohamedsulthan8027
    @mohamedsulthan8027 8 หลายเดือนก่อน

    How did you created the vm?

  • @mihaneman3129
    @mihaneman3129 8 หลายเดือนก่อน

    thank you so much

  • @stefanberggren770
    @stefanberggren770 8 หลายเดือนก่อน

    At 8:30 he says "you should only have on eprivate endpoint per vnet. Why??

  • @prasenjitgiri919
    @prasenjitgiri919 9 หลายเดือนก่อน

    how come 20 being less didnt fall in the bucket 24 being more than 22?? that explanation made no sense

    • @XtianApi
      @XtianApi 9 หลายเดือนก่อน

      The higher the bit, 20, 22, 24, etc, the fewer the amount of addresses it is referring to. For example, if you are putting a single ip address in somewhere and only want to represent that one single IP address, and nothing else, then it would be /32. If, for example, you have an address space that ends in a very low single digit number, then you are talking about a large portion of all the addresses in existence. Here is a real life idea that may help. If I give you a whole pizza, 1, then I am giving you all of it. But, if I give you only a tiny portion of the pizza, for example 1/24th of the pizza (/24) then I am only giving you a very small portion of the whole pizza. I hope that helps. You can watch a TH-cam video on how all the math works. Use online subnet calculators when you need to.

  • @user-fk9zr5mj7e
    @user-fk9zr5mj7e 9 หลายเดือนก่อน

    Thanks such a great video. I follow all the instructions and it works.

  • @zzzaaalll
    @zzzaaalll 9 หลายเดือนก่อน

    so for services , now azure included service end point right 😮

  • @SoulJah876
    @SoulJah876 10 หลายเดือนก่อน

    Nice video. If I may, I think you mixed up service endpoint and private endpoint. Service endpoint doesn't ensure the traffic travels over Azure's backbone - that's enabled by private endpoint and Private Link. Service endpoint allows a subnet to be referred to as an object in the storage account's native firewall.