![Anjan GCP Data Engineering](/img/default-banner.jpg)
- 86
- 304 791
Anjan GCP Data Engineering
India
เข้าร่วมเมื่อ 30 เม.ย. 2022
This channel is created to serve below purposes
- Educate about Data Engineering concepts and related products available on Google Cloud Platform
- Content of the videos are more focused on practical aspects of the concepts and use-cases
- Each video will have high level discussion of concepts in theory and followed by respective hands on/demos immediately
- Content of the videos are made such a way to help beginners and professionals
- Series of product wise videos are uploaded based on complexity of the concepts ( Ex: Big Query, Dataflow, Dataproc..etc)
- Please make use of playlists created based on products.
- Educate about Data Engineering concepts and related products available on Google Cloud Platform
- Content of the videos are more focused on practical aspects of the concepts and use-cases
- Each video will have high level discussion of concepts in theory and followed by respective hands on/demos immediately
- Content of the videos are made such a way to help beginners and professionals
- Series of product wise videos are uploaded based on complexity of the concepts ( Ex: Big Query, Dataflow, Dataproc..etc)
- Please make use of playlists created based on products.
วีดีโอ
Learn code free Data pipelines with Cloud Data Fusion | Packed with detailed Demos and explanations
มุมมอง 6K9 หลายเดือนก่อน
Learn code free Data pipelines with Cloud Data Fusion | Packed with detailed Demos and explanations
Serverless Change Data Capture (CDC) , Replication , Data Synchronisation with Cloud Datastream
มุมมอง 1.3K9 หลายเดือนก่อน
Serverless Change Data Capture (CDC) , Replication , Data Synchronisation with Cloud Datastream
ETL nested AVRO files into Cloud SQL, GCS using Dataflow Pipeline
มุมมอง 9509 หลายเดือนก่อน
Code : github.com/anjangcp/GCP-Data-Engineering-Demo-Codes/blob/9b8b7b5c07129188237965b266f6d73f0350d5a0/Dataflow/batch_etl_avro_data_cloudsql.py Must watch below videos before you watch this : th-cam.com/video/hWpNyGj5xVk/w-d-xo.htmlsi=is1OHok0P0XMDRsS th-cam.com/video/dKGB3d2XXiE/w-d-xo.htmlsi=CvdRfpAeyl8NYaDd
Automate GCP Project’s IAM Roles Snapshot process using Cloud Functions, Bigquery, Cloud Scheduler
มุมมอง 41310 หลายเดือนก่อน
Ref Code: github.com/anjangcp/GCP-Data-Engineering-Demo-Codes/blob/698552fb8352876303e3d41a2e3ea56ae552d103/Common_Realtime_Usecases/iam_snapshots.py
GCP Professional Data Engineer, re certified ,Exam experience and my suggestions to Aspirants
มุมมอง 69510 หลายเดือนก่อน
GCP Professional Data Engineer, re certified ,Exam experience and my suggestions to Aspirants
Selecting GCP ETL/Data processing services | GCP Professional Data Engineer
มุมมอง 78811 หลายเดือนก่อน
github.com/anjangcp/GCP-Data-Engineering-Demo-Codes/blob/3fe03fe8304dd34f3cfb3ae9ab43f47acf44b180/GCP_Data_Eng_concept_files/gcp_etl_services.png
GCP Professional Data Engineer | Storage and Database selection process, Test your knowledge
มุมมอง 45711 หลายเดือนก่อน
GCP Professional Data Engineer | Storage and Database selection process, Test your knowledge
GCP Professional Data Engineer | Selecting suitable storage and database services based on use case
มุมมอง 86311 หลายเดือนก่อน
GCP Professional Data Engineer | Selecting suitable storage and database services based on use case
Cloud Big Table - Introduction | Console , CBT - Basic Demo with Examples
มุมมอง 1.4K11 หลายเดือนก่อน
Cloud Big Table - Introduction | Console , CBT - Basic Demo with Examples
Import Bulk data from GCS into Spanner Table using Dataflow Jobs
มุมมอง 759ปีที่แล้ว
Import Bulk data from GCS into Spanner Table using Dataflow Jobs
Basic Cloud Spanner database operations using Python Client Libraries
มุมมอง 496ปีที่แล้ว
Basic Cloud Spanner database operations using Python Client Libraries
Cloud Spanner Introduction demo with examples
มุมมอง 1Kปีที่แล้ว
Cloud Spanner Introduction demo with examples
Automate Dataproc workloads using Cloud Composer
มุมมอง 2.9Kปีที่แล้ว
Automate Dataproc workloads using Cloud Composer
Composer - Airflow https operators, Extract data from web API using https operators with examples
มุมมอง 3Kปีที่แล้ว
Composer - Airflow https operators, Extract data from web API using https operators with examples
Thanks for your support !!! and More to come
มุมมอง 91ปีที่แล้ว
Thanks for your support !!! and More to come
Schedule secured data load from Google sheets to Big query|Cloud Functions,Cloud Scheduler and more
มุมมอง 1.7Kปีที่แล้ว
Schedule secured data load from Google sheets to Big query|Cloud Functions,Cloud Scheduler and more
Big Query Clustered Tables with Examples
มุมมอง 3.9Kปีที่แล้ว
Big Query Clustered Tables with Examples
Google Cloud Storage Object Versioning and Auto Class
มุมมอง 1.1Kปีที่แล้ว
Google Cloud Storage Object Versioning and Auto Class
Schedule Dataproc Workflows using Cloud Scheduler
มุมมอง 1.6Kปีที่แล้ว
Schedule Dataproc Workflows using Cloud Scheduler
Cloud Scheduler Introduction with simple examples
มุมมอง 4.4Kปีที่แล้ว
Cloud Scheduler Introduction with simple examples
Export and Analyze GCP Billing data with BigQuery and Looker Studio
มุมมอง 3Kปีที่แล้ว
Export and Analyze GCP Billing data with BigQuery and Looker Studio
Google Cloud Storage (GCS) Object Lifecycle Management with Examples
มุมมอง 3.7Kปีที่แล้ว
Google Cloud Storage (GCS) Object Lifecycle Management with Examples
Dataproc Workflow Templates | Execute Jobs DAG on Managed Dataproc Cluster
มุมมอง 2.9Kปีที่แล้ว
Dataproc Workflow Templates | Execute Jobs DAG on Managed Dataproc Cluster
Big Query Routines , Stored Procedures and Anonymous Blocks for repeated tasks
มุมมอง 3.7Kปีที่แล้ว
Big Query Routines , Stored Procedures and Anonymous Blocks for repeated tasks
Big Query Federated queries | Connect and execute Cloud SQL queries from Big Query
มุมมอง 1.6Kปีที่แล้ว
Big Query Federated queries | Connect and execute Cloud SQL queries from Big Query
Cloud SQL Python connector | Access Cloud SQL Instance Programatically
มุมมอง 3.2Kปีที่แล้ว
Cloud SQL Python connector | Access Cloud SQL Instance Programatically
Running Dataflow Batch Jobs from Templates
มุมมอง 3.4Kปีที่แล้ว
Running Dataflow Batch Jobs from Templates
How to schedule these sir
Thanks for this video. Keep making more such videos
what is this event and where it is coming from?
Hope your health is good, great dedication and commitment Anjan
great presentation, informative session tq Anjan
Thank you! This explanation help me a lot!!
Nice 👍
informative session, excellent presentation
How to do this by terraform please
Excellent usecase. Very helpful
Really good! Thank you so much
its great and systematic presentation of content, highly appreciate your efforts, god bless you Anjan.
Thank you
Nice 👍
Super useful. Thank you for the post~
Is there any way I can find path for csv file (greenhouse_dtls) file to download?
Thank you so much for your video. It has helped me a lot.
Your videos are very good and deserves good reach. Thanks for sharing all these videos.
nicee
I was assigned to GCP project from JAVA development and was told to complete the course and do the certification. Your videos are helping me a lot. Also if you are active now, please make latest tools videos also and also IAM related videos etc. It will help a lot of IT guys like me. Thank you for your videos.
bash_task = BashOperator( task_id='bash_task', bash_command='date:echo 'Hey I am BashOperator' ', ) In above code, I thing dag=dag line is missing Please suggest on this
Thank you
hola, como podria subri un archivo desde local a Cloud storage usando cloud function?, Gracias
I really enjoyed your first video in this series but please, try to keep the intro and the rest of the video the same volume. I had the volume turned up to max since the voice was soo low in the previous video, then i came to the next part and the introduction is EXTREMELY loud compared to the volume in last video. It was a very painful experience for my ears.
I need to connect to a third party API i want to do it via cloud composer in gcp basically airflow is it possible to do it and move the data to gcs how can i do ot
Awesome
Nice and excellent content, and also very well explained the concepts with screen navigation steps. Are you giving any in person or online trainings? Can I get your email id?
Yeah the voice is so bad , couldn't hear most of the words properly
How to do the a ove for 100 Gbs of data that is going to come in the buckets
Very useful, thanks
Great video Anjan! Please, tell me. Is it possible to export a previous log? I mean, is it possível to export the last 30 days log to a file or to a BQ table?
Yes you can , in the log explorer first frame the query as per your requirement , then use it for exporting while you configure the sink
@@anjangcpdataengineering5209 Thanks for your prompt answer. The problem is, there is a limit of 10.000 records per download via console. I probably have millions of them.
Sir how to reach out to you?
while creating the DAG i am getting error as SQLite cannot found , it is asking to change the configuration can you please suggest me to what could be done and suggest me the video which i can work on dependencies and fixing the error t
Is it possible to create Partition on already existed table in BigQuery?
Hi @Anjan, I am unable to create a GCP Composer environment. After around 45 minutes it shows the error "Some of the GKE pods failed to become healthy. Please check the GKE logs for details, and retry the operation." I have configured the setting and given the necessary access. I am using composer-2.7.0-airflow-2.7.3. But it failed to create the environment. Can you please help with this? Thanks.
Did you find any solution? I am facing the same problem.
Are you using a trial account? The composer would need more resources that the trial account does not provide
Thank you for providing detailed video. Can you please able provide the dataset for testing ?
Excellent Content and brief explanation
first thank you for this amazing coursers , seconde can you pleas sheare with us the presentation of all this playliste
Sound is very low. Please fix it.
Please do a tutorial for dataform, how to make data transofrmations there
Hello anjan I am getting connection lost to server after executing the external query , how can i solve it?
Hi, Could you please provide your LinkedIn Id here . Your content is superb apart from other tutorials.
Thank you very much for giving clear understating on Google Cloud Function
audio went really low in this video
Damn Good🎉
Very nice content...
A trivial question but hope you can help. To perform data engineering activities or let's say for pipelines development what Access do I need. This question arose when I asked IT to grant me access to GCP. They have asked me what access do I require and I couldn't answer. Can anyone help please?
Anyone tried this getting module not found error for bulkdeal_aggr
can you share the sample ipl data?