If the workloads are laready running on on premise using spark and if you have to migtrate them to GCP then pyspark with dataproc is useful otherwise in case of workload (ETL) development from scratch on GCP Apache beam with dataflow is reccomended , hence it is difficult to say which one is better it all depends on use cases , as a data engineer it's better to have both skills
Thank you so much.
You're welcome!
Pyspark and Apache Beam -which one is best in current market and future??
If the workloads are laready running on on premise using spark and if you have to migtrate them to GCP then pyspark with dataproc is useful otherwise in case of workload (ETL) development from scratch on GCP Apache beam with dataflow is reccomended , hence it is difficult to say which one is better it all depends on use cases , as a data engineer it's better to have both skills
@@anjangcpdataengineering5209 Thanks Sir..
Too long video