Really a nice video on streaming. One basic question I am having is whether OLAP systems are used for real-time streaming. Generally we use OLTP databases or NOSQL databases for quick response. Please lemme know whether my understanding is correct or not?
There is no limitation in software engineering 🙂 Learn as much as possible diff integration tools with Snowflake , it's advanced feature of integration like Snowpipe for streaming data ingestion , Snowflake connector with Python , Spark etc are also used many times in many pipelines
@@parikshithshivaprakash5523 By having an API call which triggers a Lambda function to return a pre-signed URL to the client which will allow upload directly to S3. That way you won't have to deal with the limit in API Gateway.
your channel is a gem to learn real time data engineering
Thank you for your inspiring comment Shayan Kabasi! Happy Learning :-)
Its a detailed video with all the explanation, learnt how to stream , thank you so much.
Glad to hear that srinivas ch! Happy Learning :-)
you are amazing, thank you so much for sharing these videos
Thank you Lexact! Happy Learning
AWS APIGW can directly write to Kenesis streams, lamda function not mandatory
Thanks a lot. it was so helpful
Glad to know the video helped you Manoj T! Happy Learning :-)
Really a nice video on streaming. One basic question I am having is whether OLAP systems are used for real-time streaming. Generally we use OLTP databases or NOSQL databases for quick response. Please lemme know whether my understanding is correct or not?
Hello Sir,
What minimum tools we should learn with Snowflake for better career growth
There is no limitation in software engineering 🙂 Learn as much as possible diff integration tools with Snowflake , it's advanced feature of integration like Snowpipe for streaming data ingestion , Snowflake connector with Python , Spark etc are also used many times in many pipelines
can we send images or binary files to stream it
Yes. you can ingest real-time data such as video, audio, application logs, website clickstreams, and IoT telemetry data
@@chinrasufamily bro how to send large files to s3 using api
@@parikshithshivaprakash5523 By having an API call which triggers a Lambda function to return a pre-signed URL to the client which will allow upload directly to S3. That way you won't have to deal with the limit in API Gateway.
@@chinrasufamily bro upto 5gb it can handle that’s it above that it cannot