Queries change faster than data: You have a large dataset, and you are exploring it by running a lot of different queries on it. Data changes faster than your queries: You have decided on a few queries that you want to run continuously (they are not changing frequently), but you have a lot of new incoming data.
I'm writing a streaming application with Flink and getting this error from Flink - org.apache.flink.runtime.io.network.exception.RemoteTransportException : Connection Unexpectedly closed by remote task manager xx1.xx2.xx3.xx4:abcd. This might indicate the that the remote task manager was lost. The configurations for Flink are as follows flink.parallelism.default = 60 flink.taskmanager.numberOfTaskSlots = 3 flink.taskmanager.memory.flink.size = 1G flink.jobmanager.memory.jvm-metaspace.size = 512mb flink.taskmanager.memory.jvm-metaspace.size = 2048mb Could anyone please help me what is the cause for the error and how to fix it.
Host's PPT slides were so vividly to illustrate the concept of Flink and its application. Its Awesome !
Robert Metzger has amazing teaching ability!...great talk
Amazing talk, learnt a ton. Moreover picked up some speaker notions, kudos Robert!
Amazing talk!
This was an amazing talk
ammm... this was a wonderful talk :-)
Wonderful talk
Veverica means squirrel in Slovenian... wondering if name Ververica has anything to do with that or is it just a coincidence?
No, this is not a coincidence ;)
“These notions of time mostly affect the latency…… and the correctness”
What does he mean by "Queries change faster than data"?
Select a,b from A; Select c,d from A and so on... here the query changes but its on the same table A which is the data
Queries change faster than data: You have a large dataset, and you are exploring it by running a lot of different queries on it.
Data changes faster than your queries: You have decided on a few queries that you want to run continuously (they are not changing frequently), but you have a lot of new incoming data.
in other words, Queries change faster than data == "heavy reads, low writes"
Good use of 31 min.
Single's Day :-)
I'm writing a streaming application with Flink and getting this error from Flink - org.apache.flink.runtime.io.network.exception.RemoteTransportException : Connection Unexpectedly closed by remote task manager xx1.xx2.xx3.xx4:abcd. This might indicate the that the remote task manager was lost.
The configurations for Flink are as follows
flink.parallelism.default = 60
flink.taskmanager.numberOfTaskSlots = 3
flink.taskmanager.memory.flink.size = 1G
flink.jobmanager.memory.jvm-metaspace.size = 512mb
flink.taskmanager.memory.jvm-metaspace.size = 2048mb
Could anyone please help me what is the cause for the error and how to fix it.
You need to check the logs of the TaskManager.
A default parallelism of 60 on 3 task slots will only work if you have 20 TaskManagers.