I have 2 procedures in the Postgres database that take 2 hours each to complete, but I want to run them parallelly without affecting any kind of db performance so that they both can finish in 2 hours and they are not dependent on each other. I have 4 cores allocated to that server. if I run multiple procedures through concurrent.futures, will it be able to complete it in 2 hours?
Your Stored Procedures run on the Database, so it depends on the database's capacity to run multiple SPs at the same time without reaching its capacity...If you are triggering these SPs from an external python script, then yes, your script can definitely trigger all your SPs concurrently and wait for them to finish parallelly. But you may want to look deeper into the capacity of your database to run multiple SPs
Sir is it possible to apply multi threading in the same function, means if I consider a loop of 4 iteration and every iteration will be in different threads..as the data are not dependendent on the others.
Hello Thanks for your comment. Yes, same function can execute in different threads. Your thread handles needs to be different so that you can join later Hope it helps 🙏
Thank you sir for replying me. But it is not doing .. def Calc_Dist(df,x,y,centroid): i=4 for i in centroid.keys(): df[str(i)] = np.sqrt((df[x] - centroid[i][0]) ** 2 + (df[y] - centroid[i][1]) ** 2) print(df) df2 = td.Thread(target = Calc_Dist,args =(df,x,y,cen)) df2.start() df2.join() print(df2.name) here it shows that only one thread is working..but i want 4 threads for 4 iteration.
It is really a great explanation. Thankyou!
Thank you so much for your comment 🙏
great thanks! However I'm not getting any output in the console from my .map functions! seems to work for you?
Thanks for your comment, it should work.. not sure why.. try rerunning or upgrading to the latest python version
I have 2 procedures in the Postgres database that take 2 hours each to complete, but I want to run them parallelly without affecting any kind of db performance so that they both can finish in 2 hours and they are not dependent on each other. I have 4 cores allocated to that server.
if I run multiple procedures through concurrent.futures, will it be able to complete it in 2 hours?
Your Stored Procedures run on the Database, so it depends on the database's capacity to run multiple SPs at the same time without reaching its capacity...If you are triggering these SPs from an external python script, then yes, your script can definitely trigger all your SPs concurrently and wait for them to finish parallelly. But you may want to look deeper into the capacity of your database to run multiple SPs
Sir is it possible to apply multi threading in the same function, means if I consider a loop of 4 iteration and every iteration will be in different threads..as the data are not dependendent on the others.
I am just confusing about this .please help
Hello
Thanks for your comment.
Yes, same function can execute in different threads. Your thread handles needs to be different so that you can join later
Hope it helps 🙏
Thank you sir for replying me.
But it is not doing ..
def Calc_Dist(df,x,y,centroid):
i=4
for i in centroid.keys():
df[str(i)] = np.sqrt((df[x] - centroid[i][0]) ** 2 + (df[y] - centroid[i][1]) ** 2)
print(df)
df2 = td.Thread(target = Calc_Dist,args =(df,x,y,cen))
df2.start()
df2.join()
print(df2.name)
here it shows that only one thread is working..but i want 4 threads for 4 iteration.
Is it possible? or I will initialize 4 different threads for 4 iteration? but this is not the generalize form.
@@sujatabasu282 There is only one thread because, you just started only one thread. (df2.start()). Create a df3, df4 & df5 also and start them.
thanks man took hours lmao
Thanks for your comment 🙏