Fantastic explanation of the process and why it is useful, thank you very much for putting this together! I like the pace, it's easy enough to pause the video/go back if I need to find a tool or setting, don't want the video to drag on forever. Well done!
Very well presented video, just like your other videos. The best part is, you explain why one should do certain things and what's going on when a certain action is done, in addition to how it's done like many others to. I wish you do more of them!
Amazing Richard! I'm another person is very appreciative of your explanations and thorough background information about the why's and the how's. Thank you!
Amazing tutorial, thanks a lot. I appreciate that you are explaining what it is you are doing, instead of summing up 'and now i click deconvolution and i set deringing to on and' etc.
Nice tutorial Richard, and excellent understanding of the process! Only comments- turning off the STF AutoStretch isn't a reliable way of finding good DPSF star samples. In many cases, the stars that show w/o a screen stretch are indeed saturated. Also, although I don't advocate driving ourselves crazy, there are other recommendations for ideal star samples. Yes, the goal is to get the average of the samples, but according to Juan C., some samples should be discarded. Refer to his forum posts, to see what part Model type (Gaussian/Moffat), R, theta, and very importantly, MAD play in sample selection, in addition to just Amplitude. I felt you were a bit off track stating that decon is primarily for an object of interest, as I think many of us also want overall 'tightening' of our stars. The purpose of the local deringing support image (mask) is not to mask the stars, but to protect only the bright/big stars from the decon settings which otherwise work very well on the majority of the stars. So, I up my midrange slider to only capture the bright guys for my 'LDSI.' Lastly, Decon's 'regularization' is actually a sort of noise reduction component of the process, separating significant structures from noise, thereby preventing weaker signal from being deconvolved. But again, nice voice, great, clean production, and a very good understanding!
Thanks for the info! I suppose turning off the auto stretch isn't a great piece of advice. I toggle it on/off to find the stars with the best SNR that aren't saturated; turning off the stretch helps me decide if a star appears saturated from the stretch or is saturated, or if it appears bright from the stretch but isn't even bright enough to be noticed in the linear image. I'll have to look into those forums posts. I've found amplitude to be pretty reliable for star selection, but to be honest I fell back on my photometry training and didn't try to learn to use those other dimensions to select stars. I should really take a deeper look now. Some of your notes I already agree with, and looking back over the video realize where I haven't been properly clear. I think going forward I'll pre-write a lot of the stuff I say so I can make sure it won't lead new users astray. Thanks for all the insight!
Amazing! Great explanation. Can I make a question? In my case, using a Dslr, what is the moment to make this process here? I saw your Dslr workflow, and you make the deconvolution after removing noise and adjusting the levels at the histogram transformation, and in this tutorial it seems better make it before noise reduction.
Thank you for such a clear and informative video! Incredibly helpful! I was wondering if you were to use drizzle, would you do deconvolution before or after applying the drizzle? Once again, thank you for such a fantastic video!
Great video on deconvolution. Could you give us a PSF plot on a star before and after this routine for comparison? Cheers and looking forward to more videos.
Thanks a lot for this and other amazing tutorials you have made available to us. One question: looking at 11:24, how do you get those images magically appear on the screen? It seems like you just summon them and they appear :)
For those images I used IRAF, a command-line program for processing and analysing astronomical images. It's meant for research purposes, I've tried processing astrophotos with it but PixInsight is definitely better for making eye popping photos! I do think it would be nice for the folks working on PI to include something like this, because the diagram can be useful in choosing stars and troubleshooting an optical setup. But they do so much already and it's such a minor tweak, maybe one day!
great video! very happy to see that you upload videos again after one video. Could you tell me which software you use to get those jpegs of the PSF functions and how accurate the mag value of this is? best regards
Thanks! To get the PSFs that I used for demonstration (aside from the Airy function which I pulled from Wikipedia) I stuck my image into a software called IRAF (Image Reduction and Analysis Facility). It's pretty much what it sounds like - software to reduce and analyse images, and it's used for scientific research pretty much everywhere. The PSF graph is actually how you're supposed to decide whether to accept or reject certain stars as PSF candidates; if it looks noisy or saturated, reject, if it looks good, accept. As far as I know, PixInsight won't really let you see a graph of one, so that's why it's good to just aim for semi-bright isolated stars and do many of them. Unfortunately, the mag values in the video are pretty much garbage. They are offset from the real magnitude of the star by some constant value, but that constant depends on the imaging setup, location, etc. The only way to find it is to perform some tests with that setup at that site on a star with a well known magnitude. Then you can just add that constant to these magnitudes (it could also be negative) and get the real magnitude.
Hey Richard! I just found out that you can get a psf function graph out of Pixinsight too, just use the "Plot 3d graph" for that and make a little Preview over a star.
In the days of film nothing like this possible. With a few short extra exposures the blown out systs could be instead obtained with nice profiles to replace.
Great video on a fairly intensive set of steps. Question: The averaged star in your PSF has many pixels and quite a few of those appear to be set near maximum brightness. My PSF view shows only about 60 pixels and only the center one is at the brighest level. I am wondering why mine turn out this way. Is there a setting for this? Or do you have any suggestion as to why my PSF does not look the same as yours?
I'm not sure, it would be helpful to have a picture of your PSF so I can compare and see what's going on. A guess would be that you need to select better and/or more stars to average, or that there's a view setting that's different.
Richard, thank you for this excellent tutorial. It really helps to understand how and why to use the Deconvolution tool. You mention a number of times that deconvolution should be done as early as possible in the workflow; would it be better to do it before or after DynamicBackgroundExtraction?
Sorry for the late reply! I don't think it makes a difference. Technically speaking, it should be the first thing that's done after calibration, before DynamicBackgroundExtraction and anything else, because the math and process is designed to work on untouched linear data. But in reality, the effect of DynamicBackgroundExtraction is too minimal to be a problem for deconvolution, and in fact can be helpful for your own eyes in sussing out the subtle details that come from deconvolution!
I used a program called IRAF, which is used by astronomers to process images. It isn't very intuitive to use unless you're used to command-line programs, but it's free. Unfortunately I don't know the software options if there's an alternative, but as long as you don't choose the brightest or dimmest stars in your image, you can be reasonably sure you're sampling good stars.
Really a wonderful, clear, well paced video. Thank you! I have the same question as expressed below. What did you use to make those neat little PSF function graphs?
Thanks! I don't know how better to phrase it, so I'll just be lazy and copy/paste from another thread: To get the PSFs that I used for demonstration (aside from the Airy function which I pulled from Wikipedia) I stuck my image into a software called IRAF (Image Reduction and Analysis Facility). It's pretty much what it sounds like - software to reduce and analyse images, and it's used for scientific research pretty much everywhere. The PSF graph is actually how you're supposed to decide whether to accept or reject certain stars as PSF candidates; if it looks noisy or saturated, reject, if it looks good, accept. As far as I know, PixInsight won't really let you see a graph of one, so that's why it's good to just aim for semi-bright isolated stars and do many of them.
Excellent, really helpful. I have a question, I never know if I need to stretch my image before to start some processes or after, at the end, (the linear or non-linear mode), could you explain what I should do and why ? Thanks a lot :)
That's a great question, and I'm afraid I don't know enough to produce a full list of PixInsight functions and their optimal location in a workflow, especially since astrophotographers use their own personal workflow. That said, there are some basic rules to live by: Most work will be done in the nonlinear regime. A lot of the information in linear data is not useful when it comes to aesthetics, because it contains very little in the midtones for most images (ie galaxies are dim, stars are bright). So stretching occurs very early on, because at the end of the day processing is about making the most of the "other" data, which is best manipulated post-stretch. Some tools assume you're working with linear data, such as deconvolution. Pre-stretch, you can make analytical approximations of the physics that produced your image data, because there is a very simple relationship between incoming light and what your detector "sees". Deconvolution is an example of one such function: it assumes you're dealing with "raw data", so any changes you make to your image prior to deconvolution will make deconvolution less effective in rough proportion to the number and amount of changes. Noise reduction has a lot of literature pre-stretch and post-stretch, and plenty of debate as to what "the best" approach is. I'm still learning in that area, so I can't advise specifics, but it's worth looking into pre-stretch noise reduction to at least get informed and decide for yourself whether to include that in your workflow. Most tools however don't make assumptions about linearity. Most common tools, such as what you find under IntensityTransformations don't. The only "necessary" tools that do are the ones that go into calibration, like ImageCalibration and the suggested settings for ImageIntegration when stacking, but you wouldn't stretch while you're doing calibration anyway.
I made them using IRAF, a command-line program for image processing and analysis. I couldn't find a way to do it in PixInsight, and am not sure it's possible.
In theory it should be done before, though I don't think it will make a big quality difference. I do it after colour calibration usually. My logic train goes like this: 1) The PSF analysis works best when all pixels have been modified in a uniform manner throughout the entire image. This is because using a well-sampled PSF would well represent what the deconvolution algorithm will simulate "removing", and you'd get the best representation possible. 2) In reality, minor differences are unavoidable across your image due to a variety of sources. "Uniform" isn't a word you normally see around raw data. But the idea still holds that the less change applied to your data, unless absolutely uniform across pixels and linear in nature, the better result you can get from the PSF analysis. 3) Calibration frames and the stacking process come before deconvolution, so they technically "harm" the best possible result. However, the signal gains and sensor corrections they provide result in a stacked image that is much more "uniform" in pixel-to-pixel nature than any individual image. Thus deconvolution performed on a raw image pre-calibration will produce a worse result than a calibrated and stacked image. So from (3) I calibrate register and stack. From (1) I would want to go straight to deconvolution, but in practice some of my colour gradients can be wonky. I could extract a luminance channel and work on that, or I could classify minor non-uniform differences added from the colour calibration process under (2). In practice, colour calibration can darken my background sky if one colour was dominant, so it's not always "free" to extract a luminance pre-colour calibration. And if I'm going to work on the RGB image, I don't want any weird colours to be exaggerated by deconvolution, associating weird and steep local colour gradients with high frequency structure. (This happens more often than not in DSLR data that isn't taken in the deep freeze of winter.) So it seems to be a "pros outweigh the cons" situation for doing it post-colour calibration (and thus post-DBE), as far as I'm concerned. Though that really lands on me pinning affected colour noise as "minor" under point (2), which can sometimes be data-dependent. But really that's where the tradeoffs end; I wouldn't push deconvolution beyond almost any other function in the workflow at that point.
whoa slow down a little Rich you're flying round the functions a little too fast for me, know this is probably aimed at intermediate Pi users but having to keep going back and trying to see how you've done a couple of the function bits a couple of times. well practice makes perfect as they say keep up the good work still using your stacking tutorial cheers
Fantastic explanation of the process and why it is useful, thank you very much for putting this together! I like the pace, it's easy enough to pause the video/go back if I need to find a tool or setting, don't want the video to drag on forever. Well done!
Very well presented video, just like your other videos. The best part is, you explain why one should do certain things and what's going on when a certain action is done, in addition to how it's done like many others to. I wish you do more of them!
Thanks. 1st time I've ever got deconvolution to work. Very helpful
Great! I've been using Decon pretty much blindfolded until now. Great to understand what is actually going on. Thanks!
Amazing Richard! I'm another person is very appreciative of your explanations and thorough background information about the why's and the how's. Thank you!
So much information and detail in a single video, thanks a lot!
A great explanation how to apply Deconvolution. Thanks for the video.
Amazing tutorial, thanks a lot. I appreciate that you are explaining what it is you are doing, instead of summing up 'and now i click deconvolution and i set deringing to on and' etc.
Thanks for your straightforward tutorial just used it on M27 at 2650mm and worked wonders.
Glad to hear it helped!
Another excellent tutorial! Clear and concise ... even for a beginner! Thanks!
Indeed, very nice! I've learned a lot about PixInsight using your videos.
Really good teaching i learn alot, please tell me you are working on more or have some more tutorial, i watch them all :)
Nice tutorial Richard, and excellent understanding of the process! Only comments- turning off the STF AutoStretch isn't a reliable way of finding good DPSF star samples. In many cases, the stars that show w/o a screen stretch are indeed saturated. Also, although I don't advocate driving ourselves crazy, there are other recommendations for ideal star samples. Yes, the goal is to get the average of the samples, but according to Juan C., some samples should be discarded. Refer to his forum posts, to see what part Model type (Gaussian/Moffat), R, theta, and very importantly, MAD play in sample selection, in addition to just Amplitude. I felt you were a bit off track stating that decon is primarily for an object of interest, as I think many of us also want overall 'tightening' of our stars. The purpose of the local deringing support image (mask) is not to mask the stars, but to protect only the bright/big stars from the decon settings which otherwise work very well on the majority of the stars. So, I up my midrange slider to only capture the bright guys for my 'LDSI.' Lastly, Decon's 'regularization' is actually a sort of noise reduction component of the process, separating significant structures from noise, thereby preventing weaker signal from being deconvolved. But again, nice voice, great, clean production, and a very good understanding!
Thanks for the info! I suppose turning off the auto stretch isn't a great piece of advice. I toggle it on/off to find the stars with the best SNR that aren't saturated; turning off the stretch helps me decide if a star appears saturated from the stretch or is saturated, or if it appears bright from the stretch but isn't even bright enough to be noticed in the linear image.
I'll have to look into those forums posts. I've found amplitude to be pretty reliable for star selection, but to be honest I fell back on my photometry training and didn't try to learn to use those other dimensions to select stars. I should really take a deeper look now.
Some of your notes I already agree with, and looking back over the video realize where I haven't been properly clear. I think going forward I'll pre-write a lot of the stuff I say so I can make sure it won't lead new users astray. Thanks for all the insight!
Very very good, thank you. PLEASE make more of these :-)
I keep planning on it!
Very well done!
Thanks Richard, your videos are very helpful!
Thanks!
Fab, thanks for doing this 👍🏻
very nice tutorial !
Amazing!
Great explanation.
Can I make a question?
In my case, using a Dslr, what is the moment to make this process here?
I saw your Dslr workflow, and you make the deconvolution after removing noise and adjusting the levels at the histogram transformation, and in this tutorial it seems better make it before noise reduction.
Excellent tutorial on the topic
cheers
Thank you for such a clear and informative video! Incredibly helpful! I was wondering if you were to use drizzle, would you do deconvolution before or after applying the drizzle? Once again, thank you for such a fantastic video!
Great video on deconvolution. Could you give us a PSF plot on a star before and after this routine for comparison? Cheers and looking forward to more videos.
Thanks a lot for this and other amazing tutorials you have made available to us. One question: looking at 11:24, how do you get those images magically appear on the screen? It seems like you just summon them and they appear :)
For those images I used IRAF, a command-line program for processing and analysing astronomical images. It's meant for research purposes, I've tried processing astrophotos with it but PixInsight is definitely better for making eye popping photos!
I do think it would be nice for the folks working on PI to include something like this, because the diagram can be useful in choosing stars and troubleshooting an optical setup. But they do so much already and it's such a minor tweak, maybe one day!
I did try hard to get into Pixinsight, but I'm back with dearest Photoshop now - so easy to use and such great results for astro processing.
Looking forward to it :). Thank you
great video! very happy to see that you upload videos again after one video. Could you tell me which software you use to get those jpegs of the PSF functions and how accurate the mag value of this is?
best regards
Thanks! To get the PSFs that I used for demonstration (aside from the Airy function which I pulled from Wikipedia) I stuck my image into a software called IRAF (Image Reduction and Analysis Facility). It's pretty much what it sounds like - software to reduce and analyse images, and it's used for scientific research pretty much everywhere. The PSF graph is actually how you're supposed to decide whether to accept or reject certain stars as PSF candidates; if it looks noisy or saturated, reject, if it looks good, accept. As far as I know, PixInsight won't really let you see a graph of one, so that's why it's good to just aim for semi-bright isolated stars and do many of them.
Unfortunately, the mag values in the video are pretty much garbage. They are offset from the real magnitude of the star by some constant value, but that constant depends on the imaging setup, location, etc. The only way to find it is to perform some tests with that setup at that site on a star with a well known magnitude. Then you can just add that constant to these magnitudes (it could also be negative) and get the real magnitude.
Thank you very much for that information! I'm really looking forward to your future videos! :)
You're welcome! I hope to put out more soon
Hey Richard! I just found out that you can get a psf function graph out of Pixinsight too, just use the "Plot 3d graph" for that and make a little Preview over a star.
amazing tutorial, thanks so much
Thanks!
In the days of film nothing like this possible. With a few short extra exposures the blown out systs could be instead obtained with nice profiles to replace.
Great video on a fairly intensive set of steps. Question: The averaged star in your PSF has many pixels and quite a few of those appear to be set near maximum brightness. My PSF view shows only about 60 pixels and only the center one is at the brighest level. I am wondering why mine turn out this way. Is there a setting for this? Or do you have any suggestion as to why my PSF does not look the same as yours?
I'm not sure, it would be helpful to have a picture of your PSF so I can compare and see what's going on. A guess would be that you need to select better and/or more stars to average, or that there's a view setting that's different.
Thanks for the amazing video :) How did you get those 3D plots for PSFs?
Richard, thank you for this excellent tutorial. It really helps to understand how and why to use the Deconvolution tool. You mention a number of times that deconvolution should be done as early as possible in the workflow; would it be better to do it before or after DynamicBackgroundExtraction?
Sorry for the late reply! I don't think it makes a difference. Technically speaking, it should be the first thing that's done after calibration, before DynamicBackgroundExtraction and anything else, because the math and process is designed to work on untouched linear data. But in reality, the effect of DynamicBackgroundExtraction is too minimal to be a problem for deconvolution, and in fact can be helpful for your own eyes in sussing out the subtle details that come from deconvolution!
great video!
How can i get these chart that u where explaining on
I used a program called IRAF, which is used by astronomers to process images. It isn't very intuitive to use unless you're used to command-line programs, but it's free. Unfortunately I don't know the software options if there's an alternative, but as long as you don't choose the brightest or dimmest stars in your image, you can be reasonably sure you're sampling good stars.
The deconvolution is by far the most complex process I've seen with such a minimal effect. I will never use it probably.
had the same feelings until I ran the DeconvolutionPreview script ... Have a try.
where can I download the PixInsight am trying unsuccessfully on the site
Really a wonderful, clear, well paced video. Thank you! I have the same question as expressed below. What did you use to make those neat little PSF function graphs?
Thanks! I don't know how better to phrase it, so I'll just be lazy and copy/paste from another thread:
To get the PSFs that I used for demonstration (aside from the Airy function which I pulled from Wikipedia) I stuck my image into a software called IRAF (Image Reduction and Analysis Facility). It's pretty much what it sounds like - software to reduce and analyse images, and it's used for scientific research pretty much everywhere. The PSF graph is actually how you're supposed to decide whether to accept or reject certain stars as PSF candidates; if it looks noisy or saturated, reject, if it looks good, accept. As far as I know, PixInsight won't really let you see a graph of one, so that's why it's good to just aim for semi-bright isolated stars and do many of them.
Excellent, really helpful.
I have a question, I never know if I need to stretch my image before to start some processes or after, at the end, (the linear or non-linear mode), could you explain what I should do and why ?
Thanks a lot :)
That's a great question, and I'm afraid I don't know enough to produce a full list of PixInsight functions and their optimal location in a workflow, especially since astrophotographers use their own personal workflow.
That said, there are some basic rules to live by:
Most work will be done in the nonlinear regime. A lot of the information in linear data is not useful when it comes to aesthetics, because it contains very little in the midtones for most images (ie galaxies are dim, stars are bright). So stretching occurs very early on, because at the end of the day processing is about making the most of the "other" data, which is best manipulated post-stretch.
Some tools assume you're working with linear data, such as deconvolution. Pre-stretch, you can make analytical approximations of the physics that produced your image data, because there is a very simple relationship between incoming light and what your detector "sees". Deconvolution is an example of one such function: it assumes you're dealing with "raw data", so any changes you make to your image prior to deconvolution will make deconvolution less effective in rough proportion to the number and amount of changes. Noise reduction has a lot of literature pre-stretch and post-stretch, and plenty of debate as to what "the best" approach is. I'm still learning in that area, so I can't advise specifics, but it's worth looking into pre-stretch noise reduction to at least get informed and decide for yourself whether to include that in your workflow.
Most tools however don't make assumptions about linearity. Most common tools, such as what you find under IntensityTransformations don't. The only "necessary" tools that do are the ones that go into calibration, like ImageCalibration and the suggested settings for ImageIntegration when stacking, but you wouldn't stretch while you're doing calibration anyway.
Thanks for the video ! :)
Have you done one on noise reduction? Don't seem to see one. Would be very helpful :)
Thank you :)
I've actually been trying to figure out what a good next video would be. Noise reduction it is! Thanks for the tip!
Hi Richard. Are you sure about your web page link? It brings you to a Stethoscope site ??
Oops! I let the domain name lapse because I hadn't done astrophotography for 2 years. I should remove references to this...
How can I see the star profile? (the pics you are showing in the vid)
I made them using IRAF, a command-line program for image processing and analysis. I couldn't find a way to do it in PixInsight, and am not sure it's possible.
should deconvolution be done before DBE or can it be done after as well?
In theory it should be done before, though I don't think it will make a big quality difference. I do it after colour calibration usually. My logic train goes like this:
1) The PSF analysis works best when all pixels have been modified in a uniform manner throughout the entire image. This is because using a well-sampled PSF would well represent what the deconvolution algorithm will simulate "removing", and you'd get the best representation possible.
2) In reality, minor differences are unavoidable across your image due to a variety of sources. "Uniform" isn't a word you normally see around raw data. But the idea still holds that the less change applied to your data, unless absolutely uniform across pixels and linear in nature, the better result you can get from the PSF analysis.
3) Calibration frames and the stacking process come before deconvolution, so they technically "harm" the best possible result. However, the signal gains and sensor corrections they provide result in a stacked image that is much more "uniform" in pixel-to-pixel nature than any individual image. Thus deconvolution performed on a raw image pre-calibration will produce a worse result than a calibrated and stacked image.
So from (3) I calibrate register and stack. From (1) I would want to go straight to deconvolution, but in practice some of my colour gradients can be wonky. I could extract a luminance channel and work on that, or I could classify minor non-uniform differences added from the colour calibration process under (2). In practice, colour calibration can darken my background sky if one colour was dominant, so it's not always "free" to extract a luminance pre-colour calibration. And if I'm going to work on the RGB image, I don't want any weird colours to be exaggerated by deconvolution, associating weird and steep local colour gradients with high frequency structure. (This happens more often than not in DSLR data that isn't taken in the deep freeze of winter.)
So it seems to be a "pros outweigh the cons" situation for doing it post-colour calibration (and thus post-DBE), as far as I'm concerned. Though that really lands on me pinning affected colour noise as "minor" under point (2), which can sometimes be data-dependent. But really that's where the tradeoffs end; I wouldn't push deconvolution beyond almost any other function in the workflow at that point.
Do I have to drizzle my picture before or after the deconvolution?
Deconvolution should be applied to the stacked image, so drizzle comes first as part of the combination process.
Thank you
You're welcome!
whoa slow down a little Rich you're flying round the functions a little too fast for me, know this is probably aimed at intermediate Pi users but having to keep going back and trying to see how you've done a couple of the function bits a couple of times. well practice makes perfect as they say keep up the good work still using your stacking tutorial cheers
Sorry about that, I'll try and pace things a little better for the next one!