Man watching it in 2020 after having watched dozens of Pixinsight videos and read dozens of tutorials and I have to be honest ... it's the first time someone explains what EXACTLY all the checkboxes and settings mean in a comprehensive manner ... Chapeau my friend!
First tutorial that explained "why" we should do a certain thing or choose a certain value, etc. This really helped a lot, Richard. Thanks! Now I need to get to work building better Flats!
Yes outstanding, others click, click and click and you can't even see the screen and not really interested in teaching, like the OPT guy. This is great!
This is an incredible, well thought-out, and well-explained tutorial. It's far above and beyond any other free video tutorials that I've seen for manual calibrating in PixInsight, and has helped me a great deal. Thank you so much for taking the time to make this and other tutorials.
Years later, and I still come back to this and your other videos. Richard you really have a gift for instruction. If you are still doing this stuff I would love to see another of your PI Vids. maybe on some of the new features? Thank you again so much for what you have given.
Thanks a lot! =) I always intend to come back and make lots of videos, but the hobby has fallen to the wayside in recent years. I've been trying to pick it back up, and when that happens (and I'm up to speed with those features) it's on my list!
Well well, now this is an amazing surprise. There are hundreds of youtube tutorials available on this topic, however very few go manually instead of batch preprocessing. Now this is a video you learn from! Not only do you tell us what to click in pixinsight, but I particularly appreciate that you explain to us why you do what you're doing. I also appreciate the speed in which you handle your files, you have a good mouse and a fast hand and obviously very capable to handle a computer. That makes it a joy to watch. I think after watching this video I can uninstall DeepSkyStacker and 'go pro'. I will recommend this video to anyone I meet who has questions on this in the Netherlands. By the way: it does not matter 1 bit that you didnt select the calibrated lights, it is totally irrelevant. Thank you for this excellent free tutorial! Subbed, liked, bookmarked, please marry me, et cetera.
Why doesn't it matter that he didn't select the calibrated light frames? I see no options that point to them, nor do I see any calibration frame selections in the star alignment tool. The Image Integration tool also has no selections for the master calibration frames. So, how does calibration play into anything, then, if you aren't adding them back into the workflow? I ran through this tutorial and still had a bunch of vignetting and a bad column of pixels, despite following it step-by-step. Once I went back and chose the calibrated lights instead of the original RAW files, all those issues went away.
Richard great job! You explain much more than integration in how PixInsight works. Some of these things I have never heard explained so clearly before. Thank you, thank you!
THANK YOU RICAHRD BLOCK for your excellent non verbose ,commonsensical, sometime humorous TH-cam videos without which I would've been dead in the water.
Beautiful tutorial. I find these tutorials the most useful and up to date than any of the others I have found online. Thank you very much for taking the time and effort to make this (and your other videos). Much appreciated!
Excellent tutorial Richard. instruction and explanations were very clear and evenly paced. Was able to follow each step with my own images with success! Thanks!!
This is a really helpful, well-explained tutorial. Thank you for putting this together! I realize it's already almost 6 years old, but it's still very helpful to me as a newcomer to PI. I do have a couple issues with it, however... As noted in other comments, the registration step should be using calibrated light frames, not the original RAW data. ...and... I ended up with a weird final stack, where the red channel was gray; (K) instead of (R). I needed to go back and debayer my Canon .CR2 RAW files with the RGGB pattern (I did this before registration, as some comments elsewhere said to do this at that point), and then I finally got a color final image.
Thanks! Yes it was a mistake to use the original images and not the calibrated ones. I'm not sure why the same process yields greyscale images for you; my guess would be a difference in RAW import settings: either something custom on one of our ends, or a default option that has been introduced in the time since the video was made. But I'm glad you have a workaround - when in doubt, explore and do your own thing!
Your videos are the best! Without your extremely in depth help, as oppossed to just speeding through your tutorials, I would still be far more lost in this progrm.
This has been a great and continues to be a great aid in my processing. I don’t do it often enough for it to get ingrained so I come back and refer to it often. Thank You
Excellent tutorial, Richard. I encountered an error when I tried to process my flats. I was getting an error: incompatible image geometry . I posted this on the forum and was instructed to: Change the default format preferences for DSLR_RAW: from View menu select Explorer Windows/Format Explorer. Double click on DSLR_RAW and the preferences window is shown. Its the "image flip" setting that matters and it should be on. Or just click the "Pure Raw" button at the bottom of the window and that will assign the correct settings. I made that change PI was happy. Thought I'd pass that along in case other encountered it. I like you method of executing the steps one at a time. I tried to use the automated pre-processing but could never tell where my errors were.
Oh wow thank you so much for that tip! In the latest update of DSLR RAW, I was getting the same error, so I ended up reinstalling the platform minus the one update. I decided to ignore the update since I didn't strictly need it, but I prefer your solution!
Thanks very much for these videos Richard. Very informative. Some day I will learn WHY I'm actually selecting all those options. But for now I'm happy my pictures are looking good.
Thank you Richard. Will try this out on the weekend. Definitely looks like a better way than batch processing. Had a lot of trouble with batch processing in the past.
Keith McLEAN-GIBB I've had issues as well, and really was never satisfied leaving calibration up to a script. I like inspecting the process at every step, making sure I'm getting the most of my data. Hope it works well for you too!
Time index 27:07, I was wondering about the calibrated, went back using calibrated for reference and targeted images. I find this video so helpful getting started and how to make cr2 files to fits. I believe this will solve my ransac errors for batchpreprocessing.
Thank you for thr tutorial. I was using the batch preprocess and the images were strage looking. From now on I will calibrate and stack like this. Cheers from Spain ;)
Excellent Richard Bloch. Please make more video's to enlighten us! ;-) There are so much total garbage 'tutorials' out there, and so precious few which really teach you how things work - like yours. Keep them coming! ;-)
In case it helps anyone, I was having my images show as Gray instead of RGB as mentioned in a few comments. I noticed that if I select "Pure Raw" in PI it defaults to the "Create raw Bayer CFA image". I changed it to "Create raw Bayer RGB image" and it was better. I still had other "color" issues so I wound up doing "de-Bayer RGB" and then selected the box "No image flip".
Still use this after first using years ago. I note that in the latest PI updates there is no longer 'noise evaluation' under weights in image integration. The default is now PSF (Point spread function) signal weight. There is also 'SNR estimate' which to me sounds like the closest to evaluate noise.
This was very helpful.. Thank you for it .. I will be taking the barks bias frames and follow your steps. I really needed it . I have lights of Barnard33 and when I tried to process it came out crappy not happy at all . So I will follow this and see Thanks again...
This is an excellent tutorial. Props and respect! The one thing I am struggling to understand, even with this great tutorial, is taking into account flats created over multiple nights. As we all know, we seldom capture all subs for SHO or LRGB in one night. How does one process the flats taken over multiple nights with ever changing optical artifacts created from breaking down and re-setting up equipment, more dust donuts, camera not perfectly aligned like the night before, etc...? Do we create master flats for each night and then create a master-master? How does one do this?
Using Winsorized Sigma Clipping with the latest version of PI results in an image full of bright pixels that appear to be hot pixels but aren’t (don’t line up with the hot pixels in a dark frame) in a dark grey background (default STF applied). Linear Fit Clipping is what is recommended in the PI book by Warren, and this results is an image with a light grey background with no visible bright pixels. The same “hot” pixels are there, but very dimmed and requires overstretching to see them. So I’m not sure which method is best without applying each one to Flats and Lights for comparison. I’ve been told that CMOS cameras do not need Bias at all, whereas CCD cameras definitely do. (Hope I got that right.)
Agree with the comments below. Great tutorial, to the point. Thanks! It helped me a lot to move faster in PixInsight, even though I am already familiar with the basics. After following the complete tutorial I have a question. Based on the comments, I read you prefer working with RGB images directly instead of grey+debayering. In case I want to import and work with pure RAW/Grey and debayer later in the process (for example to apply Cosmetic Correction); between what specific steps in your tutorial should the DeBayering process be applied? After following your tutorial I applied Debayering at the end - It looks good, but I am not sure it's the right step. Thanks for a great tutorial again!
Great tutorial. One small nitpick: You said flats can be taken using the pre-dawn sky. I think that should be post-dawn or pre-dusk. Pre-dawn is when the sky is still dark.
TheWonkyAstronomer Good call, I suppose I shouldn't throw the term 'dawn' around so loosely! I usually aim for 20 minutes before sunrise, when stars can't be detected
stacking biases th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=3m02s stacking darks th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=10m15s calibrating flats th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=11m28s stacking flats th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=17m09s calibrating lights th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=23m53s registering lights th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=26m06s register manually th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=29m41 stacking lights th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=31m56s I like this tutorial, becaus the author knows what he's doing and is explaining it very well. Thanks Richard! Since i've been working along this tutorial several times, i created some shortcuts for easier access of the different topics.
Hi, Rich. Great tutorial. However, I am having a couple of issues and I was wondering the best way to discuss them with you. Since this tutorial is somewhat old, I want to be sure that you are still helping in this venue.
Oh dear, you're absolutely right! I'll tag the video at the appropriate part with a note when I'm off mobile. Good catch! A waste of time to calibrate lights and then discard them!
@@seanberrett8980 Not a dummy at all, it seems the tag has been swallowed up by TH-cam changes in the years since it was added. I've gone ahead and re-introduced it at 26:31 but unfortunately it's buried in the CC options now. Thanks for the note!
@@richardbloch6883 I'm a brand-new user to PI. I just ran into this, as well. Suddenly realized I never used my calibrated lights for anything and I started digging around, looked in the Inside PixInsight book by Keller, and finally discovered I should be using the calibrated lights in the registration step. It may be worth rerecording this tutorial video with the proper steps, as I imagine many newbies will get tripped up by this, just as I did. Especially with the surge in new astrophotographers getting into the hobby.
Hi awesomely clear tutorial. Thanks for that. I don't understand why do you align lights that are not preprocessed ? The calibration isn't made on those frames so the final result is not calibrated no ?
Hello. I am following step by step, and I notice that after making the Master Bias, you then go to Image Integration (at 37.06)for the Darks. Should you not Calibrate the Dark frames with image calibration (with the newly made Master Bias frame), before selecting Image Integration? Thank a lot !! ☺
@@richardbloch6883 Just came back from my second field trip @ anza after testing and learning at home, your videos are amazing ! thank you for your dedication. I am still getting a bit of "rain" on my S master light, but is not showing on H and O master lights, this after using dithering with Maxim DL and PHD Dither app just recently released, I was only able to get 3 images of 4 minutes on the S channel, but also got 3 good ones with H and 5 good ones with O, any ideas on why this may be happening? ( I used 2x2 binning on my ZWO 1600 Mono Pro) do you know how this can be erased/mitigated on post-processing?. I also noticed no difference when using and not using a master dark, I used a "Single dark frame" when exposing all my lights, my setup is a Celestron HD 9.25 with a Hyperstar and autofocuser, the flats, darks and flats where originally done with "single dark frame" as well. Thank you for your help
@@williamleveson-gower5088 Unfortunately I don't have much experience with dithering or MaximDL so I'm not sure where to point you on that one. Master darks are more useful when there's more dark current, and most useful for science rather than eyes. I've also found that the quality of dark used for subtraction doesn't make much difference for data collected in winter, but does in summer.
Hi Richard. This tutorial is still a favorite of mine, but I wanted to ask a question about LRGB or narrowband CCD imaging. How would I go about creating a Master light frame for each of the filters I've used on a monochrome CCD camera for color assignment in post-processing? Lum, Ha, Oiii, Sii for instance. Thank you.
The way I do it, which is the long way round shown here, is to do the process for each filter independently. The darks / biases can be applied to all frames, but then per filter I do the flat calibration / combination, apply to relevant lights, all the way up until the lights are all calibrated and ready for registration. At this point the process recombines, and all lights are registered. I combine registered lights into master frames per filter, so for LRGB that's 4 master frames. (If you're doing a synthetic L with only RGB data, I'd wait until you have just the 3 RGB master frames before making the synthetic L.) These are the frames you can feed into LRGBCombination to make a single colour image.
Hi Richard. First, excellent tutorial. Thank you for your patient explanations. You stressed how important it was to learn to take proper Flat frames. Can you recommend a source for that information about taking proper Flats? Again, thank you!
Unfortunately I don't have a link to where I originally learned this, because it was in a hardcopy of a CCD manual, but there are plenty of good videos out there that focus on how to take good flats, and/or different techniques people use. The most important thing is to make sure your flat field is, well, "flat". You want as uniform a light source as possible. I image all night, and about a half hour before dawn I point my setup at the zenith and take some sky flats. (If by chance a bright star is near the zenith I'll point slightly off, since you want to avoid other sources of light). Another good way to disperse the light is by covering the front of the telescope with a translucent colour-neutral sheet. AstroBackyard has a good primer on good flats here: th-cam.com/video/g3zDn-8s-_k/w-d-xo.html
Very good tutorial Richard. However I have a problem, when I integrate my files, darks, bias, lights or flats Pixinsight automatically convert transform the RGB into Gray. I do not know how to solve that. Could you lend me a hand?
Thanks alot Richard just to note for others this is great if you are making images on itelescope.net. They provide calibrated light frames that have darks lights biased already applied to download so I am new at this but you can begin at Star Alignment with the calibrated light frames, and I am most grateful because the Batch Processing script is no good for me.
Excellent tutorial, I used to do the alignment and stacking with DSS to produce a TIFF file and then process with PixInsight. Now I am going to try preprocessing with Pixinsight too. I have a problem with FLATS though. Since focus changes with temperature by night and darks are affected by temperature as well, how am I going to get the right FLATS. Do you have or know a tutorial on FLATS taking? Thank you in advance and congrats.
Richard, at 25:45 I see you are hovering one image on top of the other and you see them stacked together... It doesn't work for me. What setting should I turn in PixInsight so I could put one image on top of another and see them together? Thx!
Michael Kalika Hi! In the newer download of PixInsight I think that option is 'off' by default. You have to go to EDIT > GLOBAL PREFERENCES > SPECIAL GUI EFFECTS and make sure "Translucent workspace child windows" is checked off, and hit 'apply' in the lower left (the circle). That should do it!
Question about using mono camera and filters: When I do registration using the StarAlignment module, should I be adding the files from ALL my filters, or just do them one filter at a time? If I do them one-filter-at-a-time, can I run the SA module again later and just add the 2-3 master files? Will that even work? Thank you for such a great tutorial! You should make more videos!!!!
Ha! Ok, I paused right when I got into trouble. That was right before you discussed DynamicAlignment. I guess that could be used on the masters if I decided to register for each filter individually. Excellent tutorial. You really covered all the bases!
For creating a master flat file, what do you do if you have both flats and dark flats? Do you add them all to the IC module and do them all at once, do you create them in IC separately and then integrate them together, or is there a third thing? Thank you for posting these!
Danny MacDonald If you have flats and dark flats, you'd merge your dark flats like the regular darks. Then select your flats in the IC module, and select your master bias and your master dark flat (leave master flat unchecked). Then continue on merging your flats as in the video. It's kind of the same process as using regular darks on the flats, except you're using one set of darks when you use IC on the flats, and another set when you use IC on the lights
Thanks you Richard, This is one of the best tutorials I've viewed but I'm having a problems. When I load a bias frame it comes up as "GRAY" instead of "RGB". Could you please tell me how to change this, I know it's simple but so am I. Thanks Dick
I tried this but when it came to stacking the bias frames, the result would have a white block covering the final 1/3 of the image. Almost like the orientation was turned 90 degrees. Any ideas how to fix this?
Very useful video. One question - I was following this tutorial and I get this warning when calibrating flats: No correlation between the master dark and target frames (channel 0) and the same thing for channels 1 and 2. I tried following the steps again - but I get the same warning.
This happens when are used the wrong darks, with different exposure of the flats. For this stage you have to use the darks of the flats and not the darks of the lights.
I'm trying to use these techniques for an LRGB filter set. Should I calibrate all of my Flats by filter and put them in their own master files then Integrate each filter set together and then Integrate all the filters together to make one Master Flat? Thank you.
Sort of! For multiple filters, each filter set should go through the reduction process. Darks and biases are good for them all, but you'll want to (for example) calibrate the R flats, then use the calibration frames to correct the R images, then stack all corrected R images into a single R image. The same for L, G, B. The end result should be 4 stacked, calibrated images, where each image was calibrated by its filter's master flat. In the end you'd have 4 master flats, one for each stacked image. Then LRGBCombination can merge your 4 final images into 1, but don't merge those master flats!
Excellent tutorial!! After pulling my hair like 40 days of headache. You video really help me stack my very 1st image. But I did exactly everything you did but I don't have a color image. Why? and How do I get a color image?
I am using a one shot color and followed you video step by step. when I went to image integration I get this ---- Computing image statistics: - *** Error: C:/Users/freeo/Desktop/SGP/Processing Files/regestered/barnard33_-10C_180sec_1x1_dt_193316_frame1_a_d_r_c_r.xisf: Zero or insignificant signal detected (empty image?) Now one thing I noticed was my image was not green it was a bright red never saw that before , my image was all green until I dabayerd it. I have not idea what I am doing wrong. Any ideas ?
Richard Krause Hi Richard, I did use a 'one shot colour' camera. I haven't done anything special to my bias, so I'm not sure why yours is reading as grey. If you want to change the colour space from greyscale to RGB, you can go to the top menus and go IMAGE > COLOR SPACES > CONVERT TO RGB COLOR and it should change the image for you. I'd have to do some research to see if it actually matters whether it's greyscale or not. My first instinct is to trust the guys over at PI and say that the ImageCalibration tool reads the raw image data, and doesn't interpolate to assign colour values during calibration. That's the ideal way to do it, because colour values in one-shot colour cameras are created by reading neighbouring pixel values, and when calibrating with bias, etc., one is interested in a pixel's behaviour on its own, without neighbouring pixels' influence. It would be odd for such a robust program to neglect that; but I don't yet know for sure. Of course, if it wasn't a one-shot colour camera you've used, and you've used colour filters, all your images will be read as greyscale. In that case, the bias (and darks) are colour-independent because no light reached the sensor, and one set can be applied to all your filtered images equally
I followed the entire video up to making a master flat by incorporating master darks and master bias. When I hit execute, you said nothing appeared on screen but went to your "process" file and they were there. I did the same thing, but when I clicked on my Process" folder, it was empty. I got this message thruoght the procedure...."Warning...No correlation between master dark and target frames ( Channel 0) ( Channel 1). Now I'm using an iMac and I tried doing it over again using your steps exactly, but my files were either TIF, DNG or PNG. What is wrong....I have a feeling its file types is issue.
I know my computer is just barely able to handling the processing from Pixinsight, but you have got to have a power house pc. When you hit run within seconds you have results. For me, when i hit run i go out for dinner and com back and wait some more.
I cut out the part where it runs, except sometimes I'll leave it when I'm saying something. But for deconvolution in particular my computer is a timesink like everyone!
I have another question for you. I'm getting a lot of green pixel rejection in my rejection_low when integrating my bias files. Using the exact same settings you are. What could be causing that?
I am getting a Warning: No correlation between the master dark and target frames when calibrating the flat frames. It seems to be working but wandered what this is. I am using a DSLR in RAW format but specified the output extension to be a .fit. Any idea what this is?
+Jeremy Barrett Sorry! Not sure how I missed this comment...Late answer, but basically it's a side effect of checking the 'optimize' button for the dark frame section in ImageCalibration. PI does some noise evaluation (statistics based, it doesn't just blindly trust the temperature/exposure length info), and if it doesn't find a correlation between the noise in your dark and in the things you're calibrating, it doesn't apply the dark subtraction. This is because if it did, you'd likely end up with even more noise than you started with, because you're introducing a further element of randomness into the image (it's just how those statistics work). That's part of what the 'optimize' feature does. If you don't have that checked, PI will be a little more blind about applying that dark frame, even when it probably shouldn't. This is a warning that you'll likely see more often on flats than anything else, because flats tend to be much shorter than your darks, and have very little thermal signal. Likely little enough to make no significant difference, and certainly little enough that it's hard to build a statistical model. If you're particularly worried, you could always take darks for your flats, and then when calibrating your flats use THOSE darks as a master dark, and uncheck the optimize button for that one. In that case, you'd always be dark-subtracting your flats, but be able to rely on the physics behind it rather than let PI decide what to do. But if your flats are less than a second in length, I'd argue that you'd never be able to tell from the final astro image whether you had dark-subtracted flats or not.
Hi Richard I have 1 further question. I notice that your RAW files when loaded into PI are RGB files. For some reason mine seem to be grey and I have to debayer them between the process steps of calibration and registration, to get the colour back. When I was using Deep Sky Stacker, the files always turned up as RGB files, presumably DSS debayered them automatically? Any ideas on why my files were grey? I am using a Canon 550d camera outputting RAW cr2 files. Thanks for your help?
Alec Alden Hey Alec, that's interesting...PI hasn't given me that issue before. Is it possible for you to provide a download link for one such file? I may be able to give you a solid answer instead of a number of guesses. Off the top of my head it sounds like a system setting in PI, but I'd have to check that your file loads RGB on my system before I go hunting for such a setting!
I always use this tutorial to stack my images, I have a question though, we do not debayer the images before stacking them or where do we do that exactly during this work flow?
Hi! Just joing the crowd praising you for this. It's been extremely helpful for me. However, the debayering questions is a bit puzzling. You don't debayer - why not?
+Magnus Larsson Hey! To be honest, I never found it to make a difference in my images. I know that theoretically speaking, my colours aren't going to be anywhere near real unless I do, but here's something I just processed without debayering: i.imgur.com/ZxZCuIW.jpg And here's one that was debayered: i.imgur.com/j7wyQsY.jpg I tried processing them as identically as possible, though one still managed to be brighter than the other slightly. But no real difference! So I just skip the step entirely, never really had a need for it.
Hi Richard Great Tutorial I followed it but unlike you I found that my master darks and master bias, didn't look a lot different between a single frame and the master. Now I know they should. Any idea what I am doing wrong?
There could be a number of reasons. How many individual frames are you integrating, and what was the temperature of the darks? Also, make sure your combination method is set to "average", and not "sum"
Richard Bloch Hi Richard I have re -run virtually the same data in the Batch Processing and compared images and hey presto, there are significant differences. Only about a week into PI so very much at the bottom of the learning curve. I am attempting the data again and will get back to you. Not sure how to provide you a link to the data Without tutorials such as yours there is no way that people like me would be able to take up astrophotography at all. Very many thanks
Richard Bloch Hi Richard, apologies are in order for wasting your time. When I ran things again and checked the images there were noticeable differences, not sure what I was looking at the first time. However, there are lots of files to keep track of and I can only assume that I was looking at the wrong ones! I did run into trouble with an error message "incompatible image geometry", However, I have managed to resolve this by making changes to Format Explorer to make DSLR_RAW files the default input. Thanks for your help and superb tutorial. Alec
Thank you!!! Great tutorial. However, In my case it seems that somehow the Master dark is ignored and the hot pixels are not removed. The hot pixels are clear on the master dark and on lights, but after calibration of lights with master bias and master dark they dont get removed. (I dont have any flats, so its uncheked)
B1063N I mean the calibrated images. My calibrated images still have hot pixels, I am not sure if thats ok. After registering and doing the final image integration, most of the hot pixels are gone, however, that is another process.
+B1063N I figured it out, so if you dont add any flats most of the hot pixels remain, I have usedthe same process but with flats this time and hot pixels are gone on calibrated images.
+B1063N Hey, sorry I missed your earlier comments! I'm surprised that the master dark didn't remove the hot pixels on its own, though if you're using the screen stretch function to see the hot pixels, it could be that they aren't really "hot" pixels, but just have a slightly higher sensitivity than neighbouring pixels, which is what flats fix. If they're hot in the sense that they're easily visible without using screen stretch, then that's a little more strange...
+Richard Bloch My master dark is completely black without the screen stretch function. Same as in the tutorial. The "hot pixels" of the stretched master dark perfectly match(for the most part) the "hot pixels" of a stretched light frame. However, they do not get removed, so the process flow in my case is somehow failing to fix the "hot pixels" without flats on the calibrated light subs. Oh well, as long as they are getting removed at the end it doesn't really matter. In any event I have compared this process to the usual DSS stacking on several of my images, and I find this procedure for stacking far far far superior in terms of the final stacked image.
+B1063N I'm glad you find this works for you! Just a thought on the hot pixels. Because they aren't visible until you use the screen stretch, it could be that they look hotter than their neighbours because they're just more sensitive pixels. That way, they'd show up hotter both in your lights and in your darks. When you subtract the master dark from the lights, they still look 'hot' because even without the dark current, those pixels just detected more light on account of being more sensitive. If that's the case, then when you take a flat and normalize it (which is done in the calibration part), that pixel shows up as 'hot' because it was oversensitive, and when the flats are divided, that pixel's brightness is diminished accordingly. Thus the pixel would not appear 'hot' after flats have been used. (It may be hard to see in the master flat, though, because each pixel in a flat gets a healthy amount of light, which is not the case for lights/darks.) As you say, in the end the problem is taken care of, but that's my best guess as to what's going on given what you see.
I've tried testing debayer at many stages in the preprocess workflow, but I haven't found any appreciable improvement over the automatic default debayer!
Why would my darks have so much more noise than yours? I use a 70D and I took a 2 minute dark at 800iso with the cap on the scope. When I look at it stretched there's a massive amount of noise.
John Drake Yep, that'll do it! I shot this in Canada in March, so my sensor temperature was 1C according to the EXIF data. I believe it, as I recall it was about -20 out that night. Noise doubles with roughly every 6 C increase in temperature, so we'd expect (all else being equal) the temperature to be responsible for a 64x increase in noise. My shot was 5 minutes, not 2, so I have 2.5x the noise you do due to exposure time, but 64/2.5 is still about 25x. So your shot would be way noisier just due to the temperature difference (again, 25x specifically neglects camera differences or ISO differences, but it serves to illustrate!)
Why don't you use Batch Preprocsessing for your Lights, Darks and Biases? I am still learning and someone else in a TH-cam tutorial was using that. Seems easier.
It IS easier to go through the batch calibration! But I stick to the piecemeal method for two reasons. One is really simple: I'm just very particular about what gets done and when it happens. Using the individual tools allows me complete control over every single step; the batch tool has most of the existing options, but not all, which (in rare cases) can be limiting in terms of tricky data sets or incomplete data sets. For example, if I wanted to generate a synthetic flat, I could insert it into my stream here or use it in the batch tool, but if I want to compare the effectiveness of applying different synthetic flats, the batch tool will take more time to set and use than just doing the flat calibration step two or three times here. Ultimately it doesn't matter, but I just like having one process that applies to everything. The second reason is troubleshooting. PixInsight will return errors when things go awry, and you can set the error policy to abort when detected, but there are a few behaviours that are unwanted that don't get caught as errors. The easiest example is through StarAlignment; it's possible, and I have an unfortunately non-trivial number of data sets where it happens, for the settings to look OK, and for the tool to operate OK, but it doesn't fully calculate the image displacements. It calculates a few pixels, or sub-pixel differences, when really it should be many more. With the batch tool, I won't know anything's gone wrong until the very end and (aside from this one glitch I recognize immediately) it may not be so easy to identify where along the path went wrong, and what caused it. Again, I could just go through the batch tool for most data sets until I get a "weird" one and then try this, but my personal preference is just to do the one thorough approach and to do it once, and finish the calibration process by the time I reach the end.
Thanks. That is good to know. I am just starting out with only a few images under my belt with PixInsight thus far, but if I run into a problem it's always good to know there are options.
Hum ... it seems that you made a selection mistake when opening the files, you should have registered the CALIBRATED files but you end up registering the original CR2 files and subsequently the integration takes place on the files that were not calibrated. Regards, PA
After you do all the calibrations, but before you register the images. Debayering is most accurate when a given pixel is in its original spot - once alignment starts shifting and rotating, debayering gets messy.
![How are Images Compressed? [46MB ↘↘ 4.07MB] JPEG In Depth](/img/n.gif)
Man watching it in 2020 after having watched dozens of Pixinsight videos and read dozens of tutorials and I have to be honest ... it's the first time someone explains what EXACTLY all the checkboxes and settings mean in a comprehensive manner ... Chapeau my friend!
this is one of the best videos I've encountered so far explaining the calibration and integration - thank you!
First tutorial that explained "why" we should do a certain thing or choose a certain value, etc. This really helped a lot, Richard. Thanks! Now I need to get to work building better Flats!
Yes outstanding, others click, click and click and you can't even see the screen and not really interested in teaching, like the OPT guy. This is great!
This is an incredible, well thought-out, and well-explained tutorial. It's far above and beyond any other free video tutorials that I've seen for manual calibrating in PixInsight, and has helped me a great deal. Thank you so much for taking the time to make this and other tutorials.
Thanks for the kind words! =)
Years later, and I still come back to this and your other videos. Richard you really have a gift for instruction. If you are still doing this stuff I would love to see another of your PI Vids. maybe on some of the new features? Thank you again so much for what you have given.
Thanks a lot! =) I always intend to come back and make lots of videos, but the hobby has fallen to the wayside in recent years. I've been trying to pick it back up, and when that happens (and I'm up to speed with those features) it's on my list!
Well well, now this is an amazing surprise. There are hundreds of youtube tutorials available on this topic, however very few go manually instead of batch preprocessing. Now this is a video you learn from! Not only do you tell us what to click in pixinsight, but I particularly appreciate that you explain to us why you do what you're doing. I also appreciate the speed in which you handle your files, you have a good mouse and a fast hand and obviously very capable to handle a computer. That makes it a joy to watch. I think after watching this video I can uninstall DeepSkyStacker and 'go pro'. I will recommend this video to anyone I meet who has questions on this in the Netherlands. By the way: it does not matter 1 bit that you didnt select the calibrated lights, it is totally irrelevant. Thank you for this excellent free tutorial! Subbed, liked, bookmarked, please marry me, et cetera.
Nicely stated!
Thanks for the kind words! I believe if you don't know why you're doing what you're doing, you won't know what to do when things go awry =)
Why doesn't it matter that he didn't select the calibrated light frames? I see no options that point to them, nor do I see any calibration frame selections in the star alignment tool. The Image Integration tool also has no selections for the master calibration frames. So, how does calibration play into anything, then, if you aren't adding them back into the workflow?
I ran through this tutorial and still had a bunch of vignetting and a bad column of pixels, despite following it step-by-step. Once I went back and chose the calibrated lights instead of the original RAW files, all those issues went away.
This is an outstanding tutorial. I've watched it over and over again.
I can't tell you the number of times I have actually used this a reference. Thank you so much for it. :) :)
Richard great job! You explain much more than integration in how PixInsight works. Some of these things I have never heard explained so clearly before. Thank you, thank you!
THANK YOU RICAHRD BLOCK for your excellent non verbose ,commonsensical, sometime humorous TH-cam videos without which I would've been dead in the water.
Beautiful tutorial. I find these tutorials the most useful and up to date than any of the others I have found online. Thank you very much for taking the time and effort to make this (and your other videos). Much appreciated!
Excellent tutorial Richard. instruction and explanations were very clear and evenly paced. Was able to follow each step with my own images with success! Thanks!!
This is a really helpful, well-explained tutorial. Thank you for putting this together! I realize it's already almost 6 years old, but it's still very helpful to me as a newcomer to PI. I do have a couple issues with it, however...
As noted in other comments, the registration step should be using calibrated light frames, not the original RAW data.
...and...
I ended up with a weird final stack, where the red channel was gray; (K) instead of (R). I needed to go back and debayer my Canon .CR2 RAW files with the RGGB pattern (I did this before registration, as some comments elsewhere said to do this at that point), and then I finally got a color final image.
Thanks! Yes it was a mistake to use the original images and not the calibrated ones. I'm not sure why the same process yields greyscale images for you; my guess would be a difference in RAW import settings: either something custom on one of our ends, or a default option that has been introduced in the time since the video was made. But I'm glad you have a workaround - when in doubt, explore and do your own thing!
Your videos are the best! Without your extremely in depth help, as oppossed to just speeding through your tutorials, I would still be far more lost in this progrm.
This has been a great and continues to be a great aid in my processing. I don’t do it often enough for it to get ingrained so I come back and refer to it often. Thank You
This is the best tutorial! Thanks a lot, others are just click click with no explanation.
Excellent tutorial, Richard. I encountered an error when I tried to process my flats. I was getting an error: incompatible image geometry . I posted this on the forum and was instructed to:
Change the default format preferences for DSLR_RAW: from View menu select Explorer Windows/Format Explorer. Double click on DSLR_RAW and the preferences window is shown. Its the "image flip" setting that matters and it should be on. Or just click the "Pure Raw" button at the bottom of the window and that will assign the correct settings.
I made that change PI was happy. Thought I'd pass that along in case other encountered it.
I like you method of executing the steps one at a time. I tried to use the automated pre-processing but could never tell where my errors were.
Oh wow thank you so much for that tip! In the latest update of DSLR RAW, I was getting the same error, so I ended up reinstalling the platform minus the one update.
I decided to ignore the update since I didn't strictly need it, but I prefer your solution!
Just had this issue and click on the "Pure Raw" button and it fixed it! Thanks a ton for this information :)
Richard your video rocks too! :D
That happened to me too. :)
These are the best videos, very straightforward.
Well done Richard. Appreciate it very much mate. Have used all of your tutorials and if you have the time, it would be great to see more.
Thanks Richard, great tutorial! I would not have been able to make my first PI image without your guidance.
Thanks very much for these videos Richard. Very informative. Some day I will learn WHY I'm actually selecting all those options. But for now I'm happy my pictures are looking good.
Thank you Richard.
Will try this out on the weekend. Definitely looks like a better way than batch processing. Had a lot of trouble with batch processing in the past.
Keith McLEAN-GIBB I've had issues as well, and really was never satisfied leaving calibration up to a script. I like inspecting the process at every step, making sure I'm getting the most of my data. Hope it works well for you too!
First tutorial I have found that shows a straight up how to. Thank you!
Thank you, Richard! One of the best videos to get started with PixInsight. Gonna try this now...
Time index 27:07, I was wondering about the calibrated, went back using calibrated for reference and targeted images. I find this video so helpful getting started and how to make cr2 files to fits. I believe this will solve my ransac errors for batchpreprocessing.
Thank you so much for taking the time to put together such a helpful and concise tutorial. Very selfless of you. Cheers.
Thanks for doing these video tutorials. They are extremely helpful. Well done.
I know it's been a while since this was published but it is still useful mate. Well done and thanks.
Dear Richard, thank you so much for sharing that valuable information!
Great video!! would love to see more of your work.
Thank you for thr tutorial. I was using the batch preprocess and the images were strage looking. From now on I will calibrate and stack like this.
Cheers from Spain ;)
Excellent Richard Bloch. Please make more video's to enlighten us! ;-) There are so much total garbage 'tutorials' out there, and so precious few which really teach you how things work - like yours. Keep them coming! ;-)
Outstanding tutorial... You are a great communicator.
In case it helps anyone, I was having my images show as Gray instead of RGB as mentioned in a few comments. I noticed that if I select "Pure Raw" in PI it defaults to the "Create raw Bayer CFA image". I changed it to "Create raw Bayer RGB image" and it was better. I still had other "color" issues so I wound up doing "de-Bayer RGB" and then selected the box "No image flip".
Helped me!
Still use this after first using years ago. I note that in the latest PI updates there is no longer 'noise evaluation' under weights in image integration. The default is now PSF (Point spread function) signal weight. There is also 'SNR estimate' which to me sounds like the closest to evaluate noise.
Excellent tutorial - thank you for taking the time to create this.
This was very helpful.. Thank you for it .. I will be taking the barks bias frames and follow your steps. I really needed it . I have lights of Barnard33 and when I tried to process it came out crappy not happy at all . So I will follow this and see
Thanks again...
Thanks very much for doing this. PI is more than daunting to the new user.
Richard Pattie Glad it helped =) Yes, PI has a steep initial learning curve that turns many off. But it's well worth it!
Thank's for a really good and easy to follow tutorial on this!
Very helpful video Richard. Thank you.
That is an awesome tutorial ! thank you very much. Greatly detailed and explained, as I love. I'll try it asap !
excellent tutorial Richard, thank you to share it
Great tutorial !! excellent !!
I follow step-by-step and have a great integration result !
Best ever tutorial. Many thanks.
more Vids! outstanding work!
This is an excellent tutorial. Props and respect! The one thing I am struggling to understand, even with this great tutorial, is taking into account flats created over multiple nights. As we all know, we seldom capture all subs for SHO or LRGB in one night. How does one process the flats taken over multiple nights with ever changing optical artifacts created from breaking down and re-setting up equipment, more dust donuts, camera not perfectly aligned like the night before, etc...? Do we create master flats for each night and then create a master-master? How does one do this?
Well done, thanks for taking the time to do these! Very helpful!
Using Winsorized Sigma Clipping with the latest version of PI results in an image full of bright pixels that appear to be hot pixels but aren’t (don’t line up with the hot pixels in a dark frame) in a dark grey background (default STF applied). Linear Fit Clipping is what is recommended in the PI book by Warren, and this results is an image with a light grey background with no visible bright pixels. The same “hot” pixels are there, but very dimmed and requires overstretching to see them. So I’m not sure which method is best without applying each one to Flats and Lights for comparison. I’ve been told that CMOS cameras do not need Bias at all, whereas CCD cameras definitely do. (Hope I got that right.)
Agree with the comments below. Great tutorial, to the point. Thanks! It helped me a lot to move faster in PixInsight, even though I am already familiar with the basics. After following the complete tutorial I have a question. Based on the comments, I read you prefer working with RGB images directly instead of grey+debayering. In case I want to import and work with pure RAW/Grey and debayer later in the process (for example to apply Cosmetic Correction); between what specific steps in your tutorial should the DeBayering process be applied? After following your tutorial I applied Debayering at the end - It looks good, but I am not sure it's the right step. Thanks for a great tutorial again!
Great tutorial. One small nitpick: You said flats can be taken using the pre-dawn sky. I think that should be post-dawn or pre-dusk. Pre-dawn is when the sky is still dark.
TheWonkyAstronomer Good call, I suppose I shouldn't throw the term 'dawn' around so loosely! I usually aim for 20 minutes before sunrise, when stars can't be detected
Very helpful video! Thanks a lot!
Whats the final call on evaluate noise box when integrating a master flat? Light Vortex says leave it checked... Great video BTW.
stacking biases th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=3m02s
stacking darks th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=10m15s
calibrating flats th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=11m28s
stacking flats th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=17m09s
calibrating lights th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=23m53s
registering lights th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=26m06s
register manually th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=29m41
stacking lights th-cam.com/video/zU5jJgjKuQQ/w-d-xo.html#t=31m56s
I like this tutorial, becaus the author knows what he's doing and is explaining it very well. Thanks Richard!
Since i've been working along this tutorial several times, i created some shortcuts for easier access of the different topics.
i have no idea how to do flats and when i do the fail, overcorrected in PI
That was awesome!
Hi, Rich. Great tutorial.
However, I am having a couple of issues and I was wondering the best way to discuss them with you.
Since this tutorial is somewhat old, I want to be sure that you are still helping in this venue.
I think you made a mistake when registering your lights. You need to select the calibrated lights instead of the original ones.
Oh dear, you're absolutely right! I'll tag the video at the appropriate part with a note when I'm off mobile.
Good catch! A waste of time to calibrate lights and then discard them!
@@richardbloch6883 Was that change implemented? Asking for a dummy (me).
@@seanberrett8980 Not a dummy at all, it seems the tag has been swallowed up by TH-cam changes in the years since it was added. I've gone ahead and re-introduced it at 26:31 but unfortunately it's buried in the CC options now. Thanks for the note!
@@richardbloch6883 I'm a brand-new user to PI. I just ran into this, as well. Suddenly realized I never used my calibrated lights for anything and I started digging around, looked in the Inside PixInsight book by Keller, and finally discovered I should be using the calibrated lights in the registration step. It may be worth rerecording this tutorial video with the proper steps, as I imagine many newbies will get tripped up by this, just as I did. Especially with the surge in new astrophotographers getting into the hobby.
Hi awesomely clear tutorial. Thanks for that.
I don't understand why do you align lights that are not preprocessed ?
The calibration isn't made on those frames so the final result is not calibrated no ?
And for me after calibration my frames are grey and no rgb. Should I do something you don't need to get rgb version ?
Hello. I am following step by step, and I notice that after making the Master Bias, you then go to Image Integration (at 37.06)for the Darks. Should you not Calibrate the Dark frames with image calibration (with the newly made Master Bias frame), before selecting Image Integration? Thank a lot !! ☺
Hi Richard, excellent video !!, just a quick question, on 26:53, star alignment, I believe you took the non-calibrated lights by mistake?
That's right! It was my mistake. It definitely should have been the calibrated lights.
@@richardbloch6883 Just came back from my second field trip @ anza after testing and learning at home, your videos are amazing ! thank you for your dedication. I am still getting a bit of "rain" on my S master light, but is not showing on H and O master lights, this after using dithering with Maxim DL and PHD Dither app just recently released, I was only able to get 3 images of 4 minutes on the S channel, but also got 3 good ones with H and 5 good ones with O, any ideas on why this may be happening? ( I used 2x2 binning on my ZWO 1600 Mono Pro) do you know how this can be erased/mitigated on post-processing?. I also noticed no difference when using and not using a master dark, I used a "Single dark frame" when exposing all my lights, my setup is a Celestron HD 9.25 with a Hyperstar and autofocuser, the flats, darks and flats where originally done with "single dark frame" as well. Thank you for your help
@@williamleveson-gower5088 Unfortunately I don't have much experience with dithering or MaximDL so I'm not sure where to point you on that one.
Master darks are more useful when there's more dark current, and most useful for science rather than eyes. I've also found that the quality of dark used for subtraction doesn't make much difference for data collected in winter, but does in summer.
Hi Richard. This tutorial is still a favorite of mine, but I wanted to ask a question about LRGB or narrowband CCD imaging. How would I go about creating a Master light frame for each of the filters I've used on a monochrome CCD camera for color assignment in post-processing? Lum, Ha, Oiii, Sii for instance. Thank you.
The way I do it, which is the long way round shown here, is to do the process for each filter independently. The darks / biases can be applied to all frames, but then per filter I do the flat calibration / combination, apply to relevant lights, all the way up until the lights are all calibrated and ready for registration.
At this point the process recombines, and all lights are registered. I combine registered lights into master frames per filter, so for LRGB that's 4 master frames. (If you're doing a synthetic L with only RGB data, I'd wait until you have just the 3 RGB master frames before making the synthetic L.)
These are the frames you can feed into LRGBCombination to make a single colour image.
@@richardbloch6883 Very helpful Richard. Very generous to share your time. Thank you!
Hi Richard. First, excellent tutorial. Thank you for your patient explanations. You stressed how important it was to learn to take proper Flat frames. Can you recommend a source for that information about taking proper Flats? Again, thank you!
Unfortunately I don't have a link to where I originally learned this, because it was in a hardcopy of a CCD manual, but there are plenty of good videos out there that focus on how to take good flats, and/or different techniques people use.
The most important thing is to make sure your flat field is, well, "flat". You want as uniform a light source as possible. I image all night, and about a half hour before dawn I point my setup at the zenith and take some sky flats. (If by chance a bright star is near the zenith I'll point slightly off, since you want to avoid other sources of light).
Another good way to disperse the light is by covering the front of the telescope with a translucent colour-neutral sheet. AstroBackyard has a good primer on good flats here: th-cam.com/video/g3zDn-8s-_k/w-d-xo.html
Thanks Richard! I’ll look for those sources.
Very good tutorial Richard. However I have a problem, when I integrate my files, darks, bias, lights or flats Pixinsight automatically convert transform the RGB into Gray. I do not know how to solve that. Could you lend me a hand?
Thanks alot Richard just to note for others this is great if you are making images on itelescope.net. They provide calibrated light frames that have darks lights biased already applied to download so I am new at this but you can begin at Star Alignment with the calibrated light frames, and I am most grateful because the Batch Processing script is no good for me.
Excellent tutorial, I used to do the alignment and stacking with DSS to produce a TIFF file and then process with PixInsight. Now I am going to try preprocessing with Pixinsight too. I have a problem with FLATS though. Since focus changes with temperature by night and darks are affected by temperature as well, how am I going to get the right FLATS. Do you have or know a tutorial on FLATS taking? Thank you in advance and congrats.
Hey!, the best way to take FLATS is to put a white shirt on your telescope! and use that to take the photo!
Great video, thank you!
A great tutorial... Thx
what DSLR_RAW settings are you using?
Richard, at 25:45 I see you are hovering one image on top of the other and you see them stacked together... It doesn't work for me. What setting should I turn in PixInsight so I could put one image on top of another and see them together?
Thx!
Michael Kalika Hi! In the newer download of PixInsight I think that option is 'off' by default. You have to go to EDIT > GLOBAL PREFERENCES > SPECIAL GUI EFFECTS and make sure "Translucent workspace child windows" is checked off, and hit 'apply' in the lower left (the circle). That should do it!
Richard Bloch Many thanks! :)
Question about using mono camera and filters: When I do registration using the StarAlignment module, should I be adding the files from ALL my filters, or just do them one filter at a time? If I do them one-filter-at-a-time, can I run the SA module again later and just add the 2-3 master files? Will that even work?
Thank you for such a great tutorial! You should make more videos!!!!
Ha! Ok, I paused right when I got into trouble. That was right before you discussed DynamicAlignment. I guess that could be used on the masters if I decided to register for each filter individually. Excellent tutorial. You really covered all the bases!
For creating a master flat file, what do you do if you have both flats and dark flats? Do you add them all to the IC module and do them all at once, do you create them in IC separately and then integrate them together, or is there a third thing?
Thank you for posting these!
Danny MacDonald If you have flats and dark flats, you'd merge your dark flats like the regular darks. Then select your flats in the IC module, and select your master bias and your master dark flat (leave master flat unchecked). Then continue on merging your flats as in the video.
It's kind of the same process as using regular darks on the flats, except you're using one set of darks when you use IC on the flats, and another set when you use IC on the lights
Richard Bloch That makes total sense. Thank you so much!
Thanks you Richard,
This is one of the best tutorials I've viewed but I'm having a problems. When I load a bias frame it comes up as "GRAY" instead of "RGB". Could you please tell me how to change this, I know it's simple but so am I. Thanks Dick
I tried this but when it came to stacking the bias frames, the result would have a white block covering the final 1/3 of the image. Almost like the orientation was turned 90 degrees. Any ideas how to fix this?
Very useful video.
One question - I was following this tutorial and I get this warning when calibrating flats: No correlation between the master dark and target frames (channel 0)
and the same thing for channels 1 and 2.
I tried following the steps again - but I get the same warning.
Me too! Did you fix the problem? How did you do it?
This happens when are used the wrong darks, with different exposure of the flats. For this stage you have to use the darks of the flats and not the darks of the lights.
I'm trying to use these techniques for an LRGB filter set. Should I calibrate all of my Flats by filter and put them in their own master files then Integrate each filter set together and then Integrate all the filters together to make one Master Flat? Thank you.
Sort of! For multiple filters, each filter set should go through the reduction process. Darks and biases are good for them all, but you'll want to (for example) calibrate the R flats, then use the calibration frames to correct the R images, then stack all corrected R images into a single R image. The same for L, G, B.
The end result should be 4 stacked, calibrated images, where each image was calibrated by its filter's master flat. In the end you'd have 4 master flats, one for each stacked image. Then LRGBCombination can merge your 4 final images into 1, but don't merge those master flats!
where can I download the PixInsight am trying unsuccessfully on the site
Excellent tutorial!! After pulling my hair like 40 days of headache. You video really help me stack my very 1st image. But I did exactly everything you did but I don't have a color image. Why? and How do I get a color image?
debayer your files BEFORE. They will go from Gray to RBG.
I am using a one shot color and followed you video step by step. when I went to image integration I get this ---- Computing image statistics: -
*** Error: C:/Users/freeo/Desktop/SGP/Processing Files/regestered/barnard33_-10C_180sec_1x1_dt_193316_frame1_a_d_r_c_r.xisf: Zero or insignificant signal detected (empty image?)
Now one thing I noticed was my image was not green it was a bright red never saw that before , my image was all green until I dabayerd it. I have not idea what I am doing wrong. Any ideas ?
Was this done with a "One Shot Color " camera? The header on my bias frame is "GRAY" while in your tutorial it's labeled "RGB" It is messing me up!
Richard Krause Hi Richard, I did use a 'one shot colour' camera. I haven't done anything special to my bias, so I'm not sure why yours is reading as grey. If you want to change the colour space from greyscale to RGB, you can go to the top menus and go IMAGE > COLOR SPACES > CONVERT TO RGB COLOR and it should change the image for you.
I'd have to do some research to see if it actually matters whether it's greyscale or not. My first instinct is to trust the guys over at PI and say that the ImageCalibration tool reads the raw image data, and doesn't interpolate to assign colour values during calibration. That's the ideal way to do it, because colour values in one-shot colour cameras are created by reading neighbouring pixel values, and when calibrating with bias, etc., one is interested in a pixel's behaviour on its own, without neighbouring pixels' influence. It would be odd for such a robust program to neglect that; but I don't yet know for sure.
Of course, if it wasn't a one-shot colour camera you've used, and you've used colour filters, all your images will be read as greyscale. In that case, the bias (and darks) are colour-independent because no light reached the sensor, and one set can be applied to all your filtered images equally
I followed the entire video up to making a master flat by incorporating master darks and master bias. When I hit execute, you said nothing appeared on screen but went to your "process" file and they were there. I did the same thing, but when I clicked on my Process" folder, it was empty. I got this message thruoght the procedure...."Warning...No correlation between master dark and target frames ( Channel 0) ( Channel 1). Now I'm using an iMac and I tried doing it over again using your steps exactly, but my files were either TIF, DNG or PNG. What is wrong....I have a feeling its file types is issue.
I know my computer is just barely able to handling the processing from Pixinsight, but you have got to have a power house pc. When you hit run within seconds you have results. For me, when i hit run i go out for dinner and com back and wait some more.
I cut out the part where it runs, except sometimes I'll leave it when I'm saying something. But for deconvolution in particular my computer is a timesink like everyone!
@@richardbloch6883 that makes me feel better, haha, thanks
I have another question for you. I'm getting a lot of green pixel rejection in my rejection_low when integrating my bias files. Using the exact same settings you are. What could be causing that?
I am getting a Warning: No correlation between the master dark and target frames when calibrating the flat frames. It seems to be working but wandered what this is. I am using a DSLR in RAW format but specified the output extension to be a .fit. Any idea what this is?
+Jeremy Barrett Sorry! Not sure how I missed this comment...Late answer, but basically it's a side effect of checking the 'optimize' button for the dark frame section in ImageCalibration. PI does some noise evaluation (statistics based, it doesn't just blindly trust the temperature/exposure length info), and if it doesn't find a correlation between the noise in your dark and in the things you're calibrating, it doesn't apply the dark subtraction. This is because if it did, you'd likely end up with even more noise than you started with, because you're introducing a further element of randomness into the image (it's just how those statistics work).
That's part of what the 'optimize' feature does. If you don't have that checked, PI will be a little more blind about applying that dark frame, even when it probably shouldn't. This is a warning that you'll likely see more often on flats than anything else, because flats tend to be much shorter than your darks, and have very little thermal signal. Likely little enough to make no significant difference, and certainly little enough that it's hard to build a statistical model.
If you're particularly worried, you could always take darks for your flats, and then when calibrating your flats use THOSE darks as a master dark, and uncheck the optimize button for that one. In that case, you'd always be dark-subtracting your flats, but be able to rely on the physics behind it rather than let PI decide what to do. But if your flats are less than a second in length, I'd argue that you'd never be able to tell from the final astro image whether you had dark-subtracted flats or not.
All the integrations resulting files in this video are RGB but what I am getting are all grey, can anybody let me know why is that?
Hi Richard
I have 1 further question. I notice that your RAW files when loaded into PI are RGB files. For some reason mine seem to be grey and I have to debayer them between the process steps of calibration and registration, to get the colour back. When I was using Deep Sky Stacker, the files always turned up as RGB files, presumably DSS debayered them automatically?
Any ideas on why my files were grey? I am using a Canon 550d camera outputting RAW cr2 files.
Thanks for your help?
Alec Alden Hey Alec, that's interesting...PI hasn't given me that issue before. Is it possible for you to provide a download link for one such file? I may be able to give you a solid answer instead of a number of guesses. Off the top of my head it sounds like a system setting in PI, but I'd have to check that your file loads RGB on my system before I go hunting for such a setting!
+Alec Alden Richard right, it is a global setting: Format explorer -> DSLR_RAW -> Edit preferences.
I always use this tutorial to stack my images, I have a question though, we do not debayer the images before stacking them or where do we do that exactly during this work flow?
before :). you need to RBG your files before stacking
Thank you!
Hi! Just joing the crowd praising you for this. It's been extremely helpful for me. However, the debayering questions is a bit puzzling. You don't debayer - why not?
+Magnus Larsson Hey! To be honest, I never found it to make a difference in my images. I know that theoretically speaking, my colours aren't going to be anywhere near real unless I do, but here's something I just processed without debayering:
i.imgur.com/ZxZCuIW.jpg
And here's one that was debayered:
i.imgur.com/j7wyQsY.jpg
I tried processing them as identically as possible, though one still managed to be brighter than the other slightly. But no real difference! So I just skip the step entirely, never really had a need for it.
+Richard Bloch FYI, I found removing hot pixels with cosmetic correction works *much* better with non-debayered images.
Hi Richard
Great Tutorial
I followed it but unlike you I found that my master darks and master bias, didn't look a lot different between a single frame and the master. Now I know they should. Any idea what I am doing wrong?
There could be a number of reasons. How many individual frames are you integrating, and what was the temperature of the darks?
Also, make sure your combination method is set to "average", and not "sum"
Richard Bloch Had 47 x 300s darks at between 16 - 19 degrees. I think the combination method was set at average
Alec Alden Interesting... Is there any way to send me a download link to the data? It might be easier if I toyed with it a little
Richard Bloch Hi Richard I have re -run virtually the same data in the Batch Processing and compared images and hey presto, there are significant differences. Only about a week into PI so very much at the bottom of the learning curve. I am attempting the data again and will get back to you.
Not sure how to provide you a link to the data
Without tutorials such as yours there is no way that people like me would be able to take up astrophotography at all. Very many thanks
Richard Bloch Hi Richard, apologies are in order for wasting your time. When I ran things again and checked the images there were noticeable differences, not sure what I was looking at the first time. However, there are lots of files to keep track of and I can only assume that I was looking at the wrong ones!
I did run into trouble with an error message "incompatible image geometry", However, I have managed to resolve this by making changes to Format Explorer to make DSLR_RAW files the default input.
Thanks for your help and superb tutorial.
Alec
Thank you!!! Great tutorial.
However, In my case it seems that somehow the Master dark is ignored and the hot pixels are not removed. The hot pixels are clear on the master dark and on lights, but after calibration of lights with master bias and master dark they dont get removed. (I dont have any flats, so its uncheked)
B1063N
I mean the calibrated images. My calibrated images still have hot pixels, I am not sure if thats ok.
After registering and doing the final image integration, most of the hot pixels are gone, however, that is another process.
+B1063N
I figured it out, so if you dont add any flats most of the hot pixels remain, I have usedthe same process but with flats this time and hot pixels are gone on calibrated images.
+B1063N Hey, sorry I missed your earlier comments! I'm surprised that the master dark didn't remove the hot pixels on its own, though if you're using the screen stretch function to see the hot pixels, it could be that they aren't really "hot" pixels, but just have a slightly higher sensitivity than neighbouring pixels, which is what flats fix.
If they're hot in the sense that they're easily visible without using screen stretch, then that's a little more strange...
+Richard Bloch
My master dark is completely black without the screen stretch function. Same as in the tutorial.
The "hot pixels" of the stretched master dark perfectly match(for the most part) the "hot pixels" of a stretched light frame. However, they do not get removed, so the process flow in my case is somehow failing to fix the "hot pixels" without flats on the calibrated light subs.
Oh well, as long as they are getting removed at the end it doesn't really matter.
In any event I have compared this process to the usual DSS stacking on several of my images, and I find this procedure for stacking far far far superior in terms of the final stacked image.
+B1063N I'm glad you find this works for you!
Just a thought on the hot pixels. Because they aren't visible until you use the screen stretch, it could be that they look hotter than their neighbours because they're just more sensitive pixels. That way, they'd show up hotter both in your lights and in your darks. When you subtract the master dark from the lights, they still look 'hot' because even without the dark current, those pixels just detected more light on account of being more sensitive.
If that's the case, then when you take a flat and normalize it (which is done in the calibration part), that pixel shows up as 'hot' because it was oversensitive, and when the flats are divided, that pixel's brightness is diminished accordingly. Thus the pixel would not appear 'hot' after flats have been used. (It may be hard to see in the master flat, though, because each pixel in a flat gets a healthy amount of light, which is not the case for lights/darks.)
As you say, in the end the problem is taken care of, but that's my best guess as to what's going on given what you see.
Big thanks awesome!
What about Debayer? You are processing Canon CR2 images.
I've tried testing debayer at many stages in the preprocess workflow, but I haven't found any appreciable improvement over the automatic default debayer!
Why would my darks have so much more noise than yours? I use a 70D and I took a 2 minute dark at 800iso with the cap on the scope. When I look at it stretched there's a massive amount of noise.
Also sensor was at 35-37*c... If you think it could of been that
John Drake Yep, that'll do it! I shot this in Canada in March, so my sensor temperature was 1C according to the EXIF data. I believe it, as I recall it was about -20 out that night.
Noise doubles with roughly every 6 C increase in temperature, so we'd expect (all else being equal) the temperature to be responsible for a 64x increase in noise. My shot was 5 minutes, not 2, so I have 2.5x the noise you do due to exposure time, but 64/2.5 is still about 25x. So your shot would be way noisier just due to the temperature difference
(again, 25x specifically neglects camera differences or ISO differences, but it serves to illustrate!)
Why don't you use Batch Preprocsessing for your Lights, Darks and Biases? I am still learning and someone else in a TH-cam tutorial was using that. Seems easier.
It IS easier to go through the batch calibration! But I stick to the piecemeal method for two reasons. One is really simple: I'm just very particular about what gets done and when it happens. Using the individual tools allows me complete control over every single step; the batch tool has most of the existing options, but not all, which (in rare cases) can be limiting in terms of tricky data sets or incomplete data sets. For example, if I wanted to generate a synthetic flat, I could insert it into my stream here or use it in the batch tool, but if I want to compare the effectiveness of applying different synthetic flats, the batch tool will take more time to set and use than just doing the flat calibration step two or three times here. Ultimately it doesn't matter, but I just like having one process that applies to everything.
The second reason is troubleshooting. PixInsight will return errors when things go awry, and you can set the error policy to abort when detected, but there are a few behaviours that are unwanted that don't get caught as errors. The easiest example is through StarAlignment; it's possible, and I have an unfortunately non-trivial number of data sets where it happens, for the settings to look OK, and for the tool to operate OK, but it doesn't fully calculate the image displacements. It calculates a few pixels, or sub-pixel differences, when really it should be many more. With the batch tool, I won't know anything's gone wrong until the very end and (aside from this one glitch I recognize immediately) it may not be so easy to identify where along the path went wrong, and what caused it. Again, I could just go through the batch tool for most data sets until I get a "weird" one and then try this, but my personal preference is just to do the one thorough approach and to do it once, and finish the calibration process by the time I reach the end.
Thanks. That is good to know. I am just starting out with only a few images under my belt with PixInsight thus far, but if I run into a problem it's always good to know there are options.
Question: You used the non calibrated lights. Shouldn't you have used the calibrated light when you staraligned them? This confused me a little.
Yes, a stupid mistake on my part!
Hum ... it seems that you made a selection mistake when opening the files, you should have registered the CALIBRATED files but you end up registering the original CR2 files and subsequently the integration takes place on the files that were not calibrated.
Regards,
PA
OK, this has already been reported, sorry for the redundancy.
Still a great tutorial with very important details.
Regards,
PA
+Paulo de Almeida Yes, sorry for the confusion! Kind of a silly mistake to make in a tutorial!
At what point do you debayer?
After you do all the calibrations, but before you register the images. Debayering is most accurate when a given pixel is in its original spot - once alignment starts shifting and rotating, debayering gets messy.
Thanks Richard got it working well now. Your video was a huge help.
You never debayered your images (if using a DSLR). Great tutorial though.
My final image came out colorless. Where did I go wrong?
Can you import a single raw frame and see colour?
@@richardbloch6883 yes, the subject was the Orion nebula so lots of color mostly red.
@@2010pinmaster Interesting. Are you using any format hints at any step of the process?
cool!
After I calibrate my lights they are way too dark. I lose all the detail in my image. Even the alignment process cannot find 6 stars.
Found out the problem... I had the sample format at 8-bit instead of 32-bit!