Stacking images to get star trails, finding pulsars and night mode images.

 Typically, when we take data from a source, such as a telescope, camera, or microphone, we don't expect the data to be clean. We'd expect to see a lot of noise in the data we collect, be it from other sources of data pollution, or even issues with the instrument.

For Audio data, we have noise reduction algorithms based on Fourier transformations and such. But for images, we use a process called Stacking, among many others.

You may have heard the reason for the image processing chip on smartphones. What we discuss here, is one of the many software processes that go on in that dedicated chip, in order to make your images more beautiful.

Given how small the phone camera sensors are, the data collected is VERY limited and has a lot of noise. A lot of software level processing is required to get the images we see. Similarly, to observe a galaxy that occupies next to no space in our field of view, we need some really precise instruments, and even then they are susceptible to noise. 

Here, we will explore two different stacking algorithms. The maximum stack, and the mean stack, two of the common stacking techniques we typically see.

What is Stacking?

Take an image. It is a two-dimensional array, where each index contains the information corresponding to the particular pixel in the image. 

Essentially, you take all the images you collected and put them over one another. This process of image processing is called Stacking. we generate a final image after running an operation on all the data on each pixel. So the value of Pixel (1,1) will be input for the function, and the output of the function will make the pixel (1,1) of the new image. We repeat this for every single pixel in the image, and we end up with the final image we're looking for.

There are many types of stacking. Here, we'll be just looking at one method.

The Maximum Stack

Once we have every single image taken, we have to look for the highest pixel value at a given position. Since we're looking for the maximum value, it's called Maximum stacking. There are multiple ways to achieve this.

1st, we make an empty array filled with zeroes of the same dimensions as the original image

2nd, we can take the pixel value at a position from all the images, and make a 1-dimensional array. Then we can sort this array, and look for the maximum value of the array. 

3rd, we replace the image blank array at that position with the value we got from step two.

If the image is RGB and has more dimensions, we can just repeat the process with the individual RBG values, and in the final image, we can replace the individual RGB values with the maximum we see.

Typically, this is used to increase the brightness of the image we've captured. If we take ten images in quick succession, and then stack them together in a maximum stack, we can get a brighter image.

Like everything in Computer Science, there are more memory efficient methods, and faster methods, but this is what you could try on a small set to get a basic understanding.

Another cool use of this is to generate images with star trails.

Suppose we have many many different images of the night sky, like this.

Yeah, sure it looks cool as it is. No doubt. But we can make it look cooler. 

If we take a number of such images a few seconds or minutes apart, we can stack them with this Maximum stacking. Since the lighting in the foreground will not change, they will have relatively consistent pixel RGB values. However, as the stars move in the sky, the brightest RGB value in the sky will also change. When this happens, we would essentially be superimposing the position of the stars as they moved across the night sky.

Here, I did exactly that, and this is what I got. You can clearly see the part each star took as it moved in its journey across the night sky. 

As you can see, the maximum stack has a lot of brightness where you would expect pitch black, on the earth under the atmosphere. This brightness is because of random noise values increasing the brightness due to the maximum stack. Other techniques like Mean Stacking can help eliminate it.

The Other stacking methods, and their uses

The mean stack is an effective method to eliminate the noise in an image. Noise is merely random disturbances captured on the image. It'll be there everywhere, but when you take a lot of images and superimpose them, the noise is effectively eliminated. 

Another popular method among photographers is Focal stacking. You take the image of the same thing at multiple focal lengths, and then stack them carefully, to simulate a much better depth of field of the image.