https://www.wral.com/science-behind-black-hole-image-may-have-far-reaching-applications/18323558/

The first image of a black hole at the center of the Messier 87 galaxy published this week is amazing. The technology developed to create it is just as amazing. The Event Horizon Telescope (EHT) project brought together a network of 10 telescopes across the globe to see, as project director Sheered Doeleman put it, “what we thought was un-seeable”. Einstein first theorized the existence of black holes 100 years ago while developing his theory of general relativity. Since then, there has been a lot of evidence supporting the existence of back holes, and simulations of what they might look like,  because one had not been directly observed. So what took so long? Why haven’t we used something like the Hubble Space Telescope (HST) to take this photo? While the black hole in M87 is as wide as our solar system, it’s also 55 million light years away, which makes it very, very small in our sky. Each pixel in HST’s WFC3 camera is about 1,250 times wider than the black hole. A telescope as big as the Earth was needed, so one was created, virtually.

Radio telescopes in Hawaii, Arizona, Spain, Mexico, Chile, and Antarctica were linked together in a process called Very Long Baseline Interferometry to create a single virtual telescope effectively as big as Earth, producing five petabytes of data. That is about the size of the Library of Congress’s digital collection or the equivalent of 190 continuous years of streaming Netflix in high definition. This isn’t just an achievement in astronomy, it’s an achievement in data science, particularly data reduction and image construction from sparse data. Even with so much data and all that processing, there were still a lot of gaps. This is like putting together a puzzle where 90 percent of the pieces are missing and you don't even know what the final image is supposed to look like.  So where does the confidence come from that the image published this week is accurate?  The answer lies in data science. Petabytes were reduced to terra-bytes through correlation of observations between the radio telescopes which take into account things like Earth’s rotation during the observations and the amount of water vapor in the atmosphere above each telescope. The dry conditions in the Atacama Desert of northern Chile, where one of the telescopes is located, helped make this easier. This step also helps pull useful data out of the noise, a common problem in data science.