https://phys.org/news/2018-06-de-beers-synthetic-diamondshere-theyre.html

Tiny imperfections in this arrangement, whether an atom that's in the wrong place, missing or of a different element, can lead to huge changes in the diamond's colour. For example, replacing one carbon atom in every 10,000 with a nitrogen atom would turn a transparent gemstone brown.

Getting carbon atoms to arrange in this perfect crystal is not easy and it cannot happen naturally on the Earth's surface since carbon here prefers to form crystals of graphite, the soft black, material we use in pencil leads. In this environment, carbon atoms also tend to attach more easily to other atoms such as oxygen and hydrogen than to each other. This means that even making pure graphite crystals is difficult.