I’m currently working on a personal project generating optical illusions that involve color, and one of the frustrating parts has been looking into just how computers store, transmit, and generate the colors that we see on screen. As you might expect from color being a human phenomenon, it’s not easily converted to numbers and faithfully stored by our digital systems.

For instance one thing I’m trying to do is remove all of the color – say, red – from an image. One obvious way, since the image is stored as RGB pixels, is to just set all of the red subpixels to 0. That would work but it would also distort the image in ways I don’t like. Any pixel that was gray before would now become colored since the red subpixel is 0. Colors should be drained to gray, but not grayer. This and issues like it are what led me to write something purpose-built for this, and solving the problem starts with seeing just how colors are stored on computers. We need to go deeper than just “red, green, and blue”.

Measuring color

Colors on computers are modeled as numbers using color spaces, and the base one often used to measure others is called the CIE 1931 color space. This was created during the dawn of the camera era and its creators studied how various colors could mix to produce different ones. Eventually they came up with their chromaticity diagram, which could specify the colorfulness of a color using just two numbers (with brightness as a third number). Here it is:

parabola colored grayish-white at the center and becoming a rainbow of colors radiating out, with measurement axes around it
So many colors to choose from!
Image credit: https://commons.wikimedia.org/wiki/File:CIE1931xy_blank.svg

Along the edge of the shape are the most colorful possible colors that we can see. Most of them are created by a single pure wavelength of light – the numbers along the edge are the wavelengths – though the line in the purple region requires pure wavelengths to be mixed together. The same is true of the colors inside the shape — they must all be mixtures of wavelengths. Immediately several interesting properties of colors are implied by the shape and how it’s constructed. First is that any color can be created by mixing two wavelengths together in varying proportions. A color in the white region could be created by mixing approximately visually equal amounts of 560 nanometer light and 460 nanometer light, for example. The second is that no palette of colors will let you render everything. Folk color theory says that every color can be created by combining three primary colors, generally red, green, and blue. As seen on the diagram, the colors can be mixed to create a lot, but not everything.

For this reason the colors on this and the chromaticity diagrams below are somewhat misleading — the colors can’t all be rendered on whatever display you’re viewing this on, no matter how sophisticated it is. They give you an idea of what colors are being represented but aren’t exact.

The full color space is very pretty, but it isn’t very useful for actually rendering colors. After all most computer screens just have red, green, and blue pixels. One of the major color spaces in use is derived from Rec. 709, the color space that was defined for HDTV when it was first being developed. The colors that it can display enclose a narrow triangle of the full color space:

chromaticity diagram with a triangle overlaid with vertices in red, green, and blue areas
A lot gets left out.
Image credit: https://commons.wikimedia.org/wiki/File:CIExy1931_Rec_709.svg

The edges of the triangle are a red, a green, and a blue. Rec. 709 can specify any color within the triangle by a mix of those three colors. Sound familiar? It should. This color space was derived slightly to create sRGB, the color space in most common use today and the “default” one used for most images if another one isn’t specified. sRGB changed Rec. 709 by compensating for the nonlinear output CRT monitors of the time had, so that the color values in an image could be passed unmodified to the monitor. Not even colors are safe from the specter of legacy systems.

We have started to move away from sRGB in recent years. Many computer screens now support DCI-P3, which supports a wider array of colors that look like this:

chromaticity diagram with a triangle overlaid with a triangle larger than the previous one
More colors!
Image credit: https://en.wikipedia.org/wiki/File:DCI-P3_D65.svg

A good improvement, but unfortunately not in wide use yet. I’m currently only working with sRGB on my project.

Transforming color

So we’ve figured out how to measure color, but what about transforming colors from one to another? My previous example of removing all of the red from an image, for instance? To get there we need to use yet another model: the CIELAB color space. This system models each color as three numbers called L*, a*, and b*. L* is the overall brightness of a pixel, while a* is color on the green–red axis, and b* is color on the blue–yellow axis. Each of the color axes is neutral at 0, making some transforms easy. For example all the color can be removed from an image by setting a* and b* to 0.

There are some issues, though, that are simply inherent in how our color vision works. Here are some graphs that show CIELAB at different levels of L*, showing all of the values that can be represented in sRGB space:

three color surface diagrams, showing changes in the colors available at different brightnesses
Each level has a different shape and different enclosed colors 😭
Image credit: https://commons.wikimedia.org/wiki/File:Lab_color_space.png

Note how the boundaries of sRGB space change. At the brightest spot on top, there are mainly pastel colors. Going down expands the color space some and makes them more “neutral”, while the final one has a greatly reduced color range. This is partly due to the limitations of sRGB, but also partly due to the limitations of how we see color. We simply see a lot fewer colors in dim light.

Final manipulations

Anyway, this has provided me with a lot of help in calculating color transformations. It should be noted that CIELAB is not the last word in playing with color, and other ones have been developed over time, generally making it easier to calculate the visual differences between colors. CIELAB is simple and works for what I’m trying to achieve, though.