Textures, Colors, and Bits

There's a lot of talk these days about how much space is needed to store texture data for the latest generation of game consoles. As while the Wii and the Xbox360 have stuck with the tried and true DVD format (4.7GB single layer, 8.5GB dual layer), the PS3 has gone with BluRay which provides 25 GB single layered and 50 in dual layer. Some developers are claiming to be maxing out the potential of the DVD (not to be confused with HD-DVD), however I'm willing to bet that about 98% of the time, this is simply due to poor compression and color palette use (note: I'm certainly no expert on all of this...just thinking out lout here).

Even though we now have a small, but growing number of TVs and monitors than can support beyond 24bit color, this does not mean every single texture needs to be stored with such a high color palette. Now beyond the Red, Green and Blue channels, you'll also want to store a "alpha" (or transparency) channel. This will bump some textures up to 32bits. However, giving up 8bits per pixel of your textures to memory can eat up alot of space and be quite wasteful, so some will only use a single bit for transparency, putting them in an either-or situation. This second method while much more efficient in terms of storage and processing required to render it in the scene looks absolutely awful most of the time. You find this 1bit alpha used most often in the textures for leaves, blades of grass, or a chain linked fence...all of which look absolutely awful because they cause so much aliasing that it appears the areas they're used in heavily together causes a shimmering, sparkling effect which can be quite irritating to the eye. Also since most modern anti-aliasing techniques strictly affect the polygons, and not your textures in the scene this becomes even more apparent as it makes these items stand out even more from the rest of the scene.

With the ultra powerful CPUs and GPUs found in the 360 and PS3 the processing power needed to render an 8bit alpha vs. a 1bit alpha is almost negligible at this point. However, as I started off talking about, it can make quite a difference in the amount of space required to store your textures, whether it be in memory or the physical storage media. With the higher and higher resolutions of todays textures, multiplied by a growing number of them with modern graphics engines using sometimes 8 or even 16 different textures across just one area on a model, those 7 little bits add up pretty quick.

The thing that has always puzzled me is why no one uses anything in between? The difference between a 1bit mask and an 8bit mask is very apparent, but the difference between a 2 or 3 bit mask compared to an 8 bit is not nearly as much.

side-by-side comparison

Looking at them close up makes the differences more apparent, but it also makes it more clear how much we don't really need to use a full 8bit mask for decent alpha when it comes to edge aliasing.

First up, is a 4 times magnified close up of the 8bit mask:


And then look at the 1bit's very visible difference.


However, now look at the 2bit example. It's not quite as nice as the 8bit, but a huge improvement over the 1bit...


And if we bump that up to 3bit there's even less of a difference.


I had also originally made a 4bit example as well, but there was practically no difference between it and the 8bit example at all.

Now I want to point out that for all of this I am focusing on textures that have obvious aliasing problems, like a chainlinked fence, or leaves on a tree. For other effects like smoke, fire, or maybe even hair this may not work as well and you may still need to use 8bit to look good, but then again... maybe not? ;)

So, usually a texture map is going to be stored with either with 8, 16, 24, or 32 bits (due to byte addressable memory mainly I would assume). An 8bit texture is going to only give you an indexed palette of 256 colors, or 255 if you use 1 of those colors as your transparent color. If we take 2 of those bits and set them aside for transparency (giving 4 levels of transparency) we now get 64 colors that each can be displayed at 4 levels, or even 5 if you reserve one of those colors for absolutely transparent as with the usual scheme. Generally this is still plenty of different colors for a mainly monochromatic thing like a leaf or blade of grass.

However, let's say you need more colors. Let's look at a 16 bit texture, which in most instances is perfectly fine for even high quality graphics. Now we can use an indexed palette like with the 8bit image, and that will follow almost the same principles. Let's look at using a RGB scale instead. Once again, many times people will use 5 bits for each color channel and then the extra bit for transparency [RGBA5551], which gives us the same problems as with the 255+1 color image and aliasing. What if we bump down each color channel to 4bits, and now we have 4 bits for the transparency channel as well (16 levels of transparency) [RGBA4444]. As we've already determined, that's more than enough for the issue we're looking at, but it does cut into your number of possible colors by quite a bit. Perhaps, with a 16bit texture, you'd still be best off to use a static palette, with 2 or 3 bits set aside for transparency? It would probably depend on what type of texture you're working with individually.

So okay, let's look at 24bit textures, which rarely have an alpha channel at all. Here the solution seems very obvious, cut your color channels down to 7bits rather than 8, and then use the remaining 3bits for alpha [RGBA7773].

Generally however, if you want 24bit color textures, and transparency, you just bump on up to 32bits...which once again, may or may not be a waste depending on what type of texture it is your working with. As most TVs and monitors still can not display anything higher than 24bit color (don't let your Windows display settings fool you guys, there is no such thing as a 32bit monitor), some of the newer displays will actually go up to 30bit (10 per channel) and 36bit (12 bits per channel) which is only available to you if you're using HDMI v1.3+ or DisplayPort1.1+. In such a case (which I doubt anyone at all will even consider till the next generation of systems) we could have a 32 bit texture using 10 bits for each color channel and 2 bits for the transparency, or maybe a 40bit (next byte addressable size up) with 12bits per color channel and 4bits for transparency. However, most developers will probably opt to start using floating point based color channels ending up with 48 and 64 bit textures, which will take even more space...

Another interesting point I'd like to point out while I have your attention is that rather than using a 32bit texture for super high quality color representation, why not use a 16bit texture thats twice the resolution or more, yet uses about the same space? When the texture is filtered through mipmapping and various other filters, you probably won't even be able to tell the difference at a distance, yet up close you've got even more detail then before ;)

I guess the point of all this is that today's developers don't seem to care to try and be as creative about such problems as they used to be a decade ago. If you think squeezing all those textures onto a DVD is hard, try putting them on a 16 megabit SNES cartridge... If you spend just a little more time thinking about these kinda things, I'm sure that you'll be able to fit just about as much detail into that 9 gig DVD as you have planned to plop onto that BluRay disc. The same goes for audio (10 gigs my ass, that must be uncompressed or something...have these guys never heard of Ogg Vorbis?).