Canvas Direct Pixel Manipulation

In this blog I will talk about the HTML5 Canvas element and direct pixel manipulation. Since Internet Explorer 9 RC just released this is perfect timing. While canvas direct pixel manipulation was always possible, before RC the performance of it was not optimized. So let?s get started!

To demonstrate the concept I have created a ?poor mans? picture editor which allows manipulation of an image drawn in the context of the canvas. You can:

• remove red, green, blue color channels from the entire picture
• remove the selected color
• convert a specific color to alpha (make it transparent)
• convert the entire picture to B&W

Here?s how it looks (you may recognize this picture as the background for the FishIE sample):

In my ?Getting Started with Canvas? I explained how to draw images in the canvas context. I won?t talk about that here. Instead I will jump directly into obtaining raw image data.

The first step is to retrieve the ImageData object from the canvas:

imgData = ctx.getImageData(0,0,WIDTH, HEIGHT);

The CanvasPixelArray field of the ImageData object is an actual raw pixel representation of the image:

var pixels = imgData.data;

Now let?s talk about the format of the pixel array returned by the call above. Each pixel is represented by 4 bytes of data:

• 1^st byte is Red channel
• 2^nd byte is Green channel
• 3^rd byte is Blue channel
• 4^th byte is Alpha channel

Each color is an integer between 0 and 255. Pixels are processed from left to right, top to bottom and start at index 0. If I have a 6 pixel wide picture as shown below, the red component of the top row pixel in the left most column is index 0 in the array. The red component of the pixel in the second row, second column would be in the position (or index):

6 * 4 + 2 * 4 = 32

Keeping the above facts in mind we can create a generic formula to get the color component for any pixel position:

Red = pixels [(row * 4 * width) + (column * 4)];

Green = pixels [(row * 4 * width) + (column * 4) + 1];

Blue = pixels [(row * 4 * width) + (column * 4) + 2];

Alpha = pixels [(row * 4 * width) + (column * 4) + 3];

Since column is really our X coordinate in relationship to canvas and row is a Y coordinate in relationship to the canvas, I can write following JavaScript code to obtain color of any image pixel with coordinates (tmpxX, tmpY):

var colorOffset = {red: 0, green: 1, blue: 2, alpha: 3};

rNew = imgData.data [ (4 * tmpY * WIDTH) + (4 * tmpX) + colorOffset.red ];

gNew = imgData.data [ (4 * tmpY * WIDTH) + (4 * tmpX) + colorOffset.green];

bNew = imgData.data [ (4 * tmpY * WIDTH) + (4 * tmpX) + colorOffset.blue];

aNew = imgData.data [ (4 * tmpY * WIDTH) + (4 * tmpX) + colorOffset.alpha];

To manipulate colors we have to iterate through each pixel and change the corresponding color component as needed. Here is the loop I use:

for (var i = 0; i < pixels.length; i += 4) {

?

}

The loop above is where all the magic happens. Here is what I do, but of course you can apply any color manipulation you feel like:

1. To remove a specific channel I set the corresponding color channel to 0. For example to remove green color I do:

pixels[i + colorOffset.green] = 0;

2. To remove a custom color, I alter each color channel by decreasing its corresponding value:

pixels [i + colorOffset.red] -= rNew;

pixels [i + colorOffset.green] -= gNew;

pixels [i + colorOffset.blue] -= bNew;

3. To convert the entire image to B&W I set the red, green and blue channels to a brightness value, using the formula found here :

var brightness = 0.2126 * r + 0.7152 * g + 0.0722 * b;

pixels[i + colorOffset.red] = brightness;

pixels[i + colorOffset.green] = brightness;

pixels[i + colorOffset.blue] = brightness;

4. To convert a specific color to alpha (make it transparent or remove it, in other words ), I do two things: I check if color is somewhat similar to the one selected and if yes, I set Alpha channel to 0:

if ( Math.abs (r-rNew) < 10 &&

Math.abs (g-gNew) < 10 &&

Math.abs (b-bNew) < 10

)

pixels[i + colorOffset.alpha] = 0;

Where do you go from here?

The ability to directly access and manipulate raw pixels gives you opportunity to do all kinds of things such as applying various color effects, removing red eye and so on. But this is not all ? you can also apply the approach discussed here to manipulate videos. This is a subject of my next blog.

Thank you and I hope you find this helpful.

Download source code.

Eve Brittany Lee Daisy Fuentes Asia Argento Charisma Carpenter