Filtering a Bitmap using pixels shaders in Flex

Almost two years ago, I talked about using OpenGL pixel shaders in Swing to fast-process bitmaps. I also explained this effect in a little more details in my book Filthy Rich Clients (follow the link to download the source code).

A while ago Adobe announced Hydra, now known as Pixel Bender, a pixel shader engine/language for the Flash platform. Pixel Bender is available as a stand-alone application known as the Pixel Bender Toolkit or as part of the Flash 10 player. You can also leverage Pixel Bender shaders with the newly released Flex 3.2 SDK. Looking at the presentations I really liked the language Adobe came up with. It’s very similar to GLSL but offers a few interesting features like parameters metadata and dependent values. I recently found a few minutes to play with Pixel Bender in the context of a Flex application and recreated the bloom effect I wrote in Java/GLSL.

The code is pretty simple, a lot simpler than with Java, JOGL and GLSL actually. Let’s start with the fun stuff and look at the shader itself:

<languageVersion: 1.0;>
kernel BrightnessThreshold
    namespace : "CuriousCreature";
    vendor : "Romain Guy";
    version : 1;
    description : "brightness threshold";
    parameter float threshold
        minValue: float(0.0);
        maxValue: float(1.0);
        defaultValue: float(0.5);

    input image4 image;
    output pixel4 outputPixel;

    void evaluatePixel()
        outputPixel = sampleNearest(image, outCoord());

        float3 luminanceVector = float3(0.2125, 0.7154, 0.0721);
        float luminance = dot(luminanceVector, outputPixel.rgb);
        luminance = max(0.0, luminance - threshold);
        outputPixel.rgb *= sign(luminance);

The code is pretty simple, straightforward and very similar to the GLSL implementation. Interestingly, this shader could be optimized by creating the luminance vector only once using the evaluateDependents() function. Unfortunately, Flash 10 does not support this function, only the Pixel Bender Toolkit does.

Here is now the Flex code to apply the shader on a bitmap. Note that this is much more complicated than what you would do to apply the shader on a button for instance. The purpose of this code is to load an image and then apply the shader on a copy of that image. The program thus end up with the original image and the filtered result.

var SourceImage:Class;
var ThresholdShader:Class;

// Load the bitmap and its data
var bitmap:Bitmap = new SourceImage();
var data:BitmapData = bitmap.bitmapData;

// Load the shader
var brightnessShader:Shader = new Shader();
brightnessShader.byteCode = new ThresholdShader(); = [0.75];

// Create a new bitmap to hold the filter result
var filtered:BitmapData = new BitmapData(data.width, data.height);
filtered.applyFilter(data, data.rect, new Point(), new ShaderFilter(brightnessShader));

If instead you want to apply the filter directly on a component, like an image or a panel, the code becomes simpler. The following snippet comes from the MXML file:

        [Embed(source="assets/shaders/BrightnessThreshold.pbj", mimeType="application/octet-stream")]
        private var ThresholdShader:Class;

        private function createFilters():void {
            var shader:Shader = new Shader();
            shader.byteCode = new ThresholdShader();
   = [0.75];

            myWidget.filters = [new ShaderFilter(shader)];

The code is very similar to the previous snippet but this time, the shader is applied as a filter onto the widget itself. I believe that the code could be simpler with Flex Gumbo by passing the new instance of ThresholdShader directly to the ShaderFilter instead of going through an intermediate Shader object.

Even though this example is not very useful, it gives a glimpse of what cool things should be doable with Pixel Bender, Flash and Flex. The only real issue I can see is that shaders rendered in Flash are not executed by the GPU. These shaders do take advantage of multiple cores though and really fly on modern machines with 2 or more cores.

Oh and apparently you can use Pixel Bender shaders as filters in Photoshop CS4. I’ve been waiting for an easy way to write my own filters for quite a long time, I’m sold! Something else really cool about Pixel Bender: it was designed to let you process generic data, not just images. That means you can easily take advantage of multiple cores for your number crunching operations.

6 Responses to “Filtering a Bitmap using pixels shaders in Flex”

  1. Ivan says:

    Wow. Flash and shaders… Something to think about definitely… If this is a cheap way to bump up your book, then you’ll be pleased to know its working!

  2. Or, in JavaFX 1.0 you could write “Bloom { threshold: 0.75 }” and you’re done (and it’s accelerated on the GPU automatically, when available). But I know you wouldn’t sully your good name by talking up JavaFX :)

    Yeah yeah, I know your point was to show how easy it is to write a user-defined filter in Flex/Flash. It is indeed a powerful concept. The infrastructure for user-defined filters is all there in JavaFX 1.0 (and it’s the same infrastructure we use to implement the built-in effects); we just haven’t exposed it publicly yet. In due time…

  3. Romain Guy says:

    Hey Chris, it’s the Decora thing right? I can’t wait to see it actually :)

  4. BoD says:

    …and where is the demo? :) Or at least a screenshot!

  5. I use a program called Hamachi, it creates a Seure VPN that will connect to anyone who also has it installed. There is noo need for a server as it is p2p, and also no need to forward any ports on the router.

  6. Definitely, what a fantastic site and instructive posts, I surely will bookmark your website.Best Regards!