GPU Texture Snapshot

As you know, 3D graphics engineering deals with a lot of textures. And often, you need to see what’s in those textures. Especially if they are the result of a rendering pass like your depth buffer, G-buffer… However, seeing what’s in the textures while your program is running is a bit of a challenge. Usually, you have to dump texture data into a file format that can be opened by an image viewer.

I have always found the task of viewing GPU generated textures to be complicated and unnecessary. What if I could just display the texture on the screen? Just like printf does with text!

Printf

Any programming language has the equivalent of C/C++ printf. For instance, every “Hello World!” program uses a print function to output text to a console, file or web page… I thought about what it would take to have a similar API that works for images, the same way printf works for text. The API would have to be:

• Easy to use like printf.
• Immediate. You should see your images (somewhere) as soon as the print function is called.
• Unobtrusive. It should not alter a program’s flow or dramatically affect its performance.
• It should rely on technology that is native to any system.

That being said, printf has a weakness that would be unsuitable for the goal of printing images. Every time code execution goes over a printf call, text is printed. This is often undesirable. Without any control mechanism, calling printf (for instance, in a loop) ends up flooding the text output outlet. Since images can be large in size, continuously printing them out of a running program could have a dramatic effect on performance.

Designing Printf For Images

What if you could tell a program to export raw image data at a moment of your choosing? This would be an improvement over the way C/C++ printf works. An image would be exported out of a program when the developer chooses to while the rest of the time the program would simply run unaffected.

The solution I designed, has three parts.

• Just like C/C++ printf has a command line terminal as default output, I needed an outlet for images and I think Pico Pixel desktop application would be perfect for it. Pico Pixel is an image viewer we created. It is fast, simple, supports many pixel formats (DXT, BCn, ETC2…) and file formats such as OpenEXR, HDR, KTX and more.
• A C/C++ API. Developers call functions of the API to export images out of programs. The API is written in simple C/C++ that any compiler can handle.
• A communication system between a program and Pico Pixel. Network sockets are good for the job. Sockets libraries are everywhere. They work across hardware platforms and OSes. They are very well documented.

The result is Pico Pixel SDK. The SDK is made of a few files (open source) you drop in your build environment. They should compile without much trouble. To use the SDK you instantiate PicoPixelClient object and open a connection to Pico Pixel desktop application. The method you use to send pixel data to Pico Pixel is called PixelPrintf.

SDK Integration

As I said before, calling printf in C/C++ always outputs text in the console and this is not desirable for images. To prevent images from being sent everytime code execution goes over a call to PixelPrintf, we need something called a marker.

Markers are objects used by PixelPrintf to decide whether to send image data to Pico Pixel or not. Every marker has an associated counter of integer values. While a marker's counter value is greater than 0, any call to PixelPrintf with the marker will be carried through. Each time a marker is used, its counter value is decremented by one.

When a marker's counter value reaches zero, the marker can no longer be used to send image data to Pico Pixel. Its counter value has to be reloaded before it can be used again. Pico Pixel provides a user interface to reload counters. As a result, Pico Pixel acts as the trigger for image snapshot in a program.

Here is what the code looks like.

PicoPixelClient pico_pixel_client("GlxGears");

pico_pixel_client.StartConnection();

// the last parameter '0' means the marker use_count is 0.
int marker = pico_pixel_client.CreateMarker(std::string("Color Buffer"), 0);
int marker = pico_pixel_client.CreateMarker(std::string("Depth Buffer"), 0);

// Send the defined markers to PicoPixel desktop application
pico_pixel_client.SendMarkersToPicoPixel();

// Image data is sent if and only if the marker's use_count is greated than 0.
// A marker's use_count is decremented each time the marker is used.
pico_pixel_client.PixelPrintf(
    marker,                                 // data marker
    "color-framebuffer",                    // image name to appear in Pico Pixel desktop application
    PicoPixelClient::PIXEL_FORMAT_BGR8,     // image pixel data format
    400,                                    // image width
    300,                                    // image height
    1200,                                   // image row pitch in bytes
    FALSE,                                  // set to TRUE if the data is in srgb 
    FALSE,                                  // set to TRUE to rotate the image horizontally when displayed
    raw_data                                // char* pointer to the image raw data
    );

By instrumenting a program with the code above, Pico Pixel interface mirrors the markers that have been defined.



To take a snapshop of a texture buffer, set a value greater than 0 in the marker used to export the surface. When markers in Pico Pixel are synchronized with markers in the program, images are sent whenever code execution goes over the PixelPrintf call that use markers with positive counter's values.

Conclusion

The best feature of Pico Pixel SDK is that you have a lot of control over images and the moment they are exported. When images are not exported, your program is unaffected by the presence of the SDK.

I am very excited about this feature. It makes Pico Pixel useful in a graphics developer’s toolset. The idea of sharing images over the network between applications is not a new one. However, for us graphics developers, it opens a lot of possibilities if we take advantage of it easily during development. I believe this is a good feature to have and I hope it will save you time in efforts.