Bas Schouten

Well, it's been a while since I've posted here, so it's about time I had something interesting to report! First of all I'm afraid over the last 3 months my blog gathered over 30 000 spam messages. This made it impossible for me to reasonably deal with them, and filter out useful comments. I've had to delete all comments, including possibly well-intended ones. I've since added a captcha and additional anti-spam measures, so things should be better now!

On to what this post is all about, a little over two months ago we had the Skia team visit us in the Toronto office. While there they showed us some of the neat things they're doing with Skia, and one of the things they were doing looked pretty cool! Essentially it was a tool that could take a recorded drawing stream and inspect it, it really made me feel like a powerful debugging tool for drawing sequences could help us out as well. We already have some of these tools already in the 3D drawing area (think of OpenGL APITrace or DirectX's PIXWin (soon to be the Visual Studio Graphics Debugger), however the calls seen there can be pretty hard to correlate with 2D drawing instructions due to batching and other mechanisms.

So this is what I started working on. Since Azure didn't have any recording functionality yet I had to add that first, that's mostly done now and that work is landed in nightly. As a result nightly can now produce recorded drawing streams. The way in which these streams are produced is still a little bit constrained, the following limitations apply:

• You must be using Azure content drawing, that is currently Direct2D(accelerated) on Windows Vista+ only
• The 'gfx.2d.recording' pref must be added and set to 'true'
• After the browser is restarted -all- drawing will be recorded until closing
• The stream is always stored in the file 'browserrecording.aer' in the current working directory
• Storing everything means storing all bitmaps drawn, and all font truetype data, this can make the file grow big, currently no compression is used
• It may crash when webfonts are used

These are issues we aim to fix in the future, some are easier to fix than others, so look for updates on this. For now, if you have a machine that's capable of Direct2D you can add the 'gfx.2d.recording' pref and set it to true to enable it when you're curious about how we're drawing a certain site. You can then go to the site you want to know more about and close the browser. Your working directory will now contain the 'browserrecording.aer' file (rename/move this if you don't want it to be overwritten next run!).

As this file is stored in a binary format, in itself this isn't actually very useful to you! This is why I've also created a simple debugger. This is in the early stages of development, its UI is not great, nor very intuitive, and functionality is limited at this point. Although it should prove sufficient to be useful for debugging certain types of graphics bugs. I built it on top of Qt and the source code is available here. You can currently find a stand-alone build here, and you might also need the new VS2012 redistributables available here. To make it a little easier to explain, let's start with a screenshot of the main interface.

Click for a larger version

Now I'll proceed to explain what's currently available in the interface, this might be somewhat lengthy, so if you were just here for the cool pictures feel free to browse on! It might also require some very basic Azure knowledge. I should follow up at some point with a blog post explaining some of the basic terminology.

There's currently three distinct areas in the player:

Drawing Events

The Drawing Events area to the left represents the drawing events present in this stream, clicking any drawing event in there will replay drawing to the corresponding point in the stream. Keep in mind here that moving forward in time is faster than moving backward, when moving forward we only need to execute the 'new' drawing commands, whereas when moving backward we need to replay from the start (a drawing command can't really be 'removed'). This might be fixable in the future by storing 'keyframes' every so many drawing events.

In the columns you will see the current drawing event 'ID' (its position in the sequence), the 'object' that drawing call applies to (in case of object creation or destruction the object that was created/destroyed, otherwise the DrawTarget a drawing call executed on), and the type of drawing event.

In the top there's a text input, which allows you to jump directly to a certain drawing event 'ID', as well as back/forward buttons to jump through events you've recently visited, and a dropdown box which allows you to filter on events coming from a specific object.

Objects

This is the list of objects alive at the currently selected point in the drawing stream. Asides from being useful in debugging lifetime issues and showing some (currently very rudimentary) information about the objects listed this can be used to do a number of other things. One of those things is that you can double click several types of objects to open them in the 'View' area, which I'll explain more about later. In addition to that an object can be right-clicked in order to filter the Drawing Events list based on that object.

View

In this area relevant information can be viewed on an object. In the screenshot shown above we're currently showing the DrawTarget where the UI is being drawn at this point in time. We can do the obvious zooming and in and out of DrawTargets and SourceSurfaces. In addition to that for DrawTargets we display the currently set transform to the right, as well as there being a switchable button at the top that will visualize the currently set clip (it will illuminate it in bright red!). This area, just as the objects area is live and what is shown here will update as you navigate through the drawing events.

So that about covers the basics! Under 'Tools' there is an experimental feature called 'redundancy analysis', that is currently very experimental, if you do decide to play with it keep in mind it will corrupt the heap of the player :-)! Once I've worked on this a little more I will talk about it in a follow up blogpost. Again, this is all in a very early stage and suggestions and contributions are all very welcome!

But this is all windows-only

That's true! Currently the recording side of things, that is present on mozilla-central is available everywhere if you force Azure content on, on those platforms, however fonts currently do not work outside of windows, and Azure content is not yet ready to be used for browsing yet on non-windows platforms. The player however is currently only available on windows. Although it has been developed based on Qt and should work on other platforms, it uses the Azure stand-alone build which currently only has a Visual Studio project file and only supports Direct2D.

I'm very interested in having the stand-alone library and test-suite supported on other platforms, and as such if you want to help out in this area please let me know! I'll be more than happy to provide more detailed instructions and help out.

That's all for now! I have a lot of other ideas to work out with the new recording system, I'll hopefully be able to cover more of those here in the near future.