Two Weird 'Tricks' For Testing Graphical Code

Before I do a bunch of "Extract Method" refactorings, I looked over the code and wrote this comment:

    // creates pdf context.

    // computes rowCount and columnCount from paper_size

    //    and major_grid_distance.

    // computes margin sizes (and thus origin.)

    // draws graph paper into context.

    // close context.

    -(void) drawMajorMinorGridAt: (CGPoint) origin

                     andRowCount: (int32_t) numberOfRows

                  andColumnCount: (int32_t) numberOfColumns

                       inContext: (CGContextRef) context;

What the extract methods will do, is let me draw while not in a pdf context. Code-reuse: yay!

Another way to achieve the same goal: copy & paste. While copy & paste may be quicker,  it bulks up the code and potentially doubles the number of defects that may "bug" us in the future. Technical debt.

One could argue that planning ahead would have created a better design. But... I didn't need to draw while not in a pdf context, a half-year ago.

One reason this code is so bad, is that I didn't exactly test-drive it. I ran some calculations, wrote the code, and looked at the output. In some early iterations, it produced grids with lines poking too far through other lines: I had to reduce the length of some lines, by the thickness of other lines. Similar glitches provoked a few other changes. And then it was done. I haven't touched the code in quite a while. 

Could I write some automated tests for this code? Yes. 

I could save some manually-verified pdf output as "Golden Data" and do file comparison in tests that are supposed to produce the same output. I have done something like this before. The only time the test failed in my project (without my changing code to cause the failure) was when a graphics library that my code depended on changed. 

So... few "false" failures. Drawbacks: slow tests, manual verification needed if the test fails. And manual intervention to re-create the "Golden Data" when the code is fixed or the failure is identified as not a bug. I also had to write a custom file-comparison routine in order to ignore parts of the file that change all the time (a time-stamp in the file itself.)

You do not want to do "Golden Data" test data when false failures are frequent. (Or any kind of manual intervention is frequent.)

Mocking. Yes, I could mock out the drawing code. It's a bit difficult because the drawing code below this Objective-C class consists of C functions. I would have to create a "seam" to allow compile-time, link-time, or run-time switching out of the drawing library. 

Mocking can create fragile tests. Since mocking works by verifying that the code-under-test executes an expected sequence of function-calls, and checks that the parameters passed into those functions have expected values, any change to the order of calls or the parameters passed in will cause the test to fail. This depends on how picky you configure the mock objects to be.

With some work, depending on the mocking framework, you could ignore the exact sequence of function calls, but still check that all of the calls have been made. Make the mocks too "loose" and then you're not testing anything. Make the mocks too strict, and you not only open yourself up to the possibility of frequent false failures, you also inhibit refactoring the drawing code.

There is also the danger that the functions you're avoiding calling will someday change their behavior, but the mock functions don't reflect that change. Instead of false test failures, you get false test successes. Some redundancy in testing helps avoid this problem: have some tests using mocks, have other tests using the real functions and some other form of verification, like Golden Data.)

Learn about Golden Data, mocks, and other xUnit Test Patterns here: xUnit Test Patterns: Refactoring Test Code

Test-drive safely, everyone!