TDD and Signed SSLVerifySignedServerKeyExchange

Test-driven development (TDD) done consistently throughout a project results in near-100% logic coverage. That's better than just statement coverage. It also results in testable code, better modularity, better design (if you pay attention to code smells), and faster software development.

The heart of TDD is:

1. Write a test. Running that test should fail. (Tests are often simpler than the code they test, so usually this step is pretty easy.)

1. Make the test pass by writing just enough code. (All other tests should pass as well.) 

1. Refactor. (Not done every time through this loop; remove duplicate logic and look for other code smells.)

1. Repeat until desired feature is implemented. 

1. NOTE: Checking into source-code-control can be done whenever all tests are passing. (A Continuous Integration (CI) server can also run tests: the "fast tests" created by TDD and slower system tests.)

Badly-designed code is hard to test. Most developers who try to use TDD in a badly-designed, not-unit-tested project will find TDD is hard to do in this environment, and will give up. If they try to do "test-after" (the opposite of TDD's test-first practice), they will also find it hard to do in this environment and give up. And this creates a vicious cycle: untested bad code encourages more untested bad code.

If code is only written to make a test pass (as in TDD), you can't write an if statement until you have a test that requires it. You should end up with a test for the if statement being true, and a test for the if statement being false, possibly more than one test for each branch.

TDD in C is not terribly hard. I've taught people how to TDD in C and other languages. And the suites of tests created by TDD makes it possible to refactor with more confidence.

If one or more TDD tests (and other tests, like system tests) fail, someone broke something. The fine-grain testing that is part of TDD will usually pin-point where the new bug is. Undo the changes that broke the test, or fix things before doing anything else.

(Tests that fail unpredictably indicate something is non-deterministic either in the code-under-test or the test itself. That also needs to be fixed.)

landonf demonstrated that SSLVerifySignedServerKeyExchange() is unit-testable in isolation. See his code on github. I copied his unit test for a bad signature here:

@interface TestableSecurityTests : XCTestCase @end

@implementation TestableSecurityTests {
    SSLContext _ctx;
}

- (void) setUp {
    memset(_ctx.clientRandom, 'A', sizeof(_ctx.clientRandom));
    memset(_ctx.serverRandom, 'B', sizeof(_ctx.serverRandom));
}

- (void) tearDown {
    [super tearDown];
}


/* Verify that a bogus signature does not validate */
- (void) testVerifyRSASignature {
    SSLBuffer signedParams;
    SSLAllocBuffer(&signedParams, 32);
    uint8_t badSignature[128];
   
    memset(badSignature, 0, sizeof(badSignature));
    OSStatus err;
    err = SSLVerifySignedServerKeyExchange(&_ctx, true, signedParams, badSignature, sizeof(badSignature));
    XCTAssertNotEqual(err, 0, @"SSLVerifySignedServerKeyExchange() returned success on a completely bogus signature");
}

@end

A Key Aspect of Successful Projects

Rapid responses to questions is often a key to successful projects. Extreme Programming and other agile methods like Scrum consider this to be so important as to require the domain expert / product manager / business analyst / "product owner" / "Customer" to be co-located with the developers/testers. 

Obviously some projects can be successful without co-location, but odds against success rise whenever developers can't quickly and easily communicate with the Customer.