The Purpose of Unit Tests Is Not Recreating the Entire Running Environment

One of the "Aha!" experiences/epiphanies of a programmer I was working with, was the realization that a unit test only needs to test a "unit". He had been setting up real-life data, when a little "fake" data would verify the desired behavior.

Unit testing also makes the code more robust, because side-effects that might not be seen in a system test, can be found by unit tests exercising all the edge-cases for each function or behavior being tested—particularly testing situations that are hard to replicate in the real system.

Command Pattern

(Reblogged from 11/23/2003)


Martin Fowler writes of the CommandOrientedInterface and command objects and executors:

Command oriented interfaces have a number of benefits. One of the primary ones is the ability to easily add common behavior to commands by decorating the command executor. This is very handy for handling transactions, logging, and the like. Commands can be queued for later execution and [...] be passed across a network. Command results can be cached [...]

And, of course, it is relatively easy to add "Undo/Redo" support to command objects and store stacks of command objects to be undone/redone.

In a dynamic language that supports reflection, one could create generic command objects by creating a class that retains a reference to an arbitrary object [call it a 'target'], a reference to an arbitrary method of that object [call that an 'action'], and a list of arguments to pass into that method [or perhaps only one argument, the 'sender'].

Instances of this class could be created by reading a configuration file or deserializing instances that had been created in an Interface Builder. I'm not saying that Apple's Interface Builder does exactly like this, but it is doing something kind of like this. See this page for more.

Cocoa and Interface builder allow creating powerful programs without writing (or generating) a lot of code. It's been one of the most advanced development environments for at least a decade (it used to be part of NextStep), and it still is one of the most advanced development environments.


Smalltalk, Objective-C, and no doubt Python and Ruby could generic command-object classes like this very succinctly. Java would be more difficult, but it can be done in Java, as well.


Perhaps if methods and method-calls (and classes) were more easily thought of as first-class objects [which is very far from the case in C++], tools forAspect-Oriented programming would be easier or unnecessary.

Test-Driven Design/Development

(Repeat from my old blog 2003/11/25.)

I'm about two-thirds done reading Unit Testing In Java: How tests drive the code by Johannes Link. It's a very good book. Go buy it now, even if you're not programming in Java, and then finish reading my blog. :-)

The summary form of test-driven development is:

1. Think about what we want to the program to do.
2. Write a test that shows some aspect is done.
3. Run the test, see it fail because we haven't written the code to get the thing done yet (this tests our test, giving us some confidence that our test is written correctly).
4. Write the smallest amount of code to make that test pass.
5. Run the test, hopefully see if pass. (If it doesn't, fix the problem.)
6. Refactor to clean up the code.
7. Run the test again, to make sure refactoring didn't break anything.
8. Repeat [sometimes skipping 6 and 7] until you can't think of any new tests, or until the code stops changing as a result of adding tests.

So what if we don't know what we want the program to do?

Writing a test can help us figure that out. Since the code hasn't been done yet, the test is "black-box" and does not necessarily commit us to any particular way of implementing the solution. We can write "Exploratory Tests" to investigate existing code - does calling "X" do "Y"? Write a test and find out.

I don't have time to write lots of tests.

Well, you only have to test code that you want to work. If you don't test it before you write, you'll be testing it in some fashion after you write it. Or someone else will test it, file a bug report because you didn't test it, and then you have to not only interrupt whatever you're doing and fix it, but also test it to confirm that you fixed it. And the time-delays between your writing the code, the tester finding the bug, and your trying to fix the bug means that you will be less familiar with the code, and thus fixing it will be that much harder and slower.

There is, of course LOTS of code out there that was not written this way (some of it is mine). Just browse some of the code on the internet or in your company's source-code-control system - probably most of it isn't written using test-driven techniques.

You'll probably find these things in non-test-driven code (particularly in projects that don't do code-reviews or pair-programming and refactoring):

• Dead code -- code that is never invoked.
• Duplicate code.
• Unreferenced parameters and variables.
• Variables that never vary: 'constant' variables.
• Toughly-coupled code. You can't use a class in isolation because it depends on lots of other concrete classes.
• Non-cohesive classes and methods. (Classes and methods that do "too much")
• BUGS

There is in fact a whole list of "code smells" (possible design problems) at http://c2.com/cgi/wiki?CodeSmell. TDD helps prevent some of them, and refactoring with the safety net of tests can allow you to to fix the others. With TDD and Refactoring, you don't have to live with smelly code.
I find that once my tests are passing, I almost never have to go back and fix any bugs later. This saves me a lot of time. At the worse, it takes about the same time as coding + debugging, but it's less stressful. If a bug arises during TDD, it is most likely in code that I wrote less than five minutes ago -- easily found and fixed.

For those people who say they don't have time to write tests, I ask: why do you have time to write dead code, duplicated code, and where does the time to fix bugs come from?

In traditional development and testing, you create a lot of defect-ridden code, and then test the defects out. The underlying assumption is that this is the only way it can be done. Test-Driven-Design works with a different assumption: You can put the quality in first, and not have to spend time getting defects out.

Traditional (maintenance) development, because it usually isn't supported by suites of tests, eventually grinds to a halt: the code becomes so fragile that it can't be modified without breaking something. I don't know if the IRS has depreciation tables for source-code assets, but the reality is that code eventually loses value unless you take steps to keep it from losing value. Sometimes, code becomes unmaintainable while the company is still trying to make money off of it - the result is the company loses money or doesn't get the opportunity to modify the code to make more money.

Test-Driven-Design provides a suite of tests that allows you to refactor - keeping the code maintainable so it is easy to enhance, re-use, etc., so you can make money.

If I Were Advising Someone Forming a Start-up

If I were advising someone forming a start-up, or forming a start-up myself, I'd want to learn a few things first, and then be able to consult certain experts as I continued.

I would learn from this list of books here, in a page created by Steve Blank. Note Kent Beck's Extreme Programming Explained is listed near the top of this list of books, along with The Lean Startup by Eric Ries, and The Startup Owners Manual by Steve Blank. The list of books is quite long, so I'd get summaries of these books and skim through the few I'd buy, digging deeper as needed. Reading them all would take a long time.

I would consult with Esther Derby for team-work and process improvement. And her books:

• Behind Closed Doors: Secrets of Great Management by Rothman and Derby, and 
• Agile Retrospectives: Making Good Teams Great by Derby and Larsen.

I would consult with Johanna Rothman for management and hiring. And her books:

• Hiring Geeks That Fit,  
• Hiring The Best Knowledge Workers, Techies and Nerds, and 
• Manage It!: Your Guide to Modern, Pragmatic Project Management.

There are a variety of legal issues when starting a business. Nolo Press has a lot of advice. I'd get something like The Small Business Start-Up Kit: A Step-by-Step Legal Guide.

Basic Unit Testing in Objective-C

When you create an iOS or MacOSX project in Xcode, you have the option of creating unit tests, which you can use to verify your code without manual intervention.

Here's an example...

---

//  IntroductionToObjectiveCTests.m

//

//  Created by C. Keith Ray on 1/9/13.

//  Copyright (c) 2013 C. Keith Ray. All rights reserved.

//

#import <SenTestingKit/SenTestingKit.h> // Testing Framework

// Note 1

@interface IntroductionToObjectiveCTests : SenTestCase

@end

// Note 2

@implementation IntroductionToObjectiveCTests

// Note 3

- (void)testExampleTrue

{
    STAssertTrue(TRUE, @"TRUE should be true");

}

// Note 4

- (void)testExampleFalse

{
    STAssertTrue(FALSE, @"This assert should fail");

}

@end

---

If we build and run the tests, Xcode will compile and run these tests.

Note 1: Declares a class that has SenTestCase as its parent class. Nothing else needs to go here.

Note 2: Defines the class we declared in Note 1.

Note 3: This is a method definition, or a "member function" if you use C++ terminology, that the test framework will execute. It has to return nothing ("void"), take no parameters, and has to start with the word "test" for the test framework to find and execute this method. I call this "a test" and this file has two tests in it, so far. This test contains an assertion that will pass, because TRUE is true.

Note 4: This test will fail, because FALSE is not true and the assertion STAssertTrue expects the first parameter to be evaluated to true.

The the tests are executed, Xcode will highlight the lines of code where the failing assertions are. We have just one failure; it looks like this:

The failure output tells us the expression FALSE is expected to be true (in this test), but is actually not true. The output also includes the string supplied as the second argument to STAssertTrue.

When I had Xcode generate the project and create the initial unit test code, it created two files: IntroductionToObjectiveCTests.m and IntroductionToObjectiveCTests.h. A header file (.m) to declare the interface of an objective-c test case, and a source file (.m) implementing the test case. I moved the class interface declaration to the .m file and deleted the .h file, because we don't need it. The .h would only be needed if I had written code in multiple .m files that needed to see the class interface declaration.

Why doesn't the test framework need to see the class interface? Magic. When the test code is executed, the test framework looks for classes and methods at runtime. The framework says something like "Give me a list of all your classes that have SentTestCase as their parent class, and give me the list of methods that start with 'test'. Let's execute those test methods."

SenTestingKit also outputs information to the Console. This is what it looks like for these tests:

Test Suite 'All tests' started at 2013-01-09 20:21:02 +0000

Test Suite '/Users/keithray/Library/Developer/Xcode/DerivedData/intro_objc-gfshipzbnghwbygbtraflyelmail/Build/Products/Debug-iphonesimulator/intro_objcTests.octest(Tests)' started at 2013-01-09 20:21:02 +0000

Test Suite 'IntroductionToObjectiveCTests' started at 2013-01-09 20:21:02 +0000

Test Case '-[IntroductionToObjectiveCTests testExampleFalse]' started.

/Users/keithray/projects/NSScreenCast/01_objective_c_language/intro_objc/intro_objcTests/IntroductionToObjectiveCTests.m:25: error: -[IntroductionToObjectiveCTests testExampleFalse] : "FALSE" should be true. This assert should fail

Test Case '-[IntroductionToObjectiveCTests testExampleFalse]' failed (0.000 seconds).

Test Case '-[IntroductionToObjectiveCTests testExampleTrue]' started.

Test Case '-[IntroductionToObjectiveCTests testExampleTrue]' passed (0.000 seconds).

Test Suite 'IntroductionToObjectiveCTests' finished at 2013-01-09 20:21:02 +0000.

Executed 2 tests, with 1 failure (0 unexpected) in 0.000 (0.000) seconds

Test Suite '/Users/keithray/Library/Developer/Xcode/DerivedData/intro_objc-gfshipzbnghwbygbtraflyelmail/Build/Products/Debug-iphonesimulator/intro_objcTests.octest(Tests)' finished at 2013-01-09 20:21:02 +0000.

Executed 2 tests, with 1 failure (0 unexpected) in 0.000 (0.000) seconds

Test Suite 'All tests' finished at 2013-01-09 20:21:02 +0000.

Executed 2 tests, with 1 failure (0 unexpected) in 0.000 (0.002) seconds

If you had a lot of test failures, one way to find them would be to search the console for "failed", "error:" and so on.

Agile Limerick 1

(not to be taken too literally.)

I got certified as a Scrum Master,
To make my projects go faster,
But the code was mess,
Never refactored, I guess,
So Agile, for us, was disaster.

Technical Reviews, More on Test Driven Design

Re-run from 2003.Mar.26 Wed


Scott's essay on TDD is up here and Ron Jeffries's critique is [gone]. I'll have more to say about it after reading it a couple of times.

That recent issue of STQE Magazine also has a great short essay by Jerry Weinberg on technical reviews being a learning accelerator. One thing I want to point out is that junior programmers should be reviewing the work of master programmers, not necessarily to find errors, but to learn from the master - and of course, master programmers can make mistakes too, which are often visible to junior programmers as well as other master programmers. If the master programmer is humble, he/she can learn from a junior programmer, too.

I've been reading Weinberg and Freedman's book on the subject of technical reviews Handbook of Walkthroughs, Inspections and Technical Reviews  (which was written in FAQ style - question/answer), and was a bit surprised by their recommendation that when people are being trained on how to technical (code) reviews, they should have some practice at conducting a review in the presence of hidden agendas.

Some examples of hidden agendas in code reviews: person A wants to impress person B. Person B wants to make person C look bad. Person C needs to go to the restroom, but doesn't want to say so. Person D is distracted by illness of his/her spouse.

Only Weinberg would write about hidden agendas in code reviews - too many writers and books on software development practices seem to assume that people act like machines.

On Weinberg's SHAPE forum, Charlie Adams wrote: "When people are getting tense about their software being reviewed, use Jerry's phrase, 'Yes, I trust your honesty, but I don't trust your infallibility. I don't trust anyone's infallibility.' (Quality Software Management: Anticipating Change: page 220) In my experience this has always calmed the atmosphere and allowed us to examine the code rather than the developer."

While I have done code reviews, both informal and formal, I prefer pair programming. It combines reviews with collaborative design, testing, and coding. Rather than go into all the reasons why pair programming is good, I'll point you to www.pairprogramming.com and Pair Programming Illuminated.

Available and Seeking Employment

I am available and seeking employment.

I am now extremely interested in getting back into product development and away from Agile Coaching. I am particularly interested in programming for the iPad and iPhone and Macintosh, and I would appreciate it if you, a reader of this blog, consider me for "mobile" or "Macintosh/cross-platform" Senior Software Developer positions.

As a technically-oriented Agile Coach for the last 5 years, I have helped teams reduce their bug counts and increase their rate of feature implementation. When I coach a team, I dive into the source code, quickly analyze and understand it, and help the team refactor it for testability and improved design. I help them write unit tests and teach them test-driven development; and provide instructions in other parts of an Agile software development process using Scrum and XP practices.

I have coached whole teams (managers, programmers, testers) at clients including Google using C++ and Python on Linux, IronKey using PHP, C, and C++ on Linux and Windows, Nokia-Siemans Networks in C++, and Schlumberger in C# and C++ on Windows, and Huawei in C++, and provided training to developers at these and other companies including HP.

Previous to working as an Agile Coach, I have worked on whole life-cycle projects at

• Apple, in C and C++,
• Intuit, using Objective-C, Cocoa, Carbon and C++,
• Electronics for Imaging, in C++ and Java on Mac and Windows,
• Hoffbauer Inc., my first use of Objective-C on NeXTStep (the ancestor of Cocoa and CocoaTouch),
• Ideal Learning, using Object Pascal, Object C, and C++, and
• Kodak Health Imaging Systems, in C++ on Mac and Unix.
• Pixera, a digital camera startup, in C++ on Mac and Windows.

While at Google, I contributed code to Google's open-source C++ test framework gtest and provided technical reviews of Google's open-source C++ mock-objects framework gmock.

Sleep Patterns

In Brain Rules, and other publications, we now know that people fall into various sleep patterns: "larks," who wake and are most active in early in the morning; "night-owls," who are most active at night; and the unnamed majority, who keep "normal" hours.

In an internet search, I happened to find an introvert night owl who now had the option to take meds to sleep at the same times her husband and children slept, but wanted to keep her late-night awake time as her "private time": free to write, meditate, and observe the quiet night.

It sounds like a nice idea for a short story. Does she conform to normal hours, or stay a night-owl?

A Link to my Resume

Normally I'd use the proper accents for Résumé, but just in case... my resume (pdf)

I'm considering writing a book on Test-Driven Development in Objective-C (targeting MacOS X and iOS) and/or a book on TDD in C++ and Objective-C++ [with examples targeting Windows, Mac, and a cross-platform GUI.

Book Review: "The Unincorporated Man"

"The Unincorporated Man" by Dani Kollin and Eytan Kollin is a page-turner and a good example of realistic world-building. The main character is a very rich man from our time, revived 300 years from now in a society "in which every individual is formed into a legal corporation at birth, and spends many years trying to attain control over their own life by getting a majority of his or her own shares." Much of the conflict comes from the main character trying to keep himself "unincorporated" -- the only person in that society not owned by other people or corporations.

There is a little too much exposition in some places, and it may seem to have a libertarian agenda, though it shows that corporate power can be just as much of a threat to freedom as government power.

The chapter about the VR Epidemic, and the VR Museum (which serves this society like "the Holocaust Museum" serves ours) warns that we could entertain ourselves to death, if our entertainment gets to be too good.

I recommend this book.

changing directions

My mission has been to help programmers work with less pain, but I want to go back to basics: to provide tools for creative people (including children), particularly on iPad, iPhone/iPod Touch, MacOS X.