Exploring Agile Solutions

Sleep Patterns

2012-01-27T14:29:00.000-08:00

In Brain Rules, and other publications, we now know that people fall into various sleep patterns: "larks," who wake and are most active in early in the morning; "night-owls," who are most active at night; and the unnamed majority, who keep "normal" hours.

In an internet search, I happened to find an introvert night owl who now had the option to take meds to sleep at the same times her husband and children slept, but wanted to keep her late-night awake time as her "private time": free to write, meditate, and observe the quiet night.

It sounds like a nice idea for a short story. Does she conform to normal hours, or stay a night-owl?

A Link to my Resume

2011-12-29T10:56:00.000-08:00

Normally I'd use the proper accents for Résumé, but just in case... my resume (pdf)

I'm considering writing a book on Test-Driven Development in Objective-C (targeting MacOS X and iOS) and/or a book on TDD in C++ and Objective-C++ [with examples targeting Windows, Mac, and a cross-platform GUI.

Book Review: "The Unincorporated Man"

2011-11-12T21:44:00.000-08:00

"The Unincorporated Man" by Dani Kollin and Eytan Kollin is a page-turner and a good example of realistic world-building. The main character is a very rich man from our time, revived 300 years from now in a society "in which every individual is formed into a legal corporation at birth, and spends many years trying to attain control over their own life by getting a majority of his or her own shares." Much of the conflict comes from the main character trying to keep himself "unincorporated" -- the only person in that society not owned by other people or corporations.

There is a little too much exposition in some places, and it may seem to have a libertarian agenda, though it shows that corporate power can be just as much of a threat to freedom as government power.

The chapter about the VR Epidemic, and the VR Museum (which serves this society like "the Holocaust Museum" serves ours) warns that we could entertain ourselves to death, if our entertainment gets to be too good.

I recommend this book.

changing directions

2011-11-04T15:02:00.000-07:00

My mission has been to help programmers work with less pain, but I want to go back to basics: to provide tools for creative people (including children), particularly on iPad, iPhone/iPod Touch, MacOS X.

TDD, BDD, and XXDD

2011-06-08T10:34:00.000-07:00

(repost from 2006.Aug.12 Sat)

Check out Dave Astel's paper (pdf) on TDD/BDD. Quotes:

[...] There are a score of books available on TDD, mine even won a Jolt award. So it seems that everything is rosy? Everyone who's doing TDD is fully understanding it and getting the full benefit, right?
Fat Chance!
Too few people I talk to really understand what it's really about. That means that many people who practice TDD are not getting the full benefit from it. What's wrong?
The focus on testing
Well... one thing is that people think it's about testing. That's just not the case.
Sure, there are similarities, and you end up with a nice low level regression suite... but those are coincidental or happy side effects. So why have things come to this unhappy state of affairs? Why do so many not get it?[...]
So with people thinking about testing, it's easy to come up with all sorts of negative reactions and reasons not to do it... especially when time gets short and the pressure's on.

By the way, I HATE saying TDD is "not about the testing". I have to say it now and then because of people not realizing it's about designing, rather than testing. The fact that the word "test" is in the name just helps confuse the issue. My preference for naming this design technique, is to call it "Behavior Driven Design". Or "Behavior-Spec Driven Design". With BSDD, I could say BSDD is about the behavior-specs, but mostly aboutdriving design.

Alistair Cockburn's "Executable Example Driven Design" (XXDD) is OK, but a bit long-winded. And I don't want to have say "Dos-equis Driven Design is not about beer." :-)

The problem is mental frameworks. People reject ideas that don't fit into their mental framework. You say "test" and the listener's mental framework comes up with these and other assumptions:

Testing can be done manually.
I can do implementing, someone else can do testing.
Testing looks for bugs.
Tests need something to test.
Testing is hard.
We can skip testing if we are good enough programmers, or time is short.

None of those assumptions directly applies to TDD. TDD can't be separated into tests written by one person and code written by another because the TDD cycle of test-code-refactor is a design process that one person (or a pair) goes through in cycles, where each cycle takes only minutes. TDD/BDD helps me find bugs, but that's not the purpose of TDD. TDD isn't TDD is you do the "test" part manually.

If you say "Test-Driven Development is not about the tests," that usually doesn't provide the zen-like kick in the brain-pan that the speaker intends. Instead, the phrase gets rejected because it doesn't fit into the listener's mental framework for "tests". The listener probably thinks you're nuts. He or she might just flip the "bozo bit" and stop listening to you.

If instead you say "Behavior Spec", the listener's mental framework will have fewer contradictory assumptions. For example:

Specs are written before coding.
Specs can provide examples.
Specs tell you what you need to implement.
Someone else can write specs and I can write the code.

Not all of those assumptions are true for BDD, but there are fewer contradictions to this idea of BBD that you're trying to introduce into someone's mental framework. Behavior Specs are written before the code exists (though only minutes before). Behavior Specs are executable examples. Behavior Specs tell you, more or less, what you need to implement next. There is still the contradictory idea that someone can write the Spec and someone else can write the code, but that isn't as difficult to overcome as all the other assumptions that come up with the word "test".

What part of the brain calculates how a whip works?

2011-06-02T01:32:00.000-07:00

Imagine a whip, in which each centimeter of the whip contains a data processor node, that calculates the position of the node in 3d space, the acceleration and velocity of the node in 3d space, the relative angle between this node and the next node, and the strength and angle of the pull on this node from the previous node, and so on.

Imagine each of these nodes gathering gigabytes of data, as a talented master of the whip, whose only control of this leather string of nodes comes from the handle in his hand, waves and curls the whip in the air, to snap it with precision, the tip exceeding the speed of sound, to clip a playing card or other some other target.

How much could we learn from examining and processing that data? What equations could we derive?

No doubt simulating this activity would take bank of CPUs some time to crank through the numbers. Far more time than the real-time action that took place.

How does a man do it? Practice, of course. And the more esoteric calculations are done in parts of our brain that are not subject to conscious reasoning. The parts that can predict the not-entirely parabolic path of ball thrown in baseball or football.

(See How to crack a whip like Indiana Jones )

Would it not be interesting if the simulation of a whip could be repurposed to predict movements in currency-trading, stocks and bonds, real-estate prices, national economies?

In some limited situations the number of variables might not be that different. The forces and effects, velocity and acceration and position, could have analogs in this financial world.

How would a whip-weilder of finance practice? How would he get feedback? Could it be visible enough and fast enough to allow the esoteric calculating parts of his brain reliably direct his (financially metaphoric) hand?

Objects Dependancies Are Their Own Tax

2011-04-18T04:27:00.000-07:00

This list of ways one class can depend on another is ordered from worst to best (for C++):

1. A depends on B because B is Singleton.

Avoid this. The dependency is 'invisible' and Singleton makes it worse than a global variable. Destruction is usually not possible unless a counting mechanism is used.

2. A depends on B because B is global variable.

Avoid this. The dependency is 'invisible'.

3. A depends on B because B is a file-local variable.

Avoid this. Not accessible to test.

4. A depends on B because B is a function-static variable.

Avoid this. Initialization is not multiple-thread safe. Not accessible to test. Destruction is difficult.

5. A depends on B because B is a class-static variable.

Avoid this. A class with static members and non-static members is doing two things. Separate the two responsibilities into separate classes, one of which can be a longer-lived than the other.

6. A depends on B because B is a member variable of A "by value".

Great for "value" objects, not so good for other kinds of objects. ("By Value" means by not by pointer nor by reference,)

7. A depends on B because B is a local variable of one of A's function.

Inaccessible to test. The dependency is 'invisible'.

8. A depends on B because B is a "by value" parameter in A's functions.

Not subject to faking or mocking, any data or behavior that are defined in a subclass of 'B' is "sliced-off" to be just a "B" inside the function.

9. A depends on B because B is a pointer/reference member of A, created in constructor.

Creation in the constructor is too soon for us to insert a fake.

10. A depends on B because B is a pointer/reference member of A, created in non-virtual function.

We can't override the creation inside the function.

11. A depends on B because B is a pointer/reference member of A, created in virtual function.

This allows us to override the function that creates B, using a fake, stub, or mock.

12. A depends on B because B is a pointer/reference member set by parameter in A's constructor.

We can create A with a fake, sub, or mock in testing. Dependency is visible.

13. A depends on B because B is a pointer/reference parameter in A's functions.

This makes the dependency visible in the interface. We can supply fake, stub, or mock in testing.

Coupling. Or... what you don't want your objects doing too much of.

2011-04-18T04:03:00.000-07:00

Dependencies by one class on another class can cause problems when refactoring and writing microtests. In microtests, you want awkward collaborators to be faked, mocked, or stubbed out, and some kinds of dependencies make that more difficult than others.

Here are some dependencies from best (least-coupled) to worst (most tightly-coupled).

1. Class A depends on Class B, where class B is an interface ("pure" abstract base class).

We can test instances of A by creating our own test-specific subclasses of B to pass into A.

2. Class A depends on Class B, where class B is a concrete class but the functions are declared virtual.

We can test instances of A by creating our own test-specific subclases of B, but at a price: we have to deal with B's constructors and destructor, possibly doing stuff we don't want.

3. Class A depend on Class B, where class B is concrete class with no virtual functions.

We're going to need to refactor B and/or A for testability. Either making B's functions virtual, or putting an testable Adapter between A and B.

4. Class A depends on Class B, and class B grants class A "friend" access.

Class A not only depends on the public interface of B, but also the private parts of B as well. Not recommended. Don't do this unless you are Bjarne Stroustrup. If a test needs to access private parts of B, make those parts 'protected' and use a test-specific subclass to change selected parts of B to have 'public' access.

5. Class A depends on Class B, and class B is a class declared INSIDE class A.

This can lead to mutual complete visibility between A and B, plus the scoping makes it hard to test A without B, or vice versa. Not recommended. Don't do this unless you are Alexander Stepanov.

Also affecting how class A depends on class B is the WAY the dependency is manifest. It turns out that "visible" dependencies are often better than "hidden" dependencies, which is opposite of the usual idea of "information hiding", the predecessor idea of "encapsulation".

from one twitter-reading session

2011-04-09T05:21:00.000-07:00

Stuff I found interesting in reading twitter today, mostly links to things I'd want to read later.

RT: @LeanVoices: The idea is the easy bit, execution is where excellence and hard work resides.

RT: @suziedwards: Resume blooper of the day? "Software Mythologies used include Waterfall and Scrum".

RT @markrneedham: How to type on an iPad http://bit.ly/ftr9Vl

RT @flowchainsensei: Facebook's data center play sidelines Google, Apple http://reg.cx/1NBh

Chad Fowler = "These have changed my developer life, how I perceive software
and that great technology is fun" http://bit.ly/hP2B6O

Cool, @gregturn has finished Python Testing Cookbook! #congrats http://www.packtpub.com/python-testing-cookbook/book

RT @hackernewsbot: New engine shakes up auto industry... http://www.msnbc.msn.com/id/42460541/ns/technology_and_science-innovation/

Divvy is pretty awesome. http://bit.ly/g11Hj5

RT @badbanana: If you enjoy being the 10,000th person to put your thumb into a hole, then bowling is for you.

RT @theCoachingblog: "The things we hate about ourselves aren't more real than things we like about ourselves."
(Ellen Goodman) #quote

"Managing" Velocity: The term velocity is very well-chosen. It means speed in a certain direction, w... http://bit.ly/gvikOP #agileotter

RT @capotribu: "My startup story: from big idea to thriving business in 8 short years" http://t.co/IXMCjfV < great story of real reality

Fun Fact http://dinosaurmusings.wordpress.com/2011/04/08/fun-fact/

Our schools failed many of us. Why don't people understand the role of analogy? http://bit.ly/gBRqji (see comments) #testing

Good read. RT @toddcharron: An Agile Pace http://t.co/Rn9Gzuf - reminds me of http://t.co/gwmGRX7 #agile #scrum #kanban #overtime

Best #Agile article this year: Agile’s Second Chasm (and how we fell in) http://bit.ly/hvQDpT by @williampietri #recommended

RT @tottinge @jamesshore Someone was selling us a product that allows you to "collaborate in isolation from each other."

James Shore = For the record: Rabu is not and never* will be a project management tool, a planning tool, or a replacement for a whiteboard.

See what you started, mike hill? RT @alancfrancis: I am now nekkid as a jaybird, in an apartment with 10 windows wide open, attempting to cool down. #sorrytoputthatimag ...

10 Acts of Resistance That Changed the World — YES! Magazine http://t.co/NFN2UVx

jasonlittle = we did a retrospective on why we stopped doing retrospectives. good insights, blog post coming.

I read this = http://blog.benjaminm.net/2011/02/16/argyriscasestudylearningmodelii/ = worth it

http://blog.benjaminm.net/2010/12/07/agile-argyris-double-loop-learning/

publishing co I never heard of: http://www.packtpub.com/python-testing-cookbook/book

This practical cookbook covers lots of test styles including unit-level, test discovery, doctest, BDD, acceptance, smoke, and load testing. It will guide you to use popular Python tools effectively and discover how to write custom extensions. You will learn how to use popular continuous integration systems like Jenkins (formerly known as Hudson) and TeamCity to automatically test your code upon check in. This book explores Python's built-in ability to run code found embedded in doc strings and also plugging in to popular web testing tools like Selenium. By the end of this book, you will be proficient in many test tactics and be ready to apply them to new applications as well as legacy ones.

http://www.ted.com/index.php/talks/jonathan_haidt_on_the_moral_mind.html

yourmorals.org ?

https://github.com/twitter/commons

engineering.twitter.com blog = Cross-site Scripting Protection is one hundred percent live on mobile.twitter.com

After a soft launch, we ran into some unexpected issues. Several common Firefox extensions insert Javascript on page load, thereby triggering a report. However, even more surprising were the number of ISPs who were inadvertently inserting Javascript or altering image tags to point to their caching servers. It was the first example of how CSP gave us visibility into what was happening on the user’s end. We addressed this problem by mandating SSL for Firefox 4 users, which prevents any alteration of our content.

http://c2.com/cgi/wiki?WikiDesignPrinciples

http://elabs.se/blog/15-you-re-cuking-it-wrong

http://dannorth.net/2011/01/31/whose-domain-is-it-anyway/

http://cuke4ninja.com/

http://www.infoq.com/articles/Accidental-Agilist = Johanna Rothman wrote:

I didn’t think if myself as an agilist, of course. I decided to experiment with some practices, such as pairing and TDD. I’d worked closely with a couple of people one-on-one, back when I was a developer, and I wasn’t sure if it was really pairing. So I tried pairing and TDD with a colleague, Keith Ray, at an AYE conference, and sure enough, it was just like what I had done at work. Except, Keith was much nicer than my work colleague back in the ‘80s.

:-)

J. S. Bach 300+ years of beautiful music

2011-03-20T18:35:00.000-07:00

In honor of Bach's birthday, March 21, 1685, and the #bachchat celebration on twitter. Here are some of my favorite performances of Bach's music.

My current favorite is The "Little" Fugue in G Minor, BWV 578. ("A fugue is a contrapuntal piece of music wherein a particular melody is played in a number of voices, each voice introduced in turn by playing the melody." wiktionary)

To get a feeling for how The Little Fugue introduces the melody over and over, in multiple voices, each continuing to play over the others and yet still sound "good", check out this "piano-roll" animated rendition of this piece on youtube.

The above piece is performed on a piano-like instrument with a pedals, so the pianist can play a bass melody with his feet, just like an organist. And I think you would need that with this piece. I'm not sure how someone would like all the parts with just two hands.

Now the more fun performances are these...

The Canadian Brass:

The Swingle Singers:

Leopold Stokowski's orchestral arrangement:

You can search for other renditions of this fugue on other instruments. Wendy Carlos did a recording using a Moog synthesizer in Switched-On-Bach Boxed Set. The original instrument was of course, organ.

Bach's most famous organ piece is The Toccata and Fugue in D Minor, BWV 565 which may not have been written by Bach (!) and may have originally been a violin piece (!!). See the wikipedia article here. And here's what the violin piece might have originally sounded like:

I rather like Don Dorsey's synthesized version of The Toccata and Fugue in Bach Busters.

No youtube version of Don Dorsey's version. But there is this guitar performance:

My other favorites include the Brandenburg concertos, The Passacaglia and Fugue in C Minor (which is another piece arranged by Leopold Stokowski for orchestra)...

...and several pieces from the Anna Magdalena notebook.

I'm not a fan of Bach's vocal music.

I own a "complete" collection of Bach's music (153 CDs, excluding his religious vocal music), including several recordings of my favorite pieces.

If you are struggling how to TDD something...

2011-02-09T10:05:00.000-08:00

If you struggling to think how to test-drive something with simple input and complex output, a few heuristics might help:

1. Testing pathological code (e.g. bad argument values) isn't going to drive your implementation that much, so leave that for later. Try the simplest successful case first.

2. If you really can't think how to test it, sketch out the algorithm (e.g. code-first) and figure out how you could test that algorithm. If the algorithm is 50 lines long, pretend it is 50 method calls instead: how would you test each of those method-calls? What does each line/method take as input, what is its output? Then try to go back to test-first and re-implement the code. Don't worry about public / private methods. Once your are "done" or at good stopping point, figure out which classes should own which methods, then refactor them to cooperating classes where most of the work is in public methods of each class. Move the tests to be closer to the code they test.

3. A class where there is only one or two public methods and a dozen or more private methods is exhibiting the code smell "Iceberg Class".

4. If the complexity resides in a third-party library or framework, concentrate your tests on YOUR code, assuming the third-party library or framework is correct. Unless, of course, the third-party library or framework needs tests (because it is unreliable), in which case test how your code uses the library or framework to get expected results.

5. You might have the test create the output, you verify that the output is correct manually, and then you save that output as a "gold standard" to test against automatically. I've done that for image-processing... it broke rarely, when the jpg library I was using changed.

High technical debt = slum

2010-11-23T10:13:00.000-08:00

Sometimes I think the "debt" analogy is too clean. Brian Foote and his coauthor used many analogies in his "Big Ball of Mud" paper, including "Shanty Town", but I think the best analogy for the products I've seen with large amounts of technical debt is "Slum".

Products with large technical debt are money-makers, in spite of their problems (until the problems cost too much).

Slums have slum-lords, who profit from lack of maintenance, who are ignoring the pain of the inhabitants (developers and tester), and who are also ignoring the benefits of a clean environment... (of course, you can only take an analogy so far).

Still, it takes an empowered group of inhabitants to transform a slum into a desirable place to live and work. That requires management support.

Some books I recommend to those new to Agile

2010-11-23T09:03:00.000-08:00

The following are some books I recommend to developers new to Agile software development.

This book is new, pretty complete, and easy-to-read about adopting agile software development: The Agile Samurai

This book covers the same ground as the Agile Samurai book. It's a little more in-depth. Most of it is now free on-line, but you can buy the book from Amazon: The Art of Agile Development.

This book is about test-driven development in C: Test Driven Development for Embedded C

This is the original TDD book in Java: Test-Driven Development

The original book on Refactoring.

If you have legacy code, you also need to this book: Working Effectively With Legacy Code.

Industrial logic has eLearning and in-person workshops on TDD, Refactoring, Design Patterns, and other topics. This web-based eLearning contains videos, exercises, and quizzes that are, for most people, more effective than reading a book. I would recommend both.

Going meta sometimes helps.

2010-11-19T22:48:00.000-08:00

One of the sessions I participated in at AYE 2010 was Steve Smith's session on "Power, Authority, and Teams". We did an exercise where we individually tried to solve a problem (on paper, a problem whose answer can be scored as to correctness) and then we were divided into four teams, with one observer each, to try to solve the same problem.

The team that I was on, started with some direct actions by individuals: deciding we would use a flip-chart on an easel, and moving the flip-chart away from another team (which, funnily enough, put us next to another flip-chart on an easel, so we could have avoided carrying that easel around). One partipant decided to act as scribe, and off we want, problem-solving. I'm sure it looked like pure chaos. And it was. (But not necessarily in a bad way.)

As we worked through the problem, sometimes we'd ask for a show of hands in favor of a particular part of the solution. We never actually formalized how voting would work, and certainly never had more than one third of the group raising their hands. I think a lot of us in the group were trying to gauge agreement by audio and visual cues, but we didn't go "meta" long enough to formalize how we would signal agreement. We didn't appoint roles, or vote on a leader.

You have to remember that a many of us were coaches. "Agile coaches," even. We know about dot-voting (I almost suggested it once during this chaos), consensus/thumb-voting. We know about team-formation, Satir change model, self-organizing teams. Some of us had been through Problem-Solving Leadership training. We didn't use any of those tools. I know I could have, but I didn't, and I expect some others were thinking the same thing.

After struggling with one problem solving technique for a while, one of the Europeans in the group complained that our usage of American-style measurements was making it hard for him to contribute. I wrote "1 meter ≈ 3 feet" on the flip-chart to help him out, but another member of the team proposed working in relative sizes instead of absolute sizes. We transitioned to that problem-solving style and, after a little confusion, the team seemed to jell.

We had come to an agreement on process, though never explicitly agreeing on process through any formal means.

Later, we could compare our individual scores on the answering the problem with our team scores. Our team scored higher than any of our individual scores, though we were in third-place compared to the other three teams. The team with the highest score had chosen to use computers to solve the problem. Imagine! A bunch of software people using computers to solve the problem! :-)

OK, our team had a moment of conversation, early on, where some of us agreed not to use computers for the problem-solving, as that seemed against the spirit of the simulation. Again, not a formally-ratified agreement, but no one in our team went against that decision during the simulation. We were supposed to be learning about power: how people interact with each other, so we didn't bring computers into the simulation.

The team in fourth place had a score that was lower than any of their individual scores. I really wish I could have observed what was going on in that team. Somehow their collective problem-solving skill went completely opposite of their individual skills.

I think that even if we had formalized how our team made its decisions, we probably would have come to the same solution that we did in our informal way. We might have done so faster. We might have included insights from some of the quieter members of the team, and might done a little bit better.

It's also possible that if we had been too formal, we could have done worse, perhaps much worse.

Two points I want to mention.

In a team-building exercise at AYE 2005, my team happened to be all PSL graduates and no one in the other team was a PSL graduate. My team quickly jelled. We decided to enjoy ourselves and not stress out. The other team, across the room, we could see going through "forming, storming, norming, performing" stages. We ended up coming in second-place (of two teams)... but each team was working on a different LEGO model, so the difference in performance could be the difference in complexity of the models, or differences in our LEGO-building skills. As much as we would like to have simple causes-and-effects, sometimes effectiveness in one area doesn't mean success in another.

The other point I want to mention, I just read in The Gift Of Time, in the chapter written by Willem van den Ende: "Solving the Groupthink Problem". He quotes a diagram-of-effects by Jerry Weinberg (Quality Software Management vol. 3) Rather than re-print the image, I'll try to describe it.

"Effectiveness of meta-planning" directly affects "Effectiveness of strategic planning".

"Effectiveness of strategic planning" directly affects "Effectiveness of action planning".

"Effectiveness of action planning" directly affects "Quality of Action".

"Other Factors" directly affects "Effectiveness of action planning" and also directly affects "Quality of Action".

"Change Artistry" directly affects "Quality of Action".

That may or may not speak for itself. (Probably not, but it's getting late.) Willem has a paper here that shows other diagrams of effects. Diagramming systems can be useful.

And one last thing... In a discussion with the "power" team, I listed a bunch of different ways we can have power (or influence): speaking louder, speaking more persuasively, deciding on a process for the team to follow, holding the marker and thus acting as scribe...

That last idea had some impact on the participant who had been acting as scribe. She had been thinking that by acting as scribe, she would be avoiding taking power in this simulation. But the scribe can influence what gets recorded, who gets listened to. It's a position of power. We coaches who sometimes facilitate meetings by acting as scribe, should keep that in mind.

There are all sorts of power.

7 Design Transformations

2010-11-13T13:39:00.000-08:00

Joshua Kerievsky summarizes 7 design transformation techniques at Industrial Blogic. We teach these in our Refactoring Album and use these in our daily work. It's a quick read. Check it out!

Beautiful tools

2010-11-04T18:15:00.000-07:00

Go to this page and scroll down to Figure 4. Apple's IDE graphically shows you the results of static analysis by drawing arrows over the code.

Beautiful.

Virtual Functions in C++ make TDD easier, but at what cost?

2010-11-03T10:24:00.000-07:00

Virtual functions in C++ make allow us to do mocking, stubbing, and faking, which helps us test code in isolation in TDD or just microtesting in general. A frequent objection to virtual functions is that it "costs more" than simple function calls.

The cost of a virtual function is hardly ever important, unless the function is being called in a very tight loop, or it is being called at interrupt-time, and even then compiler / linker optimization might overcome the "extra" table-lookup that makes a function virtual.

In C, calling a function through a function-pointer is equivalent to calling a virtual function in C++. Implementations of file-systems APIs in Linux and BSD Unix use function-pointers internally to do what C++ does with virtual functions.

The cost of cache-misses (on-chip cache and off-chip cache) is a LOT bigger than the cost of virtual function pointers. Incorrect branch-prediction is also a CPU-level cost that people fail to understand.

The best way to know if making a function virtual has any user-apparent cost, is to measure the application via some tool that does non-invasive timing. Often, there will be no visible effect if you make some functions virtual that were non-virtual before. (I'm talking about REAL applications, not benchmarks.)

Summoning the Demon: Testing

2010-10-23T23:10:00.000-07:00

James Bach, advocate of exploratory testing and one of the founders of context-driven testing movement. He gave a keynote in Sweden. Watch it here:

He says at one point "You got to get good at what you're doing, and you got to get credit for what you're doing." But they don't blog about it, don't talk about it on the internet. They don't establish a reputation. Then they don't have a fall-back position if they disagree with some stupid metrics that their managers want to impose on the testers.

There are lots of programmers out there blogging about their craft. Percentage-wise its probably a small number, but it seems like it's even fewer testers that do so.

James also says "I have a bunch of Google alerts. If you use certain code words in your blog I will read them. [...] If you say the name of the demon, he shall appear: and read your blog."

:-)

Hi James!

My Response on TDD list on the "excessive cost" of unit testing

2010-10-02T12:28:00.000-07:00

And also, and excessive focus on unit testing inhibits refactoring.

Not in my experience.

What if I split C into two or more classes?

So C was too large and now you're fixing it, splitting it into C and CPrime. Great! Good unit tests for C insure this refactoring doesn't break desired behaviors. If a test fails, it's doing its job, warning you that you did part of the refactoring wrong. Because unit tests are closer to code they test than integration tests, it will be easier to locate the problem and fix it.

Do I have to rework and split all its tests, so that they'll be one-class unit tests on the new classes?

No. If you do refactorings in small steps as described in Martin Fowler's book, the tests don't change (except for name and signature changes that I hope you are using a refactoring tool to do.) CPrime is indirectly tested by the tests for C.

AFTER I've split the class, and the tests are still passing, I may move some tests to directly test CPrime.

Further refactorings might remove some of the forwarding-functions from C that you would have created if you were doing the Extract Class/Move Method refactorings as per Fowler.

What if I decide to combine C and D into a single class. What will that do to my tests?

You change change all users of C and D appropriately, whether those users are tests or production code. Refactoring tools make that easy.

Avoiding the code smell "Duplicated Code" in both your test code and your production code insures that all the changes required for a Merge Class Refactoring are minimal.

And how will my tests help me determine if the new combined class is correct? If I have only unit tests for C and D, then really I won't have any way to determine if the new combined "CD" class is working correctly?

There are three activities in TDD: (1) writing a test (usually one that fails first), (2) writing code to pass a test, (3) refactoring to clean up code smells.

By separating #3 from #2, you are much less likely to have problems. You do need new skills: refactoring is not the same as "rewriting"; you can't let code smells go unfixed for very long; and you need to know how (and when) to use fakes and mocks to test how objects collaborate and also break dependencies to test in isolation.

Because tests in TDD _do_ allow collaboration between multiple objects, they might not fit your definition of a "unit test". That's one reason we at Industrial Logic call them "microtests". Each microtest only tests a single behavior, setting up the class under test with real or fake collaborators as needed.

You might find IL's courses athttp://elearning.industriallogic.com/gh helpful for learning Refactoring, Code Smells, TDD, etc.

Hope this helps,
C Keith Ray

What's the most important thing?

2010-09-07T08:36:00.000-07:00

For Agile software development, what is the one thing that's so important that anything else is a distraction?

short answer: Working software

slightly longer answer: Being able to reliably build working software.

Thoughts?

Mind Heart Body 2

2010-04-23T16:35:00.001-07:00

Mind: More time creating value!
Heart: No more debugging!
Body: Test-drive new features.

http://bit.ly/7pla9C

Mind Heart Body 1

2010-04-23T16:33:00.000-07:00

Mind: More time creating value!
Heart: No more debugging!
Body: Write microtests whenever you find bugs.

http://bit.ly/7pla9C

Test-Driving C++

2010-04-16T22:51:00.000-07:00

Reprinted from my blog in 2007.Jan.31 Wed

In most languages, when I'm using Test Driven Development to create a class, I only put into that class those methods or fields that I needed to pass a test. C++ has some exceptions to that, given how the compiler will generate aspects of a "canonical c++ class" for you.

I should explain the idea of a "Canonical C++" class. Imagine that I have this code:

class Buddy
{
public:
  Icon* myIcon;
  std::string myName;
};

Now, I didn't write a constructor, destructor, nor an assignment-operator, but the compiler did create those for me. It's as if I really wrote the following code:

class Buddy
{
public:
   Icon* myIcon;
   std::string myName;

   Buddy() // default constructor
       : myName() // invokes std::string's default constructor
   { // myIcon is not initialized, it probably has a garbage value here.
   }

   Buddy(const Buddy& other) // copy constructor
       : myIcon( other.myIcon ) // copy the variable's value
       , myName( other.myName ) // invokes std::string's copy constructor
   {
   }

   ~Buddy() // destructor
   {
   } // invokes std::string's destructor for myName.

   Buddy& operator=(const Buddy& other)
   { // assignment operator
       myIcon = other.myIcon; // copy the variable's value
       myName = other.myName; // call std::string's assignment operator
   }
};

A "canonical" C++ class has default constructor (and/or other constructors), copy constructor, destructor, and assignment-operator. These may be defined by the programmer or created by the compiler.

And this invisible compiler-generated code can be wrong, particularly if ownership of pointers or other resources is involved. Let's say that I test-drive a default constructor that sets up myIcon to point to a newly-created Icon object, and write the corresponding destructor code to delete the Icon object. It's hard to verify the "state" of an object after its destructor is called ('cuz it's GONE), but there are a few tricks to verify the behavior of a destructor that I won't get into here.

class Buddy
{
public:
   Icon* myIcon;
   std::string myName;

   Buddy()
       : myIcon( NULL )
       , myName( "no name" )
   {
       myIcon = new Icon(Icon::DEFAULT_ICON);
   }

   ~Buddy()
   {
       delete myIcon;
   }
};

SPECIFY_(Context,BuddyHasDefaultNameAndIcon)
{
   Buddy* aBud = new Buddy;
   VALUE( aBud->myName ).SHOULD_EQUAL( "no name" );
   VALUE( aBud->myIcon ).SHOULD_NOT_EQUAL( NULL );
   delete aBud;
}

This test will pass. (by the way, I'm using "ckr_spec" here, a Behavior-Driven-Design framework I've written in C++ in my spare time. I'll publish more about ckr_spec one of these days.) However, this test doesn't exercise the compiler-created copy-constructor and assignment operators. AND THOSE ARE WRONG. Nothing (besides self-discipline) prevents anyone from writing the following (crashing) code:

void crashingCode()
{
   Buddy keith;
   Buddy keithClone(keith); // calls compiler-created copy constructor
   Buddy anotherKeith;
   anotherKeith = keith; // calls compiler-created assignment operator

// destructors are called invisibly here - crash deleting the same Icon
// object 3 times. (Also leaks an Icon object, too.)
}

The compiler-created copy constructors just copy the pointer to the Icon object. They don't create a NEW Icon object. So in "crashingCode" above, the Icon object created in the constructor of "keith" gets deleted three times, when the destructors for the "keith", "keithClone", and "anotherKeith" objects get called at the end of the function.

Therefore, when I'm test-driving a C++ class, very early on, I make a decision. Is this a "value" class that is going support the copy-constructor and assignment-operator, or an "entity" class that should never be copied because the instance represents something with a persistent identity? (These are some over-simplified ideas from Domain-Driven Design.) I can change my mind later, of course.

If my class is going to allow "value" semantics, then I'll need to write some tests to assure that the copy-constructor and assignment operator function correctly, whether I've written them, or the compiler has generated them.

If I'm not going to allow "value" semantics, then I need to signal to the compiler and to my fellow programmer not to generate or use the copy-constructor and assignment-operator. Declaring them private and unimplemented is how to do that.

class Buddy
{
public:
   Icon* myIcon;
   std::string myName;

   Buddy()
       : myIcon( NULL )
       , myName( "no name" )
   {
       myIcon = new Icon(Icon::DEFAULT_ICON);
   }

   ~Buddy()
   {
       delete myIcon;
   }

private:
   Buddy(const Buddy& other);
       // don't implement copy constructor

   Buddy& operator=(const Buddy& other);
       // don't implement assignment operator
};
// the crashingCode example will not compile now.

For entity objects, quite often I don't want to allow the default constructor either, so I would declare that private and unimplemented as well.

The moral of the story is that in C++, sometimes you have to write code to prevent the compiler from writing the code for you. Just add that to your TDD/BDD development process.

Lean in a Nutshell

2009-09-13T14:10:00.000-07:00

(originally posted 2003 Dec. 31 on my old blog-site)

On the Lean Development mailing list, John Roth summarizes the heart of Lean: (my emphasis)

Lean is the opposite of fat, and the only basic principle of lean thinking is to eliminate waste. As the saying goes, all else is commentary, or at least an elaboration on that one basic thought: find a faster, better, cheaper way to add value to the product, and do it at all scales, from the total, end to end process on down to the individual practices used to construct it.

[...] In manufacturing, the first exercise is to squeeze as much in-process inventory out of the value chain as possible. Doing that will force a huge number of other beneficial changes.

In Lean Software Development, the first exercise is to go to fixed length iterations, where you have a production quality deployable piece of software at the end of each iteration. Doing that one thing will force a huge number of other changes in the process. I can't emphasize those two criteria too much: production quality and deployable. If you need further testing, signoffs or work from some other department after the iteration ends, you don't have it.[...]

Let's look at the seven core principles of lean, and compare them to Extreme Programming practices:

1. Eliminate Waste. Waste is anything that doesn't contribute to adding value to the product. One-hour status meetings waste time, so XP has 15-minute daily standup meetings instead. Excessive documentation and planning wastes time and energy, so XP recommends creating just enough documentation and planning, at appropriate levels of detail. Debugging is a waste, so XP does test-driven-development, which reduces debugging time and reduces time spent manually testing. Slow communication to/from the Customer/Domain Expert is a waste, so XP recommends having the Customer on-site.

2. Amplify Learning. Creating software is a 'learning' process. Typically developers do this learning alone, and that knowledge doesn't spread around to other developers very quickly. XP does coding in pairs to spread knowledge around faster. (Code reviews can also do this, but with higher overhead.) The Customer/Domain Expert is learning, too, as he sees his ideas implemented and developers ask him for details needed for acceptance tests. XP teams often do Retrospectives each iteration to bring learning to the whole team. Having all the developers and Customer working in a "war-room" also helps the whole team learn together.

3. Decide as Late as Possible. In Release Planning, stories are "promises for future communication" - the Customer/Domain Expert describes each feature/story in just enough detail for the engineers to provide a rough estimate. In Iteration Planning, the Customer describes the story in more detail, so the engineers can break it down into a task-list and/or rough design. The Customer/Tester specifies acceptance tests, which test the "what" not the "how", and then the programmers do test-driven-development where "what" tests and "how" coding are interleaved in a short cycle.

4. Deliver as Fast as Possible. "Simple Code" does everything necessary for the current requirements, but not more, so the team can ship it quickly. Keeping the quality high via test-driven-development avoids coding on an unstable foundation, wasting time debugging. Having the Customer/Domain Expert within range of spoken questions allows getting requirements into code in a minimum of time.

5. Empower the Team. XP encourages all team members to design, test, and code, instead of restricting those activities to a few. It encourages people to sign up for tasks, instead of assigning them to those who may not be as willing or as able. Anyone can improve code at any time, as long as the tests continue to run - allowing new learnings to be reflected in the code/design.

6. Build Integrity In. With short iterations where something isn't "done" until it passes (automated and manual) acceptance testing, and test-driven-development insuring that every line of code is tested, the XP team always builds on top of working software. In every iteration, all automated testing is repeated frequently, detecting as soon as possible any "breakage" of code. Any bugs found by manual testing or automated tests failing are not just fixed, but also force the creation of new automated tests.

7. See the Whole. Before beginning the first iteration, the XP team has a release plan. The Customer/Domain Expert has created that plan with the development team, and together they maintain and update the plan throughout the project, measuring and charting progress. Code is only implemented according to the current and previous requirements, so there is no work wasted on a requirement that may be dropped... however, engineers are not going to write code that will be difficult to adapt to that future requirement in that future comes around. To use the (not very good bridge-building analogy): the draw-bridge may not have motors in this iteration, but the builders are not going to build the bridge out of stone in this iteration, ignoring the need for mobility in the future.

By the way, I hate the "bridge-building" analogy, because writing software isn't building, it is designing. Running the compiler and linker associated "build" scripts, etc., are the "building". If building a bridge were as quick and easy as running a compiler and linker, we'd build bridges multiple times in order to test them. Any comparison of software development to bridges should be to designing bridges - an iterative, learning activity where multiple solutions are designed and tested (via mathematical/computer models or physical models) until all the requirements are met.

Measured Throughput - Where are your bottlenecks?

2009-06-10T20:13:00.000-07:00

Agile Hiring: see Esther Derby's article on Hiring for a Collaborative Team.

Johanna Rothman writes about an organization where the number of requirements implemented are always significantly less than the number of requirements requested. They've been blaming the testers (!) and not seeing that their throughput through the whole system is limited. Given the numbers that she posts, I compute that the through-put is about 3.4 completed requirements per week for project 1, 3.7 per week for project 2, and 2.3 per week for project 3.

If that company accepted that they're only going to get about 3 requirements per week implemented, no matter how long the project is, they could avoid waste in the "requirements/design" phase by not specifying more requirements than their measured through-put allows.

They could also use that figure to move to incremental development as well, and maybe learn practices that would improve throughput as well. If they used XP or another appropriate set of agile practices, they could maximize information-flow and minimize delays between requirements, implementation, and testing.

To find the bottle-neck, we need to know who's waiting on whom in this organization. Is there a queue of finished-but-not-tested work piling up in front of the testers? Then it could be that the testers are the bottleneck. One way to deal with a bottleneck is to change to a "pull system". In this case, a programmer only does implementation when tester is about to be ready to test their work. That leaves the programmers idle, but then they could use that "idle" time to improve their development process (code reviews, unit testing, etc.)

In this system, the bottleneck is the programmers. The "pull system" can extend to the requirements: don't specify a requirement until a programmer about to be ready to implement it and a tester is about ready to test it. To move out of phasist approach, work on testing (creating automated tests) and implementation can overlap -- or (the XP way) the programming can actually follow the creation of tests and use the tests as an executable version of the specification.

(Reposted: original was posted 2004.Apr.05 Mon)

Test Driven Dialog

2009-05-27T13:52:00.000-07:00

repost of my blog entry of 2003, April 08:

This is a dialog I created in July 2002, from messages on the Test Driven Development mailing list -- an email conversation between Robert Blum and Roger Lipscombe. After my editing, they didn't recognize their words, so perhaps I did a little more writing than editing. Any similarity to my favorite comics characters is purely coincidence.

Calvin: I don't do test driven programming. I write some code, and test it manually, walking through it in the debugger, checking that it works.

Hobbes: How do you know that it works? Could you express that as a test before you run your code?

Calvin: I can't think about the test until I've thought about the code

Hobbes: Changing the habit of writing code before the test can be a bother. Try this: Write the code on paper. Then write a test.

Calvin: Paper?!

Hobbes: What if you just outlined the code on a whiteboard and didn't write any real code? Could you still write a test first?

Calvin: I'd feel silly doing that alone.

Hobbes: What if you explained it to your pair partner? Could you still write a test first, or maybe have your pair partner write the test?

Calvin: That could work.

Hobbes: I notice you're refactoring without tests. What's that like?

Calvin: After I refactor, I have to test it again manually, in case I make a mistake doing the refactoring.

Hobbes: Wouldn't it be easier to run some quick automated tests before and after you refactor?

Calvin: Well duh! If I had the tests.

Hobbes: If you do test-driven development, your tests drive the design of your code, and as a side-benefit, you get a nice small set of automated tests that make refactoring safe.

Calvin: Perhaps if I start by thinking about the interfaces that my class or module would need to support, I would be able to write the unit test first? Does this sound like the kind of thing I should be trying more of?

Hobbes: I think so. If I were given just the task of writing a class or module, my first test would describe a very simple case of what I wanted from that code. The interface of this not-yet-written code is evolving, of course, during my test-driven development process. Since I'm calling the functions that I need at each moment, without a care how (or if) they are implemented, it tends to create a pretty useable interface.

Calvin: So you find out quickly if your interface functions are no good when you're forced to use them as client, in the tests, before investing a lot of work at writing them!

Hobbes: Now you're getting it.

Calvin: Tell me, when you think of a feature, is your next thought about testing or implementation?

Hobbes: I just think about using that feature. Which leads me to an example of how I would use it. Now I just write down the results I expect from that, and voila, there's the test. I'm not even actively thinking 'I have to write a test now. How can I test it?'. My main focus is how I can use it.

J. B. Rainsberger added a few more lines:

Hobbes: Have you written tests for that code yet?

Calvin: Tests?! TESTS?! I don't need tests! What do I need tests for? Don't you think I can write something this simple without using tests as a crutch?! I know it's right!

Hobbes: (rolling his eyes) Of course you do.

Calvin: Right. Now help me fix these last four bugs.

And Laurent Bossavit capped it off:

Hobbes: I thought I was doing that.

New Industrial Blogic

2009-03-10T19:18:00.000-07:00

Industrial Logic, Inc., my employer, now has a blog, built using technology from our eLearning software. (Which means it doesn't have an RSS feed yet, but I'm sure it will eventually.)

The first entry, "Do What You Love In A Down Economy" by Joshua Kerievsky, is here. I'll probably contribute blog entries there, as well continuing to post, perhaps, more personal blog entries here.

Josh has also finally made visible to the public, the page in our eLearning where graphically demonstrates technical debt. It's the movie where a printout of a single function is draped out in a hallway, 28 pages long.

Students of our training courses and eLearning have been asking Josh to make this video public for years. Now it is.

Who Tests the Tests?

2009-02-05T09:25:00.001-08:00

Q: Who tests the tests?
A: The whole team.

1. Story Tests or Customer Tests test the fully-integrated application. Those tests are developed in close collaboration with the Customer and his/her helpers (for example, Business Analysts, Professional Testers).

2. Programmer Tests, which come from doing Test Driven Development, are tested by seeing them fail (for the right reasons) before making them pass. When pair programming, two sets of eyes see those tests and the code that makes them pass.

The Programmer and Story tests are also run again many times during development. And incremental/iterative development (ala XP, or Scrum, or any of the other Agile methods) requires refactoring as the team adds new features. Refactoring will change the code, but the tests should continue to pass.

The tests continue to pass despite refactoring because the behavior of the tested code should stay the same until both the tests and code-under-test are deliberately changed. If the tests fail during refactoring, either the tests are too close to the code, or the refactoring wasn't done entirely correctly. In either case, at least one or two programmers are going to look at those tests and the associated code again. So the tests are getting another set of eyes reviewing them.

Sometimes I have written a programmer test, which I expect to fail, and it passes unexpectedly. I take that seriously. Is the test not good enough? Is the code that I planned to change already doing what I want? Good thing I have a pair to help me answer those questions.

Peer Discussion Improves Student Performance

2009-01-03T11:39:00.000-08:00

Science Daily reports that peer discussion using "clickers" helps students learn in a way that simple lecturing does not.

Clickers are simple audience response devices, similar to a TV remote control, that allow students to record their answers to thought-provoking, multiple-choice questions in class. After students answer a question individually, the instructor often asks them to discuss the question and then vote again before revealing the answer. After discussion, they usually do better on the question - but why?

The important point is that none of the students were told what the right answer was," said Su. "Even when students in a discussion group all got the initial answer wrong, after talking to each other they were able to figure out the correct response, to learn. That was unexpected, and I think that's dramatic.

Industrial Logic's Agile eLearning allows students to add comments or questions on every page, and even comment or question specific parts of a quiz. You can see a sample of this in the new videos in the Welcome Album. (See "Community and Feedback" in the track "Find Your Way: Start Here")

2008-12-29T09:31:00.000-08:00

William Wake described the essence of an automated test as Arrange, Act, Assert. I've added "Erase", to account for the clean-up that some tests have to do. You might ask why "Erase" added to "Arrange, Act, Assert" and not some word starting with "A". I think starting with "eh?" is close enough. :-)

"The thing about elves is they've got no ... begins with m," Granny snapped her fingers irritably."

"Manners?"

"Hah! Right, but no."

"Muscle? Mucus? Mystery?"

"No. No. No. Means like ... seein' the other person's point of view."

Verence tried to see the world from a Granny Weatherwax perspective, and suspicion dawned.

"Empathy?"

"Right. None at all. Even a hunter, a good hunter, can feel for the quarry. That's what makes 'em a good hunter. Elves aren't like that. They're cruel for fun [...]"

—Terry Pratchett, Lords and Ladies

In C++, with certain test frameworks, a test might be specified in a manner something like the following in C++.


TEST(TestBlurImageFilter)
{
    // Arrange
    string outfileName = NewTempFileName("TestImageFilter");
    Image* sourceImage = new Image("lena.png");

    // Act
    ImageFilter* filter = new BlurImageFilter();
    filter->ProcessToFile(sourceImage, outfileName);

    // Assert
    AssertImagesEqual("expected_lena_blurred.png", outfileName);

    // Erase
    DeleteTempFile(outfileName);
    delete filter;
    delete sourceImage;
}

Note: generally you don't want to deal with files in unit tests; working in-memory would be much faster. Also, if this is one of those frameworks that throws an exception, or otherwise aborts the test if an assertion fails, then the "Erase" portion of the test won't get executed if AssertImagesEqual failed. Let's assume that's not a problem for the moment.

Let's imagine that you then write a another test like so:


TEST(TestUnblurImageFilter)
{
    // Arrange
    string outfileName = NewTempFileName("TestImageFilter");
    Image* sourceImage = new Image("lena.png");

    // Act
    ImageFilter* filter = new UnblurImageFilter();
    filter->ProcessToFile(sourceImage, outfileName);

    // Assert
    AssertImagesEqual("expected_lena_unblurred.png", outfileName);

    // Erase
    DeleteTempFile(outfileName);
    delete filter;
    delete sourceImage;
}

Now you've got duplicated "Arrange" and "Erase" sections. And duplicated logic in tests can be just as bad it would be in production code. Fortunately, most test frameworks already have support for extracting "Arrange" and "Erase" to methods in a "test fixture". The above code could be refactored to something like the following:


class ImageFilterTests : public TestFixture
{
public:
    ImageFilterTests()
        : sourceImage(NULL), filter(NULL)
    {
    }

    string outfileName;
    Image* sourceImage;
    ImageFilter* filter;

    virtual void SetUp()
    {
        // Arrange
        outfileName = NewTempFileName("TestImageFilter");
        sourceImage = new Image("lena.png");
    }
    virtual void TearDown()
    {
        // Erase
        DeleteTempFile(outfileName);
        delete filter;
        delete sourceImage;
    }
};

TEST_F(ImageFilterTests, TestBlurImageFilter)
{
    // Act
    filter = new BlurImageFilter();
    filter->ProcessToFile(sourceImage, outfileName);

    // Assert
    AssertImagesEqual("expected_lena_blurred.png", outfileName);
}

TEST_F(ImageFilterTests, TestUnblurImageFilter)
{
    // Act
    filter = new UnblurImageFilter();
    filter->ProcessToFile(sourceImage, outfileName);

    // Assert
    AssertImagesEqual("expected_lena_unblurred.png", outfileName);
}

Not only has this eliminated the duplicated logic, most unit test frameworks will also guarantee running the TearDown method even if the test fails, so you don't have to write your own try/catch blocks or other contortions for exception-safe "erase".

You'll see that I also added a constructor to insure that the pointer variables have valid NULL values so we don't delete garbage pointers if the Image or Filter objects were not allocated successfully. (You should consider using boost::shared_ptr and/or boost::scoped_ptr if you're dealing with object pointers in C++ code and tests, by the way.)

In those C++ test frameworks where the test-fixture creation and deletion is done just before and after executing the test, the SetUp and TearDown methods can (almost always) be replaced with a constructor and destructor instead. Using that and boost::scoped_ptr to insure exception-safe object deletion would allow us to write the following code:


class ImageFilterTests : public TestFixture
{
public:
    ImageFilterTests()
        : outfileName(NewTempFileName("TestImageFilter")),
          sourceImage(new Image("lena.png"))
    {
        // Arrange
    }

    virtual ~ImageFilterTests()
    {
        // Erase
        DeleteTempFile(outfileName);
    }

    string outfileName;
    boost::scoped_ptr sourceImage;
    boost::scoped_ptr filter;
};

TEST_F(ImageFilterTests, TestBlurImageFilter)
{
    // Act
    filter.reset(new BlurImageFilter());
    filter->ProcessToFile(sourceImage.get(), outfileName);

    // Assert
    AssertImagesEqual("expected_lena_blurred.png", outfileName);
}

TEST_F(ImageFilterTests, TestUnblurImageFilter)
{
    // Act
    filter.reset(new UnblurImageFilter());
    filter->ProcessToFile(sourceImage.get(), outfileName);

    // Assert
    AssertImagesEqual("expected_lena_unblurred.png", outfileName);
}

C++ Mocking Framework

2008-12-15T10:09:00.000-08:00

Google released Google Test for C++ earlier this year, and just recently released Google C++ Mocking Framework.

These are extensively documented... check them out!

What a Tangled Web We Weave, When Don't Have Effective Tracking

2008-12-09T10:53:00.000-08:00

This is a re-post of a blog entry I wrote in 2003...

Steve Norrie points to a recent online article by Jerry Weinberg published on CrossTalk, the Journal of Defense Software Engineering here: Destroying Communication and Control in Software Development.

I'd like to mention a few of the Extreme Programming (XP) solutions to some of the problems mentioned in this article. Be aware that XP is a light-weight process, relying on people rather than technology to do the right thing. (A savvy person can undermine technology anyway.)

Requirements. One of the first areas that communication can be destroyed is by not doing requirements well... not involving the customer, thinking that requirements are a waste of time, and so on. Extreme Programming recommends involving the customer, or a qualified representative of the customer, throughout the entire project. And in addition to talking about the requirements often and in detail, XP requires writing the requirements down in an executable form - automated acceptance tests.

The configuration management system (CMS). The CMS tracks requirements, design, code, test data, test results, user documentation, etc. Jerry notes that information in the CMS can be undermined by failing to keep it up to date, restricting read-access from people who should have access, removing data or failing to put data into it. Extreme Programming doesn't require any specific CMS system (software or otherwise) for this tracking, but does suggest using the simplest thing that actually works. For code, tests, and test data, I recommend using CVS or some other source-code-management system. For acceptance test results, many XP teams record those on a white board or poster board, updated them weekly or daily, charting them over the course of the project. XP does strongly recommend using index cards (story cards) for tracking requirements during initial and weekly planning, and also strongly recommends documenting the relationship between the automatic acceptance tests and the story cards. The person playing the Customer role, as well as the whole team, is responsible for keeping track of the stories. Many XP teams also record the stories in web-accessible ways, such as a wiki.

In regards to a bug-tracking database, some XP teams track bugs the same way as stories - index cards and acceptance tests. You might think that this won't work, but consider that several XP teams have reported that their bug rate after implementing XP dropped from hundreds per six months to around one bug per month. It helps that bugs are not usually recorded until after a story is finished, and in XP, a story is not finished until it is passing its acceptance tests - this requires conversation between the tester and coder as soon as problems are noticed during the implementation and testing of the story. When a bug is recorded, it probably indicates that an acceptance test that was passing has started failing.

Weinberg recommends that you "set and enforce a policy of complete and open information at all times." Agile processes like XP need accurate information daily. Many projects keep tracking information on poster boards and white boards, visible not only to all team members, but anyone in management who walks by. This is the "Project Progress Poster" concept that Weinberg recommends in Quality Software Management: Anticipating Change. Since in my company, few people in management walk by, we keep tracking information in our wiki web pages.

Quality Assurance. Weinberg recommends "Prevent these abuses by having quality assurance report to the highest levels of management, and not to project management." In XP, QA testers are delegates of the person playing the Customer role -- the same person who defines the requirements. QA testers should not only be implementing and running the automated acceptance tests, but also running stress tests and manual testing of the product's user interface.

Weinberg reports that testing often comes too late in the project to be useful in effective risk management. (There's a phrase in XP circles: "Doctor, it hurts when I do this..."). Don't wait until it is too late. XP requires testing to start in the first iteration of the project -- the first week. This and other XP practices enables effective risk management.

The XP solutions noted here require that project management be willing to face reality at all times. One of the quickest routes to failing with XP is to not do XP. If project management destroys information, hides it, degrades it, or inserts misleading inforation, intentionally or not, it is going to be very difficult to have a successful project, no matter what the methodology used.

Science Daily Snips

2008-11-25T10:39:00.000-08:00

Hopes for cleaner, safer energy:

Quicker, Easier Way To Make Coal Cleaner "Construction of new coal-fired power plants in the United States is in danger of coming to a standstill, partly due to the high cost of the requirement — whether existing or anticipated — to capture all emissions of carbon dioxide [...] Instead of capturing all of its CO2 emissions, plants could capture a significant fraction of those emissions with less costly changes in plant design and operation [...]"

Making Gases More Transportable "Chemists at the University of Liverpool have developed a way of converting methane gas into a powder form [...] made out of a mixture of silica and water which can soak up large quantities of methane molecules [...] might be easily transported or used as a vehicle fuel."

Agile Methods Help Programmers Shine

2008-11-18T16:50:00.000-08:00

Now and then someone says something like "Agile methods won't turn a mediocre programming into a star. That irritates me, because bad non-agile methods can make a very good programmer produce mediocre results. Without taking into account their environment, no one will know how good that "mediocre" programmer can actually be.

A star NASCAR driver probably won't win the race, if his car has a flat tire and his support team can't or won't replace it. It doesn't matter how good the driver is, if the car is bad.

One simple idea - that the "traditional" way of developing an application is by implementing one complete layer at a time (whether bottom-up or top-down) - dooms a project to deliver no working features until late in the development cycle. When feedback finally arrives about the features (wrong requirements and/or wrong implementations), there is often not enough time to fix all the problems, and so the developers look bad. The testers also look bad if they don't have enough time to test, or enough time for fixes to be tested.

Turn that idea around - develop one vertical slice of an application at a time, so each feature can be tested as soon as possible - allows the customers and testers to provide feedback early in the development cycle. That can make a difference in how good the final product can be. Instead of projects running late or shipping with bugs, the project has known good features, can ship when enough features have been done, and the developers look so much better. People might say the project has star programmers.

Many programmers and testers have been trained in that "traditional" style. It's assumed in many books, schools, and corporate cultures. I call it "Unconscious Waterfall" when the style is just assumed. It takes a very remarkable developer to do good work in this environment.

In some shops where they do something they call "Waterfall" and they make it work, it turns out that they verify that many features are working correctly early in the project - there are many places where requirements, tests, and code are examined, verified, validated. Those kinds of waterfall projects are pretty rare, from what I see.

Adopt pets from the Humane Society

2008-11-05T09:27:00.000-08:00

I adopted two cats last weekend. A five-month old female and a 3-year old female. Beautiful, curious, energetic, affectionate. I'll post pictures later.

Check the web pages of your local humane society. They probably have cats, dogs, rabbits and birds. If you can, give some of them a loving home.

When a student is ready...

2008-10-26T10:27:00.000-07:00

When a student is ready, a teacher will appear. On an agile team, everyone can be a student and teacher. One team member may be more knowledgeable about the unit testing framework, but another may know more about object-oriented design. One team member may be more skilled with the IDE, and another knows the in-and-outs of the mocking framework.

Pair programming is an excellent time for these kinds of knowledge transfer. Code-reviews (particularly those conducted asynchronously over email) are a low-bandwidth medium for teaching and learning. It can happen, but teaching and learning has a lot more room to occur with in-person interactions.

The converse: "When a student is not ready, a teacher will not appear." also happens. People who reject pair programming without giving it a serious try are sometimes afraid to expose their fallibility. I didn't expect to enjoy pair programming, and when I first gave it a try, I was making typos and exposing my fallibility right from the start. And then the person I was pairing with was just as clumsy and fallible. As we continued to pair, we learned not only how fallible we are, but also how smart we are. We get to know we each as human beings. We find out that instead of being a waste of time, we make progress faster than we could if we were programming alone.

It takes courage to pair program. Do it for a while, and you'll find it also provides more safety than programming alone does.

Refactoring for Testable C++ at OOPSLA

2008-09-17T10:22:00.000-07:00

I will be co-presenting a workshop on Refactoring Legacy C++ Code to Testable C++ with Joshua Kerievsky at OOPSLA 2008. If you deal with C++ code and want to make it testable, this tutorial is for you.

This tutorial for intermediate-level C++ programmers is based on my own pragmatic experience working with legacy C++, as well as the experience of other Industrial Logic coaches. It is part lecture with slides and video, and part practice on C++ code that we provide.

I gave this workshop with Gil Broza at the Agile 2008 conference, and the class members really appreciated digging into the labs -- to the point that many of them choose to work on the code and tests in one of the labs instead of taking a break.

If you are working with hard-to-test legacy C++ code, and want to get it under test so you can safely refactor it, this tutorial will present several hard-to-test examples of C++ code and show how it can be refactored into testable code. The legacy code problems that may experience have several possible refactorings to get them under test, and we will cover the pros and cons of the most common refactorings and problems.

Looking at Good Object Oriented Design

2008-09-05T11:40:00.000-07:00

Looking at good Object Oriented design is like watching an effective manager (or CEO) who knows how to delegate the proper responsibilities to his or her team(s).

jBehave's Golden Rules

2008-09-03T10:40:00.000-07:00

I like these "Golden Rules" for contributers to jBehave:

Here are some rules that we follow when developing JBehave:

Example Driven: Only create code to solve a realistic (and preferably real) problem.
Behaviour Driven: Only create classes once you can describe and automate examples of their behaviour.
No Misbehaving: Automated examples must work before being checked into the repository, and existing tests must not be broken.

They have a new release, by the way. With "plain text" scenarios. Check it out.

SF Reading material for you and your iPhone

2008-08-21T07:21:00.000-07:00

Jeffrey Carver has made the book Neptune Crossing, available for download. The HTML version appears to be quite readable on my iPhone when rotated for a horizontal aspect ratio.

Cory Doctorow has made several of his books available for download, the best being Little Brother. He is finding that free online releases of his books are enhancing the sales of his paper editions, rather than reducing sales.

Other authors are finding the same thing. Old out-of-print books made available for download can revive an author's readership, while new books simultaneously released on the web and in paper can generate more buzz than just the paper edition will.

A little bit more about Little Brother: one of the blurbs sums it up well:

Little Brother is a scarily realistic adventure about how homeland security technology could be abused to wrongfully imprison innocent Americans. A teenage hacker-turned-hero pits himself against the government to fight for his basic freedoms. This book is action-packed with tales of courage, technology, and demonstrations of digital disobedience as the technophile's civil protest.
– Andrew "bunnie" Huang, author of Hacking the Xbox

Check it out.

Can someone do a meta-study on agile adoption

2008-08-19T10:02:00.000-07:00

Get conference proceedings from Agile2008 and earlier. Check out all the experience reports and other papers about agile adoption. What do the success reports have in common? What do the failure reports have in common? What is a typical timeline for success for transforming a project team, a division, a company?

Clean Code

2008-08-15T11:23:00.000-07:00

Clean Code (also known as "Simple Code" as defined by Kent Beck) is (paraphrased):

Correct: it passes all its tests and its tests are thorough (but not redundant) and correct.

Does not contain duplication: Duplicate logic is eliminated and localized in a few places – this may require using functional or object oriented concepts and design patterns. Duplicate data is minimized. Duplicate syntax is also minimized. No redundency.

Expresses its concepts: by using good class names, methods names, and other identifiers. Dependencies are expressed in interfaces. Methods and classes are short, readable, and have a single responsibility.

Contains no superfluous parts: it does not have dead code, unnecessary parameters, etc. No code left over from features that have been removed. No code for features that haven't been scheduled for implementation.

I find that more explanation is often needed, since some programmers will, for example, point at "expresses its concepts" by saying: this huge function in this huge class perfectly expresses the two dozen actions it performs.

How I Learned...

2008-08-09T19:58:00.000-07:00

richard durnall points to an article "How I Learned to Let My Workers Lead" by Ralph Stayer. Richard notes

There is a deeper, less tangible philosophy embeded within Lean that focuses on people, the systems we place them in and the behaviours these systems encourage. [...]

The Stayer article explains the leap of faith we have to take as leaders to create new systems where the people can determine the process.

In Lean Ops we missed this deeper philosophy. We focussed on the tools and when they didn’t work we just tried harder to enforce the application of the tools. The irony! [...]"

This correlates with several lessons that I am still learning to embody within myself and organizations I interact with.

An observation: The most well-known success story of "Lean", Toyota, shows that the corporate culture is more important than the practices. Attempts to imitate the practices without the culture of true empowerment have failed. And from what I've heard, many of their specific practices change over the course of time.

A few choice quotes from Ralph Stayer's article:

If I was going to fix what I had made, I would have to start by fixing myself.

Of course, fixing himself and his company (which, since he was the founder, was a reflection of himself) was a long and circuitous course... The "end state" he envisioned for his company was:

[...] a flock of geese on the wing. I didn't want an organizational chart with traditional lines and boxes, but a "V" of individuals who knew the common goal, took turns leading, and adjusted their structure to the task at hand. [...] Most important, each individual bird is responsible for its own performance.

Note the difference from his "end state" and how he attempted to get there:

I spent those two years pursuing another mirage as well -- detailed strategic and tactical plans [...] We tried-to plan organizational structure two to three years before it would be needed -- who would be responsible for what and who would report to whom, all care fully diagramed in boxes and lines on charts. Later I realized that these structural changes had to grow from day-to-day working realities; no one could dictate them from above, and certainly not in advance.

Now that is one of the lessons I learned when I attended Jerry Weinberg's PSL and participated in the experiential simulations he designed. Everyone who attends tends to learn different lessons -- often, the lessons they most need to learn. Jerry creates an environment that enables us to observe problems, without him having to didactically explain what's going on. He was teaching this before "Agile" or "Lean" became buzzwords in the software development community.

Trying to assign decision-making (or not) responsibilities and define a team structure before beginning a project is a something like buying equipment and organizing a team for to play softball, without knowing that the sport you're actually going to be playing is basketball.

[...] The early 1980s taught me that I couldn't give responsibility. People had to expect it, want it, even demand it. So my end state needed redefining. The goal was not so much a state of shared responsibility as an environment where people insist on being responsible.

To bring people to that new Point B, I had to learn to be a better coach. It took me additional years to learn the art of coaching, by which, in a nutshell, I mean communicating a vision and then getting people to see their own behavior, harness their own frustrations, and own their own problems.

Coaching is an art I'm still working on. As an Agile coach, I can model the behavior I'd like to see in others, observe, communicate, offer feedback, ask for feedback, and sometimes give advice. It turns out that managers (even CEOs) have little more power than a coach does:

The debacle of ordering change and watching it fail to occur showed me my limitations. I had come to realize that I didn't directly control the performance of the people at Johnsonville, that as a manager I didn't really manage people. They managed themselves.[...]

Stayer describes how top management had the responsibility of checking the quality of the product (tasting the sausage) and he made the change that the workers who are responsible for creating the product would become responsible for checking quality, with the expectation that quality would remain high.

The employees self-organized taking ownership for quality and defects, and to remedy the defects. This led to better quality and employees asked for more information and more responsibility including changing training, HR, and salary structures: paying for people taking on more responsibilities rather than simply job tenure.

Check it out.

Last day at Agile2008

2008-08-08T04:59:00.000-07:00

I will be on the "Legacy Wrapup" panel with Michael Feathers today.

Industrial Logic at Agile 2008

2008-08-01T09:21:00.000-07:00

My Industrial Logic colleagues will be out in force at the Agile 2008 conference.

In order of appearance...

Tuesday

Brian Foote: "Agility, Evolution, Emergence, And The Primordial Ooze"
Mike Hill: "Throwing the Agile Transition Party"
Brian Foote: "Big Ball of Mud"
Joshua Kerievsky : "Ten Terrific Transition Tips"
John Tangney: "Mastering Selenium"

Wednesday

Mike Hill: "Clean Code Clinic: Dealing with CRRAP - Microtesting Legacy Code"
Joshua Kerievsky, Gil Broza: "Crafting User Stories – Four Experts And The Audience Weigh In"
Joshua Kerievsky: "Refactoring Strategies & Tactics"
Gil Broza: "Agile Clinic - Bring Your Toughest Challenges"

Thursday

Gil Broza: "The Secrets of High-Performance Agile Implementations"
Keith Ray, Gil Broza: "Refactoring for Testable C++"
Brian Foote: "Patterns Poster Children"
Joshua Kerievsky: "Estimating Considered Wasteful: Introducing Micro-Releases"

Tuesday - Friday

Joshua Kerievsky, Mike Hill: "Programming with the Stars"

next week - Agile 2008

2008-07-29T13:43:00.000-07:00

Next week I'll be at the Agile 2008 Conference, where I'll be co-presenting a workshop with Gil Broza on refactoring legacy c++ code for testability. I might even blog from the conference. Hope to see you there.

Three rules of TDD?

2008-07-29T10:50:00.000-07:00

A recent object mentor blog entry said:

We had an interesting result during that class. One group was practicing Bob’s three rules of TDD (paraphrased);
Write no production code without failing tests
Write only enough test code so that it fails (not compiling is failing)
Write only enough production code to get your tests to pass.
But they ended up with a bit of a mess. Following the three rules wasn’t enough. These rules are guiding principles, but those three rules mean nothing if you forget about clean code, refactoring and basic design principles [...]

Maybe the cause of the mess were the rules they used. I remember the rules more like this:

Write enough test code so that to get a single test failure.
Write enough production code to get all your tests to pass.
Refactor until the code is Simple/Clean. (See elsewhere for the Rules of Simplicity/Clean)

As Brett Schuchert points out in that blog post, leaving out the refactoring step puts you onto the road to certain doom.

Hello again!

2008-07-28T15:07:00.000-07:00

I'm moving my blog Exploring Solution Spaces away from being hosted on .mac into blogger.

More later!