C. Keith Ray

C. Keith Ray is developing software for Sizeography and Upstart Technology. He writes code for multiple platforms and languages, including iOS® and Macintosh®.
Go to Sizeography and Upstart Technology to join our mailing lists and see more about our products. Keith's Résumé (pdf)

Tuesday, March 31, 2015

Conflict Haiku and other Writings

(Originally posted 2003.Jun.18 Wed; links may have expired.)

Some good papers by Rick Brenner of Chaco Canyon consulting... the Point Lookout email newsletter.

Conflict Haiku: "When tempers flare, or tension fills the air, many of us contribute to the stew, often without realizing that we do. Here are some haiku that describe some of the many stances we choose that can lead groups into tangles, or let those tangles persist once they form."

When Naming Hurts: "One of our great strengths as Humans is our ability to name things. Naming empowers us by helping us think about and communicate complex ideas. But naming has a dark side, too. We use naming to oversimplify, to denigrate, to disempower, and even to dehumanize. When we abuse this tool, we hurt our companies, our colleagues and ourselves."

Let Me Finish, Please "We use meetings to exchange information and to explore complex issues. In open discussion, we tend to interrupt each other. Interruptions can be disruptive, distracting, funny, essential, and frustratingly common. What can we do to limit interruptions without depriving ourselves of their benefits?"

Full list of past issues: http://www.ChacoCanyon.com/pointlookout/topicalarchive.shtml

Monday, March 30, 2015

Unit Testing In Java, Duplicate Code in Java GUIs

(Originally posted 2003.May.28 Wed; links may have expired.)

Johannes Link announces that the book Unit Testing In Java by Link and Frohlich, is now available for purchase, even though Amazon says it isn't yet. Link says the book teaches Test Driven Development in Java, covering GUIs, web-applications, databases, multi-threaded applications and other advanced topics. A sample chapter is available here: [PDF, 28 pages] http://www.bhusa.com/bookscat/samples/1558608680/1558608680.pdf?mscssid=W2B78S2RV83D8K9W260JPX1W0DUJ9DC7.

The sample chapter on test-first Swing GUI programming isn't that different than other writings on Java GUI test-first. Near the end of the chapter they discuss JFCUnit, and briefly mention AWT Robot, neither of which they recommend for typical test-first unit testing. I will order the book and read it.

One nit I'd like to pick: the authors make the JFrame subclass an ActionListener, listening to the "Delete" button and the "Add" button. This is very common in Java books, but that doesn't make it right. Making the JFrame an ActionListener on multiple widgets means we have some tightly coupled pieces of code (reformatted - I like the braces to line up):

public class CatalogEditor extends JFrame
    implements ActionListener, ListSelectionListener 
    private void addAddButton() 
        addButton = new JButton("Add");
... etc. (similar code to add "Delete" button)...

    public void actionPerformed( ActionEvent e ) 
    { // blech... nasty.
        if ( e.getSource() == deleteButton ) 
        else if ( e.getSource() == addButton )

This is nasty because it couples the JFrame too tightly with the buttons it is listening to. If you add another button, you also have to change the actionPerformed method. You would rather not have to change multiple pieces of code when you add "just one more thing". There is a better way (using anonymous subclassing) - don't declare the JFrame to be an ActionListener; instead, pass anonymous implementations of ActionListener to the buttons like so:

public class CatalogEditor extends JFrame
    private void addAddButton() 
        addButton = new JButton("Add");
        addButton.addActionListener( new ActionListener() 
                public void actionPerformed( ActionEvent e ) 
... etc. (similar code to add "Delete" button)...
    // now there is no need for an actionPerformed method that uses 
    // cascading if statements.

I also want to point out the code duplication we have: creating the "Add" button is almost the same code as creating the "Delete" button (not shown). That kind of duplication seems to be common in Java GUI programming, but it is really unforgivable when you have lots of dialogs and buttons. What can we do about it?

First, we could have a DialogCreator class that reads the button names from a string list, creates them, and adds them to the ContentPane of the JFrame subclass (which it could also instantiate). It could also read the layout name and other information needed to lay out the GUI. In order to get the "Add" button's listener to call the method "addButtonClicked" in the JFrame subclass, we need to use the Java reflection API. Here's the pseudo-code:

class ListenerCaller implements ActionListener
    public ListenerCaller( Object objectToCall, String methodNameToCall )

    public void actionPerformed( ActionEvent e )
        ... reflection to call methodNameToCall on objectToCall...

Notice that the ListenerCaller class doesn't care what kind of class it is calling. You could connect your GUI buttons to your model classes directly this way. Apple's Cocoa does something similar - the Cocoa widget classes (menus, buttons, etc.) take an object (called "target") and a method selector (easily created from a string) and call whatever object and method that you specify. You would normally do this specification in the Interface Builder.

class DialogCreator
    public void CreateDialog( textfile or string list )
        String jfClassName = read JFrame subclass name from textfile;
        JFrame jfObj = ... reflection to create object given jfClassname;

        String layoutName = read layout name from textfile;
        // also read number of arguments, etc... this can get complicated.
        Layout lay = ... reflection to create object given layoutName, 
             number of arguments, argument types, etc.;

        jfObj.getContentPane().setLayout( lay );

        while not end of buttons
            String buttonName = read button name from textfile;
            String methodToCall = read method to call name from textfile;

            JButton aButton = new JButton( buttonName );
            aButton.addActionListener( new ListenerCaller( jfObj, methodToCall ) );
            jfObj.getContentPane().add( aButton );

A single DialogCreator class like this removes button-creation code from all your Swing GUI code. Given that GUIs have a nested hierarchical structure, you probably want your text file to reflect that structure as well, using XML, and creating all kinds of widgets, not just buttons. It turns out that someone has already done all this work (and more) and called it XMLTalk.

I haven't used XMLTalk myself, because I haven't needed to do very much Java GUI programming, and I'm not sure about its license agreement. I haven't downloaded the source, but I have looked at the white papers and other documentation. It is worth studying to see how ValueModel adapters and other helper classes can be used to decouple your GUI code from the model code, and to decouple some of your model classes from each other. I should point out that XMLTalk is based on concepts first(?) implemented in VisualWorks Smalltalk. You should check out their product as well.


(Originally posted 2003.Jun.02 Mon; links may have expired.)

Previously, I used my review of the sample chapter of the book Unit Testing In Java by Link and Frohlich to launch my diatribe against the typical Java-book method of creating widgets in windows/dialogs. Johannes Link send me this comment:

Subject: Unit Testing in Java

Hi Keith,

Compliments on being so quick in reviewing the sample chapter of "Unit

Testing in Java". That's really impressing. 

One thing I'd like to comment on: You're perfectly right when saying that

implementing ActionListener in the window class itself usually leads to loads

of duplicated code. The code in the book, however, is in a temporary state. As

long as there are only two buttons the code is easier to read (IMO) than

when using anonymous inner classes.  Refactoring to a different solution would

certainly take place during one of the next enhancements. The point is: Having

the tests in place allows you to perform this refactoring on stable


BTW, your blog really is worth reading.

best wishes,

Johannes Link

When to do refactoring is certainly worth further exploration. Martin Fowler has the "three strikes" rule to remind him to refactor away duplication when a third instance is seen. Extreme Programming recommends "once and only once" to refactor after the second instance is seen. But when your current task is test-driving a dialog or widget-filled window, at what point do you refactor to a dialog-creator class - after the second button, or after the second dialog, or never?

Sunday, March 29, 2015

La Internacia Lingvo

(Originally posted 2003.Jun.08 Sun; links may have expired.)

When I was in high school, in the 1980's, I discovered a book on Esperanto at a library and studied it. I bought a few books, and even acquired (I don't recall how) a record album of folk songs recorded in Esperanto. It was interesting, but I never had anyone to speak it with. For academic credits in high school and college, I took Latin, another language I didn't use for speaking purposes. In case you haven't heard of it, Esperanto was one of the first "artificial languages," a language created to be culture-neutral, to enable international communication. It has very simple rules of grammar, and a vocabulary adopted from root words of many European languages. Esperanto is reported to be ten times easier to learn than a typical European language.

Created over a hundred years ago, Esperanto was not just a language, but a movement for international understanding and peace. It was denounced by Hitler and banned in Nazi Germany. The USSR in 1938 shot or deported Esperanto speakers. Pre-WWII Japan mistreated and sometimes executed Esperantists. In the USA during the McCarthy era, Esperanto was associated with Communism (and perhaps some American Communists were using it as a 'secret language'.)

Perhaps at its height of popularity, between one and ten million people spoke Esperanto world-wide. In recent years, it has been reported that Klingon and Elvish are two artificial languages with more speakers than Esperanto, though I think more people are actually fluent in Esperanto than Klingon.

Why am I talking about Esperanto? Partly because I just read an article about it in the San Francisco Chronicle, which reports that about 25 people in the Bay Area get together to speak it monthly. It quotes ELNA vice president of public relations Charlie Galvin as saying "For me, I like Esperanto because it's like a secret club. I like the fact that few people know it. It makes it special." That is so depressing, in light of the intent of Esperanto's creator.

Esperanto will always be a little-known language because there is no economic reason to learn it - and that's just fine with the Esperantists. The Esperanto-enabled travel service is free and non-commercial. In this day of international organizations (both commercial and non-commercial), Esperantists are no longer making the case that that a simple-syntax, easy-to-learn language could help people communicate with each other on a level playing field, with value gained in commercial as well as non-commercial arenas.

Another reason I'm talking about Esperanto is that it reminds me of Smalltalk. Like Esperanto, Smalltalk has a simple syntax that is pretty easy to learn. Esperanto was invented by an idealist wanting international communication. Smalltalk was invented by idealists who thought it could be a computer language easy enough for children. Like Esperanto, few people today have an economic reason to use Smalltalk. No big companies are hyping Smalltalk the way Sun hypes Java, or Microsoft hypes C#. Even IBM, which sells a Smalltalk environment, is putting more energy into Java than Smalltalk.

The rules of grammar for Esperanto can be written down in few pages, including all the exceptions. The rules of Smalltalk's grammar are also simple. This PDF explains Smalltalk syntax in relatively few pages: http://www.speakeasy.org/~podenski/stug/reading-smalltalk.pdf.

Smalltalk syntax is unusual today, though I find it easier to read than Perl or Ruby. Objective-C is a language that adds object-oriented constructs to the C language using a syntax similar to Smalltalk. One of my coworkers complained "why isn't Objective C using Java or C++ syntax?" Because it predates C++. When Objective C was invented, Smalltalk was the leading OO language. Now Smalltalk is one of the language reported as dead or near-dead by Wired magazine.

Objective-C is regaining mind-share these days because it is the recommended language for implementing GUI applications on MacOS X using the Cocoa class library. There is an economic reason for people to adopt that language -- it is the easiest way to put a really nice GUI on non-gui unix-compatible code on MacOS X. At least five books on Cocoa programming have been published in the last year on Cocoa, and every one teaches the OO parts of Objective C in the opening chapters.

Unlike some Smalltalk implementations, Objective C on MacOS X plays well with others. Linking C libraries and Objective code together is trivial. Even though the Cocoa class library is implemented in Objective C, you can write Cocoa-using applications in Java, Python, AppleScript, Ruby, and a Smalltalk-like scripting language called FScript. (C++ is too "static" to easily interoperate with classes written in dynamic languages, but Apple's version of 'gcc' allows mixing C++ and Objective C code if one is careful to cope with the differences in object allocation and deallocation.)

I would use Smalltalk it if had native GUIs on MacOS X and Windows. If it interoperated with Cocoa on MacOS X. If it easily interoperated with C libraries. If it could be used to implement shared libraries (TWAIN plugins) and other non-application code on the two platforms I need to support, and not have to be written in "dialect dependent" forms on each platform. If it didn't freak out my coworkers. If it wasn't a "secret language."

Saturday, March 28, 2015

Refactoring is Not Rewriting

(Originally posted 2003.May.26 Mon; links may have expired.)

I'm concerned that people are getting sloppy about their usage of the word "refactoring." I had a short dialog with someone asking about refactoring, bug-fixing, and testing, where the questioner revealed that he had not read Martin Fowler's book "Refactoring: Improving the Design of Code." If he had read the book, then he would have known that refactoring is improving the design without changing behavior, and could have avoided problems from combining bug-fixing with design improvements.

If you're writing or fixing code, particularly in an object oriented language, you must buy and read Refactoring: Improving the Design of Existing Code by Martin Fowler.

That book and Design Patterns by Gamma, et. al. are essential reading no matter what methodology you might be using.

In test-driven design, refactoring is the design step: test, code, refactor, repeat.

Martin Fowler has a blog/wiki (a bliki) here: http://martinfowler.com/bliki/, which I'm adding to my blog-roll. (I'm delighted to see I'm already on Fowler's blog-roll.) I'd love to make my site into a bliki, but unfortunately, my site is static (updated only when I'm adding a new entry) - there's no cgi. My time is full with working, reading, and blogging, so I probably won't get around to writing my own bliki software (though it would be a good exercise.)

That questioner I mentioned was asking about refactoring, bug-fixing, and testing legacy code. I forgot to point him to Michael Feathers, currently working on a book titled Working Effectively with Legacy Code, and the mailing list discussing the book, http://groups.yahoo.com/group/welc/, but I'll do that after finishing this blog entry.

Here's my advice on this topic.

First, keep refactoring and rewriting (bug-fixing) separate. These two activities may be done only five minutes apart, but you're wearing a different "hat" during each activity. Refactoring is improving the design of the code, while preserving its behavior. Bug-fixing is changing the behavior.

Since unit-testing legacy code is almost always difficult, try writing some automated acceptance tests before doing other changes. The acceptance tests will hopefully not need to be changed much during refactoring and bug-fixing activities.

When confronted by legacy code, my first step would be to review the sources, fixing up formatting problems - indentation and spacing. Badly-designed code is often badly formatted. This gets me familiar with the code without changing its behavior. This helps me understand the code. Doing this might seem too boring to do with a pair programmer, but the alternative would be to do code reviews, which are even more boring. If automatically reverse-engineering class diagrams and other UML diagrams is possible, I would do that.

If I have a bug to fix, I recommend writing an acceptance test or unit test that shows the bug's effect. Then track down where the bug is - what defects in the code cause the bug. I know from experience that it is really hard to get into the habit of writing the test before doing the debugging; I do think this test-first habit is a valuable one.

When you've found the code-defect, it can help a lot to do a little refactoring to make fixing the bug easier. Run the few tests that you now have and do manual testing to make sure that the refactoring hasn't changed any behaviors or broken anything. Then fix the bug, and run the tests to confirm that the bug is fixed. Then maybe do a little more refactoring in that area of the code and re-confirm that no behavior has changed from that refactoring.

At the beginning of this bug-fix/refactoring process, you will have to move very slowly - manual testing will take a lot of time. Trying to do too much at once will only slow you down when you find you've broken something and need to back-track. Don't forget to use source-code-control - check in frequently.

Resist the urge to refactor working code until at least the worst known bugs have been fixed.

For more on refactoring, check out http://www.refactoring.com/.

Friday, March 27, 2015

Strengths and Management

(Originally posted 2003.Jun.15 Sun; links may have expired.)

I just read Now Discover Your Strengths by Buckingham and Clifton, and I'm currently reading First, Break All the Rules by Buckingham and Coffman. The books are based on extensive data collection and analysis by Gallup.

The point of First, Break All the Rules is that great managers treat each of their employees as unique individuals, encouraging their strengths, and working-around (rather than fixing) their weaknesses. The "rules" to be broken are "treat everyone the same" and "concentrate on fixing flaws".

Gallup has evidence that good managers are the primary reason for employee job-satisfaction, and poor managers are a primary reason for employees leaving a company. Managers are also a primary reason for productivity and profits (or losses) at the worker level. Of the Twelve Questions that measure employee satisfaction, the top five are directly related to how managers work with their employees.
  1. Do I know what is expected of me at work?
  2. Do I have the materials and equipment I need to do my work right?
  3. At work, do I have the opportunity to do what I do best every day?
  4. In the last seven days, have I received recognition or praise for doing good work?
  5. Does my supervisor, or someone at work, seem to care about me as a person?
Now Discover Your Strengths identifies thirty-four "themes" or basic strengths (which I'll call "StrengthFinder types") that people have in varying degrees, and provides an on-line test to find your top five strengths. It says that the strength that managers must have to be good managers is "Individualization," which is the natural ability to observe and identify people's strengths, style, motivation, and other unique qualities. Ironically, in a book about 34 generalized "types", it says that people with the talent for "Individualization" are impatient with generalizations and "types."

Unlike MBTI, the StrengthsFinder types are not pairs of opposites. Both type systems can help people who are not strong in Individualization to learn to see people as different from each other. There's probably some statistically-relevant relationships between MBTI types and StrengthsFinder types, but I'm not aware of any documentation on that yet.

This link explains the 34 types and give advice to students with those strengths

Thursday, March 26, 2015

Experience-based list of traps for those new to XP

(Originally posted 2003.May.04 Sun; links may have expired.)

Alex Chaffee has started a wiki page http://c2.com/cgi/wiki?XpPitfalls, making a list of pitfalls often encountered by teams new to XP. Erik Hanson made "major additions".

For example: "Problem: Refactoring is hard. Really hard. Solution: Practice, practice, practice."

This item is probably meant more for entertainment: "Problem: OAOO can lead to a nest of small objects. Writing new code or debugging you end up getting lost in a Smalltalky maze of pointers. Solution: Stop the madness. Trust your design instincts. Solution: Don't hire former Smalltalk programmers :-) "

Wyatt Sutherland reports a XP+Scrum success story for Blue Cross Blue Shield of Tennessee, on the XP Mailing List. A software project that supports BCBSoT's customer base of millions, managed via index cards and agile techniques.

Wednesday, March 25, 2015

Don't Impose Improved Practices; The Team Must Improve Their Own Practices

(Originally posted 2003.Jun.14 Sat; links may have expired.)

Reading the book Lean Development by Mary and Tom Poppendieck has reminded me that successful group-improvement comes from self-directed (self-improving) groups -- and that the principle of self-improvement is more important than the specific practices.

They have an example of a software team within a large company that decided to improve their own processes. That improvement was noticed, and a committee formed to copy their improved practices to other teams in the company -- which never worked as well for those other teams as it did for the first team.

They also say the same thing happened (is happening) as NUMMI - the auto plant owned by GM and run by Toyota. Originally it was GM's worst plant, but within a year of Toyota taking over its management, it became one of GM's most productive plants — low absenteeism, high quality (almost as high as Toyota's plants) — with substantially the same people. GM tried to copy the practices to other plants, never with much success. The principle of continuous, self-directed-improvement is what makes NUMMI different, and which GM doesn't seem able to copy.

Tuesday, March 24, 2015

Rewards, Teamwork, and Ropes Courses

(Originally posted 2003.May.02 Fri; links may have expired.)

Esther Derby, on her blog, mentions a few reward-structures that work against teamwork. She writes:
Many companies throw up organizational barriers to collaboration:
  • reward structures that focus on individual achievement
  • recognition of individuals for group efforts
  • disbanding groups that do work well together (the "seeding" principle)
  • promotion of individual stars
  • And most "team building" misses the mark. Ropes courses, pep talks, and inspirational posters are a crock.
    Once upon a time, I and my coworkers were sent to a Ropes Course once. One thing being emphasized was the fact that old people, when asked what they would have done more of, never said they'd spend more time at the office. (Those who sent us on this course probably didn't know about this lesson ahead of time. Or maybe they did.) We also learned about providing "back up" to each other (support), not using joking put-downs on each other, and were given some time to think about our individual priorities.

    Many of the employees were energized and tried to improve things at the office when they came back from the Ropes Course, but became frustrated and bitter because, while many of the employees who came back did change (more teamwork and respect among coworkers), the managers who came back from the course didn't change: they continued a long history of backstabbing and turf-building.

    Some employees quit to get better jobs and spend more time with their families. The company's president, who wanted this course to help turn the company into a focused team, did not achieve her goal: the managers under her did not change their behavior, even though the employees did change, for a short while, until they realized that the managers were not providing "back up" to their workers. They probably could have used Retrospectives, Charters, and all sorts of good things, but as long as the managers were rewarded for back-stabbing and turf-building, the situation would not have improved.

    One of my priorities was/is "fun". (The IXP value of "Enjoyment".) The IXP mailing list has been discussing this, including other names like "Satisfaction". The most fun I have had in work comes from working in teams of respectful coworkers, with collaborative design (and since I've discovered XP, pair programming).

    Pairing can be tiring, and with the some people it can be VERY tiring, but the fun comes from getting work done quickly, with someone to provide "back up" (catching my mistakes), and with me providing "back up" to the pair when he's keyboarding. Getting code to pass the latest test (or sometimes, getting the test to fail) is a shared triumph, repeated many times during the day. Getting our changes checked into source code control several times a day is also a shared triumph, particularly when you can announce that this check marks the completion of a story/feature.

    Even formal code reviews can be a little bit fun, if the people in the review respect each other, and have a sense of humor about the mistakes we can all make.

    Check out more about Pair Programming at http://www.pairprogramming.com

    Monday, March 23, 2015


    (Originally posted 2003.Jun.12 Thu; links may have expired.)

    Ron Jeffries has written on his discomfort with the Prime Directive of Project Retrosepectives here: http://www.xprogramming.com/xpmag/jatPrimeThis.htm.

    The Prime Directive of Project Retrospectives is "Regardless of what we discover, we understand and truly believe that everyone did the best job they could, given what they knew at the time, their skills and abilities, the resources available, and the situation at hand."

    Jeffries writes:

    We wound up, not in a "wonderful place", but in an OK place.... I don't get it. I don't get how accepting badness as "we did our best" leads to learning. There must be something after the Prime Directive. What's the Second Directive?

    The prime directive isn't setting the stage to accept that nothing could be improved, it is to set the stage to look for improvements without getting stuck blaming people for what they or others think they did wrong. Once you're in fault-finding mode, everyone will try to defend themselves, trying to re-direct fault-finding onto others, and will stop sharing experiences that the group could be learning from.

    We can apply Dale Emery's excellent essay "Creating Empathy" to the example posed in Ron's essay: Ron wrote some code non-test-first. It took longer than expected to debug, so that it could have been done test-first in the same amount of time. Why would he do this? He says that he had a deadline; people were asking "where the heck" is this code? He didn't know how to throw characters at the GUI code in his tests... and so on. His deeper reasons were to not let down those people; to deliver as soon as possible. The prime directive helps us assume the intentions were good, even though the results were not.

    Perhaps a facilitator in a retrospective would remind the team to not worry too much why Ron didn't write that code test-first, and help them realize what the consequences were, and let the team identify things to do differently in the future.

    I know of a "kick-butt" manager whose every question seems like an attack. When he says "Why did you do this?" It doesn't come out as "I'd sincerely like to understand reasons why someone would do this," it comes out as "Why would anyone be so stupid to do this?" This attitude is very demoralizing to his employees. He may not realize that when he focuses a barrage of these kinds of questions onto his employees, they become demoralized and ineffective for hours or days afterwards. This is not a way to keep the project work going smoothly. If the job market was better, many of those employees would quit.

    If a retrospective was attempted with that "kick-butt" manager involved, I hope the facilitator would be strong and experienced enough to kick that manager out of the retrospective - no learning is going to happen otherwise.

    The XP mailing list had many good responses to Ron's essay. To quote just one, George Dinwiddie wrote
    The reason they didn't do a better job is perhaps not in the list of items you're considering (knowledge, skills and abilities, resources and situation) but certainly there was something that led them to do what they did. Of course, you can attribute it to their laziness or orneriness, but in my experience this tends to lead to non-productive confrontations.

    It can be much more productive to search for what can be done better in the future rather than searching for the reason(s) that one couldn't do better in the past. Just because, for reasons that are not clear, this was the best I could do at the time, is no excuse to quit striving to do better in the future. And that striving can be done without identifying
    character flaws or laying blame.
    Ron asked what is the "second directive?" Several people identified that as talking about the future, identifying what things you would do in the future that would be different from what you did in the past.

    Sunday, March 22, 2015

    New Skills for Developers

    (Originally posted 2003.May.03 Sat; links may have expired.)

    Extreme Programming requires new skills for both Managers and Developers; in today's entry, I write about some of the new skills required for Developers.

    In typical "traditional" development (which is often waterfall / code-n-fix), a developer needs the skill to design from scratch. To some extant, this skill is taught in school and in books. Some developers master it better than others. Unfortunately, it doesn't help too much with maintenance, and about 90% of developer work is maintenance. Debugging is a very valuable skill for this kind of work, but only learned from experience. Testing is done after development, if it is done at all, (typically just manual testing is done) and very few projects keep automated tests running in maintenance mode.

    Traditional Development State of Code
    Design from scratch Good
    "Patch" for new feature Less Good
    "Patch" for bug fix Even Less Good
    Repeat Patching for years Eventually Declared Unmaintainable

    In Extreme Programming, a developer doesn't have to make a good design from scratch. During the Test-Code-Refactor cycle, the developer's first stab at code doesn't have to have a really good design, it just has to pass the test. Then the developer "puts on his refactoring hat", and improves the code to good design. The project is in maintenance mode after the first week, and the discipline of refactoring keeps the design good throughout its life. Keeping the tests up-to-date is also required. It also turns out that pair programming and this test-first approach mostly removes the need for debugging.

    Extreme Programming State of Code
    Write a Test No Code Yet
    Code To Pass Test Minimal Code, Possibly Poor Design
    Refactoring Good Design
    Write another Test No Changes to Code Yet
    Code To Pass Test "Patched" Code, Possibly Poor Design
    Refactoring Good Design

    If you compare the two methods, you see that XP requires testing skills, which are not emphasized in traditional methods, and it requires refactoring skills, which isn't (or wasn't until recently) taught in school, and is only described in very few, recently-published books.

    Experienced developers, particularly those who have spent time in maintenance, know that patching leads to unmaintainable code, but many of them haven't learned refactoring, the "cure" for patching. Younger developers, particularly those that got started in the dot-com boom, haven't even had time in maintenance, and of course they are still learning how to do design from scratch.

    I've spent the last several days teaching a developer how to write code test-first. Writing the test for a nonexistent class or method is fairly easy, and writing the code to pass that test is fairly easy. It can even have a good design, since it isn't being patched (yet).

    Writing a test to drive modifying the behavior of an existing class can be more tricky. In some cases (equality functions), we had to regard a "passing" test as a failure, in order to follow the XP rule of never modifying the code unless there is a failing test.

    One of my exercises for teaching is to have the student write a "prime factors" function, where I supply each test, and the student writes the simplest code to pass the test. The hard part in that lesson is learning to refactor "if" statements away to keep the design good. Remember the XP rule for good design: "no duplicate logic".

    Sometimes, at the end of the day, getting the tests passing is such hard work, that we get tired and don't do the refactoring. We check in working code, and then the next day arrives and we forget to do the refactoring, moving onto new tasks instead. Pair programming is tiring enough that the experts don't recommend pairing more than six hours a day. The XP practice of "sustainable pace" is meant to keep you from programming while tired, when your judgment is poor.

    An XP team might want to have a practice where developers coming in the next morning review the code that was checked in the previous day, looking for opportunities for refactoring, and doing it first thing, while the energy is available to do it. The next-day perspective could help provide the distance needed to recognize that yesterday's wonderful code isn't as wonderful as we thought.

    Refactoring is similar to rewriting, but more disciplined: you can only make small, behavior-preserving changes. This discipline is a new skill.

    Brooks said "expect to throw one away", Clarifying later that he meant small modules not entire systems, because throwing away entire systems leads to waterfall.

    Cockburn said says it is easier to modify a working system than to write a new system from scratch. I say that it is easier to modify a working system than a non-working system.

    Ed Yourdon said that any piece of code needs to be rewritten three or four times to be constituted as an elegant, professional piece of work.

    Mary and Tom Poppendieck said that for prose, we take this [rewriting] for granted, why not for code?

    Saturday, March 21, 2015

    Reward Structures

    Originally posted 2003.May.01 Thu; links may have expired.

    Is your company's reward structure counter-productive?

    A recent issue of Inc. magazine had a article where a sales manager said, based on his experiences, that he'll never recommend using commissions to reward salesmen, because it damages the company that does it; specifically it works against team-work and causes the company owners to not trust their salesmen. Check it out: http://www.inc.com/magazine/20030501/25416.html.

    Some managers complain that pair-programming prevents them from seeing clearly which programmers are accomplishing more than others, to be able to reward their individual accomplishments. Emphasizing individuals over teams rewards territoriality, rewards competition more than cooperation. Cooperation is more productive than competition at getting products developed and shipped. Do you really want to discourage cooperation?

    When programmers are working solo, how do you know they are actually doing something useful? Could it be that someone is working alone because what he's doing is NOT useful to anyone else?

    Software is a team effort. Even when its done by a bunch of soloists, they are still using each other's work by building on each other's libraries and code. Could it be that the guy who demos a flashy application has only done some of the GUI, and the functional parts of the application were done by his coworkers? Are you going to reward the flashy demo guy because he can show you something more visible than the other programmers who wrote the functionality?

    Robert Martin wrote on the IXP mailing list:

    In one case we transitioned a team that had technology project leads. There was a lead for the database, a lead for the middleware, and a lead for the UI. (I'm not kidding!) When we transitioned them to XP these folks rapidly realized that the practices of pairing and collective ownership were diluting their power base. Two of them made abortive attempts to seize power back, and wound either resigning or being fired. The team was much happier once they were gone.

    Are you rewarding last-minute, late-night heroics? When you do XP well, you have measured progress every iteration, and you can predict how much scope can be accomplished a long time before the end of the project. Check out what happens when rewarding heroism meets XP...

    In another case we had a cowboy. This fellow used to rescue the project at the eleventh hour by pulling all nighters. His manager learned to depend upon this behavior and just "knew" that the cowboy would save him at the last minute. As we transitioned to XP the cowboy actively sabotaged the planning meetings and the iterations. He did not want the team to make progress. We went through three iterations of *zero* velocity. (Count them, zero.) And then the manager dissolved the team and reformed it months later in a different way. (I'll leave it to you to predict how it was restructured.) -- Robert Martin.

    Do you think that even before XP came along, that this cowboy might have been doing some sabotage to make sure his heroism would always come into play? I think XP just made it more visible.

    Ken Auer, who runs RoleModel Software, a consulting company that does XP, writes:
    I've always had a reward structure here that was based on the concept "share the risk share the reward" while still acknowledging that "socialism doesn't work... Some people do provide more value than others". At first, I could only come up with a "how much money did we make due to your billable hours" measures, with a few other criteria to make sure it wasn't abused. Worked fine while everyone was billable most of the time. When it wasn't, it had people lobbying for billable time against each other e.g. "I think I should be on this project (instead of Joe) because...".
    Finally, I came up with a formula for individual value to the company that could be identified, peer-reviewed, and steps taken to move the number up. That number was used to figure out shares in the pool of money. It wasn't perfect, but it completely got rid of all of the bad behavior (fighting against each other) and produced good behavior, i.e. I got help figuring out how to divvy up resources for the best of our company, the client, and the individuals.

    Amy Shwab reminds us:
    In the January Harvard Business Review (all about motivation) there was a wonderful piece about the longer term effects of performance metrics and tying results to those metrics. The company, indeed, got what they were rewarding but it did not translate into what they wanted or needed. It fragmented the organization and served the customers very poorly. The most important things are often the most difficult to measure -- like 'fun'.

    Developing intellectual property like software is always a learning experience. External rewards can hinder learning. Kohn said "Rewards are most damaging to interest when the task is already intrinsically motivating." Check out "Punished by Rewards? A conversation with Alfie Kohn".

    Rachel Davies and John Nolan will hold a workshop on this topic at the Agile Development Conference

    Thanks go the people mentioned above, and Brad Appleton and Steve Berczuk on the XP mailing list.

    Friday, March 20, 2015

    Stop the Insanity

    (Originally posted 2003.Jun.06 Fri; links may have expired.)

    Some people very experienced in software development looked at my Extract Method refactoring example and wondered what was so unusual about doing that. One asked "Isn't this just obvious, and what any sane person would do?" Apparently not.

    There are tons of "insane", unrefactored code in companies everywhere. Even in my own current XP-ish project, there are developers who sometimes write (or more likely copy-paste-modify) long methods with duplicate code, and they have more than five years of experience developing software.

    For many people, it seems like their tolerance level for hard-to-read, hard-to-maintain code is high, possibly because their high intelligence lets them sort things out in their minds - at least when they're familiar with the code. If only they would sort it out in the code itself.

    I imagine those "tolerant" people get irritated by us "intolerant" Extreme Programmers, who want the code to be so easy to read and enhance that it looks like it was easy to write. There's an old joke in programmers circles - "If it was hard to write, it should be hard to read" - that is unfortunately too true.

    If you really want to get insane code - long methods, lots of duplication - measure your programmer's productivity by counting lines of code. You'll get what you measure.

    Thursday, March 19, 2015

    Code Reviews versus Pair Programming

    (Originally posted 2003.Jun.09 Mon; links may have expired.)

    On the XP mailing list today, Ron Jeffries responds to a question about code reviews:
    Frankly, code reviews are so much worse than pair programming that a dose of them would make me fly to pair. Let's see if we can replicate my experience.

    Here's one path through a network of a million decision points:

    To do code reviews, everyone has to read the code beforehand, unless you're
    doing a walkthrough, see below. I'd ask everyone to come together
    physically to the review. Then I'd ask them to report truthfully how much
    time they spent reviewing the code. Early on, I would report truthfully
    that I had spent zero or very little time, in hopes of getting others to
    admit the truth. When they admit the truth, I'd dismiss the meeting and
    reschedule it.

    Dismiss and reschedule is the appropriate response to lack of preparation, according to Freedman and Wienberg.
    Then, after a while, the only alternative is a walkthrough, since no one is
    preparing effectively for the review. So we do walkthroughs for a while.
    They are intensely boring, and few people stay involved. Note in your mind
    the people who are present but not involved. At the end of the session,
    say, holding your hand up, "Who else had a real problem staying engaged
    with this walkthrough?" If there's honesty in the room, hands will go up.
    Prompting may be necessary. Then: "Any ideas?"

    Surely someone will think of "doing this in smaller groups or one on one".
    Try it. Ask the team whether "we should empower the one-on-one folks to
    change the code, and under what circumstances." Don't mention that this is
    pair programming.

    Try an experiment. You're "interested in collaborative programming".
    Interested parties should come to the room to help. On the screen, start
    writing a program. Ask for help with it, get the room to pair with you. Get
    stuck (no need to fake this if you are me). Someone will start telling you
    what to do. Don't get it (no need to fake this either). Get them to come up
    and do it ... grabbing the chair that is accidentally beside you, while you
    move over.

    Note that reviews often find things. Observe how many of them are resisted
    by the original programmer, or are "too much trouble to fix now".

    Build a few BVCs
    [Big Visible Charts] relating to time spent prepping, in the meetings, number of useful suggestions (by person if you can do it without problems), number of changes made in response to suggestions, ...

    Code reviews are intensely painful, in my experience, and we were trained
    by Freedman himself. There will be no need to set them up to be perceived
    that way, though it will take honesty among the group to express it. After
    doing enough code reviews, which take way more than half the groups' time
    by the way, a team who has heard of pair programming should be begging to
    pair. About all you have to do is make sure that no one treats the review
    session as nap time, and that you are early in recognizing the people who think it's a waste of time. Because they're right.

    I have done reviews on a project that was building software for a medical device. We prepared for the code review (and also reviewed unit tests). We found issues (like not enough unit tests for specific parts of the code). The programmer corrected issues after the review, and had a second review (a walkthrough with the coder and one reviewer) to confirm that the issues were corrected. We signed off pieces of paper to have the paper trail that the FDA would expect. It wasn't the most exciting work, but we were one of the best teams, people-wise, that I've ever been privileged to work with. Truly respecting each other and working together very well.

    Coding and reviewing this way was very slow - for eight person-hours of programming, expect at least a half an hour of preparation (coder and three reviewers - so two person-hours of preparation), one hour of review (times four people - so four person-hours of review), one person-hour correcting issues, and another half-hour of walkthrough (times two people - one person-hour). So for 8 person-hours of programming, there was another 8 person-hours of overhead: reviews, correcting, and walkthrough.

    As is often said, "basic XP" isn't intended for safety-critical applications. But I could see the following working in that environment: pair-programming and test-driven development for four hours (eight person-hours), and a one-hour walk-through by two people who didn't do the pair-programming (two person-hours). I would expect very few issues to be found in the walkthrough that hadn't already been found (or avoided) and corrected in the pair programming, and therefore there would be much less need for correcting issues. So 8 person-hours of (pair) programming would have an overhead of 2 person-hours of walkthroughs.

    While I was working on the medical device project, we didn't have acceptance testers nor automated acceptance tests. If we had been doing XP, we would have had better testing -- better unit tests, and real acceptance tests. I didn't stay on that project through completion - when our (well-liked but not very political) manager was removed from the project, and our team leader quit, almost everyone else on the team quit too. I was one of the last ones to leave, accepting an offer from Apple Computer, Inc. and moving to California. I heard the the project eventually finished, with half the man-power and twice the planned schedule. Very likely the acceptance testing was done at the end of that project.

    Wednesday, March 18, 2015

    Selling the Invisible

    Originally posted 2003.May.31 Sat; links may have expired.

    I found the book Selling the Invisible by Harry Beckwith to be a very interesting and inspiring read. Those of us working in software development deal with the invisible all the time. While some of us may write shrink-wrap software, and can point to the box as proof that our product exists, even that product can be mostly invisible: anti-virus software, for example, should be invisible until it tells you it found and eliminated a virus (and maybe even then).

    Selling the Invisible is about marketing and a lot more: some of the best marketing comes from having a good product or service, and happy clients or buyers. It comes from maintaining good relationships. I recommend this book. It gave me an idea for my company that could be profitable, and I've only read the book once through so far.

    "Management" is mostly invisible - the worse management is, the more visible it is. The best managers (I'm told) often seem like they are neither busy nor doing "any work," because they've done a lot of work to keep potential problems from turning into actual problems. Managers could do with reading this book, considering their bosses, peers, customers, suppliers, and direct reports as "customers", all of whom to maintain good relationships with.

    Software development methods can be invisible too. Particularly agile methods that don't produce many artifacts other than the software itself. The recommended "big visible charts" (in particular the scrum burn-down chart adapted for XP) help make the invisible stuff visible, and help keep a good relationship with the customer.

    In fact, I thought Kent Beck wrote a message in the IXP mailing list that sounded like the "positioning statement" exercise of Selling the Invisible, but now I can't find it. I'll attempt to do the exercise here, using me as an XP coach/project leader. The positioning statement answers the questions of Who, what, for whom, what need, against whom, what's different, and what benefit.

    Keith Ray is a coach and project leader for small to medium-sized projects (up to 10 programmers) which need to ship good quality software, quickly and/or frequently, whether the desired features are vaguely defined, changing, or very specific. Unlike many in this industry, Keith Ray has been developing software for over 15 years, and has participated in many successful projects in several product areas, creating and shipping good software at a predictable pace. Keith Ray uses good, context-specific, practices to enable you to get the important features you want as soon as possible.

    You notice I don't say "XP" or "agile" in this statement - people are looking for solutions to their problems, and don't care about the techniques I use (I hope).  What's your positioning statement?