Reuse and Modularity

(Originally posted 2003.Apr.02 Wed; links may have expired.)


TWELVE|71 wrote a bit about reuse and modularity. "Modularity is the idea of having a tire that can be swapped on and off without affecting the car. Reuse is the idea that we can take a tire from an old car and use it without thought on a new car."

Off the top of my head, I think software reuse projects often fail because (1) the needs of all the client projects are not considered when making the 'reusable' module, (2) the module is not documented well enough, (3) people on the client projects are motivated to not reuse modules, perhaps because they are rewarded for hours worked or lines of code written (or some other counter-productive measure), or (4) they are under time pressure, and don't have time to rewrite their app to use a module that they seem to be able to do without.

Take the reverse approach. Instead of creating a module and telling projects to use it, create a small "Extreme Reuse" team: one to four people who join projects to (1) help them get things done (2) look for code that could be extracted for use by other teams (3) refactor project code to create and reuse shared parts. The Extreme Reuse team needs to join several projects before extracting code from any of them, in order to know all of their needs. How can a few people join a team and be immediately helpful and productive? Pair programming.

Bryan Dollery has described this and other aspects of how to reuse parts here.

I don't know anyone trying a "Reuse Team"; please let me know if you are. My small team works on several projects. We have code common to these projects in some separate directories, and it is mostly unit-tested. (Yes, we're not 100% pure XP.) We make minor changes to this common code, and avoid breaking the clients by continuing to pass the unit tests, and by building and testing the client projects.

Keith Ray is developing new applications for iOS® and Macintosh®.. Go to Sizeography and Upstart Technology to join our mailing lists and see more about our products.

Speaking of Objects as cooperating, independent, agents...

(Originally posted 2003.Apr.01 Tue; links may have expired.)


Chris Uppal, on the Dolphin Smalltalk newsgroup, writes about his "Great Leap Forward from Java to Dolphin Smalltalk." He describes the big difference...

If you are like me, then you are currently thinking of a big difference between Smalltalk and Java being that Java stores code in files, whereas Smalltalk keeps it in the image. That's sort of true, and I'll get back to it, but, for a minute, just forget about code, it's not important (really!). What matters is objects. 

The image is the place where the objects live. Technically, the image is a garbage-collected heap that can be saved to file, and later restored, thus saving and resuming the state of a computation. Technically that's true, but it isn't at all a helpful way to think about it. A more organic metaphor works much better. I think of the image as a deep, murky, pond where objects move around in the depths like fish. It's an important part of the metaphor that the objects are independent of me. Even if I designed and wrote the classes, once an object has been created it has an independent existence. I can talk to it, I can ask it to perform operations, but it is separate from me. In a sense it is "my" object, but it is only "mine" in the same way that a pet, or a rose bush, or a table, could be "mine".

The image is where the objects live. Not the code, the objects. We'll get back to the code in due course, but not yet. The Smalltalk environment is just a place where you can talk to objects; no more, no less. Oh, sure its got class browsers, debuggers, editors, etc, but that's all tinsel. What matters is that it is a place where you can interact with the objects.

I'll get back to the "tinsel" later too, but for now, I want to talk about the one part of the environment that isn't just a productivity aid: the workspaces. Workspaces are the medium through which you talk to objects. You can describe workspaces as "containing snippets of code" which you execute, but IMO that's exactly the wrong way to think of it. A better picture (slightly tongue-in-cheek) is as a kind of singles bar, where you can meet objects, talk to them, check them out, get to know them. Each workspace has a number of objects that are (temporarily) living there; they are the values of the variables in the workspace. In most cases they'll die when you close the workspace, but until you do they'll survive and you can talk to them. I keep some workspaces hanging around for days if they contain objects that are important for what I'm doing. The way you "talk" is by sending messages written in the Smalltalk programming language, but that's almost incidental. The important thing is that you are communicating with them using an interactive text-based medium, like using an IRC [chat] channel. [...]

Another way of interacting with objects is to use the Inspector(s). They give you a much more nuts-and-bolts, low-level, view of the object -- a more intimate view, if you like. I, personally, don't think that the Smalltalk world has yet woken up to what inspectors could be, but the current implementations (like "flipper" in Dolphin) do at least allow you to see inside the objects.

I wish C++ had inspectors... but C++ throws away a lot of information when you compile the code. In the crappy debuggers that C++ programmers have to live with, we often can't view the run-time contents of an object properly and easily. The VC++ debugger for example, should be able to know that the object pointed to by a base-class pointer is actually an instance of a derived class, but it doesn't display the member variables that belong to the derived class -- just the member variables of the base class.

An image will contain many objects, some long lived (living, perhaps, for decades), most very short lived indeed. Some will be simple or trivial, like Strings and Points. Others will have complicated internal structures, and/or complicated behaviour. But they are all objects, and they all live in the image, and you talk to them in workspaces.

A Squeak Smalltalk image contains objects that have been "alive" since 1984 or earlier, because Squeak was derived from a Xerox/Apple implementation of Smalltalk-80. The objects in the image "sleep" when saved to disk, and awaken when restored from disk. This means that some objects in Squeak have been alive longer than some programmers have been.

Classes are one particularly interesting kind of object. Remember I'm still not talking about code (that comes later), I'm talking about the objects called classes. Just like any other objects, you can invite them to join you in a workspace:

[I modified the code slightly here...]

aclass := String.  

and then you can use the magical Smalltalk object-oriented IRC to talk to them:

aclass name. "--> #String" aclass allSubclasses size. "--> 3"
and so on. So classes are objects, and they live in the image.
[...]

Code is how we tell objects how to behave. It's text in the Smalltalk programming language. We're programmers so we care about code; when we wrote the tools for looking at objects, we naturally designed the tools so that we could also see the associated source code. For instance our special tool for looking at classes (the class hierarchy browser) allows us to see the source of the methods, to change the source and recompile, etc. That's natural for us as programmers. If we weren't programmers then we'd want different tools, and we'd be interested in talking to different objects. Such systems, built for non-programmers, are called "applications", but they are still just Smalltalk -- tools for talking to objects that live in an image. (A big difference is that the "image" of an application is typically not persistent, unlike the image of the IDE).

Back to code. Granted that the most important thing is the objects and how they behave, we still do care about the code. We want to organise it, back it up, put it under source code control, etc. A class is an object that lives in the image, but the source code for that class is something else. For all sorts of reasons, we want to keep that outside the image. The way that Dolphin organises source-code is via Packages. A package is a collection of the source code for classes and methods (and a few other things too, which don't matter here) that is kept outside the image in one or more files. You can load the package into the image, which will create an actual Package object, and class objects corresponding to the source-code. Or you can "uninstall" the package, which really means killing the Package object and the Class objects.

So a package is just a way of collecting related source-code together. [...] The package mechanism is relatively simple; it could be improved, but I find it adequate for my needs. Package files are text files, you can edit them with vi, or notepad, or whatever. Occasionally I do that if I want to make particularly sweeping changes to the source. Of course, if you do that then you have to install the changed version into the image before it'll do anything useful.

Notice how very different this way of thinking is from the way that even the best Java IDEs encourage you to think. When I started out in Smalltalk I was thinking of the IDE as if it was a Java IDE. I though of it as a tool that allowed me to write code, and had features to allow me to browse and test the code. After a year or so I realised that I'd turned the picture upside down completely, and in the process had revised my conception of what Object-Oriented programming is all about. As a Java (or C++) programmer I had pretty much thought my .java (and .cpp) files were the classes, and I thought that creating classes was what programming was about. I now think of the objects as being the important thing, and the classes as very secondary, hardly more than an implementation detail.

I feel that that has made me a better programmer. Of course it's not possible to know for sure, but if it has, then it all comes down to Smalltalk's workspaces...

The original message can be found here

Chaos, Order, and Software Development

(Originally posted 2003.Mar.31 Mon; links may have expired.)

Kevin Kelly published Out Of Control: The New Biology of Machines, Social Systems, and the Economic World in 1994.

Jim Highsmith (James A. Highsmith III), wrote Adaptive Software Development: A Collaborative Approach to Managing Complex Systems in published in 2000, before he read Extreme Programming Explained: Embrace Change by Kent Beck, published in 1999.

The book on Scrum was published in 2001, and Highsmith's Agile Software Development Ecosystems was published in 2002.

What do these books have in common? Order (or the semblance of purpose) emerging from independent agents in situation that one might expected to be purely chaotic.

Kevin Kelly's book covers the most ground, of course, from bee hives and ant colonies, boot-strapping ecosystems, competing/cooperating agents within our brains, distributed control within robots, evolution and genetics, genetic algorithms, and so on. The theme is that order can arise "by itself" (it emerges, rather being designed up-front.)

The agile software authors are saying that a good software product can arise with a minimum of planning up-front. However, creating this order is not 'random', it arises from the constant thinking and re-thinking of the people involved through the life of the project.

Some people have objected to a lack of up-front planning or designing, making the analogy to a "hill-climbing" algorithm that gets stuck on a local maxima (getting stuck on a small hill, when the goal is a larger hill, a valley away). The difference of course, is that the hill-climbing algorithm is stupid, whereas many people are smart. People can see the big picture, and can do a little planning of refactoring to get from the current design to the desired design, even though the refactorings may temporarily go through a poor design along the way (but with all tests still passing!)

In a recent web-search, I came upon a paper written by a member of a group researching independent software agents. The paper was about how Extreme Programming is helping them write software successfully, allowing new programmers to become productive members of the team quickly, and how the process allows them to increase software re-use. That paper is here: Using Extreme Programming for Knowledge Transfer. Using XP to continue research like those described by Kevin Kelly... I don't have my copy of XPExplained or Adaptive Software Development handy, but I would expect Kelly's book, or books he references, to be referenced in the bibliography of Kent's and Highsmith's books. I enjoy feedback loops like that.

Daily Standup Meetings

(Originally posted 2003.Mar.29 Sat; links may have expired.)

Laurent Bossavit writes: I regularly hear from people who have experimented with daily meetings such as Scrum ("Daily Scrum") or XP "Stand-up meeting") recommend. With no exceptions, everyone says that such meetings are incredibly effective in getting issues solved quickly, gathering momentum within the team, etc. My experience is about the same, although I prefer brief, informal "huddles" to formal meetings.

With a three person team, I tried daily standup meetings, but didn't find them that useful, since we were co-located anyway. An attempt at doing combined standup meetings with two unrelated teams was even less useful. It seems that having a common goal and related work makes daily meetings productive. That's probably why weekly "status meetings" of unrelated teams reporting to one manager are very unproductive. Like Laurent, I prefer brief huddles whenever information needs to be shared.

Do the Hard Thing

(Originally posted 2003.Mar.28 Fri; links may have expired.)


Watching the TV show Boston Public, Principal Harper was a telling a student something like "when you have a choice, pick the hard choice. Nine times out of ten, it will be the right one." That rings true.

In developing software, writing tests is hard, so XP does it all the time. Design is hard to get right, so XP does it all the time. Communicating requirements is hard, so XP does it all the time (by talking to the person playing the role of Customer.) By doing these things all the time, we make them easy.

In corporations, telling someone the truth can be hard; standing up against peer and management pressure to avoid reality can be hard. Doing the right thing, tactfully, is hard, but more rewarding than living in the hell caused by ignoring reality.

Please Ignore the Elephant in Your Living Room

(Originally posted 2003.Mar.27 Thu; links may have expired.)

The subject of legacy code appears on the Test Driven Development mailing list periodically. My advice is to use test driven development for writing new code or bug fixes, and leave the rest of the legacy code alone, writing tests for old code only when you need the support for refactoring.


David Brady, on that mailing list, writes: "being behind schedule with 250,000 lines of monolithic, untested, difficult-to-test code is the PERFECT time to start learning how to test. You just have to ease into it one step at a time and be prepared for a long journey." My emphasis on the "one step at a time."

Technical Reviews, More on Test Driven Design

(Originally posted 2003.Mar.26 Wed; links may have expired.)


Scott's essay on TDD is up here and Ron Jeffries's critique is there. I'll have more to say about it after reading it a couple of times.

That recent issue of STQE Magazine also has a great short essay by Jerry Weinberg on technical reviews being a learning accelerator. One thing I want to point out is that junior programmers should be reviewing the work of master programmers, not necessarily to find errors, but to learn from the master - and of course, master programmers can make mistakes too, which are often visible to junior programmers as well as other master programmers. If the master programmer is humble, he/she can learn from a junior programmer, too.

I've been reading Weinberg and Freedman's book on the subject of technical reviews Handbook of Walkthroughs, Inspections and Technical Reviews (which was written in FAQ style - question/answer), and was a bit surprised by their recommendation that when people are being trained on how to technical (code) reviews, they should have some practice at conducting a review in the presence of hidden agendas.

Some examples of hidden agendas in code reviews: person A wants to impress person B. Person B wants to make person C look bad. Person C needs to go to the restroom, but doesn't want to say so. Person D is distracted by illness of his/her spouse.

Only Weinberg would write about hidden agendas in code reviews - too many writers and books on software development practices seem to assume that people act like machines.

On Weinberg's SHAPE forum, Charlie Adams wrote: "When people are getting tense about their software being reviewed, use Jerry's phrase, 'Yes, I trust your honesty, but I don't trust your infallibility. I don't trust anyone's infallibility.' (QSM 4: page 220) In my experience this has always calmed the atmosphere and allowed us to examine the code rather than the developer."

While I have done code reviews, both informal and formal, I prefer pair programming. It combines reviews with collaborative design, testing, and coding. Rather than go into all the reasons why pair programming is good, I'll point you to www.pairprogramming.com and Pair Programming Illuminated.

Test Driven Development is about Designing, Not Testing

(Originally posted 2003.Mar.25 Tue; links may have expired.)


In a recent issue of STQE Magazine, Joel Spolsky wrote that Test Driven Development (TDD) doesn't substitute for "normal" testing. It seems like he doesn't understand that test driven development is about low-level design, not testing. Programmer Tests are a happy (and intentional) side-effect of the design and refactoring process. It is to avoid this misunderstanding that I prefer to call TDD "Test Driven Design".

Ron Jeffries and Scott Ambler had a little spat on the Agile Modeling Mailing List about TDD, not about whether it constitutes "design", but on how much design "up-front" it entails. Scott started it by writing here "An important observation is that both TDD and AMDD [Agile Model-Driven Development] are based on the idea that you should think through your design before you code. With TDD you do so by writing tests whereas with AMDD you do so by creating diagrams or other types of models such as Class Responsibility Collaborator (CRC) cards."

Ron replied "Does TDD suggest that you "think through your design before you code"? I see no such thing in TDD. In TDD we write ONE test, then make it work, then write another." [He's leaving out the refactoring step here, which is another area of design in TDD.]

Maybe Ron doesn't think writing each test is "thinking" or "designing", but I do. At the risk of being snide, I assert that each test represents more thinking than a lot of programmers do when they write code without tests. Perhaps Ron's extensive experience has made his designing unconscious.

When writing the test, you think about the API, the goal of the API, and how to verify the goal is met. That's design. Before you start writing the tests, you think about whether to extend an existing class (and its tests) or to start a new class and new tests. That's higher level design. After you write a test and make it pass, then you look to see if there is duplication or other design smells to be refactored away. Still more design. Perhaps Ron thinks this refactoring step is the only design step in TDD.

Check out Kent Beck's book: Test-Driven Development: By Example for an introduction to TDD. Unfortunately, only a very experienced (zen-master-level) programmer like Kent Beck can take the refactoring step of TDD (remove duplication) and derive all the other good design principles from that. So read Robert Martin's book Agile Software Development: Principles, Patterns, and Practices, which not only uses TDD extensively in its copious examples, but also documents design principles that every programmer should know.

Writer Ferrets by Richard Bach

Writer Ferrets: Chasing The Muse, by Richard Bach. Quite a delightful little book about writing, publishing and writer's block. Satisfying while reading, but less satisfying in retrospect. I'll be reading the other Ferrets books as I find them. It makes me wonder how much of the book is autobiographical... but not enough to ask the author and possibly get a disappointing answer. Does anyone have a translation of the runes before and after the title page?

Fun in the Workplace

(Originally posted 2003.Mar.22 Sat; links may have expired.)


I read some years ago in the book A Great Place to Work that a good company allows people to not "be a part of the family" -- it's ok if the workplace is "just a job" -- though I hope it's a job done well.

I don't play pool or foosball at my office; I'm not in the company softball team; and while I wanted to get together with the radio-control-car guys, I was too busy and didn't want my rc-car to get muddy. My wife is still recovering from playing in a softball game for her office a week or more ago.

My preference for "fun" at the office would be people getting together to learn about software methods, design, and so on (topics I plan to write about in this blog). Getting a group of coworkers together to ride go-carts is not my idea of fun.

It appears that some people in Germany are blaming their dot-com collapse on "fun" in the workplace. Noted in Laurent Bossavit's blog pointing to article on Mair's End The Fun.

Mair's office rules seem to have gone too far in the other direction - uniforms required, no calendars or pictures on the walls, half-hour lunches. And this is in an advertising agency?!

I fail to see how uniforms get creative work done better or faster. Doesn't anyone blame to dot-com crash on bad business plans, profit-mongering brokers, and credulous investors?

A Great Place to Work says that the core truth of what makes a company successful, is mutual trust between employees and management. Quoting the preface:

[a great place to work] requires the direct involvement of senior management who must insist that fostering an exceptional work environment is an explicit goal of the organization - one that is on a par with other explicit goals like making a profit or providing high-quality products or services. Being a great place to work cannot be a fad of the month or it will rightfully appear to employees as another version of management by manipulation.

C May Be Better Than C++ For Your Embedded Application

I have done some coaching of programmers writing C and C++ for embedded systems. The C programmers' target environment didn't have a heap, so C++ new/delete and associated smart pointers were out (and thus, most of C++'s standard libraries were out). But C was an improvement on assembly language for their productivity and unit testing.

The C++ programmers, though they called their target environments "embedded," really had lots of memory and full-fledged Linux or Linux-like operating systems. So they could use C++ instead of C for programming. But my experience helping them has showed me that very few programmers know how to write code that uses C++ safely (as in exception-safe code). Often their use of C++ was a hindrance to productivity, testability, and execution-efficiency.

If I could go back in time to advise those C++ embedded teams on the choice of their programming language, I'd recommend C instead of C++. It's seems like it is harder to write untestable, inefficient code in C, compared to the nightmares I've seen in C++. I've noticed that C programmers seem more pragmatic than many C++ programmers. The C++ programmers often have rules in their heads about the "right" way to code, that they have to unlearn if they want to write testable, efficient code.

If average-skilled C++ programmers could be trusted to use C++ as a "Better C" (which is how C++ used to be marketed), I would recommend that.

I'm currently programming in Objective-C with ARC. I could be blending Objective-C with C++ (just change the source file suffix to ".mm") , to take advantage of C++'s "Better C" advances in type-checking, but so far, I'm OK with the C part of Objective-C. Unit tests have found some of the issues with C's less-powerful type-checking the C++ compiler could have caught.

ARC takes Cocoa's semi-manual memory management and automates it, so I no longer have to write retain, release, or autorelease. If I didn't have ARC, I would seriously consider using C++ smart pointers to manage Objective-C memory usage. Some would view smart-pointers plus Objective-C as a worse solution than explicitly using retain and release, but manual or semi-manual memory management is very error-prone.

Book Titles that may be More Interesting than the Books they Came From

A list of titles I found interesting, when looking at a mish-mosh of old books, some time ago.

• Labors of Love
• Proud Destiny
• Just In Case
• Natural Acts
• Education and Ecstasy
• The Cannibal Galaxy
• A Year in Upper Felicity
• The Land of Milk and Omelets
• Fly Boys
• Indian Killer
• The Formula
• Moon Deluxe
• Four Doctors
• The Outsider
• The Fourth Dimension is Death
• Dropping Your Guard
• Dark History
• The Crystal Towers of the Moon

and a title I could have used when writing about refactoring:

• From Jungle to Garden: cutting down long methods and large classes.