haruki zaemon

Continuing with the business rules examples thread, James looked at the examples of using logical to achieve a “compensating retraction” and asked me “so that’s all very well and good but what happens if I’m not asserting a fact? What a happens if instead, I’m sending an email to my broker?”

My first reaction was that this is a separate problem. The fact that I was potentially sending an email based on the SellOrder seemed like an implementation detail that I didn’t want cloduing the simple fact that, under certain conditions, I wanted to indicate my desire to sell.

After a little discussion, we came up with the following solution which, IMHO, elegantly maintains the atomicity of rules and the separation of concerns. I’ve taken some liberties with the syntax and I’ve not actually tried this in JRules but it does serve to demonstrate the concept:

rule BrokerInformedOnNewSellOrder {when {?order: SellOrder();not SellOrderActive(order == ?order);} then {sendMessage("Sell stock");assert SellOrderActive(?order);}}

As the name suggest, this simply informs the broker on any new SellOrder. The key here is that we introduce a new fact SellOrderActive. If we see an order that isn’t active, we’ll send a message and assert that it is now active.

(NB. sendMessage() isn’t syntactically correct for JRules but you get the idea.)

Next we need to know what to do if the SellOrder is retracted (either explicitly or implicitly):

rule BrokerInformedOnRetractionOfSellOrder {when {?active: SellOrderActive();?order: SellOrder() from ?active;not SellOrder(?this == ?order);} then {sendMessage("No longer sell stock");retract ?active;}}

This rule says that anytime we think we have an active order but the SellOrder itself no longer exists, retract it and send another message to the broker indicating that we no longer wish to sell.

As usual, I’ll re-write this rule in JESS. Thanks go to the creator Ernest Friedman-Hill for clarification on the exact syntax:

(defrule broker-informed-on-new-sell-order?order <- (sell-order)**(not (sell-order-active ?order))**=>(sendMessage "Sell stock")(assert sell-order-active ?order))(defrule broker-informed-on-retraction-of-sell-order?active <- (sell-order-active ?order)**(not ?order <- (sell-order))**=>(sendMessage "No longer sell stock")(retract ?active))

Or, interfaces are no excuse to build broken software.

I’ve ranted about this previously and I see the problem recurring so often I feel compelled to have another go but this time from a slightly different perspective.

Lets start by agreeing that it’s best not to be constrained by someone elses API. They introduce constructs and ways of “doing stuff” that don’t conform to my ideal. Assuming we do agree, it then follows that mocking out someone elses API is not only a bad idea but usually plain pointless.

Enter Java’s interfaces and C++ pure-virtual classes. They are truly wonderful things. You don’t have to write any “real” code just to have one. No code means no tests required. And best of all, they provide functionality just the way you want it. They’re the ideal black box. Not as ideal but potentially almost as good, are the methods themselves. Even concrete implementations can and should provide me the ideal functionality.

So what do I mean by “ideal”? Well, say I’m implementing a bit of code and all of a sudden I realise “doh! here I’m going to need to do X” where X is conceptually simple but implementation complex functionality. I have a few choices: I could go straight into implementing X; or; I could choose to “pretend” that X is actually implemented exactly the way I want it to be.To achieve this I can either add a method to an existing class or I could introduce an interface. Either way, the important point is that I want this thing to do what I need in the ideal way. I don’t care that it doesn’t exist yet. But I don’t want my code polluted with knowledge of how it’s actually going to be implemented. I just want to call it and have it do it’s magic.

So we “stub” it out and push on. Eventually we finish our lovely piece of code. We stand back and it looks good. It’s readable. It’s understandable. But there’s still that X we haven’t yet nutted out. No problem. We can still test the class using mock objects so we can still maintain the illusion that X actually exists.

At some point we will feel the pain. We need to actually implement X so it can be deployed as part of the application. So now let’s say that X really has the potential to be stupidly complex but for now we don’t actually need to solve every possible case. We create a SimpleX or DefaultX or my much despised XImpl class. What we don’t do is create a MockX class simply because it’s “not the real thing”. What does “not the real thing” mean anyway? It’s there. It’s working code. It does what I need. It may not be as flexible as I want it to be but it’s NOT A MOCK!

And this is where the title of the entry hopefully becomes apparent.

As far as I’m concerned, every line of code I write IS PRODUCTION CODE. Period! If the customer was happy with the functionality of the system, they could disband the team and deploy the application right now and it would be production code. Let me repeat. EVERYTHING IS PRODUCTION CODE.

Code I write is rarely written with the intention of one day throwing it way! It’s a production implementation of the limited functionality that I need this minute. If I need more functionality at some point, I’ll add it. This may require me to intoduce more abstractions, more interfaces, more methods that potentially do as much as required to get the job done. I’m pushing the unimplemented bits out as far as needed so that my overall design continues to adhere to my ideal. This i’s not to say it won’t or can’t be binned at some point but that occurs as a consequence of an evolving application. It happens to any piece of code whether I actually decided upfront that it was likely to be binned or not.

There is always The Law of Leaking Abstractions to consider. But. by way of the obligatory analogy, just because there is the chance someone might run into my car, shouldn’t prevent me from actually driving. It just means I need to be vigilant and aware of what’s going on around me when I do so.

And because I’m sure I won’t have made any sense whatsoever up till now, I’ll confuse you even more with a real example.

Take pico container. Hold for the moment your predjudices on IoC/Dependency Injection/etc. That’s irrelevant to the point. The point is that pico container provides a very simple implementation of an interface. No bells, no whistles, no loading configuration from files. Everything is configured by code at run-time. But it does the job. It’s just not as flexible as maybe I’ll need down the track. But if it’’s all I need right now, then it’s, well, all I need.

Then I decide that what I need is something that can be configured from an XML file. Do I a) throw away all of pico container because it doesn’t do what I need; b) cut and paste the pico container code into a new class, re-using as much as I can, or c) just simply add the new functionality (in this case by way of nano container)?

I’ll let you be the judge but anyone who chose to have written pico container as throw away code because they knew one day they’d actually need to load configuration from files and therefore it was throw-away code deserves to be forced to program in binary for the rest of their natural life.

(BTW, I didn’t write pico or nano container I’m just using them as part of an example. That’s why it’s called an example)

Yes, it is true that things aren’t always ideal. But this is very rare in my experience and will only occur at the hopefully thin boundaries between your code and someone elses API! If this isn’t the case, then WTF are you doing? It’s your software. You created it. How is it that it doesn’t at least pretend to do exactly what you want?

FWIW, This discussion is really a demonstration of how I choose to interpret XPs do the simplest thing that could possibly work.

From: haruki_zaemon@mac.com To: haruki_zaemon@mac.com Subject: Re: Question Date: Sat, 1 May 2004 09:12:29 -0400

Here is my icq list. +++ Attachment: No Virus found +++ MC-Afee AntiVirus - www.mcafee.comarchive.zip

We (all?) strive to build great software but how many of us look back on code we wrote even as recently as 6 months ago and still think “wow that’s some great software”?

Open up that CVS client and download some source code you haven’t looked at for a while. Make sure to start with the project you thought was the “best thing you’d ever written.” I’d be very surprised if your thinking on software development hasn’t changed sufficiently in that time to make you whince when you see the “cool” things you were doing back then.

I look back on videos of me performing Aikido technique and the same thing happens. “Yikes!” I think to myself “I can’t believe I used to think that was good! And to think I was teaching others that at the time!” My students learned long ago not to treat anything I say as gospel. To critically analyse what I’m teaching and understand it for themselves, not just parrot what I say. They know full well that it will probably change after a long weekend of contemplation.

We grow, we learn. Our thinking changes. In some cases it swings back and fourth like a pendulum trying to find that “sweet spot”. So don’t get down on others because they don’t produce the code that you’d like. Similarly, don’t be perterbed when your mentor/coach/teacher suddenly starts explaining to you, with all the enthusiasm of a child with a new toy, that yellow is the new brown.

Because we don’t live in an ideal world, when it comes down to it, at any point in time we really are just doing the best approximation of the least crappy solution we could possibly build.

Today I have to vent my spleen in the hope that I’ll save some poor hapless souls from venturing down the frivolous path that is the JRules Business Action Language (BAL).

First up, a recap. If you haven’t read any of my previous blogs on the subject, we’re in the process of converting all our validation to JRules and we have some, IMHO, pretty cool stuff happening now but I’ll blog about that another time. The rules are in a repository because realisticially the only way to write them using the BAL is using the repository (and associated tools). We need to use the BAL because it’s the business user friendly language and without it most if not all the business case for using JRules in the first place goes out the window. I estimate that we would be in the order of 5 times more productive if we could write these by hand using IRL in a text file!

Having all our rules in the repository makes it almost impossible to be agile/iterative/whatever you want to call it. Forget having multiple developers modify them because the rules are not maintained on an individual basis (ie they’re essentially all in one dirty great big file). Forget being able to apply domain refactoring (namely rename and move). And who the hell wants to write a custom Ant task to get the damn things out of the repository to do any kind of testing on them? Believe me when I tell you that we tried every which way to skin this little cat. Everytime we came up with a solution to one problem, another presented itself. It’s not that we couldn’t go ahead and write the rules mind you. It’s just that we knew if we did we would be heading down a path that would ultimately violate the entire premise on which we had based our approach. The greatest minds on the planet couldn’t have helped get around the plain and simple fact that the BAL should be classified as legally dead!

And so it was with great pleasure that we called in the project oncologist and removed the festering pustule that is BAL, hopefully, once and for all! From now on we will be maintaining the rules by hand in a text file (just like java). Because it’s a text file, CVS and IntelliJ can do their merging magic (just like java). Because it’s a text file, we can rename methods, classes, etc. with ease (just like java). The list goes on but I’ll spare you :-)

The upshot is that I managed to get 2 days worth of (previously estimated) work done in around 30mins! Just to be sure, I passed around a print-out of the rules (now in IRL) to garner peoples opinion and EVERYONE agreed: It was good! Oh, and readable. Which was actually going to be my point :-) What’s even better, is that almost straight away, people noticed a few mistakes in the rules that nobody had noticed when it was written in the “nice and fluffy” language.

I mean, when it comes down to it, the idea that end users will modify these rules is about as likely as the idea that end users can just create their own Crystal Reporst straight from the database - YEAH RIGHT!!! Even if they do want to, I reckon 30 mins of training would bring our business analysts up to speed with writing IRL. In the worst case, we can always convert rules from IRL to BAL on an as need basis if/when there is a requirement to do so.

In summary, JRules as an engine is great but the BAL sucks the big one! IRL IS readable (with a teeny bit of extra work) by end users and works great in an agile environment such as ours. BAL doesn’t. It (BAL) might work once you have a mature application but we don’t. Tools such as Eclipse and IntelliJ are making agile development that much easier. JRules IDE (as distinct from the engine which is just fine) is definitely a step backwards.

When I started on this project (as is usually the case when starting on any project) there was much to learn and many of the design decisions were unclear at best. Some seemed downright ludicrous.

Unfortunately for my team members, I’m not one to just start copying what everyone else has done - I need to understand why. After 6 weeks or so I started to feel that some of the stuff I was seeing was deliberate and some was just plain wrong. But that still didn’t explain to me why the deliberate stuff was the way it was.

This week James Ross returned to the project (YAY!) after being seconded by another team (BOO!) on the same floor. James is the technical architect for the project and therefore, IMHO, the one charged with having the overall “vision” for the design.

So, all week the focus of my constant ranting and questioning was turned fairly and squarely on him (poor bastard!). Still, no matter how many questions I asked or how much I ranted, every answer seemed only to address a single question which feelt uncomfortable to me - James is one of the best technical architects I’ve ever come across.

And then as we (James and I) walked out the office door on friday night still in the thick of an argy bargy, he made a seemingly off the cuff and devastatingly simple remark: “it’s an insurance application form. I’ve tried to get that across to everyone on this project so many times now” he exclaimed with a sense of exsasperation “except for you I guess?” Things immediately began to make sense! In one fell swoop, he had addressed almost all of all my issues.

XP has as one of its core practices The Metaphor. It had never been so clear to me how important the metaphor really is and the reason nothing seemed to gel was that I didn’t have it.

It’s been a while since I was on a project where I didn’t have the metaphor in my head. What also became blatently obvious was that when I have been the lead on a project, that’s pretty much where it’s stayed (in my head) and I haven’t communicated the metaphor effectively to every team member. It’s something I’ve usually written down as a series of “principles” that, whenever we have a design question, we can refer back to for guidance.

The metaphor is the highest level of abstraction in the application design fractal and therefore it’s paramount that every member of the team, either existing or new, fully understand what the metaphor is. Of course this is predicated on there actually being a metaphor in the first place! ;-)

Scarcely any degree of judgment is sufficient to restrain the imagination from magnifying that on which it is long detained. – Samuel Johnson

So “why then do I need all these unit tests for my code?” I read with amusement. Why can’t I just write functional/acceptance/integration tests and be done with it? The arguments are ludicrous. Saying that there are people we know who obsess about unit testing yet seem to write crap software and/or go overboard with seemingly useless tests in no way proves that it’s a flawed concept. Holy crap! if we applied that kind of logic to the entire industry we’d have stopped doing software development years ago.

We have to be very careful what we think we’re arguing about. I’d probably get upset if a Windoze weenie told me that Java was slow when what they really mean is that Java seems slow on Windoze! It may well be the case that Java is slow. But the fact that it’s slow on Windoze hardly justifies saying that Java as a whole is slow.

The fact remains that what most people do IS integration and functional testing and therefore most people know very well how to do this. Just because most people don’t know how to do proper unit-testing doesn’t mean unit-testing is wrong. It just means we work in a very immature field. Hell most people (myself included) don’t know how to do good software design. But we’re getting better at it!

Unit testing has proved itself time and time again in other fields. Do you think Ferrari needs an alternator to prove its engine design? Does the alternator manufacturer need an engine to prove its design? No. Likewise, every component (unit) of your computer has been individually tested before it’s placed into the PC your using to read this. Every single one. And then some! The components that make up the hard disk were individually tested before being used to assemble the hard disk. But does the PC manufacturer care about the components inside the hard disk? No. Of course not. Why? Because to them the hard disk is the smallest unit.

And so it is for software. Each “layer” in a system can be thought of as both a higher level of abstraction than the layer below it, yet a lower level of abstraction than the layer above. Sounds obvious right? Because software is and should be fractal in nature, it looks essentially the same no matter the magnification or level of granularity.

Do you test whether the JVM can actually increment a variable using the pre-increment operation? No! Do you test that the String class behaves correctly? Do you test to see if “your favourite framework” does its job? Probably not. At best you can safely believe it works as advertised. At worst, you write some integration tests to prove (or not) that your understanding of the tool matches reality.

Go have a quick search on Google and see what types of testing go on in electronic, automative, aerospace and just about every other field of real engineering I can think of. Not this trumped up Research and Development we like to call Software Engineering. Here’s a completely random start.

So when it comes down to it, the same people that tell me that “the network will never be transparent” because if I’m building a nuclear power plant, I can’t afford to just “ignore” network failures, also want me to believe that I should overlook a development and testing regimen that has proven itself time and time again in other, more rigorous, disciplines!

The project I’m on has 1200+ unit-tests that take around 11 seconds to run. Then we have a smaller yet substantial number of end-to-end regression tests, an even smaller set of end-to-end acceptance tests for each iteration and a tiny number of integration tests with external systems. So maybe we’re not your beloved “Thought Leaders” but we definitely have plenty of experience doing other than unit-testing.

We have zero bugs that we know of. We have things the users don’t like because they changed there mind or the developer didn’t understand the requirement. But we have rarely, if ever, encounter unexpected behaviour in the system. When new code is added, it quite clearly breaks the build if the developer has made a mistake. I can’t tell you how many times it’s saved my ass. We’ve definitely not encountered the types of problems usually associated with complex systems whereby strange interactions between objects causes software failure. I put this down to well defined and tested interfaces, whether they be “artifical” or not.

We perform around 20 or so automated integration builds a day running the full suite of tests and each developer runs the unit tests countless times each day.

The best part is, our system is nicely de-coupled. We can quite happily replace bits of our system with little or no impact because the unit-tests defined our interfaces.

Oh and as for writing bazillions of lines of code, actually my experience is that we end up with less code. In fact I find myself deleting more code everyday as we refine our abstractions, usually driven by testability. But that’s just my experience.

I know the software I produce today is, at best, average but the average has gone up since I started working - at least in the product development (as oppsoed to consulting) circles I keep. And it will continue to go up as we mature as an industry.

I’ve seen a few blog entries around of late demonstrating nifty things you can do to achieve setup/cleanup in your unit tests using statics or classloaders etc. And whilst I admire the creativity and ingenuity, I have to say that just like testing private methods, to me it smells.

Firstly, If unit tests rely on static state and class loaders for setup and cleanup then the tests are just plain broken and wrong. The ultimate goal for me is to be able to run test classes in parallel. Just splitting them up to achieve this won’t help because you likely have no way of knowing which test classes interact with which static parts of the system. The good old abstract static factory is a prime example of this. Unit tests should probably create any data and mock any infrastructure they need for the test. They certainly shouldn’t be relying on the behaviour of a class loader! If there’s too much infrastructure to mock, thats a smell too!

Next, ideally we’d like to run test methods in a single test class on a single instance of said test class, implementing setUp() and tearDown() as appropriate. Maybe the idea of a single instance isn’t always practical (I can’t think why) but just assuming the tests will be run in a new instance isn’t acceptable either. What could I possibly want to do in a constructor that I couldn’t do in setUp()? Again, relying on the behaviour of the test runner to create new instances or the class loader to unload static state is just wrong!

There should be no assertions in tearDown(). Assertions failures in tearDown() will mask any failures in the test itself making it all but impossible to track down the actual problem.

And lastly, avoid extending other test classes as much as possible. Remember, any tests that are defined in the super class wil also be run in the subclass. Not only do we end up redundantly running the super class tests but if you have inadvertently overidden some public or protected methods (say setUp() or tearDown() for example) and forgotten to call the super method of the same signature, all hell breaks loose.

As a general rule, excessive refactoring of unit tests can make them difficult to understand. Sure it might be testing the code but it WILL make maintenence of the tests very difficult and surely hinders the ability of anyone to get a handle on the expected behaviour/role of the class under test.

I feel so stupid! I honestly can’t believe it’s taken me this long to realise what a drongo I’ve been.

Many years ago (well probably 3 or 4 years ago) a work colleage of mine at a new job was was using a password scheme that, at that time at least, was quite common amongst his colleagues (yes I know that by inference, they were now my colleagues also hehe). Anyway, this scheme involved replacing vowels with numbers. So, for example, “Hello” would become H3ll0.

At the time this seemed to me like a sensible thing to do so I blindly followed along. But just now it suddenly occurred to me how crack-headed (no pun intended) and brain-dead that was. I mean if the whole point was to prevent brute-force dictionary attacks then its clear (to all but me it seems) that attempting to obfuscate a password with a deterministic algorithm such as this is just pointless.

Fortunately I think I only ever used it on one machine I had at home those many years ago. But I wonder how many others continue to use it?

After a long week, Achilles finds he has too much time on his hands. His friend the Tortoise takes pity and indulges him with a bit of IM’ing.Achilles:I’ve done nothing but read blog entries this weekend.Tortoise:You must be bored! Anything interesting?Achilles:I just read an entry that reminded me of some stuff I refactored during the week.Tortoise:Do you ever get any real work done?Achilles:Now that Java has a LinkedHashSet can you think of any reason to use a simple List except for “performance” reasons?Tortoise:Won’t it just look like a List?Achilles:Sort of but, importantly, it’s also a Set. Since when do you actually mean to add the same item to a List more than once? I’m being pedantic here.Tortoise:But it happens, life is full of duplicates.Achilles:I’m sure it does but I can’t think of many examples where that’s actually what you want. It just seems too often people use Lists when they should actually be using a Set. Clearly, Lists are useful but ArrayList has to be the most abused Collection class aroundTortoise:People generally think in terms of Lists - it’s a simple concept.Achilles:Yes, but people also think that AND and OR mean exactly the opposite. What we think and what we mean aren’t always the same and programming is about expressing what you mean.Tortoise:Do people really think about the correct Collection type to use?Achilles:No, they probably don’t but they should.Tortoise:I try to but I can’t guarantee that I won’t be lazy and default to ArrayList.Achilles:Exactly! And then the code ends up iterating over stuff and assuming a particular order on things that have no order. I see it all the time and this damn CollectionUtils.isEquals(Collection, Collection)code just makes it worse. Its ludicrous. It basically allows you to compare a Listwith aSetand see if the contents are the same. Which is just wrong! AListand aSetare not the same thing. They are symantically very different and thinking that it's just a matter of comparing the contents is, IMHO, flawed.</td></tr><tr><td align="left" valign="top">Tortoise:</td><td>Which takes us back to your original question - if you want to allow duplicates then you can't use aSet, so when would you want to allow dups?</td></tr><tr><td align="left" valign="top">Achilles:</td><td>Very rarely I suspect. In fact how often do you ever want to allow duplicates and how often does order really matter? Part of the problem I think is a misunderstanding of what equals(Object)actually means. It implies substitutability and therefore must be reflexive. But many people don't realise that theirequals(Object)method isn't so that we end up witha == bbutb != a.</td></tr><tr><td align="left" valign="top">Tortoise:</td><td>I've not seen that happen.</td></tr><tr><td align="left" valign="top">Achilles:</td><td>It usually happens with inhreitence and using instanceofinstead of class comparison.</td></tr><tr><td align="left" valign="top">Tortoise:</td><td>You must have looked at a lot of shitty code!</td></tr><tr><td align="left" valign="top">Achilles:</td><td>You mean you can't tell? Why do you think I bitch so much :-)</td></tr><tr><td align="left" valign="top">Tortoise:</td><td>Ok what if I have a situation where it is possible to have more than one object of the same type and content? ThatCollectioncould not be stored in aSet, correct?</td></tr><tr><td align="left" valign="top">Achilles:</td><td>Correct. So you just want a Collection, not a List. I repeat NOT A List.</td></tr><tr><td align="left" valign="top">Tortoise:</td><td>Then what implementation class do I use?</td></tr><tr><td align="left" valign="top">Achilles:</td><td>The implementation can be a Listbut the variable should be aCollectionas inCollection things = new ArrayList();code because a List implies ordering and so far you haven’t mentioned anything about order being important.Tortoise:Ok so then I decide that ordering is important.Achilles:Sure make it a List but the key thing is that you don’t just assume that order is important because then people will try and write tests assuming something about the order and then they’ll build screens assuming something about the order, etc. etc.Tortoise:I’ve just remembered…I added a method to compare Collections (for that domain object) to see if there had been any changes - there is no check to see if they are the same implementation of Collection so i could be iterating over a List and a SetAchilles:Why can’t you just call Collection.equals(Object)? Thats what it’s for.Tortoise:On the Collection?Achilles:Yes. I see people writing “convenience” methods for comparing Collections all the time when they already have an equals(Object) method that does a perfectly good job.Tortoise:I assumed that it wouldn’t do a deep comparison.Achilles:It iterates over the contents, calling equals(Object) and or checking object identity (whatever is appropriate for the Collection). I use assertEquals(Object, Object)code on Collectionsall the time.<tr><td align="left" valign="top">Tortoise:</td><td>Hmmm, that didn't get picked up in the tech review.</td></tr><tr><td align="left" valign="top">Achilles:</td><td>Probably because everyone on the project usesCollectionUtils.isEqual(Collection, Collection)code!

On the current project I’m on, and no doubt on many of yours, we have restrictions on the number of parameters you may declare for a method. This, hopefully, forces developers to re-evaluate what they are passing around. For example, use a DateRange instead of dateFrom and dateTo.

Unfortunately there is a simple way to subvert that process by simply declaring everything as “setters” which typically have only one parameter. Then all the lovely detail becomes much harder to see as it’s hidden in the morass of the aptly name mutators (yet another reason I dislike getters and setters).

Declaring service dependencies in constructors allows me to see them all in one go. It immediately becomes apparent if a class depends on “too many”. Something that is much harder to see when you use setters. It also allows us to construct objects in a valid state with all the obvious benefits.

Another advantage to using a constructor is that only the creator need know about the dependencies. In our case it’s our ServiceRegistry. Once an object has been constructed (either by the registry configuration for singleton services defined by interfaces or by calling ServiceRegistry.newInstance(Class) allowing the construction of any class that depends on a service) client code is unaware of the dependency.

I’ve recently been converting large swathes of code on this project from static lookups and setter-based dependency injection (DI) to constructor based and in the process finding stuff that was not previously apparent. These include classes that depend on too-many services, classes that depend on services they just shouldn’t and classes that are quite fragile because callers didn’t realise they had to call the setters in a particular order! None of these was in anyway obvious previously.

The down-side of course is that constructor parameters make inheritence a pain. If you have a deep and/or wide class heirarchy, you will need to declare the dependencies in the constructors of all the sub-classes. Not only is this tedious but it necessarily exposes the dependency to classes that probably don’t directly make use of the service.

My counter to this is simple: don’t have deep and/or wide class heirarchies. I rarely find the need for them and they’re usually a code smell. The fact that we can extend classes to inherit behaviour doesn’t mean we should. No doubt a topic for another rant :-)

As always there are exceptions. Sometimes you just can’t get around the need for setter-based DI, usually when you are constrained by someone elses API. Most noteably in our case is the fact that (for reasons I don’t want to go into urgh!) we’re forced to use versions of third-party libraries (Struts) that want to directly construct our classes with no factory mechanism. This means we have Actions that must have a default constructor.

You may have noticed a distinct lack of the ‘T’ word. I deliberately stayed away from describing how all this makes classes more testable and is a natural consequence of doing TDD anyway because if you’re doing TDD you already know this and if you don’t like TDD (really?) I didn’t want to put you off the idea by implying this was all about testing, which it’s not.

Some of the arguments against the tsting barrow I push are that you end up with essentially “legacy” tests. Tests that you have to continually maintain. And often tests that are brittle.

I was having a discussion with a colleage the other day and he asked me if I thought it was really necessary to test everything given developers often write crappy and brittle tests anyway. Whats worse is that inexperienced developers often go through unecessary hoops to build the tests and/or production code and possibly end up writing software that is just as bad (or maybe worse?) than if they hadn’t tried to do test first (or at all).

I couldn’t believe it but you know I’d never really thought about it too much. I had just believed my own BS and assumed that, yes, you need tests for as much as you can possibly test.

And you know what?

I still believe it!

Crappy tests almost always (I can’t think of an exception but there invariably is one) imply crappy code. That means that the brittle tests are likely testing brittle code. Now, brittle tests are bad because when I make my seemingly innocuous change, I invariably end up breaking some of these brittle tests when really it shouldn’t have. But then again, if I assume brittle tests means brittle code, the chances are I really have broken the underlying code! And how would I have know that if the tests hadn’t been there in the first place?

So I still stick by the mantra. Test everything. Even if (especially if) it’s crappy and brittle ‘cause you’ll be doing yourself and everyone else on the team a favour by ensuring they know when they’ve broken your code.

I leave developer re-education/career re-assignment as an excercise for the reader. Though I hear a 2x4 comes in real handy. I have the scars to prove it ;-)

An amusing topic coming from one such as myself that can quite clearly never shut up!

I’ve been bitten a few times this week by a low signal/noise (or a high noise/signal which ever you prefer) ratio. The first was earlier in the week when I was supposed to give a talk (more noise) and demo at the monthly MXPEG meeting.

Like most geeks, I’m continually tweaking my system. And as most geeks know, tweaking is really a euphemism for breaking something. I’m constantly updating the patches on my system and installing the latest version of this and that. Each time I perform the awesome Gentoo emerge -uD world it spews forth volumes of logging. So much logging that it’s not only hard to see if there are any messages worth reading but they fly past so quickly I probably couldn’t read them even if I tried. So after a very short time one starts to ignore pretty much all of them. Hey if it doesn’t complain at the end then it’s probably ok. Right?

So there I am at the meeting trying to get my machine to connect to the network and failing dismally. I gave up and vowed to track it down when I got home. Eventually I discovered that there were some config files that needed updating and the messages were there for all the world to see saying just that. But because I had just ignored them all I never noticed.

Similarly, we had a bunch of checks on our code base that were flagged as warnings not errors. At some point, that list of warnings got so big we all just started to ignore them. Then one-day I accidentally checked in a copy of the build file that stopped breaking the build on errors (over zelaous find and replace) . Nobody noticed. Why? Because everyone had just grown accustomed to ignoring the output because it contained so much noise. The answer was of course to either remove the warnings as we clearly didn’t care about them, or turn them into errors so we train people to look at all messages.

James and I worked on a project where there was so much logging going on, nobody noticed all the stack traces and NPEs etc. being spewed into the server logs. In fact there were so many errors that they turned off the “page me if there is an error” handling. This of course just made the problem worse.

JavaDoc comments are another pet hate of mine. We did a refactor the other day that involved a rename of a class and the subsequent getters and setters on the domain object and process beans, etc. We ended up spending so much time updating the comments from “Sets the value of the product details” to “Sets the value of the vehicle details” that I wanted to murder someone. JavaDoc comments convey little or no more than the method signature should convey. I emphasise SHOULD for good reason. Sure, there are some special cases such as “will return the default if no explicit value has been set” for example but if you feel you need to explain what getCustomer does, then throw a GetAGripException or find a new profession.

All this stuff is noise. The problem really is that you have been allowed or have chosen to ignore it. You can’t afford to. Don’t just ignore it or it will come back to bite you later on. Of course, just saying “well we ignore it so maybe we can just stop doing it” isn’t the answer either. Maybe it’s there for good reason. Find out why it’s there and stop needing to ignore it before it starts to take it’s toll on yours and the rest of the teams poductivity.

P.S. Can anyone tell I’m back on a project again? LOL

I wrote previously on my initial investigation into JRules. Having been using and playing with it in a production code environment for a bit now I thought I’d follow up with some observations thus far. Some of these no doubt relate to other rule engines such as Drools but as I’ve never used them in the real world, so to speak, I’ll have to wait to hear back on that.

One of the biggest hurdles we had to face was the so called Business Object Model (BOM). This is a mapping from your domain model to user-friendly “natural language” expressions. Our first cut on this was to load up the model in the rule editor and start mapping away. This soon became tiresome, error prone and very, very brittle. Even slight changes in the heirarchy meant wiping out rules and starting all over again.

Next we tried generating the BOM directly from our domain model but that too smelled of maintenence nightmares. So instead, we simplified the whole process by effectively flattening the heirarchy. We represent as many facts as we can at the highest possible level by “exploding” the domain model (for rule execution only). We also assert services such as navigation state, current time, calculation engines, etc. in the same way.

At first this seems odd, especially to a hardened OO developer. But after some time it soon becomes apparent that it makes a lot of sense. To start with, it’s much easier to do the mapping. We no longer need to worry about the relationships between classes and can focus on mapping the properties of individual classes. This makes our mapping much less susceptible to change. It also becomes much easier for the end users because they tend to think of pieces of information (facts) with less regard to relationships than we do. As in The customers address as opposed to the address of the customer of the order.

One huge bonus is that we no longer have to spend large amounts of time making the “nice” natural language mappings actually understandable. By that I mean tweaking the “translation” so that it reads in plain english. Because there is far less navigation going on, we decided we can probably not even worry about the language mapping until later, making it much faster for developers to work.

As an aside, I find it incredibly annoying that JRules doesn’t seem to allow you to reference constants (public static final fields). Instead we are forced to create virtual methods for each of them with a translation back to the constant. DUH! Please someone tell me it aint so.

Don’t allow the rules to hit the database if at all possible. Not something we even entertained but another project in the same building is doing this and what do you know, it performs like crap. Relatively speaking that is. We’re using Hibernate so the intention is to have all our reference data modelled as objects, cached and asserted just like all the other facts.

The cool thing is that all this is kinda like good old IoC/Dependency Injection/Whatever it’s called this week. Nomenclature aside, you tell the rules everything they need to know. They don’t call out to services by calling a ServiceLocator. They don’t call static methods, etc. Instead they simply declare they need a particular service or are interested in a particular fact, and the rule engine does the plugging. Neat, and for all the same reasons that IoC is good for “normal” java code.

IlrRuleset and IlrContext are relatively expensive to create so we are using a very simple pooling mechanism to manage them. Don’t forget to reset the context when it’s returned to the pool or it’ll hold references to objects that you probably no longer care about.

Another thing we found was that it’s best that the application not worry too much about what rules may/may not be applicable. Similary, don’t try and put too much “behaviour” into the rules. It’s tempting to want to tell the rules engine (either explicitly or via some facts) that certain rules don’t apply. The problem here is that we start to embed knowledge of our application into the rules and vice-versa.

Let the rules do their job. Namely, assert and retract facts based on certain conditions being met (or not as the case may be). The application is then responsible for performing some kind of action based on the state of the facts after firing the rules. This is probably somewhat controversial as it would be totally possible to implement an entire system in rules. (Actually, not only would it be possible but it would be pretty cool too). However in our case, and no doubt in yours, the rules are not application specific. In our application they represent company-wide knowledge about the way they do business. In fact the longer term plan is to publish these rules as an enterprise accessible service.

And finally, the non-deterministic execution of methods on your domain model during rule evaluation. By this I mean you can’t guarantee that a given method will be invoked once or many times (or at all?) and especially not in what order. This isn’t usually a cause for concern as you will probably be calling simple getters but in the few instances that isn’t the case beware. Even something as innocuous as obtaining the current time can be an issue. More often than not, you want to treat the rules as a batch and to be evaluated and fired against a fixed point in time. So for example, we assert a [Clock](https://harukizaemon.com/posts/2003/12/07/when-is-a-clock-not-a-clock/) implementation that returns the same value whenever (ie. no matter how many times) getCurrentTime is called.

Every two weeks we have our iteration close during which we hold a mini retrospective. We break our observations into four

• What Worked Well?;
• What Went Wrong?;
• What Puzzles Us?; and;
• What to Do Differently Next Time?

Mike Melia made an observation that, when one works too late, they are prone to making stupid mistakes. Therefore, it’s often better to go home, get a good nights sleep and tackle the problem the next day. Sage advice. Golf-claps all ‘round.

Of course being the bunch of smart-arse, lazy, meeting loathing developers that we are, we decided that it was such a good point we could probably generalise it a bit and save ourselves a lot of time for the next and subsequent iteration close meetings:

• What Went Wrong? People made mistakes;
• What Puzzles Us? Why do people make mistakes?;
• What to Do Differently Next Time? Don’t make mistakes;

And of course the one we all hope for:

• What Worked Well? People made fewer mistakes.

I think everyone will agree though that David (get a blog soon you git!) Pattinson does do a pretty good iteration close/kick-off especially as he is continually interrupted by people throwing stress balls around and knocking cans of coke off tables. Sheesh. Who would do such a thing I ask you? :-P

I found an interesting (to me anyway) implementation detail in Suns JDK LinkedList and HashSet code.

Compare these two. The first is deep in the bowels of LinkedList.indexOf(Object) which is used by the implementation of contains(Object):

if (o.equals(e.element))
    return index;

The next is from HashMap.containsKey(Object) which is used by HashSet to implement contains(Object):

return x == y || x.equals(y);

Notice anything interesting? No?

Well I know it’s not much but HashSet which one would think probably only honours the equals(Object)code+hashCode()contract, also checks for object identity. On the other hand,LinkedList, which I would have thought probably only checks for object identity, actually only honours the equals(Object)code+hashCode() contract.

Why is this a problem? Well I found a class (in a 3rd-party library) that never returned true for equals(Object). I only found it because I was writing a test and wondered why when I added instances of the class to a LinkedList, contains(Object) always returned false even though when adding them to a HashSet, contains(object) always returned true.

I had expected List implementations to at least check object identity. Unfortunately it behaves exactly as the documented, says so I’ll just have to deal with the real problem and shoot the developer who wrote the broken code instead. But saying so upfront would have spoiled a good rant :-)

And the moral of the story is? Tests are good; Assumptions are bad; Test your assumptions; and; Sometimes the documentation is worth reading but usually only after the fact when all the subtleties hidden therein become apparent.

I thought this article was quite amusing, especially the comparison between dining at the JavaHut versus the Perl Cafe.

So allow me to give another perspective:

…Dusting yourself off, you walk across the street to the Perl cafe. The person at the door asks what kind of seat you want, what you want to eat, and how you want to pay. They sit you at your seat, bring over your food, collect the money, and leave you to eat in peace. You look down and realise that what they brought you wasn’t what you ordered. In fact looking around, it seems everyone is eating the same dish. It’s then that you realise no matter what you order at the Perl Cafe, all you ever get is spaghetti.

I know, I know but I just couldn’t resist ;-)

I’ve often joked that procrastination is the mother of invention. If I put something off for long enough, it’ll either cease to be a problem or a more elegant solution will come to me. Certainly this doesn’t always hold true nor is it always that simple but it does work out that way often enough for me to at least continue believing my own rhetoric :-)

The idea is to continue pushing the implementation of a solution further and further away. By that I mean, instead of thinking about the nitty gritty detail, start off by thinking in terms of the simplest interface possible and code to that. Imagine that it actually worked even if you’ve hard `d some behaviour.

It’s nothing new. It’s what good software developers do all the time (one hopes). It used to be called top down design. These days it’s what TDD exponents typically do.

The important thing for me is that it IS important to sit around and think through the problem as fully as possible. Think about all the possibilities. What could we do. What would happen if we did option a versus option b? What might the customer want to do?

Yes that’s right, I’m suggesting actually solving a possibly imaginary problem. Shreaks from the crowd. The BIG difference is I’m not actually going to implement any of this. All I’m trying to do is validate that my assumptions, my abstraction, my simplification of the problem, my procrastination on implementation, my (some would say blind) adherence to YAGNI isn’t going to come back and bite me if at all possible. And more to the point, trick those 95% of unused brain cells into doing more than just suck nutrients from my blood stream.

Again I stress I’m not going to do anything about it. I’m going to defer implementing anything until as late as possible. Hopefully by then my brain has been feverishly working away in the background mulling over all the things we came up with in the brain storming session (as distinct from design session) and magically come up with an elegant solution.

Sometimes it may be necessary to break the problem into smaller chunks. Flip the question around. Rephrase it. Try and push it into a different mould. See if all your assumptions still hold. Playing simple semantic games can help make a problem seem more familiar. Try calling it a Displayable instead of a Page. Maybe (and this is where the domain modelling dudes possibly get upset) Finance really is a product just like MotorVehicleInsurance even though the customer doesn’t see it that way because it’s not core business.

And you know what? More often than not, the problem just dissapears. We’ve abstracted the problem such that it no longer seems like one of many disparate, unrelated problems. It starts to conform to some other problems we’ve seen or implemented. Maybe we can tweak an existing implementation to support the extra behaviour.

To me, that’s what problem solving is all about. It’s about thinking through the problem sufficiently enough that it no longer seems like an isolated case. Some of the worst code I’ve ever seen was obviously written as a mass of solutions to completely (perceived) isolated problems.

I’ve always loved the idea of rule-based applications but never really had the opportunity to build one. And I have to say I’m having a lot of fun using a rules engine on this project. Since we dumped the BAL in favour of the IRL in JRules, productivity has sky-rockected. FWIW, TDD and rules-engines are a perfect match (a fact I’ll blog more about in the coming days). I’m in geek heaven! About the only thing I wish I had now was an IntelliJ plugin (like the AspectJ one).

So anyway, my intention is that step-by-step, I’ll try and document my progress starting with a little of what I discovered today by way of a rather contrived example:

rule MessageIsGeneratedOnSignificantStockMarketIndex {when {?index: Index();evaluate(index.value > 3000);} then {assert Message("Today the stock market rose above the psychological 3000 barrier");}}

This example says that whenever the stockmarket index rises above 3000, we assert that it was a significant event. (Actually JRules has some other nifty stuff to do with associating timestamps with events but I’ll blog about that another time.) The important thing to notice is the assert keyword. This asserts a new fact into the “knowledge base”. This new fact will remain “forever” or at least until another rule retracts it.

Simple assertions such as this are great when you know that the asserted fact will always be true independent of the triggering condition. In the example, it will always be true that at some point in time, the stock market index rose to a significant level, even if the index drops again.

But what if we have a fact that only holds while the condition holds? In such a case, we’d need a “compensating” rule to retract the fact when the condition changed. This could get quite ugly. Thankfully, JRules provides a neat solution:

rule SellOrderRaisedWhenStockValueReachesMinimum {when {?stock: Stock();evaluate(?stock.value >= 30);} then {assert **logical** SellOrder(?stock);}}

This rule says that we will place a sell order for any stock that rises to 30 dollars. The key difference here is the use of the logical keyword. This tells JRules that the assertion onlyholds while the triggering condition holds. That is, while the stock value is at least 30 dollars, the sell order remains. However, if the stock value drops below 30 dollars, JRules will automatically retract the fact for us. What’s even better is that if the assertion of the SellOrder causes other rules to fire and therefore assert more facts, all those that were declared as logical will all be retracted as well. How cool is that?!

In our application, probably >99% of all rules will use the logical form of assert. This allows quite complex interactions between essentially independent rules.

If you find yourself having to structure your rules with priorities and worrying too much about the interactions between rules, it’s likely your individual rules are doing too much. Ensure your rules should be as atomic as possible. Seperate “inference” rules from “do stuff” rules". And don’t be tempted to simply change the state (ie property values) of existing facts. Instead, always assert new facts (as we have done in the examples above).

I thought I’d also show you the same rules using JESS.

(defrule message-is-generated-on-significant-stock-market-index(index (value ?value))(test (> ?value 3000))=>(assert (message (text "Today the stock market rose above the psychological 3000 barrier"))))(defrule sell-order-raised-when-stock-value-reaches-minimum?stock <- (**logical** (stock (value ?value)))(test (>= ?value 30))=>(assert (sell-order (stock ?stock))))

You’ll note that in JESS, the logical keyword is associated with the condition (or LHS) rather than the action (or RHS).

In many ways JESS provides a richer environment than JRules but I admit the syntax is less obvious to novice users.

A procedure for critical evaluation; a means of determining the presence, quality, or truth of something.

My original post was (supposed to be) asking the question, when is it justifiable NOT to have a test (either new or modified) AND in particular what consititutes a test in the first place. It seems to me that many developers see refactoring as an excercise that somehow doesn’t need tests. Also, the term test has become synonymous with JUnit and this seems to me somewhat short-sighted.

Unfortunately though, I think I failed to communicate this clearly. At no time did I suggest that tools would take over the world and do everything for me. In fact quite the opposite. It’s especially the non-obvious forms of anything for which I want someone to write a test precicely BECAUSE I can’t get a tool to check for it automatically.

Andy Marks suggested that maybe rename is one refactoring that doesn’t require a test. I’m sure there are many others - introduce constant, inline local variable, etc. to name but a few - and though I probably agree that these are special cases on a simple gut feel level, I’m not convinced there has been a rigorous argument put forward.

Because, fundamentally, I don’t believe you when you say it’s an innocuous change :-) Hell I don’t trust myself. And I think you’ll find that the more experienced a developer becomes, the more sceptical he/she is about saying something like “oh yeah that’s trivial.”

It’s like speeding in a car. You wouldn’t speed if you thought you were going to kill someone right? You drive fast because you think you can handle it. And the speed you think you can safely handle is probably lower than my 18 year old cousin even though you can probably safely drive faster because you have more exprience.

And therein lies the problem. Letting people arbitrarily decide when it’s appropriate is dangerous especially on a team of inexperienced developers who by definition don’t know what they don’t know. But maybe just as much on a team of experienced developers because they believe they know everything ;-P

In light of that I think it’s fair to say I want you (and myself) to have to “justify” the code change with a breaking test of some kind (preferably automated) if at all possible.

Take the example of double-checked-locking. If I see it, I want to remove it because it’s a broken pattern. But I don’t want to have to write a test for it cause that’s pretty involved (like writing thread-safety checks!) So instead we have a machine identify them and on the basis that we all agree it’s a bad thing AND the test for it (ie checkstyle) barfs, we are all happy to change the code.

So, can we consider this to be a test? If so, then I’m satisfied that I have a broken test and on that basis I am justified in making a modification to the code. After the change, the test no longer breaks and as long as no other tests broke as a consequence, we’re satisfied.

On the other hand, if we don’t consider it a test, where is your breaking test to change the code? And are we satisfied that we don’t need one. Again, I don’t believe you when you say “it just needs to be changed and it’s only a comment so it won’t break a thing.”

From my perspective, the code base is a picasso - if you think you need to change something, you’d better have a damn good reason to do so. And just saying you don’t like impressionist paintings isn’t one of them :-)

Being forced to justify a change forces you to think rigourously about the underlying problem. Irrespective of delivering functionality, at it’s core, that’s why we believe TDD produces good quality code. And a bonus is that when you pair with a less senior person, you’ll be forced to explain to them what you’re doing. In fact as a junior developer, I won’t let the senior partner in the pair get away with just telling me it’ll be ok. I force them to stop and explain what it is they’re doing AND make them write a test (first)!

And lastly, adding a test helps to ensure the issue will never return to the checked-in code base. Without the failing test, someone may decide to just change it back when they see a seemingly unecessary change or maybe even because they don’t understand the change. At least with a test, it will force them to (re)consider.