haruki zaemon

James mentioned that one of his colleagues had been compiling stats on various projects and how they rated in terms of Cyclomatic Complexity (A.K.A. McCabe’s Complexity). Quite rightly, everyone on the team was ecstatic to find that the project ranked top of the list (ie. lowest average complexity) in a field of various open and closed source projects.

Tools such as Checkstyle and PMD (to name but a few) make collecting these kinds of statistics very easy. What’s more, incorporating these tools into your developer and continuous integeration builds makes enforcing limits on whatever metrics you like almost trivial. And that is exactly what we had done.

Comprehension and testability of code vary with complexity. Overly complex code is difficult to maintain, difficult to test and more likely to contain bugs (hidden flaws). Experience on previous projects (rather than just theory) had taught us that a low value for cyclomatic and it’s cousin NPath complexity was indeed a good predicter of problematic code. On these projects, we had always cranked the threshold right the way down and had found this to have a positive influence on the code base.

In true Pavlovian style, and with nothing but ultruistic intentions, we decided to make cyclmatic complexity (along with dozens of others) a constraint on this project as well. It obviously follows then that our project would place at or near the top of the list. We had made it a constraint, enforced it in our build and therefore the only possible outcome was a low average.

But as always, the devil’s in the detail. It soon became apparent that like so many other checks, it is possible to subvert the process. The worst part is that few developers do so out of malice. More often than not a check will fail and the developer will (often quite ingeniously I might add) code their way around it. Without knowing any better, it is quite easy to inadvertently increae actual complexity whilst adhering to simple metrics.

I have a love-hate relationship with development tools. In the right hands they are invaluable but you can’t just give inexperienced developers tools and expect them to do as well. Tools + Monkeys != success. In fact as we discovered, left unchecked the opposite is much more likely to occur. Luckily for us we had a much better ratio of experienced/inexperienced developers so we caught most of the problems early and entered into intense education (ala Clockwork Orange - just kidding). I think it’s safe to say that in the end, the benefits certainly outweighed the potential dangers.

While certainly interesting, relying on a single metric can be very dangerous. Blind adherence to any number of metrics can disatrous. In general it’s often far safer to let tools do the grunt work to find the smells and even make recommendations, then employ the expertise of more experienced developers to deoderise. It’s also a good idea to use several metrics in combination that address different, sometimes competing, concerns.

All good things must come to an end they say. Tomorrow (or today I guess) is the last day on my current project. It is no exaggeration to say that it’s been one of the two best projects I’ve ever worked on.

After six months of development I’ve managed to see it through to code freeze and although I’d have liked to see it go live, that’s a luxury I will unfortunately miss out on this time. As strange as that sounds, there is nothing like nursing your own creation into production. I’ve always found production support of my own baby far easier than the actual development effort and certainly easier than supporting someone elses. Watching real, live users pounding away at your product is always good for the soul in what can often be a soul destroying lead-up hehehe.

I’ve laid my fair share of turds in the code base, I just hope I’ve obfuscated them well enough. Everyone has worked really hard to keep the code as clean as possible, employing as many checks and tests as we could. There are always things you wished you could have done differently but all in all I think it will survive the ultimate test - maintenence.

Technically, we used constructor based IoC really effectively. We have pretty good unit test coverage. Integration tests are up there too. Functional test coverage is arguably too good but hey I’ll live with that. We have out of container tests, in container tests, automated builds and deployments, code checks, cruise control, you name. Thankfully no one managed to find a way to squeeze any AOP in though they gave it their best shot :-P. In the last couple of days I’ve even been seen partaking in the writing of perl scripts. Oh and of course nothing gave me more pleasure than to see all 10 developers happily writing declarative business rules!

The people I’ve worked with have been just sensational. I’ve certainly learned a lot more than I’ve given especially from people like Mad Dog Murdoch, Uncle Dave with his little nuggets of wisdom (or as he would put it “corn in the stool”) and of course JRO who never ceases to make me realise why I’m just not cut out to be a “Technical Architect”. Marcus, Trashy, Triggles, Phil, Jimmy, Happy, Big Daz, Bazza, Victor and Vorgahan not to mention my backup singers Andwea and Neats. I don’t know how they did it but they all managed to endure my endless singing (if you can call it that) interrupted by bouts of ranting and the occasional tap dance.

It was gratifying to see customers, testers, BAs, developers and management all working together to get this thing out the door. It hasn’t always been easy but then “nuffin is”. In the end though I think pragmatism (along with threats of violence - just kidding) won through and we all just worked to ensure we got the job done.

I’ve quite a bit to digest and hopefully blog about in the coming days and weeks but before that I’m going to get my last 8 (ahem 10?) hours out the way then jump on my motorcycle for a couple of days and enjoy the sunshine that seems to be blessing this lovely city. Come Monday it’s back to work on a new project. For as a wise developer once said “It ain’t all sunshine and party pies.”

The first time I met James I was amazed to see him sitting mere inches from the monitor. As I moved closer to see what it was all about I saw the text and graphics on the screen zooming in and out. It was enough to make my stomach turn.

James is legally blind. It’s actually not as bad as it sounds (it does come with some perks like free public transport hahaha) and he uses a great bit of software called ZoomText to magnify the display. But the best bit is the Text-To-Speech (TTS) feature. Highlight a paragraph of text (for which there is a short-cut key combination) and Davros reads the text back to you. You can also adjust the speed at which text is read back. I can just understand it at about medium pace but James has it turned all the way up to Chipmunks.

Recently, James wondered what source code would sound like and, to his delight, discovered that not only did ZoomText read the source code back as intelligibly as (perhaps more so than) a human, it even understood CamelCase text, separating the words appropriately.

Like most developers, I guess, I’ve always tried to adhere to one of the longest serving mantras of the development community - to write readable code. As you’ve probably guessed, James and I pair program quite a bit and now that he uses ZoomText to read back our code, never has this meant as much to me as it does now.

As I’m coding away now, there is this little voice in the back of my head (yes I am nuts but that’s a different voice) constantly reading back the statements, constantly reminding me of what it will sound like when James uses his TTS. And I think it’s made a difference. I think because of it I’ve changed the way I name methods, classes, etc., and even the way I structure statements, hopefully, for the better.

I watch with interest as the Rule Engine chatter begins to increase. I truly believe it’s an area much ignored by the great majority of developers.

If you’re not aware, there is a JSR in the works to provide a common interface for integrating rule engines. In its current form, JSR-94 provides little more than a common interface for creating a context/rule engine and marshalling objects in and out. It is trivial to implement this yourself.

The fact is (pun intended) that the JSR provides little more than would result in developing a system that makes use of a business rules engine keeping in mind requirements for testability of rules and loose coupling (a.k.a. good design?). Only the JSR is considerably more verbose.

To illustrate, we have a large number of rules in our current system and a very small but useful set of interfaces:

public interface RuleEngineFactory {public RuleEngine createRuleEngine();}public interface RuleEngine {public void reset();public void addFact(Object fact);public void addFacts(Set facts);public void execute();public Set getFacts(Class type);}public interface RuleEnginePool {public RuleEngine getRuleEngine();public void returnRuleEngine(RuleEngine ruleEngine);}

Add to these a few very light-weight implemenation classes (and some Dependency Injection for good measure) and you have pretty much everything you could need from an integration standpoint.

public class JRulesRuleEngineFactory implements RuleEngineFactory {private final IlrRuleset _ilrRuleset = new IlrRuleset();public JRulesRuleEngineFactory(Reader reader) {if (!_ilrRuleset.parseReader(reader)) {throw new RuntimeException("Error parsing rules");}}public RuleEngine createRuleEngine() {return new JRulesRuleEngine(new IlrContext(_ilrRuleset));}}public class ThreadLocalRuleEnginePool implements RuleEnginePool {private final ThreadLocal _engines = new ThreadLocal() {protected Object initialValue() {return _ruleEngineFactory.createRuleEngine();}};private final RuleEngineFactory _ruleEngineFactory;public ThreadLocalRuleEnginePool(RuleEngineFactory ruleEngineFactory) {assert ruleEngineFactory != null : "ruleEngineFactory can't be null";_ruleEngineFactory = ruleEngineFactory;}public RuleEngine getRuleEngine() {return (RuleEngine) _engines.get();}public void returnRuleEngine(RuleEngine ruleEngine) {assert ruleEngine == getRuleEngine() : "ruleEngine not allocated from the current thread";ruleEngine.reset();}}

We theoretically have plugability of rule engines but to believe that this might be useful or even practical is naive at best.

Unfortunately, (or fortunately depending on your perspective) the biggest part of using a rule engine is in analysing and writing the rules themselves. Granted, many engines use a Rete Algorithm but to suggest that all Rete-based rule engines are the same is akin to saying that two Java applications are the same. JRules and JESS both use a Rete network and both are implemented in Java but the language, tools and behaviour (not to mention performance characteristics) of each differs sufficiently to render the conversion process rather less than trivial.

Surely, few of you would be imagine that a switch from using JSPs to say Velocity in a system of any significant size would be an overnight job. Similary, a switch from Struts to say Tapestry or JSF would be non-trivial. All of these technologies attempt to solve essentially the same problem but all come with a slightly different design philosphy. No matter how standardised the interface, if the behaviour of the system on the other side is different, the illusion breaks down.

Anything that lowers the barrier to entry for those wishing to explore the use of declarative rules is to be applauded however there are far more important problems for an organisation to solve than transparency of the underlying rule engine implementation. Atomicity, testing, managability, education, analysis, configuration, understandability, to list but a few. It is no coincidence that these are largely non-technical.

Many years ago now I read the first edition of About Face by Alan Cooper. At the time, Dave and I were working on an HR application which was subsequently sold to a another company and is still being sold and (I presume) used, today.

Not to ding my own chain (much!) but it still rates as one of the best apps I’ve ever built. I’m sure if I looked at the source code these days I would whince but I still think we made some pretty good technical achievements. However technical merit aside the one thing that really made an impression on me and continues to make an impression was the usability of the application. Not only in terms of business functionality but also the way it simplified the way users performed their day to day tasks.

Alan Coopers most excellent insights were enlightening to me at the time and certainly influenced a lot of the design. But I can’t take credit for the usability nor functionality of the application. No for that we have Dave to thank. Besides having a brain the size of a planet, he is an exceptional business analyst. “Oh we have really good business analysts” I hear you cry. I’m sure you do in which case you’ll appreciate my definition of a business analyst.

Picking on Dave once again, he has an amazing ability to actually analyse a business. By this I mean try to really understand what the customer does; Why they do it; Determine if their current business practices even make sense; How their shiny new software might actually make their life easier; and; most importanty to convey to (AKA convince) the customer why his ideas will work. I’ve heard of CEOs walking away from meetings asking their staff how this guy knows so much about their business. And I know for a fact that he had very little prior knowledge. He just knows how to ask the right questions to get to the heart of their business.

Traditionally (though I have little data to suggest this isn’t still the norm) business analysts will sit with the customers and essentially document what the customer does. Workflow, day to day tasks, etc. From this they then write story cards or use cases (whatever is flavour dejour) that form the basis of the application design. These then go to the developers who consult with the customers on what exactly needs to be done, screen designs, etc. and then off they go to build the software. Unfortunately, the net result is usually a computerised version of some ancient manual system that is barely better than what they had and in many cases worse!

Maybe it’s because the skills of which I write are rare but I’m not sure where the notion that customers should design the software comes from . The idea that customers know what they need (or even want) is just plain ludicrous. In most cases, business people understand what drives their business. They understand what their competitive advantage is and where they could gain new business if only they could do X or had Y. Surely it is the BAs job, nay duty, to come up to speed with the business and from that explain to the customers what would make their life easier. Surely that is where BAs add value. They understand software AND the business.

Even as a developer, I see it as my responsibility to go and talk to the BAs and customers if I see inconsistencies or if I think the application flow or business rules can be improved. I’m sure their are those who wished Id shut up sometimes but I still think it’s worth it. Which brings us right back to where we started. It’s very rare that the end users can design a piece of software that actually does what they need but it’s equally as likely that developers will, on their own, design totally unusable software. So go read the book :-).

Oh, and this paper if you have the time.

Have you ever read any C. J. Date? No? Hardly suprising. In fact most of my colleages couldn’t give two hoots when it comes to data modelling (aka ER modelling) as distinct from object modelling. I’ll admit that for many it’s not the sexiest of topics.

Some months ago, Dave and I were discussing object modelling and database design. Daves assertion was that you need to do a good data model before your object model. I countered that I believed a good object model necessarily produced a well formed and normalised data model. On reflection it seems we were both right.

You see I was under the illusion that a good understanding of relational theory and alegbra was ubiquitous. I naively believed that all software developers knew what a normal form was, what a primary key actually meant and why relational integrity enforced via foreign key constraints IS important! WRONG!

Relational database management systems (RDBMS) are based on mathematically provable theory. SQL for example is a bastardisation (though not Dates exact words) of the relational algebra which in turn is derived from predicate calculus. Interestingly RDBMS and Business Rule Engines have much in common so it’s not suprising that Date, an amazing logician, seems to spend a bit of his time these days writing on business rules. But I digress.

Given this basis in rigourous theory and mathematics, I find it amazing that as an industry of self-proclaimed engineers so few of us seem to have any clues on the matter, preferring instead to stay in the relative adhocery of object modelling where we can all become pioneers by inventing and re-inventing this months rules of thumb or best practices for good OO design.

Many of the available ORM tools force you to either screw your object model or screw your database model and being the anal retentive person that I am when it comes to both topics I find this rather discouraging. Mostly because I’d be willing to bet my left nut that most developers out there don’t even realise that they’ve ended up with either an object model that violates some pretty basic principles of good OO design or ended up with a data model that is just plain wrong.

Granted, DBA’s are supposed to look at our data model and find the flaws but so far as I can tell, most DBAs these days are there to enforce BLLSHT NMG STNDRDS THT N N CN NDRSTND or to tell us what columns need indexing.

I’m all for surrogate keys over business keys, etc. but the fact remains that until the world chooses object oriented database management systems (OODBMS) in favour of the good old RDBMS don’t delude yourself into believing that relational theory isn’t important.

I was interested to see Martin Fowlers recent entry on Notifications. If you’ve ever used Struts (gasp) or similar “framework” you’ll already be familiar with the concept so it’s certainly nothing new but Martin has a fantastic ability to document and explain things in clear, unambiguous terms.

The most interesting thing to me was this statement “You should use Notification whenever validation is done by a layer of code that cannot have a direct dependency to the module that initiates the validation.” and how this relates to the use of a rule engine within an application.

One of the biggest mistakes we’ve seen in using rule engines is to allow the business rules to become dependent on other than the business domain. For example, allowing rules to know or depend on what screen is currently displayed. Business rules should be statements of fact about business information not application workflow or navigation state. As much as possible, we want business rules to survive changes to the form, flow, layout and even number of application/s that depend on them.

Business rules make inferences about the business information (facts) presented to them. Some of these inferences will be new facts for other rules to consume and some will be facts for the caller to consume. It is this second class of facts which we classify as Notifications and that which the application collects and proccesses. At any point in time, some of these notifications will be relevant to the application and some will not. Some will cause the application to alter it’s workflow, screen layouts, etc. and some may safely be ignored. The critical thing to understand is that it is the applications responsibility to filter the notifications.

For example, imagine we have an application that collects data on a customer. The data is collected over N (where typically N > 1) screens according to the business workflow requirements. After each screen of data is collected, the user hits Next to proceed at which time the state of the domain is asserted into the rule engine, the rules are executed, and the notifications processed. Now lets imagine that one of the rules states that a customers date of birth is required. You’ll note there is no mention of a screen here meaning that until the date of birth is filled out, the application will receive a notification indicating some problem with that field. Rather than emed knowledge of the application into the rules, the application instead filters out any notifications that are not relevant. In this case by checking to see if the field specified in the notification exists on the curent page or not. If after filtering, there are no notifications, the application can proceed to the next screen; otherwise a message is displayed and the user cannot proceed. Finally on the last page, the application can check to ensure that there are no unfiltered messages before allowing the user to save. You can even get fancy and have the application take the user directly to the appropriate screen, something that would be difficult to achieve if the business rules were dependent on navigation state.

Another area we have used this approach is with authorisation. We have rules that assert Permissions (a type of Notification) based on the business data that the application can use to determine what a user is or isn’t allowed to do. Again, the rules make no reference to screens or assume anything about the calling application for that matter. They simply state the facts as presented and inferred. The application then checks the results for the existence of the desired permission and if present the user is allowed to proceed; if not the user is denied access to that particular function. This also makes rendering links, enabling/disabling buttons etc. very easy while maintaining the ability to define the rules in purely business domain (ie application-agnostic) terms.

The concerns of a client application are typically to do with workflow and appropriate use of screen real estate. Business rules on the other hand are concerned with statements of fact about the underlying business data. Notifications allow the business rules and application workflow to vary independently according to these different concerns.

I recently attended a half-day seminar thingy put on by Enterprise Java Victoria. During the day we had presentations from Mark Hapner and Gavin King along with a few vendors and some large-scale J2EE shops. Robert Watkins, James and I (apparently I talk too much and need to learn to shut up - like you didnt know that already) were among a dozen or so people fortunate enough to be invited to sit on various panels. All in all it was well attended and I think most people got something out of it.

Most of the talk centred around EJB Three-Dot-Oh and Es-Oh-Ay but there was also discussion on topics such as developer training, heavy versus light-weight containers, Java Web Start, etc. During all this, the one thing that struck me was the distinct lack of audience participation. We (the panels) were pretty much there to give our views (such that they are) on the future of J2EE but no matter how much I or Gavin or anyone else ranted, I kept wanting to ask the audience “so what problems are you having with J2EE?” I’m astonished the question never came up. Then it occurred to me that these people were here to find out what the next big thing in Java was going to be. And therein lies the problem.

If you’ve realised anything from reading this blog (apart from the fact that I’m a loud mouthed opinionated git) it’s that I really do feel we as an industry (being IT) do more to justify our own existence and “innovate” than to actually think about what our customers need. I love playing with new tools and frameworks and whatever else seems to be in vogue at the moment as much as the next developer but I also feel we (yes that includes me) persist in creating the technology equivalent of whiter toothpaste and softer toilet paper. FUD drives our industry as it does fashion. Vendor X or “Thought Leader” Y come up with some new fandangled tools/methodologies/practices. Quite often its not even new. Then these ideas are disseminated to the masses (being we the developers) in IT shops who then, upon seeing this bright shiny new toy, go about selling it to the business to justify the R&D required to work out how we might actually use it. What’s more we often shoot ourselves in the foot by over promising and under delivering.

Pause to blow off steam…

Why does the customer care about the latest technology unless it specifcally addresses some business need? “Why do they even care if it runs on Java or not?” was one of my questions for the throng and didn’t that go down like the Mars Beagle Lander on a bad day! Surely the customers have problems not to do with technology but to do with the way they conduct business. They want to do business more effectively. They want to be more flexible. Shouldn’t we be trying to find out what they do and help them do it better? Feel where their pain is, then go away and work out how we might make it go away. In doing so we would encounter a world of hurt and pain which should lead us to ask the tool vendors to help us out. Instead we end up with BEA telling us that what we really need is to turn everything into a web service and let business users clickedy-click to put together an application.

“So,” I asked the audience, “who here has built their own house or their own car?” amazingly one person replied “yes” to both questions. What about build your own furnitiure or made your own clothes or even serviced your own car? The materials are all readily accessible. Do commercial pilots construct their own planes or astronauts their own rockets? Surely they could, they’re some of the most highly trained people around but that’s not their job, that’s not where they add value.

Just take a look at SourceForge. We are very good at building IDEs and frameworks, etc. Well maybe not the last but certainly IDEs and tools in general. Whether you like it or not, Visual Studio really set the benchmark that all subsequent IDEs had to match and, thankfully, have now surpassed. It’s no surprise that IntelliJ and Eclipse , etc. are so good. We as developers understand developers. We understand how we work. IntelliJ doesn’t get in my way. It does just what I need and no more. It takes away the tedious tasks without trying to do my job for me. What we need is to build end-user software with as much thought as this. Not try to solve cool networking and infrastructure issues over and over and over again.

Sure, SOA and interoperability are laudible goals but they’re hardly new. Whether you like it or not, EDI has been around for decades, CORBA for longer than RMI, telnet longer than HTTP. I’m happy to listen to our Thought Leaders tell me their vision for computing but don’t try and convince me it’ll solve a bunch of problems that quite frankly I just don’t have right now.

I’ve taught martial arts for over a decade now and it’s only just occurred to me that some people turn up because they want to be told what to do. Maybe they lack the capacity to learn for themselves or maybe they just don’t want the responsibility but if I spend all my time teaching and none of it training, my reflexes get slow and I forget all the little things that one learns when actually forced to use technique in a meaningful way.

Neccessity they say is the mother of invention. If our Leaders long ago stopped actually dealing with customers and developing real code then they also stopped feeling the “pain” associated with developing business software. No amount of thinking makes up for this.

Now for something completely geeky so as to fly directly in the face of my previous post, I quite literally just received notification that a Java feature request I had voted for (along with 861 others lol) has finally been closed off.

It may not seem like much but I’ve quite often felt the desire to further restrict the return type of an overidding method and soon, according to the release notes at least, I’ll be able to do just that in the up and coming 1.5 ahem 5.0 release of J2SE.

A clear candidate for this is clone() meaning I will finally be able to do:

public ThisClass clone() {
    return (ThisClass) super.clone();
}

No longer requiring callers to perform the cast themselves.

Of course removing redundant casts is but the simplest and most obvious use. Anyone who’s ever been forced to code on top of a “generic framework” implementing/overriding Object doStuff(Object) throws Exception methods will know what I’m talking about :-).

I guess it comes down to your preference for static versus dynamic typing in many cases and while I’m not averse to a degree of dynamic typing, I do like it when my APIs can be more explicit without the need for all that useful™ JavaDoc.

Lately I’d come to think of Websphere 4 as a steaming pile of crap but then the other day I read an article which changed my mind; Human waste is actually useful for something.

We have an application that runs within the web container against a DB2 database. For development, we use OrionServer which, sensibly, leaves the isolation level at READ_COMMITTED by default. For production, we’ll be using Websphere 4 which gives a default transaction isolation level of REPEATABLE_READ.

Sidebar: There are four basic transaction isolation levels:

• READ_UNCOMMITTED - Let me read anything even if it’s not been committed yet; Actually you should be committed for using this;
• READ_COMMITTED - Only allow me to read stuff that’s been committed; The sensible choice for the descerning developer;
• REPEATABLE_READ - I want to lock every row I’ve read because somehow I think I might want to read the same stuff again with the exact same results in the exact same transaction; Also known as please Sir can I have a deadlock; and;
• SERIALIZABLE - My users are nostalgic for the response times they enjoyed during the good old days of 1200/300 baud accoustic couplers. Yay! Bring on those deadlocks. And the only way to change the isolation level in Websphere 4 is to deploy an EJB and specify the isolation level in the custom descriptor (I assume this is pretty common but no doubt not specified in the J2EE standard - anyone?). All very well and dandy IF YOU HAPPEN TO HAVE AN EJB FROM WHICH TO DO THIS!

breath…in…out

Unable to get Websphere to cooperate in any sensible fashion (no I don’t count EJB’s as sensible) we turn to the database. Ahh good old DB2 you old thing you..hey…hey…nudge nudge…wink wink. What can you do for us?

Bugger all that’s what! Well, not quite. A good-old Google search reveals some interesting stuff. It seems I can go in and set a server-level parameter DB2_RR_TO_RS which basically says “hey mister DB2, if that Websphere kid comes around here asking for REPEATABLE_READ, just smile and say yes yes and send him blissfully on his way but don’t actually do anything. ok?” Even more interesting, we found this solution right on IBMs website for their own Portal Server Software! And what do you know? It worked. No more deadlocks.

So that should be it right? Wrong! The problem with this little solution is two-fold: Firstly, it’s at the server instance not dabase instance level so it affects EVERY database; and secondly; it’s not a supported option in DB2 version 8.1 (works in 7.1) which we’ll be using for production. Instead, 8.1 supports so-called Type 2 indexes which supposedly solve the same problem. The index solution seems a little fragile to me though. I bet my bottom dollar someone will add a new table and forget to add the appropriate indexes. BLAMMO!

Back to square one. We try every combination of locklists and maxlock parameters, you name it. None work and frankly although they would have decreased the likelyhood of deadlocks, they don’t actually solve the underlying problem namely that that REPEATABLE_READ is a retarded default.

But what can we expect? I mean we aren’t doing it the J2EE way. “Please Sir, can I have READ_COMMITTED” we feebly plead. “How can you have any READ_COMMITTED if you don’t have any EJB’s?” Websphere replies cruely.

In the end it looks like I’ll need to dust off the old train book, wrap our Hibernate persistence stuff in a session bean and hopefully get back to doing some load testing.

Even after all that, compared with the performance of OrionServer, Websphere seems to mysteriously turn my P4 into a Casio wristwatch. Given the amount of CPU power required to run it, I think we’re going to need to find alternative means of power.

UPDATE: Having jadded the websphere code, it seems that the default isolation level for Sybase and Oracle is a more sensible READ_COMMITTED. That of course doesn’t help us. It also turns out that once you’ve called an EJB, even a nop method, the database connection is then set to whatever isolation level was set for the EJB! Oh we’re having fun!

As you’re probably aware, some time ago I ported Simian to C# and ever since I’ve been maintaining two versions of the source code. This was becoming tiresome to say the least and recently I’ve been toying with installing VisualStudio.NET under VMWare to see if I can use J# instead. Not only did this idea urk me “just because”, but the thought of installing anywhere up to 3GB worth of software that I would be forced to run under Windows just didn’t impress me in the slightest.

If you’re not up with the latest on mono you’re probably unaware that the site has been overwhelmed for days now since the release of 1.0. Almost hidden within the distribution is IKVM.NET “an implementation of Java for Mono and the Microsoft .NET Framework.” It includes the following components:

• A Java Virtual Machine implemented in .NET
• A .NET implementation of the Java class libraries
• Tools that enable Java and .NET interoperability

So, having just recently emerged the latest packages under Gentoo, I figured what the heck, I’ll give it go.

First off, I tried ikvm. It’s like a java.exe replacement. Where I would usually run:

java -jar simian.jar

I can instead run:

**ikvm** -jar simian.jar

Which will run simian under the .Net framework instead. And it worked. My jar was happily running under .NET with no code changes! But it gets better. The next step was to try ikvmc the compiler. Yes you heard it, a “static compiler used to compile Java classes and jars into a .NET assembly.” So I gave it a whirl:

ikvmc simian.jar

It read the Manifest and determined where my main method was, “converted” all my classes and spat out an executable: simian.exe. So I ran it and once again, it worked first time. It just blew me away! No need to convert to C#. No need to installed Gigabytes of wizard-ware and best of all, the tools run under linux as well as windows.

So for a bit of extra bandwidth required to download the supporting DLLs, I can finally ditch all that lovingly hand-crufted C# code and deliver on just about any platform you can imagine!

I can’t vouch for it’s suitability with regards to your project but it’s definitely worth a look if only for it’s pure geek value. I’m sure yet another layer of indirection is exactly what you architects out there have been looking for. Just think, you too could have a Java application running under IKVM under .NET under Windows under VMWare under Linux ;-P

One of the managers at the client I’m currently working for observed that I hadn’t blogged quite so much recently as in the past and wondered if I was, perhaps, suffering blogstipation. But today I received a good dose of editorial cod-liver oil.

I mentioned some time ago our success in speeding the build with respect to JUnit execution times. That has worked sensationally well for us for some time now. Unfortunately, as more and more tests were added we started getting OutOfMemoryErrors forcing us to generate multiple suites which in turn slowed the build.

Our short term solution was to stop instrumenting the build. This bought us some time but that was in no way considered a viable long-term solution. So finally today, after much distruption to the cruise build, we decided to get to the bottom of it. We already knew that JRules classes have some quirky behaviour that meant special care was needed to ensure it can free all working memory before being garbage collected. So we took a punt and looked at the tests involving rules.

On inspection, it did indeed look as if we had forgotten to clean up. Voila! No more memory errors. But then something struck me. I had just recently implemeted a solution to this for our mainline code for this very problem that ensured we were cleaning up in the finalize method of the same class we were using in our tests. This implied there was something else going on. Something most peculiar.

Unconvinced we had found the real cause of the problem, we decided to back-out the change and try something else. This time we made sure that the objects being used were candidates for garbage collection by explicitly setting the instance variables to null in the tearDown() method. We ran the tests again and bingo! Problem solved. But hang on a second. Surely the test classes themselves are being garbage collected? Surely clearing the instance variable was redundant?

A bit of digging around and we were soon scruitinising the JUnit TestSuite source code. And most fascinating reading it was too. Right there in plain Java for all the world to see: A Vector (how Java 1.1); a constructor that creates and stores (in said Vector) an instance of the test class for each test method it finds; and; a run method that simply iterates over an Enumeration of the instances, running each one in turn.

No wonder we were running out of memory! The bloody test class instances, all 2000+ of them, and the data they were holding on to were never released until the suite had finished.

A bit of decoration and a custom test suite class later and it’s bye bye to our OutOfMemoryErrors. I wonder what the next gotcha will be?

Today I shall depart from my usual software development rants. It’s not often that a piece of hardware tickles my geek bone but I found something recently that did just that.

I have a 1.5Mbps DSL connection to the internet and run everything on a wireless laptop running Gentoo Linux and Windows under VM Ware for the times when I absolutely need to test something on windows.

The only piece of equipment I need to connect to my laptop is my ancient HP Laserject 5MP printer. Or should I say, needed. On Thursday I picked up a NetGear PS101 Mini PrintServer. A tiny (half-cigarette packet-sized) device that attaches directly to the printer and then via a CAT-3 cable to my hub. Granted it’s not wireless (though NetGear and LinkSys do have them) but at AU$115 it’s sensational value for money.

It supports both static and DHCP (the default) IP address assignment. It comes with Windows drivers (if you really feel the need hehe) but importantly took no more than about 5-mins to get working with my CUPS installation on Gentoo. I just needed to work out its IP address and internal print server name and set CUPS to print to: lpd://ip_address/internel_print_server_name

Don’t forget to set your iptables configuration to allow printing on port 515 (printer):

> iptables -A OUTPUT -p tcp --dport printer  -m state --state NEW -j ACCEPT

And you’re done. Sweeeet.

I have previously made a comparison between rule engines (and the RETE algorithm in particular) and SQL databases. Business rule languages are declarative as is SQL, both being based on predicate calculus. Both suffer (or at least have suffered) similar problems in terms of performance and optimisation.

I recall many years ago, tuning my queries within an in inch of their (or my more likely) life. Re-ordering the WHERE clause, changing JOIN conditions, even changing the order in which columns were returned.

Thankfully these days, even the simplest of SQL database engines have some form of optimisation built-in. High-end systems such as Oracle have very sophisticated optimisation techniques. I can pretty much write any old SQL (with caveats) and know that I’ll get at least acceptable performance in most cases.

The RETE algorithm (and it’s successor RETE-II) is amazingly good and rule engines have also come a long way but certainly not as much as ye-olde RDBMS. So there are still some things you need to consider when writing rules.

Without going into too much detail, the RETE algorithm builds a network of nodes representing the conditions of your rules and the matching facts. In general, the smaller the network, the better the performance.

The first thing to note is that any rules sharing common conditons are optimised into a single node. However, with many rule engines, this is sometimes only possible if the conditions are listed in the same order. So for any N rules having M conditions in common, order the conditions so that the first M are the same.

Now that your are conditions are in the same order, you’ll be interested to know that the exact order is in fact important. Because each condition is like an SQL JOIN, you need to place the MOST restrictive conditions first. That is, place the condition that is LEAST likely to be matched FIRST. This is no doubt familiar to anyone who has ever tuned SQL.

Iimagine we’re trying to find two people with the same parent. We could do this (JRules code examples, just ask me if you want to see JESS as well):

?a: Person()Person(getParent() == ?a.getParent())

This has one glaring problem: It’s essentially a cross product! So we need to fix it:

?a: Person()?b: Person(getParent() == ?a.getParent())evaluate(?a != ?b)

Now, as we’ve shown above, the number of conditions evaluated is also important. Anytime we can short-curcuit the conditions, we save ourselves another join. So once again, we can re-write our conditions:

?a: Person()Person(?this != ?a; getParent() == ?a.getParent())

I’ve found these simple techniques can result in the difference between rules running in seconds versus OutOfMemoryErrors!

To be continued…

I’ve just found out that Amazonian women cut off their right breast so as to be able to use the bow better. Guess I’ll stop lusting after Amazonian women from now on!

In other news, an old friend of mine, Mike Lee, was down from Sydney today and passed on a piece of obscure Java knowledge.

He had spent some considerable time debugging a junior developers code, trying to work out what was going on with a static initialiser. He couldn’t work out why it was executing just prior to, and every time, the constructor of the class being called.

Eventually he worked it out. The developer had mistakenly omitted the static keyword from the initialiser block.

Well blow me down, Java has anonymous constructors (well that’s the name I’ve given them at least)! Yup, that’s right, this is perfectly legal Java:

class Foo {
    private final String _bar;
    {
        _bar = "Hello, World";
    }
}

In fact, just like static initialisers, you can declare any number of anonymous constructors and they will all be run (in declaration order) prior to invoking a “regularly” declared constructor.

So, to all you JLS weenies out there who already knew this, you don’t deserve to get a life. For the rest of us mere mortals, WTF?! :-)

Ever waded through seemingly useless JDK JavaDoc screaming “I know getBlah() returns a Blah but what do I actually do with one once I have it?! How do I use the swags of java.io classes? And what in the world would I ever want a StreamTokenizer for?

So maybe you’re a smarty-wishbone-legs and know everything there is to know about using the JDK APIs but, there’s always something new to learn.

Sometimes forums and newsgroups can be a great help but more often than not I don’t feel like wading through 18 pages of discussion to find the answer. Sometimes I just want a succinct example.

Enter the Java Almanac. I’ve found it to be a good source of JDK examples. It generally seems to be pretty up-to-date with new examples added all the time and is a great starting point for developers who are new to Java and struggling to come to grips with the vast number of APIs.

Recently I’ve been treating my ideas as if they were drawn on an Etch-A-Sketch. A deceptively simple concept that has helped me let go of ideas that were leading nowhere but to which I had an emotional attachment that prevented me from considering anything else. Just shake and you’re back with a clean slate. :-)

Remember the good old days when Crystal Reports was going to save the world? That’s right. By about now (2004) every user on the planet would be writing up their own ad-hoc reports straight from the database. All they needed was the database schema and away they would go.

What? You mean your business users aren’t doing this? Really? Say it ain’t so!

No, the truth is it never really eventuated the way we (the IT industry) had envisaged. End users just don’t get fully normalised data structures. FWIW, most people I work with probably don’t understand why 25,000,000 + NULL == NULL so how did we ever expect users to? Oh and let’s not forget the IT manager who has enough knowledge of the system to be dangerous. He has a big picture of the database schema on his wall and knows just enough SQL to build queries that do multiple table scans over the millions of rows of inventory data, causing the DBMS to not so quietly tell every other use of the system to please get nicked :-) The plain fact is that writing reports typically requires as much help from IT infrastructure as to make it an IT task.

So now that Business Rules are looking more cylindrical with that silver sheen each day, some of us naively believe that our so called “Business Users” should be able to code up/modify rules for direct inclusion into a production system all by themselves. To me, this is an even bigger problem than reporting.

All the problems that plague end-user reporting apply. Users don’t understand our lovely, normalised, domain model. They surely don’t understand why they get a NullPointerExceptions when adding numbers. And when it comes to knowing the difference between and and or you can forget it!

What’s worse is that Business Rules are used directly by the application to “reason” on appropriate behaviour. By comparison, with the exception of the table scan problem and of course any bad decisions that might be made based on incorrect data, reporting seems rather innocuous.

Now, if I said to the man (and in this case yes it is a man) who writes the cheques, hey how about we don’t test any of this code before we put it into production, I’d get the sack immediately. And quite rightly so I might add. Application components interact in subtle and non-obvious ways that necessitate large-scale unit/functional and integration testing. Business rules are no different. Actually they can be worse. We have many years of collective experience managing essentially procedural languages such as Java, C, C++, etc. Most developers I know think a lisp is a speech impediment and surely wouldn’t know a prolog if they tripped over one :-)

Just like with reporting, we can try going down the path of writing views and buulding neato tools to try and make this stuff more like human readable languages and structures but when it comes down to it, like reporting, business rules require about the same (if not more) intervention from IT departments when end users write them as when the developers themsleves write them. The “best” tools in the world won’t solve real problems with allowing end users the ability to directly modify business rules. I mean, glasses aren’t much good if the patient is blind.

Business rule maintenence is and should be an IT responsibility. However, the rules must be representable in a way that makes it easy for an end user to verify the translation from written/spoken languages. Appropriate use of the lower-level JRules or OPSJ languages makes this a reality without the need for tools to render rules from/to “plain english”. With a tiny bit of coaching, our business users are finding they can understand the rules sufficiently to know when we’ve made a mistake. In fact this iteration, our business rep has indicated he’d like to try his hand at writing one. No points for picking the irony in that.

I admit to having been influenced greatly by much of Martin Fowlers writing over the years. It’s always thought provoking at the very least but this entry on his bliki made me do a “What The?”

There is no way I (or any other software loud-mouth) has that much influence.

There is no way I can believe that he actually believes this. Clearly, people such as Martin do have that much influence. That’s why they are employed by organisations such as ThoughtWorks. That’s why they draw huge crowds at conferences. That’s why developers the world over buy their books.

So maybe we could re-phrase that sentence? Perhaps it should read:

There is no way I (or any other software loud-mouth) should have that much influence.

But the fact remains that he (and other software loud-mouths) do and as a consequence lots of people will blindly do exactly what they say. The operative word here being blindly.

He is quite right though. People are selective when appealing to authority and people as a whole should take more responsibility for their own thinking.

It’s becoming dull explaining the difference between Encapsulation and Hiding. Maybe universities need to teach English as part of their computer science courses?

**v. encapsulated, encapsulating, encapsulatesv. tr.**1. To encase in or as if in a capsule.2. To express in a brief summary; epitomize: headlines that encapsulate the news.

**v. hide, hidden, hidesv. tr.**1. To put or keep out of sight; secrete.2. To prevent the disclosure or recognition of; conceal: tried to hide the facts.

Encapsulation does not mean that my classes need to have a high proportion of private methods to demonstrate good OO. It means the methods and data on a class should be cohesive - share and participate in common purpose and responsibility. Encapsulation has little to do with making all fields private and adding respective getters and setters.

A class has certain well defined and cohesive responsibilities - think normalisation (1BNF, 2BNF, etc.) but for classes - and those responsibilities are, by in large, public. Read encapsulation. The implementation of those responsibilities is hidden. The fact that some of the implementation lies is private methods is incidental and occurs for clarity and maintainability. Too many private methods is often an indication that a class has too much or wildly varying responsibility.