Igor's Blog

Modeling for Domain Evolution

2009-07-16T13:08:00.010-05:00

A constant goal of mine is to express intent while developing software systems. In the begging, I spent a lot of time trying to create a better Object Oriented Design hoping that the abstraction level would be enough to clearly represent a business model. Of course, this wasn't enough and I continue looking for better abstraction levels.

The next step was my obsession with Design Patterns and Frameworks. Common understanding of the problem, at least among developers helped a lot but business people still couldn't easily relate a system solution to the actual business model.

Continuing in that direction led me to discover the incredibly valuable book - Domain Driven Design and just a little later the somehow related concepts behind an interesting but not very popular framework called "Naked Objects". Domain Driven Design was huge advanced towards expressing intent and introduced an invaluable concept about ubiquitous language.

At that point, the level of abstraction was pretty good in relation to designing domain models, but specifying the behavior of a system was somehow problematic. Again, mostly a change in the perception and the way of thinking rather than technology advances led me to the concept of Behavior Driven Design. Specifying system behavior in an executable way was an impressive step but a little hard to achieve with the current static programming languages that most of the systems were build at that time.

Didn't take me long to discover the disruptive power of dynamic languages like Ruby and all meta programming concepts that came with them. Additionally, I discovered a much different way of designing APIs in the form of Fluent Interfaces. That allowed me to express business intent through a software solution in an almost natural english language form that business people could start understanding.

Interestingly enough, it turned out that even this is not enough because of a simple fact - business people use a natural language only to define different specific languages that better describe their business domains. Well, not surprisingly this realization led me to the concept of Domain Specific Languages (DSL).

After the initial euphoria of designing and implementing Domain Specific Languages, I start looking for the same impressive tool support that I was used to while working with other well established general purpose programming languages. It turned out that there is not a better time for that with current release of most of the main players in this area (Intentional Software, JetBrains, MS Oslo, Eclipse xText,...) delivering solutions for a really disruptive concept summarized as Language Workbenches.

In order to visually represent this path towards delivering better software solutions, I created "My Curve of Abstraction Towards - Expressing Intent & Ubiquitous Language"

I don't know if DSLs and Language Workbenches would be successful in practice, but those concepts would definitely change the way in which I approach and design software solutions. In most of the system that I have built until now I used programming languages to specify a domain model and the behavior of a system. Recently, however, I worked on software solutions that had to be model for domain evaluation. Basically, I used programming languages to define a business model definition and structure via other languages and notations, or so called Language Oriented Programing.

This is actually the topic in my presentation that I gave recently called - "Domain Specific Languages- ... a forgotten interface metaphor with renewed tooling support".

Agile within Constraints: Embrace Kanban Limits

2009-07-15T19:15:00.011-05:00

Every time when I try to introduce Kanban ("visual card/board") Lean system in an Agile team I got the same question - Kanban looks exactly the same as the current index card board, so what is the difference? As I mentioned in my previous post, the main difference that really defines a Kanban system are limits. Logically, the next question to follow is why we need limits.

Well, there is a problem that I see occurring often with the basic index-card task board. There is nothing to prevent accumulating a big pile of story cards that are work in progress. Usually, the bottleneck could happen in multiple places. Sometimes, it is the huge backlog that is constantly and prematurely prioritized, other times it is the familiar pattern of busy developers in the beginning of an iteration and busy BAs and QAs at the end of the iteration. More often than not a big story card queue for testing is left beyond a current iteration without any timely feedback.

Actually, an iteration, which is central part of an Agile practices is a natural time-boxing constraint to how much Work in Progress can be accumulated. However, the time-boxing limit doesn't help in optimizing the overall team workflow and throughput, doesn't avoid premature work.

Let's take a common example of an agile story board:

I see some bottleneck from the pile of story cards in the testing backlog on the picture above but I can't tell for sure if this is a problem. I don't know what the team testing capacity is because looking at the story board I don't see visual indication of the current limit - the time left to the end of an iteration.

So, I have to wait to the end of the iteration to know for sure if there is a problem and I would still not find out where the bottleneck is. Even if all the story cards are completed on time, the basic index-card task board with time-boxing limit show team progress during an iteration but doesn't indicate whether the team has optimal throughput. Also, accumulating a big backlog at some stage of the work in progress (as the big pile of cards in the testing backlog on the picture above) could delay feedbacks (development going in wrong direction until BAs/QAs complete review/testing) and consequently allow for premature work to be done.

At the end, one of the goals of a development team should be to maximize overall team throughput and reduce waste through constantly optimizing team workflow and enhancing teamwork. A Kanban Lean system is one mechanism that can be used to control a workflow and indicate waste in form of bottlenecks.

Kanban is what enables a pull system for just-in-time work. Very simply, there is a queue of work, which goes through a number of stages of development until its done. When someone needs new work to do, they pull it from their upstream queue. Not much different from an Agile story board until here. However, the important element, as I mentioned above, is different type of limits. There are two basic limits – Queue limits and Work in Progress limits.

Queue limits try to achieve just-in-time work and are designed to avoid premature work. The limit should be large enough to keep the team busy (i.e. there is always something in it for the team to start work on), but small enough to avoid premature prioritization (i.e. having things sitting in the queue for too long before they are begun). The later is what pressingly is missing in a typical Agile team story board.

Work In Progress limits are designed to reduce multi-tasking and maximize throughput. The former concept of reducing context switching and performing task sequentially to yield results sooner is very familiar to Agile teams. The later, however, is usually overlooked in typical Agile settings. Maximizing overall throughput by decreasing Work in Progress is often seen in various areas. Simple examples of this effect are traffic jams, where traffic speed reduces as more traffic builds up.
Limiting Queue size and Work In Progress like this can seem unusual for teams, and there is often a worry that team members will be idle because they are having no work to do, but are unable to pull any new work. However, the Queue and WIP limit sizes can depend on type of work and size of a team and should be adjusted to achieve maximum flow and reduce bottleneck. One easy approach is to adjust the WIP limits to the number of available people/pairs in a team. For example, if there are 6 developers in a team and they work in pairs, the size of development Work in Progress limit should be 3. The development Queue size limit should be large enough to keep continues development Work in Progress (so at least 3 story cards) but not big enough to allow premature prioritization on every developer pull (so this would require some adjustment and depends on the Average Developers and BAs Completion Rates as well as the number of BAs. I think that twice the size of WIP could be a good start as a queue limit).

Finally, the consequences of using a Kanban system could be a little surprising or even controversial and could include eliminating the Story Card Backlog (because the immediate queue is the only work of interest), time-boxed iterations (i.e. Sprints) can be eliminated as well because work is pulled as necessary, and consequently estimations can be eliminated because work is not planned into iterations.

Now, most of the questions at this point would be along the lines that if an Agile team isn't estimating or planning within iterations(Sprints), how it can make reliable commitments in front of business product owners.

As a start, Kanban could help you eliminated iterations, estimations and backlogs but can work equally well within a time-boxed limit and with a reasonable backlog size. However, the real answer to that questions is Cadence Lean system, which is an approach to achieve commitment and reliability with a Kanban system and for which I will continue in further post on this topic.

Agile within Constraints: Identify

2009-07-09T23:55:00.005-05:00

All parents know what is going to happen when a kid walks in a toy store. That is why, every parent knows to better set some kind of expectations and constraints (like "…you can choose a toy but only one ..., one toy or nothing.") before entering a toy store (actually any store).

Setting the right expectations and identifying all constraints can help achieve a given goal. Working within clearly identified constraints could make our solution leaner and more efficient.

Interesting example of that idea is the classical children book "The Cat in the Hat". The author of the book, Dr. Seuss, had very clearly defined constraint - the book could only use a vocabulary of 225 words, so that beginning readers could read it. Dr. Seuss used clever combinations of the 225 words and a lot of illustrations to create a very interesting long lived story.

Of course, a lot of similar examples could be found everywhere, especially in technology field. The HTTP protocol over TCP has been commonly criticized to be very constrained one and yet, exactly that constrained nature of this protocol is considered to be the main reason behind the wildly scalable and successful word wide web.

Interestingly enough, most of the advices that one could get about web application scalability would come down to embracing system constraints and rely on them rather than try to overcome them.

In fact, a constraint is a restriction on the degree of freedom you have in providing a solution. However, such a restriction on the degree of freedom could be utilized to convert complexity into simplicity. To illustrate that ask yourself how your perfect house would look like, or where you would like to be professionally in 5 years, or .... Easy questions but hard answers. The reason - prefect implies no constraints but perfect is hard if not possible to achieve. Now, if you include some constraints like the "perfect" house should not be more than 3 times your yearly salary, you would probably compromise somewhere but you would get to a solution much more easily.

Embracing constraints to help provide leaner solutions is something that I am currently excited in retrospect to Agile. In fact, this was the main difference that I discovered while exploring Lean software development in recent years. The first time when I realized that was while working with Kanban ("visual card/board"), which is an essential part of Lean Software Development and for which I would write more in my next post.

URL Crawlability as Architectural Design Goal

2009-03-17T11:40:00.008-05:00

A lot of times during a system design a lot of architectural design decisions are based on architectural goals like performance, security, scalability, usability and so on. I believe that url crawlability (or with other words - RESTful urls) is often neglected (especially in Java, PHP, ASP.Net….) but should be considered as one of the top priorities among all other architectural design goals for every web enabled system.

As usual, Ruby on Rails excels in this area too and I could only learn from it but in the realm of Java, even if you want to do it you would have incredibly hard time. Almost all of the java web frameworks fail to provide a way to work with RESTful urls. A big exception to that is Stripes framework. The REST support is still missing, although it would be trivial to be implemented but RESTful urls are definitely there.

A lot of nice thing could be said about Stripes but better read it from their web site.
The only thing that I would like to add about this framework is that I had the chance to work with it for the last 9 months extensively (after working more than a year with Rails) and I can say that, in my opinion, this is not the best Java framework right now, but probably this is the ONLY Java WEB framework right now (at least from the ones that I have evaluated – and this list is very long!).

I went a little off point but I just wanted to say that currently I consider url crawlability/RESTful urls as one very important architectural design goal with a lot of benefits coming out of it and my first criteria evaluating any web framework is whether it could support RESTful urls or not.

Is this an important consideration in your design?

What I am excited about applying in the realm of software development lately

2009-02-23T21:07:00.003-06:00

Recently, I had to answer to the questions:

What insight have you had lately in the realm of software development that you are excited about applying?

The answer turned out to be a little longer which made it a good candidate to post on my blog. So, there it is:

Well, there are a lot of interesting ideas that I am passionate to apply or improve in the technology area. Most of them are a few years old and there are successful examples of applying them, but still the majority of IT industry doesn't embrace them either because these ideas require a lot of effort and discipline or because it is mostly different mindset and vision.

If I can summarize most of the ideas in one sentence, it would be – using the web as a platform coupled with lean and lightweight business model to give the ability to strategically position a company as an middleman or automated intermediary between the user and his or her online experience, between customers and producers, between data providers and data consumers.

That includes a high accent on remixable data sources and data transformation, allowing users to control their own data and at the same time use their collective intelligence to generate new content or improve on the quality of existing data. (representing data in different formats –html, xml, json .. and on different platforms – mobile, browsers, desktop, real time positioning…., gather comments, ratings, reviews, adding own content, products…)

Support lightweight programming models that allow for loosely coupled systems and enabling constant changes and enhancements. I believe that by using lightweight frameworks, simple architectural design and more dynamic type programming a technology team could enable a business model to be more flexible and efficient. (conventions over configurations, simple web services –REST, syndication of data, autonomous distributed systems with smart ends )

For this to happen, I would think that the technology part of a company should concentrate on the core competency and re-use data or infrastructure services of others for everything else (storage services, virtual serves, emails, cloud computing and so on.)

Further, I think that businesses would have hard time in today's very competitive global economy if they don't go far and beyond for their users. From a technology point a view, I am very fascinated to implement features like real time monitoring of user behavior to find out what is valuable for them and what could be improve. Implementing features that would allow to enable and cooperate with users and not to direct or control them – simple user interface, intuitive searches, intelligently customizing application based on users behavior and data consolidation, different data formats, widgets, web service interfaces, content syndication and so on.

Last but definitely not least, I am very passionate about software development practices and methodologies. Exciting ideas that I would like to try or improve on are – extremely frequent releases, different types of collaboration and intelligent gathering of various project information including natural processing language algorithms to improve communications and common understanding.

Well, what are you excited about in the realm of software development recently?

The Curve of Development Experience

2009-02-22T23:13:00.005-06:00

I start realizing recently that there are lots of debates about software best practice, design patterns, different frameworks or libraries. What happens usually is that inexperienced software developers are usually obsessed (I was) with learning design patterns and the best frameworks out there.

After a few years constantly trying to apply every possible design pattern or framework to any problem, I start realizing that I spent more time fighting with those frameworks than I benefited from them. Not to mention that using design patterns and so-called best practices could severely increase the complexity of a given code base if not used in the right context.

At some point, you start asking yourself not only about the benefits but also about the negatives that a given framework, a design pattern or a best practice solution could bring. At this point, you are more inclined to go with the simplest possible solution – build whatever you need by yourself.

Of course, sometimes you see more values in some frameworks or using design patterns and you start playing very balancing game. So this is what I call "the curve of development experience".

Observing Behavior

2008-01-28T21:24:00.000-06:00

I had to write very simple counting method today. I started with a unit test that pretty much executes a method (which increment a count) and asserted that the count was incremented. This is the test that I wrote:

public void testShouldRecordSystemException(){
     exceptionMonitor.recordSystemException();
     assertTrue(exceptionMonitor.getCurrentCount());
}

and this is the code that I was intending to write:

public void recordSystemException() {
storeNewTotal(++currentCount);
}

The problems with this test are:
- exposing internal state and breaking encapsulation with getCurrentCount() without anybody using it. (Yes, I know that I can make it protected instead of private, but this is not good enough for me. Changing private methods to protected in order to be tested is a topic, hugely discussed already in the blogsphere and I don’t have too much to add. I share the opinion that exposing private methods to protected just for testing looks like design smell and I try to avoid it. However, from time to time, I could make a method protected just for testing if this is the most pragmatic thing to do).
- This is pure state based testing. Although, sometimes state based testing is the better solution in many other cases verifying behavior is leading to much better design.

As a result, this small test reminded me about one presentation that I gave in our office about a year ago. The presentation is about “Mocking Abuse” but the final point of that presentation is that there is a middle ground between state base testing and interaction base testing (using mocks) that I call “verifying observable behavior”.

What that means is that this test should be changed to observe the behavior instead of verifying the object state:

public void testShouldRecordSystemException(){
         assertEqual(1, exceptionMonitor.recordSystemException());
}

What happened here is that the method recordSystemException() changed its return type from void to int:

public int recordSystemException() {
   storeNewTotal(++currentCount);
   return currentCount;
}

The first question that comes to mind is how to verify that the method executes correctly because you can pass the test above with implementation like:

public int recordSystemException() {
    return 1;
}

This is where the thinking about testing state vs verifying behavior should change. In practice, in order to verify behavior, we perform series of steps. Similarly, in mathematics we have so called “Mathematical induction”. Very simplistically put, a given statement is true for all natural numbers if the statement is true for 1, 2 …. n, and n + 1. Similarly, just one verification for verifying observable behavior is not enough:

public void testShouldRecordSystemException(){
     for(int n = 1; n< 5; n++){

       assertEquals(n, exceptionMonitor.recordSystemException());

     }
}

Now, this test should make me implement the right behavior.

That is not a ground braking technique or something not widely known. This is just a change in the way of thinking about design and testing. What is more, in functional languages like Ruby and Python this kind of thinking is encouraged from the fact that every method (function) has always a return value, which allows for observing the behavior of every method.

So, back to the presentation that I mention earlier, it is about something similar. Generally, in our desire to avoid state based testing (which sometimes leads to poorer design, braking encapsulation and so on) we go to the other extreme with interactive testing using mocks (exposing implementation being one of the negative consequences). I already wrote way back about this development behavior in Stubs or Mocks and State or Behavior.

This presentation is building the case on top of those two blog posts that mocks are not the new silver bullet and there is a middle ground between state based testing and interactive based testing that I call verifying observable behavior.

Mock Abuse!

As usual, any feedbacks are welcome!

Job Title

2008-01-28T21:17:00.000-06:00

For a long time, I am struggling to find my job title (yes, we put whatever titles we want in our company). Coming from a highly business (financial) background, I had hard time defining the title of my position in the IT world.

I can’t call my self true “Software Engineer” because this title applies working in very technically oriented domains. “Software Developer” sounds good but since it is widely accepted term, it means different things to different people. Finally, I picked up “Business Developer”. This title was supposed to mean that I am very interested in solving business problems with any means possible (including prioritization, optimization of business workflows and processes and so on). Most importantly, however, it is about abstracting business models and managing the complexity of a business through a sophisticated use of software as a tool.

Yes, you can imagine that this didn’t work out the way I intended to. Almost of all of the people thought about “Business Developer” to mean a person who develops the business of a company in a way of increasing the market share, market awareness and finding new clients (or something like that) .

Later on, I realized that although sometimes I do provide small business optimizations and improvements most of the time I just solve pure software problems. Very often, I am solving those types of problem through heavy use of Google search. I search for similar problems and how other people solved them or didn’t solve them and then try to modify and apply the solution for my problem at hand. That’s why I decided that my title should be not something like “Java Developer” or “Ruby Developer” but rather “Google Developer”, meaning I can solve the problem in any language and tool with some extensive research on the web.

Well, “Google Developer” probably means that I am a developer working for Google, so logically the next one coming to mine is “Search Developer” or “Research Developer”. As you can see, both titles have different meaning than what I want to imply. For a while, I just give up and I came back to “Software Developer”.

Not far ago, I was involved with a huge enterprise project. What I reconfirmed there was that usually the business people give requirements according to which a new software system should be built. However, based on many miscommunications, misunderstandings, different context, simple mistakes, a lot of compromises and the amount of time that takes to build an application the abstract business model together with the business workflows and processes built by the (gigantic) software team are not exactly as the business envisioned them (and sometimes even dramatically different).

Very often the business (since it spent so much money for this) is forced to use and adjust to this new (different) business model. After a while, the whole business behavior of a department or an organization is determined by this new business model.

So, there you go! It came out to me that no matter whether the business likes it or not, I am building and redesigning the business model, workflows and processes while building new software business systems. Based on this reasoning, I am now considering the titles “Business Modeling” or “Business Transformer”.

And yes, they are still not on the target. “Business Modeling” is very good but it should be “Business Modeler”, which doesn’t sound good. “Business Transformer” sounds more like Robocop or something like that.

Finally, I am back to just “Software Developer” whatever this means. It is difficult to escape the common understanding (misunderstanding) for what you do or want to do.

Any other suggestions or similar thoughts?

Filtering has_many Associations in Rails

2007-02-22T23:11:00.000-06:00

... intro

A few months ago, I read Jamis Buck's blog entry Extending ActiveRecord associations. It's about filtering and memorizing the results for has_many association based on some conditions. I found this to be useful and I bookmark it. Sure enough, it happens that I just needed similar functionality. However, it didn’t work exactly as I thought. I decided to post it here, so if I am doing something wrong, hopefully, somebody can correct me or in case somebody else has similar issues.

So, this is an exact copy form Jamis’s blog entry of :


class Project < ActiveRecord::Base
  has_many :tasks, :dependent => :delete_all do
    memoized_finder :active, "status = 'active'"
    memoized_finder :inactive, "status = 'inactive'"
  end
end


class Module
  def memoized_finder(name, conditions=nil)
    class_eval <<-STR
      def #{name}(reload=false)
        @#{name} = nil if reload
        @#{name} ||= find(:all, :conditions => #{conditions.inspect})
      end
    STR
  end
end

... evaluation

I created a file “/lib/ memoized_finder.rb” and included it in “/config/environment.rb”:


# Bootstrap the Rails environment, frameworks, and default configuration
require File.join(File.dirname(__FILE__), 'boot')
require File.join(File.dirname(__FILE__), '/../lib/memorize_finder')

Rails::Initializer.run do |config|
... ...

as it was mentioned in the comments for this post.

Then, I specified the expected behavior using rspec:


context "Project has many active or inactive tasks" do
  fixtures :projects, :tasks  
  
  setup do
    @project = projects :project_with_task
  end

  specify "should filter out active tasks" do
    active_tasks = @project.active_tasks
       
    active_tasks.size.should_be_eql 2
    active_tasks.each {|t| t.status.should_be_eql 'active'}
  end
end

When I ran the specification for the first time, I had:
“undefined method active_tasks for #”

... investigation

I didn’t expect that. I mean many people commented on this blog entry but nobody mentioned something about that. Well, I didn’t know what I am doing wrong (I still don’t know what I am doing wrong) but I did a quick search for more on ‘class_eval’ and found this blog entry: http://neeraj.name/2007/01/31/ruby-class_eval-in-detail/

This example from that blog led me think that I need the class object, since the class_eval method starts with class object. This is excerpt form that blog:


def add_method(obj)  
  # Get the class of the object.  
  obj.class.class_eval do  
    # Add a new method to the class.  
    def to_string  
      to_s  
    end  
  end  
end

... action

So, I introduce clazz parameter to the module:


class Module
  def memoized_finder(clazz, name, conditions=nil)
    clazz.class_eval <<-STR
      def #{name}(reload=false)
        @#{name} = nil if reload
        @#{name} ||= find(:all, :conditions => #{conditions.inspect})
      end
    STR
  end
end

I had a failure again, but this time I had the method active_tasks defined. The problem was that it didn’t recognize find() method. This make me believe, that this module should be probably included in the Project class somehow, but I am not currently aware of how.

... revelation

Anyway, the problem is not only that it can’t recognize find method, but that the find method should be executed on the Task class, which is from the other side of the has_many association. So, I passed this class as a parameter as well:


class Module
  def memoized_finder(clazz, name, conditions=nil, condition_class=nil)
    condition_class ||= clazz
    clazz.class_eval <<-STR
      def #{name}(reload=false)
        @#{name} = nil if reload
        @#{name} ||= #{condition_class}.find(:all, :conditions => #{conditions.inspect})
      end
    STR
  end
end

Well, this time the specification executed property but didn’t meet the expectation, since the number of active task for this object were more than the expected ones. This was due to the fact that the method that was specified would’ve retrieved all of the active tasks for all projects and not only for the current one. Specifying the expected behavior first was beneficial again! Without this specification, I could’ve thought that I am done and I would’ve moved forwards.

... modification

So, I have to send the id of the Project as part of the sql condition. Somebody had the same question from the blog comments - how to send the id of the Project object, since if you specify id in the association block code, this would not be the id of the Project object. The suggestion is to use escape character. The code should look like this:


class Project < ActiveRecord::Base
  has_many :tasks do
    memoized_finder Project, :active, "project_id = \#{id} AND status = 'active'", Task
    memoized_finder Project, :inactive, " project_id = \#{id} AND status = 'inactive'", Task
  end
end

Well, this didn’t work since the expression \#{id} was evaluated in the condition to #{id}. This is because we have: #{conditions.inspect} in the eval_class method evaluation. I am not sure why we need the inspect method, but once I removed it, it start working.

... actually

I just want to mention here that I don’t use strings for defining the status of the tasks (or whatever my actual class is). Instead, I use enumeration through the “has_enumerated” rails plugging. I prefer to work with reference tables for things like those and avoid duplication in the code or in the database. So, I have additional class:


class TaskStatus < ActiveRecord::Base
  acts_as_enumerated
end

... finally

This changes the code in the Project class to:


class Project < ActiveRecord::Base
  has_many   :tasks do
    TaskStatus.find(:all).each do |status|
       memoized_finder Project, status.name.downcase.pluralize , "project_id = \#{id} AND task_status_id = #{status.id}", Task
    end
  end
end

... epilogue

I am not sure this is the best solution but it is working for me and somebody else could find it useful. Otherwise, you can comment on how I can improve it. Thanks!

Is Java Back in the Game!?

2006-05-30T23:43:00.000-05:00

Well, I don’t believe so. However, the JVM could have some chances. Let me step back a little after such a flaming introduction and present various and mostly unrelated points.

What is all about?

As many others in my situation for the last year or so, I have been working in the world of Java but daydreamed about exciting and shining worlds like Ruby or its enjoyable "resort" Ruby on Rails. Although I have some issues with some of the perception in that new world, most of the innovations started happing there because of the Rails explosion.

...is there any hope?

Looking more and more in my old world, I started realizing that I, as well as many others would not be able to escape from it, at least not any time soon. Enterprise software development is more about politics and questionable higher level decisions than real values, passion about what you are doing and ability to "get real".

...alternatives?

WOA, REST and “are web services become a web services again” look promisingly exciting. It is not so much about language and technologies, it is about new way of thinking, designing and using systems. Sure, there a lot of interesting innovation there, but most of it is overtaken by extremely complex specification and vendor implementation of “enterprise” tools and frameworks for generating unmanageable code that doesn’t work as expected.

… actually, it is even worse...

Although there are some important battles that could be won, the overall war with the “old enterprise world” is not going well. The hope is fading away even faster if we combine the application world (stand alone applications) with the integration (service oriented) world where the wide spread opinion is that we need even more vendor tools that solve magically all our problems.

...!?!?!?!?!?!?!?!?!

Well, the Ruby and ROR outburst was big and definitely started a (r)evolution:
From a blog comment:

“Java community was so tied up with doing things the 'right' way. People won't change their way of thinking before something else comes up which shows them otherwise. So thanks to RoR for opening some blind eyes in the conservative community.”

However, Java enterprise world has grown too much to be overthrown too easily or fast enough. What is more, Ruby/Rails is a (r)evolution but it is still not powerful enough:

- “My pet peeves with Rails were the same like most other people back then: i18n, oracle support, integration with web servers wasn't as simple as deploying a WAR file.”

- From Rails performance tuning presentation:

“Ruby interpreter is sloooow:
Deeply rooted in 60’s technology:
• no byte code, no JIT, interprets ASTs directly
• doesn’t perform any code optimization at compile time
• GC performance is erratic

Ruby’s GC collects and is garbage!
Designed for batch scripts, not long running server apps
• tries to minimize memory usage
• simple mark and sweep algorithm
• uses malloc to manage contiguous blocks of Ruby objects”

- From “Ruby threads are week":

• slow as hell, threads tend to die unexpectedly
• documentation and sample code limited to a handful of pages in the Pickaxe (lack of people using should have been clue #1)
• non-OO (in Ruby! an int in Ruby is an object!), all examples use code blocks, subclassing Thread can have unexpected (to me, with my Java threading biases)
results
•many core libraries, and more painfully (in that I lost several weeks of my life, and some hair) ActiveRecord aren't thread safe.”

(Well, Hibernate session is not thread safe either but this is different story.)

While Ruby/Rails hit Java very badly (C# as well to some degree but rapidly increasing the pressure with C#-3), the JVM is holding almost indestructibly against Ruby VM and doing pretty well against CRL as well.

So, if Java/JVM world is still sustaining big hits from outside (even from .NET these days) can it sustain a (r)evolution from inside?

...wakening the sleeping giant.

“In many respects Groovy is the best combination of the Java platform and dynamic language design. Groovy has been quiet, but like a sleeping giant Groovy will awaken and effectively take over as the dominate dynamic programming language…..”

I omitted the last part of the sentence “…after it becomes a JCP standard.” because I disagree with that part. Although JCP standard could help with the adoption from the conservative corporate/enterprise world I think that standards can actually make Groovy worse. Anyway, it sounds exciting. Let see more.

Groovy: The Sleeping Giant, an Elaboration

At first look, Groovy looked to me as Ruby without the Perl with obvious Java perception.
Sounds very attractive to me so let me take deeper look into the language:

Mix and match - interfacing with java and using the enormous amount of Java APIs and libraries directly.

“But, it is there where Groovy's true strength lies in its ability to cleanly interface with Java.” Basically, we can use existing java classless dynamically, test or extend them directly in Groovy. This is one of the best features in Groovy. We can introduce Groovy incrementally on old/new projects or just for specific tasks like configuration, tests and so on. Actually, test are very compelling candidate to be written in Groovy:

- faster feedback (removing the compilation step)
- test code doesn’t have the same strict requirements as the application source code.
- much more productive when doing exploratory testing or scripting behavior specifications of a system.
- writing test in DSL is very appealing.

Another good feature here is the ability to extend interfaces. It is kind of nonsense for a dynamic language but it is very beneficial to provide Groovy implementation of existing Java interface and use the Groovy implementation in the java code.

Ability to create DSLs with Groovy.

“In addition there are other, largely undocumented, reasons why Groovy has huge amounts of potential:

Meta-programming - Groovy has the ability to inject methods, constructors, properties, fields, etc. into ANY java class at runtime. Most of what AOP is used for nowadays can be achieved with Groovy's Meta-programming capability. No byte-code manipulation a la AspectJ, this is awesomely powerful and in combination with Groovy's support for closures and builders it makes it possibly to create mini-DSLs

Power Features - Although there has been a lot of argument about the optional parenthesis, and the optional semi-colons etc. There are very good reasons for these things being optional: For combining closures and method calls to allow builder syntax. Builders and closures are very powerful, especially when you consider you can pass the same closure to a multiple different builders. Why you say? Well one may output a tree structure as XML another might render a GUI from it, I'll leave the rest to your imagination.”

“… the rest to your imagination”. Ruby does have similar features, but this is one of the first things that I looked for in Groovy. Ability to create DSLs is actually the real power of Ruby and it looks that Groovy can do it too.

... is that all?

Of course not, but these are the most compelling features that made me look even deeper into the language. There is a lot more at the Groovy web site. Especially impressive (at least for a Java guy) are:
- Grails Object Relational Mapping (GORM)
- Groovy SQL
- GPath
- Groovy Markup
- Groovy SOAP

This is it!

I feel very excited and I think I like the name Groovy a lot. I am going to play around with Grovy. The most obvious choice is to start with Quick Start from Grails framework.

Grails is inspired by Ruby on Rails and built with Groovy. From the Grails presentations:

Why Groovy?
- Meta-Programming
- Closure Support
- Syntactically Expressive
- Java Integration

Built on very compelling and solid foundation:
- Spring IoC, MVC and WebFlow
- Hibernate
- SiteMesh

Are you familiar with these frameworks? Me too!

When working with Grails, you don’t understand that these frameworks are there but if you want more flexibility (configuration), you can always use them directly.

The reality...

Downloaded the framework, set up a home variable and I am ready to go.

… the build.

Positives:
- It is working as expected (as in RoR).
> grails create-app

create-app:

[input] Enter application name:

Negatives:
- Ant! This was great a few years ago but it is now clear that xml is not the best way to write build files. Groovy could’ve eaten its own dog food and create something similar to Rake (probably something like GrAnt!? It is not too late though.).

- I can’t specify the name of the application directly instead, I have to wait to enter it.

... the domain. Oooh, this could take a while.

Positives:
1. I can optionally define property types in a domain class:

class Book {
@Property Long id
@Property Long version
@Property String title
@Property String author
}

The @Property thing is not the best way to do it but I can live with that when I have the options to define types for domain object properties.

Why the option to define Types is so important to me?
This is something that is not possible in Ruby and it is interesting to explore whether it could be beneficial in Groovy in some way.

1.1.) Domain Driven Design

This allows frameworks like Grails to use an approach where everything is driven by the application - you create the domain class first and then Grails will create the different database objects.

This is absolutely the opposite of the RoR approach where you are leveraging an existing database object and creating the domain class on top of it. One of the reasons is that you can’t optionally specify types in Ruby. Since the current relation databases work with types, the only choice is to go from the stricter to the more loosely couple system, or db-to-ruby. Well, this is an issue only if you don’t fill comfortable defining your domain model in a database schema.

It is still an issue at least for me but I believe that the second point is relevant for almost everybody.

1.2.) Powerfull IDEs and refactoring

“The fact remains there are still huge benefits to compiled languages such as Java especially in large systems where the benefit of refactoring and powerful IDEs becomes clear.”

I am not very sure for the compiled languages but the refactoring and powerful IDE do matter enormously. Unfortunately, you have to give up on this for dynamic languages. There is really impressive progress with IDE’s like RedRails. However, the dynamic nature of the language constraints a lot what such IDEs can do. The problem is that a tool or many times, even a person can’t figure out what is the type of a variable.
Example:

class Song{
length
name
author
}

Is the “length” a string or an integer? Is the name a string or is a type Name? Many times, you don’t need to know but some times it would’ve been nice to know. What about if you can mix statically typed with dynamically typed? What about if we have the option to specify a type if we feel like doing it? Do we get the best from the both worlds? I don’t know but we can do that in Groovy:

class Song{
Integer length
String name
Author author
}

The option to declare the type (at least in our domain objects) can bring the power of IDEs (refactoring, auto completion and so on…) into dynamic languages.

2. Domain objects don’t extend any base framework object.

Positives:

This one is so obvious by itself that I am not sure what to add more.

Negatives:

2.1.) Grails generated toString, hashCode and equals method as well as id and version property. The method basically uses only the id property in an object.
This is good to have but the problem with generation is that you have to maintain afterwards and usually this is clear indicator of typical code smells. Let say that I generate 20 domain objects, all of them has the same three generated method as well as id and version property. Let say I use a lot of caching and I have problems with stale data and uniqueness of objects. I see the version property in the objects and decide that the id plus the version number define the uniqueness of an object. Oh, boy… I have to go and copy and paste the three methods with included version property to all 20 classes.

One way to refactor this is to have base object, something like “IdentifiableObject” where we can define the id and version property as well as the three methods defined above. I know that the framework creators don’t want to create bad association with having base objects but the difference here is that the base object is not part of a framework.

Having a base object like that could be beneficial in situation (I have encountered very often) when you want to perform an operation to all entity objects (identifiable objects) and disregard the value objects (entity and value definition according to Domain Driven Design book).

2.2.) Another negative was that there was a unit test generated but it was empty. I didn’t see test for the three generated methods mentioned above!?

controllers:

Is there anything to be said after I saw this?

class BookController {
@Property scaffold = true
}

which auto-generated at runtime the necessary actions and views:
• list
• show
• edit
• delete
• create
• save
• update

I don’t know if you spotted but the controller again doesn’t need to extend a framework base class. Good to mention here that you can inject services in the controller (through Spring although you don’t understand this) just with using naming conventions.

view.

The view is similar to RoR but without the scripting code inside. Everything is a proper xml which is very nice. On a different topic, I would much prefer something like Google Web Toolkit at this point but this is different story.

….conclusion.

There are lots of nice things in Groovy or Grails but there are many not so nice too. A short list (but very critical) includes: not very good error messages, currently not debugging support, not IDE support (…yes, there is something there but at least for Eclipse the plugging is far for even reasonable solution).

All these issues could be resolved but the biggest issue for me is that there is still no well defined community with clear vision and commitment behind Groovy. One example is that I tried to look for mixins in Groovy (a very important feature for me) but the only thing that I found is a proposal to include them in the language and nothing else.

Despite that and the fact that some frameworks integrate already with Groovy including Spring 2.0, RIFE, NanoContainer, and Drool, I am cautiously optimistic about Groovy and Grails and I will continue my further exploration.

For the two people left that still reading this long post, do you file like Grooving?

How do you keep the build time low?

2006-05-29T15:17:00.000-05:00

I was reading: http://jroller.com/page/peter_pilgrim?entry=my_javaone_2006_conference_second1

where Rod Johnson was quoted:

“It is possible to write unit test for applications in J2EE that run under five minutes. If your unit test take place longer than this, then it is wrong.”

I little more relax time is stated in Martin Fowler's Continuous Integration article:

“For most projects, however, the XP guideline of a ten minute build is perfectly within reason. Most of our modern projects achieve this. It's worth putting in concentrated effort to make it happen, because every minute you reduce off the build time is a minute saved for each developer every time they commit. Since CI demands frequent commits, this adds up to a lot of time. “

I completely agree with both of them although I would prefer the five minute constrain. Slow builds could have huge impact on the productivity of an Agile/XP team. Some of the reasons for slow build times are if we use database fixtures (as the accepted practices in RoR) or if we don’t bother to use stabs/mocks when appropriate in our test. All of that could be disregarded by many as unnecessary complication but from my previous experience on various projects using db fixtures or not using stabs/mocks have been an issue at some point or another.

An example is the fact that the build for my current project that has been continuing for the last year and a half, on average 6-8 developers, more that 1000 classes, about 100 domain objects with up to 83% unit test coverage (at times :-)) is taking just a little more that 4 min and about 1 min of that is xdoclet generation. The real time is 3 min!!! It is not even a pure unit test build. Our build includes testing the persistence for all domain objects with the database that would be in production as well as the instantiation of Spring IoC container inside the unit tests.

In the beginning we didn’t use stubs/mocks and everything was “freely” accessing the database. After the second month of the project, the build (probably about 20-25% of the current size of the project) was taking already more than 4 min! I have seen/heard about many projects where the unit test build is taking from 30 -1.30 min. I strongly believe that built times like that are unacceptable for an XP team.

I haven’t had the chance to use RoR on something more than a personal learning projects and I wonder how much time is a build taking on an average enterprise project using RoR or how do you overcome this potential problem in RoR projects?

Leverage Application Framework Integration with Spring

2006-02-24T23:15:00.000-06:00

Written by: Igor Stoyanov and Dragos Manolescu

ABSTRACT

This article summarizes our experience in simplifying application frameworks integration using Spring framework as well as provides some overview of Spring framework. We will try to show that Spring is a powerful framework, which helps you build a loosely coupled flexible architecture. It helps in decoupling components and greatly simplifies a software project. By using more pragmatic approach, Spring can boost considerably developers productivity and efficiency. In very simple and elegant way, Spring framework can address most infrastructure concerns of typical applications.

KEY POINTS
Spring Framework provides:

ability to build loosely coupled architecture
simplify application development
leverage application framework integration
reduce complexity
improve testing

STRUCTURE OF ARTICLE

Intro: Common problems across many applications provided fertile grounds for software reuse. Using frameworks can help solve these problems and give you better productivity
and helps developers focus on solving business problem to deliver more business value to customers.
Spring Overview: Introduction to most important feature of Spring relevant to framework integration and reducing complexity of an application.
Using Spring Framework Integration: Greatly simplifies the integration of independently-developed frameworks within a single application. Spring reduce complexity by coding against interfaces and not depending on application servers.

Intro

Building business applications entails solving two types of problems. Some of these problems are unique to each application. For example, a health care management system catering to teachers has to accommodate coverage periods that span over an academic year rather than the typical calendar year. It also has to accommodate plans negotiated at the school district rather than at the employer level. These characteristics make it different than the typical heath care management system.

Other problems are common across many applications. For example, applications that have a presentation component must deal with user interaction. Applications that store objects into a relational database must deal with object-relational mapping. And so on.

The problems common across many applications provided fertile grounds for software reuse. Code reuse techniques such as class libraries and frameworks already provide the building blocks for many recurring problems. Their availability shortens development times and helps developers focus their energy on the unique aspects of their applications.

As the software platforms such as J2EE and .NET mature, the number of frameworks available for them increases. Frameworks represent skeleton applications that tackle specific problems. Examples include (in no particular order) application integration frameworks; common services frameworks; messaging frameworks; application architecture frameworks; connection frameworks; reference data frameworks; persistence frameworks; logging frameworks; caching frameworks; authorization frameworks; portal frameworks; rules management frameworks; optimization frameworks; business process management frameworks; data management frameworks; presentation layer frameworks; and so on.

With an increasing choice of available frameworks developers face the problem of using more than one framework in their applications, each of which addresses a specific problem. However integrating many frameworks while following good design practices and maintaining consistency is challenging. So how do you build applications that combine several frameworks?

Lightweight Containers

Since frameworks provide large scale reuse, their usage typically affects the application’s structure (i.e., it has architectural impact). Until recently developers had limited options for providing integration among objects and frameworks within their application. In J2EE the only framework that partially addressed some of these issues was the Enterprise Java Beans (EJB) specification. For several years EJB represented the only framework for managing application tier objects.

The specification covered many nice features, such as declarative transaction in container management of business objects. However the difficulties related to managing and testing business transactions as well as other problems caused the emergence of alternatives: the lightweight containers.

These lightweight containers take some of the successful ideas from EJB to bring a greater level of sophistication to non-EJB architectures. They are similar to EJB architectures in being centered on a layer of managed business service object. However, this is where the similarity ends. Instead of running inside an EJB container, business objects run inside a lightweight container. Lightweight containers manage the business objects as Plain Old Java Objects (POJO) running within the container.The most popular lightweight containers are Spring framework, Pico container and HiveMind.

Lightweight containers employ Aspect Oriented Programming (AOP) interception to deliver enterprise services. For example, we can have a business service implemented as POJO that doesn’t have the responsibility to perform transaction management.

class EmployerService {
public Collection findAllEmployers(){}
public void save(Employer employer){}
}

Then we can declare the methods of the service to be intercepted via AOP and be transaction managed. A configuration file will look like this:

<bean id="transactional-employer-service" class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean">
<property name="transactionManager">
<ref local="transactionManager"/>
</property>
<property name="target">
<bean class="EmployerService"/>
</property>
<property name="transactionAttributes">
<props>
<prop key="save*">PROPAGATION_REQUIRED</prop>
<prop key="*">PROPAGATION_REQUIRED,readOnly</prop>
</props>
</property>
</bean>

We can define different transaction mangers for the "transactionManager" property in "TransactionProxyFactoryBean" without any changes to our "EmployerService" class as well as define the transaction propagation strategy for the different methods in the same class.

Unlike EJBs, service objects managed by lightweight containers don’t usually need to depend on container APIs, meaning that they don’t need to inherit from a specific framework class. This makes the (re)usable outside of the container, which in turn greatly simplifies testing and improves reusability. If transaction management is the only requirement then implementing service objects as POJO may not appear as a real advantage. However, it becomes incredibly beneficial once the application developers bring in frameworks for security, persistence, and remoting, to name just a few.

Spring Overview

Spring is an open source framework for managing business objects. It provides integration of the middle layer of an application’s components. Spring relies on two ideas for achieving this integration: Inversion of Control (IoC) and Dependency Injection(DI).

Inversion of Control removes the responsibility of looking up resources or services from the business objects, and places it to the container. Instead of making a library call, an IoC container like Spring injects the needed resources or services into the domain objects. This Inversion of Control with dependency injections addresses the problem of wiring an application’s layers in a loosely coupled manner.

One possible solution to the above problem relies on a Service Locator that hides the underlying container. This solution achieves the look up service with Inversion of Control. Service Locator pattern implements IoC with Dependency Lookup. For example:

public void SomeBusinessObject{
DataSource dataSource;
public SomeBusinessObject(){
dataSource = ServiceLocator.lookup("java:comp/env/jdbc/MyDataSource");
}
}

In contrast, Spring provides Inversion of Control through Dependency Injections. This solution is typical of the Spring framework.Injecting a resource or service into our business objects requires either setter methods (for setter injection) or constructor (for constructor injections). The following example demonstrates the latter:

public void SomeBusinessObject{
DataSource dataSource;
public SomeBusinessObject (DataSource dataSource){
this.dataSource = dataSource;
}
}

The Spring configuration file defines the actual data source:

<bean id="jdbcDataSource" class="org.springframework.jndi.JndiObjectFactoryBean">
<property name="jndiname">
<value>java:comp/env/jdbc/MyDataSource</value>
</property>
</bean>

<bean id="someBusinesObject" class="SomeBusinessObject">
<constructor-arg><ref bean="jdbcDataSource"/></constructor-arg>
</bean>

When Spring first initializes the object this configuration injects "jdbcDataSource" into the business object.

Spring uses application context objects to handle the location and configuration of resources such as Session Factories, JDBC data sources, and other related resources, as well as inter-object dependencies. This makes these values easy to manage and change. Dependency injection and interface implementation allow us to lower the coupling between different architectural layers. For example, this provides a mechanism for injecting mocked services or recourses for testing. As a whole, Inversion of Control with Dependency Injections solve very elegantly all problems related provides a simple and consistent approach in handling configurations as well as lowers the complexity of all other application layers.

Another very important feature that Spring provides is Declarative Transaction. Spring's declarative transaction demarcation can be applied to any POJO target object. The choices for a back-end transaction manager range from simple JDBC-based transactions to full-fledged J2EE transactions via JTA. The application service interfaces and their implementations are not dependent on Spring or Spring transaction management in particular.

The Spring framework provides fine grained transaction management and flexibility in specifying transaction strategy. During runtime, Spring frameworks intercepts (via AOP-like interceptors) calls to service methods manage the transactions according to the declared transactional properties of the service methods. For example, in the above EmployerService example we declare transaction propagation for the two methods:

<property name="transactionAttributes">
<props>
<prop key="save*">PROPAGATION_REQUIRED</prop>
<prop key="*">PROPAGATION_REQUIRED,readOnly</prop>
</props>
</property>

The second declaration (which includes the "readOnly" qualifier) marks a retrieval operation, when a business transaction doesn't need to be committed or rollbacked. Spring supports additional transaction strategies and can apply them not only at the method level but also to a group of methods or classes.

Inversion of Control with Dependency Injection and support for Declarative Transaction are Spring framework features that can greatly reduce the complexity of integrating different application layers as well as fosters better OO design and greatly improves testability. Let’s look in more details at some of the key benefits of building applications with Spring.

Using Spring for Framework Integration

For the past twelve months, we have been building a large enterprise Java application. We used Spring to integrate Hibernate (ORM framework), Tapestry (component based MVC web framework) and Acegi (security framework). Let’s take a closer look at using Spring for integrating independently-developed frameworks with the same application.

- O/R mapping integration

Spring framework supports integration with variety of O/R mapping frameworks. Out of the box Spring integrates directly with JDBC or ORM solutions like Hibernate, JDO, TopLink, and iBatis. In fact, the Spring data access architecture allows it to integrate with any underlying data access technology. Spring and Hibernate are a particularly popular combination.

For the implementation of our DAO layer, we used Hibernate framework as well. Spring provides integration with it in the following ways:

Spring offers easy, efficient and safe handling of units of work such as Hibernate Session. Generally, ORM frameworks use the same "Session" object for efficiency and proper transaction handling. Spring can transparently create and bind a session to the current thread, using either a declarative AOP method interceptor approach, or by using an explicit "template" wrapper class at the Java code level. Thus Spring solves many of the usage issues that affect many users of ORM technology.
We used Spring to provide an abstraction around the Hibernate "Session" objects, thus wrapping the session in the form of one line helper method - getHibernateTemplate(). Since Hibernate Session is implementation of Unit of Work pattern, it needs to use the same Session object for efficiency and proper transaction handling. Spring provide thread safe access to this Session object by putting in it on ThreadLocal.
Spring provides resource lookup and allocation for DAO. Spring intercepts the creation of DAO implementation objects at the application start up and injects Hibernate session into DAO. Example configuration will look like that:
<bean id="exampleDAO" class="ExampleHibernateDAO>
<property name="sessionFactory">
<ref local="hibernateSessionFactory"/>
</property>
</bean>
Hibernate session is injected into ExampleHibernatedDAO object when created. This removes from the object the burden to know how to look up for any resources that it needs.
Spring wraps exceptions from the Hibernate layer, converting them from proprietary (possibly checked) exceptions, to a set of abstracted runtime exceptions. This allows developers to handle most persistence exceptions (which are non-recoverable) only in the appropriate layers, without annoying boilerplate catches/throws, and exception declarations. For example, using Hibernate directly will result in code like this:
public void save(Object obj) {
try {
Session session = SessionManager.currentSession();
Transaction tx = session.beginTransaction();
session.save(obj);
tx.commit();
session.close();
} catch (HibernateException e) {
log.error("Save failed.");
} finally {
try {
SessionManager.closeSession();
} catch (HibernateException e1) {
log.error("Problem closing session");
}
}
}

Spring on the other hand wraps all of the exception handling as well as the transaction operation in one line method invocation. The above example using Spring will look like:

public void save(Object obj) {
getHibernateTemplate().saveO(obj);
}

You can still trap and handle exceptions anywhere you need to but they are nicely wrapped.

- Service layer Integrations

The service layer benefited the most from Spring:

Spring provided transaction management for business objects. Our service objects are stateless. However, very often services access shared resources as Hibernate Session. Thus we had the ability to configure the service object with either declarative transaction management, AOP method interceptor, or an explicit 'template' wrapper class at the Java code level. Spring provides seamless framework support for any of the solution above. However, we used only declarative transaction since this approach was sufficient for all services in our project.
Spring injected DAO and other business resources into service objects. We employed dependency injection when we needed particular business object or resource. Consequently our development team did not have to implement the Service Locator Pattern in order to request a particular service.
Spring encouraged coding against interfaces. Using this basic OO principle yielded a pluggable design. This allowed developers to mock or stub service objects during unit testing.
Spring provided an alternative to implementing service as Singletons. The proliferation of Singletons reduces testability and causes lots of other problems. In addition, implementing a multi-threaded Singleton is more complex than the single-threaded GoF implementation. Spring provides thread safety through the use ThreadLocal. By default, all bean declared in Spring configuration files are singletons.
The following example shows our use of declarative transaction management, dependency injections of DAO, and coding against interfaces for a service. It also instantiates the service as a singleton:
<bean id="transactionManager" singleton="true" class="org.springframework.orm.hibernate3.HibernateTransactionManager">
<property name="sessionFactory">
<ref bean="hibernateSessionFactory"/>
</property>
</bean>

<bean id="transactionProxyTemplate" abstract="true" singleton="true" class="org.springframework.transaction.interceptor.TransactionProxyFactoryBean">
<property name="transactionManager">
<ref local="transactionManager"/>
</property>
<property name="transactionAttributes">
<props>
<prop key="update*">PROPAGATION_REQUIRED</prop>
<prop key="set*">PROPAGATION_REQUIRED</prop>
<prop key="store*">PROPAGATION_REQUIRED</prop>
<prop key="*">PROPAGATION_REQUIRED,readOnly</prop>
</props>
</property>
</bean>

<bean id="transactionalExampleService" parent="transactionProxyTemplate" singleton="true">
<property name="target">
<bean class="ExampleService">
<property name="exampleDAO">
<ref bean="hibernateExampleDao"/>
</property>
</bean>
</property>
</bean>

This bean factory configuration files specifies the following:
- Spring will create as singletons all of the above beans, excluding the abstract "transactionProxyTemplate".
- Spring intercepts the creation of "transactionManager" object and injects the bean declared with name "hibernateSessionFactory".
- The abstract class declaration of "transactionProxyTemplate" class provides declarative transaction management for all children object methods starting with declared prefix name. For example, assume a bean declared as a child of "transactionProxyTemplate". If this child object has method name starting with "store..." then it is guaranteed that it will be provided read and write transaction management during invocation of this method as well as thread safe access to all common resources that this method or any other nested object methods work with.
- Spring also intercepts the creation of "transactionalExampleService". The implementation of this object is "ExampleService.class". This object has "transactionProxyTemplate" as a parent, which means that it inherits all properties and transaction management declared in the parent. Also, a bean declared with name "hibernateExampleDao" would be injected via setter method at the creation time.

- Application/Façade layer integration

We didn’t really need Spring here for anything but integration. We used Spring to inject the services into facade objects. Since our façade objects kept the state of the current user progress, they hat to be instantiated as new objects every time. In order to do that, we explicitly had to specify singleton property to be false. For example:

<bean id="facadeTemplate" abstract="true" class="BaseFacade">
<property name="exampleService">
<ref bean="transactionalExampleService" />
</property>
</bean>

<bean id="exampleFacade" class="ExampleFacade" parent="facadeTemplate" singleton="false" />

In this configuration file, we have defined the bean "exampleFacade" to be created as a new object every time when it is requested by another object. Since the parent "facadeTemplate" has property with name "exampleService", an implementation object of ExampleService interface will be injected during creation time.

- Tapestry Page Controllers integration

We needed the Spring context to access façade or service objects in our controllers. We used Spring as Service Locator /Lookup pattern in order to locate business objects. For example, accessing the façade objects would be wrapped the following way:

SomeFacade façade = getBean(someServiceName);

The implementation of this method is the follows:

protected Object getBean(final String beanName) {
return getApplicationContext().getBean(beanName);
}

private ApplicationContext getApplicationContext() {
return (ApplicationContext) getFromGlobal(Engine.APPLICATION_CONTEXT_KEY);
}

which obtains the Spring context from the web server.Alternatively we could define services or resources declaratively. Fore example, the same can be achieved with the following declaration in Tapestry page specification file:

<property-specification name="someService" type="SomeService">
global.appContext.getBean("someServiceName ")
</property-specification>

In addition to facilitating the integration of frameworks Spring also affects the complexity and testability. In closing let’s take a closer look at these qualities.

Other Benefits of Using Spring

- Reducing Complexity

Spring is both comprehensive and modular. Spring has a layered architecture, meaning that it can be chosen to use just about any part of it in isolation, yet its architecture is internally consistent. Since Spring is designed so that applications built with it depend on as few of its APIs as possible, most business objects in Spring applications have no dependency on Spring. Consequently all business objects can be implemented as POJO. In contrast, many other frameworks (including) require implementing some framework base classes. This introduces coupling between the domain and the infrastructure.

By using Spring framework, we were very successful in reducing overall complexity in our project. Spring greatly simplified the integration with Hibernated framework, as well as provided very consistent and straightforward way of using services or resources among the different application layers.

- Improve Testing

The use of POJO objects in Spring applications can run without modifications outside the container and thus are much easier to test. Spring's inversion of control approach makes it easy to swap implementations and locations of resources such as Hibernate session factories, datasources, transaction managers, and mapper object implementations (if needed). This facilitates isolating the area under test.

Our project we had a lot of business functionality related to historical data and effectively of this data. For this reason, in order to have reliable tests, we had to use different implementation of our TimeService interface. In tests, we overridden the default implementation and that allowed us to move the time according to a particular test needs. This is the default implementation:

<bean id="timeService" class=" TimeService"/>

Then in a different Spring configuration file, which is loaded only during testing we override the previous declaration:

<bean id="timeService" class=" TimeServiceStub"/>

Also, since our DAO weren’t transaction managed, we wired them for the test to be transactional. The application dao was declared as:

<bean id="hibernateExampleDao" class="ExampleDAOHibernate">
<property name="sessionFactory">
<ref local="hibernateSessionFactory"/>
</property>
</bean>

and the bean used in tests looks like:

<bean id="transactionalHibernateExampleDAO" parent="transactionalProxyTemplate">
<property name="transactionAttributes">
<props>
<prop key="store*">PROPAGATION_REQUIRED</prop>
<prop key="save*">PROPAGATION_REQUIRED</prop>
<prop key="delete*">PROPAGATION_REQUIRED</prop>
<prop key="*">PROPAGATION_REQUIRED,readOnly</prop>
</props>
</property>
<property name="target">
<ref bean=" hibernateExampleDao"/>
</property>
</bean>

- Vendor Independence

Different server solutions have different performance and other characterizes, and there is no perfect one size fits for all solution. Alternatively, it may be found that certain functionality is just not suited to an implementation using a particular server. Thus it makes sense an application be decoupled from the vendor-specific implementations of container managed transactions and lookup resources. If there is ever a need to switch to another server implementation for reasons of functionality, performance, or any other concerns, using Spring now can make the eventual switch much easier.

However, most places seems to have already invested in a vendor’s stack and are not very likely to changed form one to another. Even in cases like that, since Spring is modular as well as free open source framework just about any part of it can be used and benefited. In our project Spring enable us with completely portable solution. The production environment, user acceptance and quality insurance testing were done on WebSphere application server. However, the developers of the team developed and tested on very light web server - Jetty. This saved a lot of time of deployment and configuration during development.

Another very important benefit was that by using Jetty server together with Tapestry frameworks, we were able to see the changes on the screen immediately after refresh without any redeployment. Spring provided out of container transaction management declarative authentication with Acegi Security framework integrated with Spring, database connection pooling, property configuration manager. All of them were integrated by Spring out of the application server container.

Summary

Spring is a powerful framework, which helps you build a loosely coupled flexible architecture. It helps in decoupling and greatly simplifies a software project development by removing dependency on the complex specification of application servers. By using more pragmatic approach, Spring can boost considerably developers productivity and efficiency. It can address in very simple and elegant way most infrastructure concerns of typical applications as well as allows you to seamlessly integrate with other framework used in a project.

Acknowledgements

I would like to thank to Dragos Manolescu, an enterprise architect at ThoughtWorks, for his great help and contributions to this article. Without his consistent guidelines I wouldn't be able to put it together.

Reference

Dragos Manolescu, Framework Abuse: http://micro-workflow.com/node/4
Martin Fowler, Dependency Injection: http://www.martinfowler.com/articles/injection.html
Rod Johnson, Expert One-on-One J2EE Design and Development (Programmer to Programmer), 2002
Rod Johnson, Expert One-on-One J2EE Development without EJB, 2004
Spring framework: http://www.springframework.org
Spring AOP: http://www.springframework.org/docs/reference/aop.html

Are there bugs in agile project?

2006-02-06T22:38:00.000-06:00

(Updated)
"The main aim of XP is to lower the cost of change."

In many agile projects, one of the most difficult questions is how to deal with defects when they weren’t supposed to exist at all.

Different types of defects...

There are two broad categories of defects - implementation defects and requirements defects:

implementation defects refer to defects in architecture, design, coding, installation, or any other aspect of the technical implementation of a software development project. Implementation defects are a major source of project trouble in traditional methodologies but no so much in agile practices.

requirements defects include all types of requirements, functional or story card specifications, acceptance testing (probably FIT) and any other means to define what the software is supposed to do and, from a functional perspective, how it is supposed to do it. Requirements defects are very relevant for "building the right software" and as such they are the most disastrous and costly.

The best we can do...

Agile development practices are sometimes thought of as an undisciplined approach to software development, lacking such things as effective defect management. On the contrary, agile development does much to reduce the incidence of defects in the first place:

we have all types of automated tests - unit, integration, regression, functional, acceptance and so on.
we do TDD to have very fast feedback and to ensure that the code is thoroughly tested.
when we pair we have the benefit of more than one person watching the quality of the code.
we do continues integration in order to uncovers integration issues early.
we have short iteration cycles in order to verify that everything built until this point is fully functional according to the story cards and reviewed by the customers.
we put the customers (users) in the center of the project to ensure adequate client involvement and proper set of project priorities.
we communicate with the customers (users) constantly to ensure we evolve our understanding of the business domain, challenge assumptions, and make informed choices and decisions.

Should we even bother...

Since we apply very rigorously all practices listed above, we expect that very few defects of any type are introduced and the system is in relatively defect-free state at any given time.

Not so fast...

It turns out that most of the time we still end up with a lot of defects at the end of a project (mostly requirements defects) and still spend time and money doing the not so much exciting work of fixing bugs. Finally, we are only humans and we can make mistakes.

So, how do we deal with resolving defects in agile team in the most effective way?

In theory ...

Even the software engineering books very clearly state that "the earlier within the software life cycle a problem is discovered, the cheaper it is to fix". Also, fixing a defect in a much later stage of the project could be a prohibitingly expensive thing to do. This is definitely proven on many occasions to be true.

The strange thing is the fact that not too much has been written about that problem in agile literature. One of the most essential agile principles, however, is that an application delivered by the end of every iteration should be absolutely functional and ready to be used by the customers (even with very limited functionality). Iterative development is perhaps the single most important vehicle for managing requirements defects and in my opinion this should exclude the existence of any them.

In practice ...?

But how many agile projects we know that end up having bug trucking systems with hundreds of bugs there waiting to be fixed. So, we finish the story cards for an iteration on time, we clap hands for one more good iteration where we deliver the most business value to the customer but we don’t bother too much dealing with bugs.

If we go back to the statement above that the defects are getting much more expensive to be fixed in latter iterations or phases of the project it could turn out that we probably didn’t deliver as much business value to the customer as we would’ve liked.

Actually, most of the bugs are left for so called "bug fixing" or "application readiness" iteration(s) at the end of the project. This represents huge risk for a project, since we haven’t made any, even rough estimation for how much time it would take for these defects to be fixed. In reality, we would not be able to do any estimation since we don’t know how many more bugs will be discovered. More importantly, the time and the complexity for fixing current defects will increase in the time to the end of the project.

Is this the right way...?

Of course, there is politics involved here too. Everybody wants to present nice graphs and charts in front of the customer but how is this better than using the waterfall way of thinking?

So, what now...?

Since agile are not some well established processes (thankfully) but rather helpful guidelines, there is no one way to deal with this issue. Every agile team should find its own way to do things through constant retrospectives. Based on my experience while working in agile teams, there are two main assumptions that tend to lead to the bug problem in an agile project:

The assumption that every software development project will indisputably need bug trucking system.

The bugs are not prioritized together with the story cards in relation to the business value they represent.

Is this even open to debate...????

The first assumption is very similar to the assumption that we need a relational database in every project we are about to start. Probably, most of the time we do need some kind of database, but we should not just assume and implement some kind of infrastructure that is not driven by business needs (this is how we end up with so call "technical cards" that can not be justify with any business value for the customer but I am leaving this for other time).

Did we overlook communication...?

Since every tool imposes some kind of process to be followed in some degree, an agile team should be particularly alert when picking tools to work with. Overall, an agile team should choose "people and communication over tools and processes". Sure enough, the bug trucking tools do a good job of keeping record of what kind of defects has been discover and much more information that we will ever need.

However, we should determine whether we loose the communication within the team by using such tools. What happens usually in practice is that a BA, QA or a user discovers a presumable defect in the system and enters it in a bug tracking database. Much later, probably at the end of the project, a developer opens the same bug to work on. By that time, all of the communication and most of the information about this bug is lost. The reasons could be many:

the most important case is when we have requirements defects where a lot of functionality is built on top of wrong assumptions. This should not be so much severe in well designed OO systems but still it is going to be much, much more expensive than fixing a defect around the time when it was discovered.
most if not all of the bug trucking tools require owners assign to a defect. This is very close to code ownership and doesn’t stimulate the other team members to participate in the knowledge sharing while resolving a particular defect.
it can’t be reproduced since many times the cases are very data specific.
many people that had worked on a particular functionality have left the project already.
developers and BAs have a clearer picture of what is going on with the system when a story card is implemented but they have hard time recall any issues latter on.

As a whole, later defect detection and repair is much more expensive, which is totally against one of the main aims of Agile development to lower the cost of a project.

Defects and feature request are...

Recent innovative software development methodology is Defect Driven Design (D3, pronounced "D Cubed"). The idea is to remove the two major phases in a project development phase (as some call it - defect creation) and maintenance phase (also called defect removal) and have only one phase called probably defect implementation phase. You can read more about this idea but the main point was that there is no distinction between a defect and a new feature request, story card, enhancement or whatever you want to call it.

What provides more business value...?

Even though D3 is still very open to debate, the idea that defects should not be separated from and prioritized differently than story cards is not easily ignored.

Usually customers don’t have problems prioritizing new functionality in regard to importance and business value for them. However, we would not see the same level of confidence when we ask customers, whether they prefer to have new functionality added or some previously prioritized and not currently working functionality fixed. Of course, the answer strongly depends on case by case basis but that’s why, we should not make assumption so that to eliminate defects from the prioritization process and treat them as something different than story cards.

Is there another way...?

It is not a big surprised the fact that there is not one efficient way to deal with defects in different agile teams. The same should be true for the assumption that bug tracking database is the best solution for every software project. We can have or try many possible scenarios:

some kind of white board (either real or wiki based) where everybody can see current development of defects, the progress and all related communications.
customers and anybody else involved in acceptance testing can use some kind of recording test tools when they do manual testing. Firefox has very simple and user friendly plugging for Selenium test for example. With such a tool, a customer can very easily record any defect and then send the resulted failing test file. The file could be included immediately in the build and thus fixed as soon as possible. No time lost for reproducing errors or misunderstanding or some kind of blaming.
creating story cards for defects that are prioritized, estimated and accepted as any other story card is very appealing scenario. This will ensure that all defects are analyzed, discussed, prioritized and fixed in proper manner and time. There could be slight variations like creating story cards with letters or numbers in the name. Example, Search For User, Search for User- b or Search for User When something happens and so on.

How would we know it is better...?

Iterative development and team retrospectives after every iteration are perhaps the single most important vehicle for managing defects. Through retrospective and fast feedback we can determine how effectively we manage defects and whether we detect and fix them as early as possible. The goal is very clear and even more easily confirmable - we should not have bugs fixing iterations at the end of an agile project.

What actually I wanted to say...

The whole thing was to reassure myself and convince at least one more person that:

many times we don’t need buck trucking databases since these kinds of tools almost always expose some process to be followed and the huge benefits of communication in an agile teams is lost.
defects are actually missing or not properly working features of the system and as such we should not make distinction between story cards and defects.

Update:
A very good addition to this is "... on features vs task post":

Instead of recognizing that defects must be prioritized and planned as stories, as Jon's post suggests, defects are still considered "exceptions" as per traditional thinking about software development. There is a tendency to avoid treating them as real work items.

My Ruby Tipping Point

2005-12-07T00:29:00.000-06:00

(Updated)

I was going to ramble about the best place to do business validations in a java application but I ran into this post. Todd says it very well. The only place where business logic belongs to such as business validation is in the domain objects. Not in the view, not in the controllers, not to even talk about putting them on facades, services, DTOs or some similar concepts that java community tends to like a lot.

Ruby on Rails shows perfect example of good OOD with putting so elegantly all validations in the model. I struggle with similar concept in java but the situation there was different story. Most of the web frameworks provided some kind of validation in the view/controller part that were too tempting to be used. As a result, we have some of the validations in the view/controller, some in facades/services and sometimes in domain objects. The result is not very impressive. We have a lot of problems and possible defects if we have rich domain objects with a lot of collaboration among them since most of the business logic is placed outside of the domain.

Throwing exceptions in the setters method or much more useful Notification pattern - domain object use notification object to collect information about errors during validation (implemented similarly in Ruby on Rails) are possible alternatives to have the validations in the model. So, the examples are out there and very convincing.

Then I read the comments in the Todd’s blog and I realize that java community would never get it. Some of the comments are stating that the validations should be separated from the domain objects - preferably in xml file; others point out that putting validations in domain objects make validations not reusable and so on. Similar java thinking could been seen in this very long discussion in the Spring framework architectural forum about domain objects being bind to the view in a web application is very bad and we have to use DTO and facades. All that was the tipping point for me.

In my previous post “Learning Ruby on Rails", I wrote that I like java because of the community. It looks like this is not the case any more. I really like the Ruby way of thinking and a very good example is the Human Interfaces. We can achieve great things with Java as well but the community thinking is not going in this direction, even with frameworks such as Spring built around the principle of simplicity. My only hope is that Ruby would never become the same type of “enterprise” language as Java currently is.

Update

Martin Flowers’s ContextualValidation is good addition to the validation topic.

“But one thing that I think constantly trips people up is when they think object validity on a context independent way such as an isValid method implies.”

This is much related to the first comment below:

“…often some of my domain objects will be used across different projects and business rules vary based on the context.”
As well as all other similar issues about validation in different user cases, when domain objects are used in batch processing with slightly different business rules and so on.

Martin gives an excellent answer for that:
“I think it's much more useful to think of validation as something that's bound to a context - typically an action that you want to do. Is this order valid to be filled, is this customer valid to check in to the hotel. So rather than have methods like isValid have methods like isValidForCheckIn.”

Dependency Injection OR Service Locator in Domain Objects with Spring and Hibernate frameworks

2005-12-01T21:53:00.000-06:00

The Problem

Currently, many java applications use Spring framework together with Hibernate. One of the advantages of that is that we can implement our domain and service objects as POJOs. Most of us are already convinced of all the benefits that such a type of architecture can have. In architecture like that, domain objects are managed by Hibernate and service objects are managed by Spring. Also, another advantage of using Spring specifically is Dependency Injections (DI) of services and resources.

The problem is how to retrieve/find/locate services or resources in domain objects since they are managed by Hibernate and not created through the bean factory of Spring framework?

Note: we should differentiate between domain, infrastructure and application services (Domain Driven Design book - http://domaindrivendesign.org/book). Using infrastructure or application services in domain objects is rather not appropriate. The problem I am most concerned with here is in the case when we need to invoke domain services inside of domain objects.

Can we avoid the problem?

Probably! The only way to avoid this problem is not to deal with any services/resources in domain objects. We can do this if we coordinate all domain-service cooperation from a service layer. In this case, we don’t need to bother with dependency injections, service locators, etc. This makes a lot of sense when we invoke services in an asynchronous fashion and especially in SOA /EAI types of architectures.

However, there are some disadvantages:
- We deprive ourselves from some of the benefits and power of OOP/OOD.
- Sometimes, service behavior fits very nicely into the domain model when we have a rich domain model rather than an anemic one http://www.martinfowler.com/bliki/AnemicDomainModel.html and http://wrschneider.blogspot.com/2005/01/avoiding-anemic-domain-models-with.html

Why do we get to the problem?

When I start a project I always try to have very well separated service layer from domain objects (usually enforced with different compilation modules for domain objects and service layer). However, in practice (at least until now) it always turns out that at some point I will need to use some kind of service or resource in my domain objects (the simplest example is Time Service. A Time Service is usually needed to give synchronized time when the application is deployed / clustered to more than one server. Also, if you need to play with the current time in your tests, this is a perfect solution. Usually when you deal with some kind of payment/billing systems, you will definitely need something like the above). There can be numerous examples of domain objects using some kind of service. Usually, most of the applications that have more complex domains use some kind of services or resources in domain objects.

Common solution to the problem

In the past when faced with this sort of situations we have usually used some form of Service Locator within domain objects in order to retrieve/locate services or resources.

However, these days we routinely use Dependency Injection with some form of Inversion of Control lightweight container (Spring for example) to wire up practically everything else in an app - and we would like to use a consistent approach if at all possible. Unfortunately, our domain objects are constructed by Hibernate and not Spring.

The most common solution (and the only one that I have seen in practice for now) is to use Spring framework DI to inject services into the service layer and to wire up practically everything else in your app beside domain objects. The domain objects, on the other side use Spring-based service locators to find further services/collaborators they need to do their job.

This could be workable approach since both the DI and service location mechanisms use the same underlying infrastructure (Spring bean factory), so they can share the same configuration files etc. Also, this common solution to the problem has been kind of described in the javadocs for Spring’s BeanFactoryBootstrap class with the statement “one singleton to rule them all”. In other words, allow singleton access to the bean factory or application context that holds all of the service layer objects.

However, as I stated above, I would like to use a consistent approach if at all possible for retrieving services. Not to mention all problems that comes with using Service Locator pattern. I guess using Service Locator with DI injections is a little better than the old style Service Locator but at the end, this is still a singleton. At least there is only ONE singleton in the entire app instead of one for each service object.

Example implementation of the common solution with Service Locator and DI

Before I continue further, let me show brief example of Service Locator using Spring DI. ServiceManager is an implementation of Service Locator pattern in this example. Its conceptual implementation comes from the statements “one singleton to rule them all” and “another level of indirection can solve any problem in OO design” (I don’t fully agree with this statements but it is hard to argue against them). The ServiceManager class has reference to all services that are needed by domain objects. All services are injected by Spring and ServiceManager is singleton instantiated by Spring creation factory on application startup. Something like this:

public class ServiceManager { private static ServiceManager ourInstance; private IReferenceDataService referenceDataService; private ITimeService timeService; ………… //other service declarations public ServiceManager() { ourInstance = this; } public static IReferenceDataService getReferenceDataService() { return getInstance().referenceDataService; } //setter for Spring injection public void setReferenceDataService(IReferenceDataService referenceDataService) { this.referenceDataService = referenceDataService; } …… //other service setters/getters protected static final ServiceManager getInstance() { return ourInstance; } }

and the Spring mapping:

<bean id="serviceManager" class="service.impl.ServiceManager">
<property name="referenceDataService">
<ref local="referenceDataService"/>
</property>
<property name="timeService">
<ref local="timeService"/>
</property>
<property name="securityService">
<ref local="securityService"/>
</property>
.......

</bean>

Many people do consider that this is perfectly fine solution and there are not any problems with it. If you are some of these people, probably the rest would not be of such an interest for you.

Issues with using Service Locator with DI

I have encountered several problems with such an implementation:

1. We end up with more than one way to retrieve a service, mock or stub.
In some not domain objects, we get a service through Spring injections, in others we get it through static call to ServiceManager class. I feel this approach could bring a little confusion. This inconsistent approach of retrieving services could be a problem but I guess it could be argued that this is more of a question of “Enabling Attitude”: http://martinfowler.com/bliki/EnablingAttitude.html rather than a problem with using Service Locator together with DI. However, I found out that this could be an actual problem in many practical situations. At the end, remembering when to use what is still an indicator for a “code smell”.

2. Another major problem is that ServiceManager is very hard to be tested, mocked or stubbed.
The example implementation could enable us to mock specific services provided by the ServiceLocator through sub classing the ServiceLocator class and providing methods for overriding of a particular service.

public class ServiceManagerStub extends ServiceManager {
        public static void overrideTimeService(ITimeService timeService) {
            getInstance().setTimeService(timeService);
        }
        public static void overrideSecurityService(ISecurityService securityService) {
            getInstance().setSecurityService(securityService);
        }
        .................
        //other service overridings
    }

Before we congratulate ourselves with successfully solving infrastructural issues that already has been solved by a DI lightweight container, which by the way we already use, I will try to point out some new problems that are introduced with this solutions.

The problem comes from the fact that we still use static class methods to retrieve services. So when we set up a mock service in a single test (ServiceManagerStub.overrideTimeService(getMockedTimeService()), we have to remember to reset it in the tearDown method. Again “we should remember”. Of course many times this is not done since running a single test class in an IDE would not cause a problem. Then we run all test from the build. At some point, we have a situation when a test which passes in an IDE fails in the build. The reason is that this test could be about 10-20 test classes after the one where we set up the mocked service. Some of the objects in this particular test use ServiceManager class to retrieve the same service that we mocked previously and we forget to reset it. Of course, we don’t have even a clue that a mocked service is invoked instead of a real one and we can not even think that 10-20 tests before our test, someone else have set a mocked implementation of the service we need.
Anyway, to make the story short, if somebody has used this kind of ServiceLocator, he/she should be familiar with this type of problems.

3. Issues with dynamic class loading.
Any implementation of ServiceLocator that uses calls to static class methods will have problem with releasing the static reference from the memory potentially with every dynamic class loading in a J2EE application server on hot redeploying. I have definitely had a lot of class loader issues with WebSphere 5. (Not with Jetty BTW)

4. One of the consequence of using ServiceLocator is that we hard-wire it.
This makes reusing of the domain objects that have hard coded dependence on the ServiceLocator often impossible due to dependencies on the particular application’s initialization and service infrastructure. In some of the cases, we were trying to reuse our domain objects in another application that will deal mostly with integration and will not use our current service infrastructure.

5. Another more general problems
- ServiceLocator cut against interfaces, by forcing you to know about specific class.
- Using ServiceLocator in the beginning could not be a big problem. However, it could turn out that ServiceLocator class is access simultaneously by many threats and it is a performance bottleneck. In cases like that we can't switch to using a pool of such objects for any of the callers that have hard-coded dependency on ServiceLocator class. Velocity and Struts frameworks, for example, have found that this initial assumption can end up being wrong and restrictive.

I guess that we can find a solution to all problems related to ServiceLocator and most of them depend on how it is implemented but should we have to deal with all these problems on the first place when we already have an implemented solution in form of DI.

Alternative Solutions

1. Passing a service as a parameter in the method call.
Passing a service as a parameter would remove the problem of dealing with dependency injection, service locators, etc and could be an elegant solution in some cases. However, in most of the situations, especially when we have rich domain object model this would not be a workable solution.

I suspect that following this approach for a more complicated domain would result in behavior moving itself inexorably out of the domain and into our service layer - leaving us with the ubiquitous anemic domain model. Usually, my goal is for the service layer to be very thin - in the majority of cases each service would just find the domain object(s) it needs and then call a method on the domain - all the behavior would be part of the domain.

Also, when the object behavior is not controlled by ‘Transactional Script” ( http://martinfowler.com/eaaCatalog/transactionScript.html ) but rather than the flow of control emerges from object behavior in rich domain model, we usually have situation when one object delegate to another object, which call a third object and so on. In situations like that, we would need to simply propagate the service in every method in the chain if this service happens to be needed in the last one, for example. Something like this anti-pattern description: http://www.picocontainer.org/Propagating+Dependency .

2. Using Hibernate to set the services/resources
This is probably the most compelling solutions since Hibernate anyway mange our domain objects. There are a couple of ways to do this. A god solution using Spring Sandbox classes is described here:
http://jroller.com/language/taranis/20050204?language=nl

Another solution is to use the currently available hibernate interceptors:
http://forum.springframework.org/archive/index.php/t-10547.html

Example of using AuditInterceptor to inject TimeService in domain objects can look like that:

<bean id="time-service" class="service.TimeService"/>
<bean id="audit-interceptor" class="persistence.AuditInterceptor">
<property name="timeService">
<ref bean="time-service"/>
</property>
</bean>

<bean id="hibernate-session-factory" class="org.springframework.orm.hibernate3.LocalSessionFactoryBean">
<property name="dataSource">
<ref bean="jdbc-data-source"/>
</property>
<property name="entityInterceptor">
<ref local=" audit-interceptor"/>
</property>
<property name="mappingResources">
<list>
<value>
/somepackage/SomeDomainObject.hbm.xml
</value>
... ... ... ... ...
</list>
</property>
</bean>

The problem with using hibernate interceptors is that they will be fired up only when the domain object is either saved or retrieved from the database. Of course by the point you need to save a domain objects, you don’t need the service any more. So, if you need to inject services or resources in objects that are only retrieved from a database (never create new objects and then persist them) this approach will be very elegant solution.

However, when we create a new domain object, this interceptor is not going to be fired and we won’t have the resources or services injected in the newly created object.
Needles to say, that this is the most common scenario in practice. We will definitely have many places where we need to create a new object, perform some business operation on it and then persist it to a database. In situation like that we need to have/inject/retrieve any services/resources before we push it to Hibernate through session.save() for example. As some of the people from Spring framework suggest (http://opensource2.atlassian.com/projects/spring/browse/SPR-431) we can use Spring via the ApplicationContext to create the domain object instance, populate it into our system via a lookup-method (to keep coupling low), and then push it into Hibernate.

3. Using Srping Bean factory to create your domain objects
In order to inject a service during creation of a domain object, we can use the Spring factories BeanNameAutoProxyCreator and ProxyFactoryBean:
http://www.springframework.org/docs/reference/aop.html#aop-pfb
And some opinion about the difference between ProxyFactoryBean and BeanNameAutoProxyCreator http://erik.jteam.nl/?p=8
As the name suggests (although they use AOP) they are still factories. This means that you can’t create an object just with “new”, but you have to use some kind of factory (in this case Spring bean factory). The other problem is that in order to use these Spring bean factories or Spring AOP for that matter, the lifecycle of a domain objects should be managed by the Spring container. If this is not a concern for you, then these bean factories together with Spring AOP will do the job.

So, if we don’t have problem with Spring container managing the life cycle of our domain objects, we have the ability to intercept the creation of a domain object despite the fact that the creation of this object is managed by Hibernate as well.

One problem with this approach is that we have to create all of our domain objects via
factory and our domain objects will become dependent on Spring framework. Since I usually try to keep all business logic in pure POJO, having domain objects dependent on Spring is somewhat problematic. Another problem will be unit testing our domain objects. We should initialize Spring application context before running any test. This can slow a great deal our unit tests and we can loose one of the biggest benefits of unit testing – very fast feedbacks. Not to mention that the task of refactoring the creation of all domain object from using “new” to using factory is almost impossible to be done in respects of agile iterative approach.

Also, many people don’t think that this is very efficient solution either. Some of the arguments are about performance issues of Spring AOP implementation: http://www.theserverside.com/news/thread.tss?thread_id=30681 .

So, if the biggest problem in this approach is the dependency on Spring bean factory, then we can work around it through using directly AOP in order to inject services in our domain objects.

4. Using AOP (AspectJ) to inject services/resources in domain objects during their instantiation.
Finding resources or services (Dependency Location/ Look Up) should not be a concern of a domain object to deal with. One of the basic principles of OOP is that one object should have only one responsibility and should do it well. Looking up for resources or services is clearly a cross-cutting problem. This should be a very good candidate for AOP. We can use AspectJ in order to inject services/resources in DO both when they are loaded and when new ones are constructed.

One issue with this approach is that the use of Spring DI for everything else and AspectJ for DO is not a consistent approach. Still, in my opinion it is better that a static call in DO to class method in Service Locator object. Also, you can integrate very easily AspectJ with Spring:
http://static.springframework.org/spring/docs/1.2.x/reference/aspectj.html#d0e4554

It looks like finally we have some feasible solution to the problem. Not so fast! According to my experience in dealing with customer enterprise applications and also according to many other opinions on AOP (one of them: http://beust.com/weblog/archives/000180.html) most of developer community (as well as IT as a whole) remains very skeptical about a direct adoption of AOP. The idea of using open source aspect oriented compiler could be very problematic on many enterprise projects.

As a whole, I think that this is the least popular solution and the hardest one to sell it out to other developers, architects and especially to customers.

5. Using Hibernate to set the services/resources when retrieving DO and setting them directly when a new DO is instantiated.
We can inject services/locators as with Hibernate interceptors when we retrieve objects from a database. When we create a new object, we can have the needed services/resources as a constructor parameter or just to set a property to the newly created DO.

Again we have at least two places that will deal with injecting services/resources – Hibernate interceptors as well as every place in the code where we create a new DO, which needs services. This probably will make us use a factory for creating DO, which brings us back to the solution with Spring bean factory managing our domain objects. Very informative discussion about this approach could be found here:
http://forum.springframework.org/showthread.php?t=12221

Conclusion

This is the first time I feel that there is some inconsistency of using Spring framework for DI. To me, all this seems to beg the question - If Spring framework doesn’t provide effective way for DI in the domain model - and we want the majority of behavior to be in our domain model – then how we have to deal with this problem?

Anyway I still think that Spring framework is a great framework. In my opinion using AOP directly with AspectJ is the best approach for DI in the domain model. However, since AOP is not even close to wide spread adoption, probably the alternative solution to AOP in java5 – annotations could have the potential to solve the problem in less powerful but still elegant way (http://weblogs.java.net/blog/crazybob/archive/2004/09/where_is_aop.html).

What approach are you using in your projects?

Interactive Tests – Mocks to the Extreme!

2005-11-17T22:38:00.000-06:00

If you do Behavior Driven Development (BDD) or in less extend applicable if you do Test Driven Development (TDD), you will definitely get to a situation where you need to know that a certain method has been call but you are not interested in what is going on inside of this method. You don’t want to check whether the state of an object has been changed. It is responsibility to another unit test – “Try to use only one assert per test” – (Behavior Driven Development)

So, Nothing new here. I can use a mock object and specify the expected behavior I need. However, the complication comes when the method that I want to mock is specified in the same class I am testing. Example of this would be some service object that has method to retrieve data from a database and another method in the same class that filters this collection. How do we mock the database access method in order to verify the behavior of the filter method?

public class SomeClass {
           Collection retrieveSomeData(){ // goes to db.}

           Collection filterBySomeCriteria(){
                 Collection all = retrieveSomeData();
                 // apply some rules to filter
                 return filteredCollection;
           }
}

If it is not easy to verify the behavioral specification of a specific context, then it should be refactor via Template, Strategy or some other form of collaboration. This is definitely very good principle that I follow a lot. One of the primary reasons for me to use BDD/TDD is to be able to design easily testable and verifiable behavior as well as a simple overall abstraction of a problem.

Another solution is to create something like a proxy object, which overrides the method with some stub implementation. This is definitely possible and it is a technique that I used to do a lot in the past when testing methods that use Singletons objects. (Service Locator Patter - thankfully not any more with DI). However, it is as unnatural as to try to set specific data to db and then retrieved before verifying.

Sometimes however, you probably don’t want to refactor similar implementation to another collaborating object (because of “Cohesion”, “Completeness”, “Convenience”, “Consistency” or some other reason). In situation like that, we can very easily create a proxy of the object under test and stub/mock the method that we want. We can create such an object by ourselves, similar to 2). However, much more convenient is to use some framework like EasyMock.

In EasyMock, we can mock not only against interfaces (much preferable) but against actual classes and even the class under test. The example above would be:

    public void testShouldFilterBySomeCriteriaWhenThereAreMoreThanOneElements() {
        // create mock control for this class
        MockControl someClassControl = createClassControl(SomeClass.class, new Method[] { 
           SomeClass.class.getMethod("retrieveSomeData", new Class[] {}) });
        // create the actual class under test.
        SomeClass someClass = (SomeClass) someClassControl.getMock();
        // expected returned result for the mocked method.
        Collection retrievedData = Arrays.asList(new Object[] { object1, object2 });
        // set the expectations
        someClassControl.expectAndReturn(someClass.retriveSomeData(), retrievedData);
   
    classControl.reply();

        assertEquals("One entry should be filtered", 1, someClass.filterBySomeCriteria().iterator.size());

        // since we need this more as a stub rather than a mock,
        // it is not advisable to verify the expected behavior
        // anyway, if you really need to
        classControl.verify();
    }

As everything else, we should be careful not to abuse this kind of testing.

Learning Ruby and Ruby on Rails

2005-11-14T22:27:00.000-06:00

Why:
After all the talk about Ruby/Rails. Also, so many highly respectful people in Java community raved so much about Ruby/Rails. Probably, there is something.

Background:
I am Java guy. I have one-two year experience with .NET as well but I don’t want to go back there (at least not voluntarily). Why? I like a little more methodical oriented Java community (many could read this as unnecessary complex but still). Also, it turned out that I am not so much Java developer rather I am Eclipse/IntelliJ developer. However, I like the simplicity in design, so I thought I could learn many valuable things from Ruby/Rails and probably become very passionate about them.

When:
When I found out a comparably decent (for dynamic language) plugging for Eclipse – (the same debugging as in Java, some auto-completions as well) and when I find recommendation for good books – “Programming Ruby” and “Agile Web Development with Rails”. Why so late? I have 1-2 years with Python and Plone. I didn’t think that there could be something so much new or interesting.

Progress – Learning by comparison:
I am learning Ruby/Rails with by reading various sources like blogs, books, articles, example applications and various tutorials. I can’t help but compare with all of my previous experience in web development. However, my experience with Ruby/Rails continues to be somehow mixed. Ruby so far is very comparable to Python (fortunately without the indentation stuff). I am still not so much after dynamic languages but I kind of like many features in Ruby and so far, Ruby is more than I expected.

Rails framework, on the other hand was a little disappointing for me (at least for now).

Left aside all of the limitation of Active Record mapping compared to Data Mapper/Identity Map/Unit of Work patterns implemented by Hibernate (http://www.theserverside.com/articles/article.tss?l=RailsHibernate), I still don’t understand why our domain objects should inherit from Active Record base class. So many problems because of framework inheritance (also seen in Entity Beans!). What's more, how we unit test our domain objects without going to a database?

The whole approach of defining the model in a database first as well as thinking of database state instead of object behavior looks to me like we are going way back.

After having successful implementation with pure html template (Tapestry), it is not clear to me why we are going back to early days of php, jsp and asp with spaghetti mix of ruby code within html tags.

Also, after working two years with component based web framework like Tapestry, JSF and even ASP.Net, it is very hard for me to accept that I have to iterate through elements every time when I just want to have a simple table, for example.

On the positive side, I never saw something even close in Java to the controller/listeners implementation in Rails. I was really impressed by this simplicity.

Next
Overall, I am still excited to continue learning Ruby and Ruby on Rails. I see some benefits in creating not to complex websites, creating fast prototypes, huge potential in testing and even in the integration where we need more “relax” and definitely more simple way of doing things. Besides, if Ruby turns out to be big and there are exciting projects to work on, I don’t want to miss them.

Stubs or Mocks - State or Behavior - Testing or Design: Evaluated!

2005-10-30T23:50:00.000-06:00

Many discussions about my previous blog entry Stubs or Mocks - State or Behavior - Testing or Design, made me write some summary and also share my experience on that controversial but kind of important topic.

The main arguments for using state base rather than using more interactive based unit tests are:

Most of the times the unit tests don’t catch all problems.

Some extracts:

“...things often go wrong where one part of a system communicates with another (e.g. business logic connects to a database). The mocked out functions are usually called correctly, but something else is going wrong (e.g. permissions have not been set on the database).” (from comments)

“Why we should use mocks when we could test for free all underlying layers?” (other developers conversations).

“In essence state-based tests are not just unit tests, but also mini-integration tests. As a result many people like the fact that client tests may catch errors that the main tests for an object may have missed, particularly probing areas where classes interact. Interaction tests lose that quality. In addition you also run the risk that expectations on interaction-based tests can be contradictory, resulting in unit tests that run green but mask inherent errors.”(Fowler).

2) Coupling test to implementation.
3) Coupling to the implementation also interferes with refactoring, since implementation changes are much more likely to break tests than with state-based testing.

The first argument is very debatable since there are so many different opinions on the definition of what is unit test. I completely agree with the definition of “unit test” from Wikipedia:

“Limitations
Unit-testing will not catch every error in the program. By definition, it only tests the functionality of the units themselves. Therefore, it will not catch integration errors, performance problems and any other system-wide issues. In addition, it may not be easy to anticipate all special cases of input the program unit under study may receive in reality. Unit testing is only effective if it is used in conjunction with other software testing activities.
It is unrealistic to test all possible input combinations for any non-trivial piece of software. A unit test can only show the presence of errors; it cannot show the absence of errors.
Separation of Interface from Implementation
Because some classes may have references to other classes, testing a class can frequently spill over into testing another class. A common example of this is classes that depend on a database: in order to test the class, the tester finds herself writing code that interacts with the database. This is a mistake, because a unit test should never go outside of its own class boundary. As a result, the software developer abstracts an interface around the database connection, and then implements that interface with their own mock object. This results in loosely coupled code, minimizing dependencies in the system.” (Wikipedia)

For the rest of the arguments I will try to show the evolution of the style of unit testing in my current project and all problems related to that.

(All resemblance to real people or code is absolutely on purpose!)

Phase 1: Mostly state based testing – not using mocks or stubs.

We didn’t think to much about our tests in the beginning. The tests were mainly state based without any consideration for behavior driven design, mocking or stubbing some parts of the system or for the time of our build.

Consequences:

Since we didn’t mock or stub the database access, our test became a little slow for the current code base.

The rational of not using stubs or mocks was that by using Hibernate, we can use level 1/level 2 Hibernate caches, which would mean that we would not go to the db very often. Even if we query the db directly, we shouldn’t worry about speed with today’s state of computers and databases.

So, a little slowness in our tests did make us think about some kind of mocking or subbing but since we didn’t have any other problems, we didn’t change anything.

Our tests were strongly dependent on the test data in the database.

Since, we were doing mostly state base testing without using any mocks or stubs, our tests strongly relied on test data in the db. The same was true for the unit test in layers not immediately related to the database. Tests from many different layers started failing when something is wrong with the db or there were slight modifications in the test data. I guess that test in isolation principle didn’t stand any more, and we had hard time discovering where was the underling problem?

An example would be when we retrieve some reference data and verify the size of the retrieved collection. Once we check this size in DAO layer, then we check the same in the service layer since the service layer call dao object. Finally, we check for the same size in the view controller class where we construct the drop down component for the UI. You can imagine how many test will fail if we add one more reference data entry in the underling database. This was just because we tried to do state based testing instead of more interactive one.

Actions taken:
        Since we started having so many problems after simple changes to the test data in the db and tests became a little slow, we decided to use stubs or/and mocks but only for the database access layer. So, we had persistence tests that tested the database access and our DAO. We used mocks to verify whether a service has called a dao. Also, we used stubs when we didn’t care if the object has been persisted or not. When I say stubs, this is a little conditional. Sometimes, we used static stubs, but most of the time we used mocks as stubs - without verifying the behavior after using it.

Phase 2: Mostly state based testing – with using mocks or stubs only for database access in the layers above it.

   Consequences:

1. OO design became a “little” too corrupted.

1.1  The state of the objects was exposed just to be tested without any other need in the application.

We had the following code in our application:

public class Name {

//public for testability
public Name(){
}

public Name(String firstName, String lastName){
}
…………

//public for testability
public void setLastName(String lastName){
………………..
}

//public for testability
public Long getId(){     }
}

Of course, this was the immediate consequence of the state base testing. When we created the test first, we didn’t think what BEHAVIOR we need, rather we thought of how to verify that the state of the object was set correctly. (We should think very carefully whether we are doing OOP, if we need to create/expose some getter/setter or any kind of internal state for that matter just to test if something went well)
It turns out, that this is common pattern on many other projects, so not many people were concerned about it.
(And yes, Hibernate can work not only with public but also with private/protected constructors, getters and setters)

1.2. Instead of building rich Domain Model  with objects that collaborate among each other, we end up having Anemic Domain Model .

Since testing was very coupled with the state implementation of the tested classes, we started “treating objects like new age data structures: Ask an object for information about its state, process that, make decisions based on that, then do something to/with the object. This is a classic approach from the old days or procedural programming. It is NOT OO.” (Astel)

“By pulling all the behavior out into services, controllers and so on, essentially end up with Transaction Scripts, and thus lose the advantages that the domain model can bring.” (Fowler)

Some of the developers (very few) start asking questions about the consequences of not having true Domain model and what we could’ve changed to fix it. We came to the conclusion that more behavior driven design through interaction testing could’ve prevent that:

“One of the hardest things for people to understand in OO design is the "Tell Don't Ask" principle, which encourages you to tell an object to do something rather than rip data out of an object to do it in client code. Interaction testers say that using interaction testing helps promote this and avoid the getter confetti that pervades too much of code these days.” (Fowler)

However, this problem didn’t appeal to the rest of the developers as an important problem.

2. Based on the complicated business logic, too many bugs were introduced because of not properly collaborating and encapsulated object in our model.

As the business logic growth in our application, we had too many defects and the implementation of the story cards was apparently harder and timelier to implement than before.
This was as a direct consequence from the Transaction Script implementation of our business model. Yet, no any actions were taken.

      3.   Problems with chaining  of methods.

“Interaction-based testers do talk more about avoiding 'train wrecks' - method chains of style of getThis().getThat().getTheOther(). Avoiding method chains is also known as following the Law of Demeter. While method chains are a smell, the opposite problem of middle men objects bloated with forwarding methods is also a smell. (I've always felt I'd be more comfortable with the Law of Demeter if it were called the Suggestion of Demeter.)”(Fowler)

I guess we had much more problems with chaining of methods than forwarding them. For our web application, we used web framework (Tapestry), which binds the object to the view components in configuration file. As a consequence of the state base testing, we had a lot of “train wrecks”. Unfortunately, our IDEs (Eclipse, IntelliJ) couldn’t successfully refactor those “train wrecks” in those configuration files. One can imagine the consequences of that in a web base application.

Test in isolation.

“If you introduce a bug to a system with interaction testing, it will usually cause only tests whose primary object contains the bug to fail. With a state-based approach, however, any tests of client objects can also fail, which leads to failures where the buggy object is used as a secondary object. As a result a failure in a highly used object causes a ripple of failing tests all across the system.” (Fowler)

Not only that failing tests rippled through the whole application, but we had hangover cards because of problems with refactoring or JUST FIXING NOT PROPERLY WORKING METHODS. Since, we used state based testing style, we used to verify the consequence of some behavior rather verify the actual behavior.

Example of this would be when we have an object that keeps some state. Let’s say that keeping the state in the proper form is major part of an application. Many operations lead to changes in the current state of this particular object. When we use state base testing, we don’t very that we change/revert/switch/override the state but rather we verify what is the state after performing some of these operation.

As it usually happens, some of those methods weren’t implemented as the business wanted. After fixing them to work properly, the ripple effect of failing test all over the application was kind of awaking call.

Actions taken:
All these problems made our team think more in terms of Domain Driven Design. We started having domain discussions. Also, we concentrate more on the collaboration and behavior of the objects in our Domain Model. We went back to the essential principles of OOP and start thinking more about behavior not so much about state.

Phase 3: Shifting from state based testing to more interactive style of testing.

Consequences:

     1.  Complex Fixture Setups
“With state-based testing, you have to create all the objects that are going to be involved in responding to the stimulus. While the example only had a couple of objects, real tests often involve a large amount of secondary objects. Usually these objects are created and torn down with each run of the tests.
Interaction-based tests, however, only need to create the primary object and mocks for its immediate neighbors. This can avoid some of the involved work in building up complex fixtures.” (Fowler)

As a consequence of building more sophisticated domain model and more complicated business logic, we started having a lot of complicated fixture setups.
Of course we had “Object Moder”, although we called it “Domain Fountain” as more appropriate name in our test suites. However, we still had very complex setups since most of them were very concrete to a particular test.

Action taken:
As a result of that, we started substituting state based with interactive based testing. We set up and verified mocks all over the place. We used mocks almost everywhere when the set up was more complicated. Also, in this phase we liked the interactive based testing so much so that some developers refactored all previous tests that used dynamic mocks as stubs from not calling verify to call verify of the mock expectation. We clearly moved to the next phase.

Phase 4: Accent on the interactive style of testing. Some initial steps towards Behavior Driven Design.

Consequences:

1. Coupling tests to implementation.
“This coupling leads to a couple of concerns. The most important one is the effect on Test Driven Development. With interaction-based testing, writing the test makes you think about the implementation of the behavior - indeed interaction testers see this as an advantage. State-based testers however think that it's important to only think about what happens from the external interface and to leave all consideration of implementation until after you're done writing the test.”(Fowler).
As Fowler pointed out, I think that this is an advantage. Since unit test is the clearest form of white/”window -(Rod Johnson)” box testing, I see interaction-based testing as the right thing to do. Also, in “pure” unit tests you should not be dependent on the implementation of the secondary objects as this is the case in state base testing.
2. Using Mocks instead of Stubs.
This was one of the most painful mistakes that we did. We used mocks when we want to verify some expectation. However, we used mocks even when we didn’t care whether a particular method of a secondary object had been call. What is more, we used mocks instead of stubs in more than 80% of the cases.
“Coupling to the implementation also interferes with refactoring, since implementation changes are much more likely to break tests than with state-based testing.” (Fowler).
As a direct consequence of that after every refactoring, we had very painful test fixing sessions.
Phase 5: Behavior Driven Design with very careful consideration on Interactive based testing.

1. Use Mocks only when we verify whether the behavior of the mock object has     been called. All other case, use stubs.

In this phase, we use mocks only when we really need to verify the expectation. In all other cases, we use static stubs. In scenario like that, we don’t have problem of failing test when the implementation is fine, but some of mock expectations are not met.
Using interactive based testing with more consideration will bring all the benefits of that kind of testing but will safe us a lot of troubles with refactoring.

        2.  Behavior Driven Design.

Since, we saw a huge damage on our OO design with using state base testing, we try to avoid this type of testing. Try having “one assert statement per test” (Astel).
Anyway, this is much better explained in Dave Astel’s article about BDD.

Actions Taken:
I think that I would’ve been satisfied on this level. Unfortunately, our project ran out of time and we never really got to this last phase. … well, probably in the next project we can skip the first couple of phases.

References Used:

1. Fowler, Martin – Mocks aren’t Subs - http://www.martinfowler.com/articles/mocksArentStubs.html#CouplingTestsToImplementations

2.   Astels, Dave – Behavior Driven Development -
      http://daveastels.com/files/sdbp2005/BDD Intro.pdf
3.  Wikipedia – Unit Tests -  http://en.wikipedia.org/wiki/Unit_testing

Stubs or Mocks - State or Behavior - Testing or Design

2005-10-27T21:06:00.000-05:00

A colleague of mine, Siva post a short blog entry Mocks are evil!, which became very interesting debate about OO programming and design, how we thing about verifying and designing our objects as well as the standard argument about Mocks or Stubs. I think that it is very interesting debate that I will repost on my blog as well. Siva is arguing that using Mocks is not always the best thing to do. I completely agree with that but I disagree about the importance of state testing for good OOD.

Siva said: Mocks are evil!
-------------------------------------------------------------------------------------------------

Igor said...

This could be seen in two ways:

1) We are testing the implementation rather than verifying behavior.

And this is exactly what we do. Example, my method should call dao in order to persist to db. I don’t want to know if we actually persist the object, I just want to now whether particular method of the INTERFACE of the dao is called. Interface identifies behavior. That’s why I want to verify this behavior.

2) As our interactive based tests are tied to implementation, refactoring becomes a pain. Refactoring will take more time as we need to fix interactive based tests.

Since we don’t care about the actual implementation of the method, this could save us a lot of troubles when we change, refractor or substitute different objects (implementation) for the tested interface method. If we have problems with the mocks when we refractor, this means, the behavior has changed and that is what we are testing. If we don’t care about the behavior then we are probably better off with stubs.

3) It puts us in wrong perspective. Instead of thinking in terms of behaviors in our unit (tests / contexts) we end up thinking about implementation like whether this method calls that method.

If you use real object, then you think about the state of this object and not for their behavior, which is no any different than thinking about db tables.

Otherwise, I feel your pain. Probably, we should use them more rationally.

-----------------------------------------------------------------------------------------------------

Siva Jagadeesan said...

I don’t want to know if we actually persist the object, I just want to now whether particular method of the INTERFACE of the dao is called.

Igor I accept with u. This above scenario is a good example for interactive based testing.

Take this code snippet for example.

public void doSomethingAndSave(){
doSomething();
getDao().save();
}

For the above method, we might have these following behaviors to verify.

Behavior 1) shouldCallDaoSave
Behavior 2) shouldDoSomething

Behavior 1 is perfect for using interactive based testing. By using mock we could avoid going to database.

In behavior 2, we do not care whether it is calling getDao().save(), all we want to verify is whether it is doing “something" that we specify. In this situation state based testing is better than interactive based testing. By using dummy stubs, we do not have to change this behavior even if we decide not save anything after doing “something”. But if we used interactive based testing for this behavior, then we would have to fix this behavior (even though nothing has changed in behavior that it specifies) when we refactor this method not to save anything.

------------------------------------------------------------------------------------------------
Igor said...

In 1) you are interested in how the dao will behave. That’s when you need a mock to verify its behavior as you pointed out.

In 2) you have behavior specification that tests exactly “doSomethingElse”. So, in this case you don’t need to mock dao because you don’t verify its behavior. You should have one assert/verification per test, so you testing/verifying the dao at all. You just need to stub it. However, I don’t think that state testing has anything to do with this situation. If you stub it, you don’t check for the state at all.

I believe that state testing is not good approach to OO design. It makes us think in terms of data holders (aka db tables) instead of thinking about object behavior and object collaboration. Since, one of the principles of OOP is hiding the state of the object and providing specific behavior, we should avoid thinking of the state of the object. That’s one of the major reason why most of the object that we see are data holders with public getters and setters. I would prefer to see object without any public getter or setter rather with methods giving me some behavior.

So in summary, I completely agree that we should use mock with great deal of consideration. Also, most of the times stubs would be more appropriate. Yet, don’t confuse using stubs with state testing. I completely disagree that state testing can be any good for OOP.

----------------------------------------------------------------------------------------------------
Siva Jagadeesan said...

Igor I agree with you :)

Quote from Martin’s article about state based testing,

“…kick your primary object with the behavior you want to test. Once it's done you then need to find out if everything went well. This you probe by using a bunch of assert statements, both against the primary object…”

http://www.martinfowler.com/articles/mocksArentStubs.html#State-basedTesting

In my article I am not advocating State based testing… I am more concerned about verifying behavior of an object. How we verify behavior depends on framework we use. If we use Junit, unfortunately we have to verify behavior by asserting the state of object.

I have not looked at JBehave, it will be interesting to see how this framework enables us to verify the behavior of object.

------------------------------------------------------------------------------------------------
Igor said...

Martin’s article is classic but it is a little outdated already. As I pointed out, I don’t think that state base testing brings value for creating good OOD. (Probably for QA testing yes, but not designing).

You can use JUnit with EasyMock or JMock and verify behavior without any problem. It is not design to verify behavior but with a little more effort you can achieve Behavior Driven Design with JUnit as well.

I agree on everything else besides the testing of the state of an object. Since the state should be encapsulated inside of the object, I don’t think that trying to test it will evolve to a good Object Oriented Design. You shouldn’t be allowed to ask an object for its state; rather you should delegate some responsibility to an object and verify whether it perform it correctly.

----------------------------------------------------------------------------------------------------
Siva Jagadeesan said...

You can use JUnit with EasyMock or JMock and verify behavior without any problem. It is not design to verify behavior but with a little more effort you can achieve Behavior Driven Design with JUnit as well.
Using EasyMock or JMock with Junit , we will be still testing implementation not the behavior.

I agree on everything else besides the testing of the state of an object. Since the state should be encapsulated inside of the object, I don’t think that trying to test it will evolve to a good Object Oriented Design. You shouldn’t be allowed to ask an object for its state; rather you should delegate some responsibility to an object and verify whether it perform it correctly.

Lets take classic object Lamp.
public class Lamp {
private boolean on;
public void switchOn(){
setOn(true);
}
public boolean isOn() {
return on;
}
private void setOn(boolean on) {
this.on = on;
}
}
Might be I am missing something here. How can we make sure “switchingOn” behavior worked correctly without verifying the state (on/off) of Lamp?

-----------------------------------------------------------------------------------------------
Igor said...

This example is very similar to the problem that they describe in famous article about OOD - “The Mark IV Special Coffee Maker”. http://www.objectmentor.com/resources/articles//resources/resources/articles/CoffeeMaker.pdf

We are writing a program here, and programs are about behavior! We should think about behavior. What is the behavior in class Light? It is the behavior of a system that is the first clue to how the software should be partitioned.

Clearly, from you design the Light object just wants to be turned on or turned off. Thus we might put :

void switchOn();
void switchOff()

method in class Light. I think that the return value of these methods is the problem here.
So, if the behavior that you want is to swtichOn the light and then verify that everything went successfully:

public interface Switchable{
boolean switchOn();
}

public class Lamp implements Switchable {
public boolean switchOn(){
//implementation that we don’t care
}
}

Behavior Specification will look like this.

public void shouldSitchOnTheLightSuccesfullyWhenEverythingIsOk(){
Ligh light = new Light();
shouldBeSuccessfull(ligh.switchOn());
}

That’s it. I don’t need to check the state of the light in order to verify that the behavior specification of this object.

Later on, if you need additional behavior to check whether the light is on you can add the other method:

public boolean isOn();

However, do you really want to know that? I would try to avoid this method since “One of the core tenets of object-oriented programming is encapsulation. It’s one of OO’s most powerful ideas. More powerful than inheritance. Unfortunately it’s also one of the most ignored.” From Dave Astels’s blog: http://blog.daveastels.com/?p=55.

If you put isOn() method just to test switchOn() without actually needing it then you reveal to much from your code without needing it. From the pragmatic guys http://www.pragmaticprogrammer.com/articles/may_04_oo1.pdf:

“The best code is very shy. Like a four-year old hiding behind a mother’s skirt, code shouldn’t reveal too much of itself and shouldn’t be too nosy into others affairs.
But you might find that your shy code grows up too fast, shedding its demure shyness in favor of wild promiscuity. When code isn’t shy, you’ll get unwanted coupling.”

Your testing and behavior specification are tightly coupled to the implementation of Light object. “This is a classic approach from the old days or procedural programming. It is NOT OO. This is just one example of the approach of treating objects like new age data structures: Ask an object for information about its state, process that, make decisions based on that, then do something to/with the object.” (Astel’s)

All we should know is how to ask the light to turn on and whether it went successfully.
The proper approach is to ask the Light to turn itself on. Let me know if you were able to do that. We don’t care about it state at all.

This is a little aside but:

“As a side note, writing code this way makes it much more understandable.” (Astel’s)

Also, from Dave Astele’s brilliant article about Behavior Driven Design (BDD) http://daveastels.com/files/sdbp2005/BDD%20Intro.pdf:

“First stop thinking in terms of tests. As Bob Martin has been saying for a few years “Specification, not Verification”. What Bob means is that verification (aka testing) is all about making sure (i.e. verifying) that your code functions correctly while specification is all about defining what it means (i.e. specifying) for your code to function correctly.”

So based on the above, if you argue that in my behavior specification I would not be sure whether the light is actually turned on right now (verify that my code works correctly), I would say that I just want to now that light has been turned on successfully (not exactly interested if it is on right now.)

Anyway, this debate is going on for the last 10-12 years of OOP, so I think will continue have arguments about that. Still, if we care about the state than what is the difference between OOP and database.

-----------------------------------------------------------------------------------------------------
Siva Jagadeesan said...

Igor it is a great explanation. I think my problem is my state of mind. Even though I am familiar with OOP concepts, I am relaxing those OOP rules to make it testable. This is wrong. I should consciously do behavior driven design.

Instead of State Based Testing, how about Behavior Based Testing?

---------------------------------------------------------------------------------------------------------
Igor said…

P.S. After Siva contacted Martin Fowler about all that, he stated that he is still unconvinced about this style of testing since mock verification couple a test to the implementation of the object under test. So, I have to say the article “Mocks aren’t Stubs”, from which I learned a great deal about design and testing as a whole is not outdated. Rather, my view about object state testing has changed a lot since I read this invaluable article and I tend to have different opinion on that now.

Thanks to Martin for his significant input on that topic and his replay.
Siva, thanks for the interesting topic and exciting debate.

Spring Framework Overview

2005-04-18T22:37:00.002-05:00

Achieving better object oriented design and ability to use TDD by using Lightweight containers.

by Igor Stoyanov.

Many J2EE web applications contain too much code in the web tier. There are various reasons for this, but I think the most important are:

- Too few books emphasize the importance of a thin web tier. There are many books discussing MVC frameworks, emphasizing on the importance of properly designed web tier, separating control logic from presentation logic but much less of how the web tier should related to the rest of the application.

- There are many web frameworks but there’s no framework for business objects.

- Putting business logic in the web tier is often perceived as the quickest path to a solution. The previous point helps to understand the cause of this common misconception. However, it is a misconception, as working with a more sophisticated architecture will quickly show.

Until recently, the only framework that addresses partially some of these issues was EJB. Although, there are many problems and complexity related to using EJB, this was the primary framework for managing business tier objects. There are many nice thing specified in EJB such as declarative transaction in container management of business object, scalable remote architecture, very fine grained declarative security. However, the complexity of implementation, problems with performs in some cases, difficulties to test and too much overhead form majority of J2EE applications cause the emergence of lightweight container for managing business transaction.

The most popular lightweight containers are Spring framework, Pico container and HaveMind. These lightweight containers take some of the successful ideas from EJB to bring a greater level of sophistication to non-EJB architectures. They are similar to EJB arhcitecures in being centered around a layer of managed business service object. However, this is where the similarity ends. Instead of running inside an EJB container, business objects run inside a lightweight container. Lightweight containers aren’t tied to J2EE, have negligible startup cost and eliminate the deployment step of EJB.

The beauty of lightweight containers is that the business objects that they manage are POJO running inside of the containers. They are managed via AOP interception to deliver enterprise services. Unlike EJBs, they don’t usually need to depend on container APIs, meaning that they are also usable outside any container.

All these factors make lightweight containers the preferable choice if using Agile Methodologies together with TDD. According to agile, the design and implementation should be driven by the business requirements not by unnecessary complex specifications that have to be implemented. According to TDD, this is currently the only choice in order to achieve superior design and quality of the software application. This is of course if you have the freedom to choose technologies to work with.

I will look more closely at the currently most advanced lightweight container – Spring Framework. There are so many things that Spring Frameworks provides to make development easier that it is very hard to be described or summarized. I will talk about the most important of ones:

- Interface-implementation Separation

EJB enforces a separation between interface (component interface of business methods interface) and implementation (bean implementation class). This is actually one of the best things in EJB, as programming against interfaces rather than classes is a best practice with many beneficial consequences, such as “pluggability”.

Most lightweight containers do not enforce such a separation. However, they do recommend doing it, but in my experience, giving more freedom to developers doesn’t always provide better results. At least in this situation, I would rather prefer EJB approach of enforcing implementation against interfaces.

- Inversion of Control (IoC)

Most business objects have dependencies, such as other business objects, data access objects and resource. Because of that, there is a need to look up other managed object and resource, and therefore must dependent on a container to satisfy that. In satisfying this look up services lies the major difference between EJBs and lightweight containers as Spring framework. Both architectures achieve this look up service by Inversion of Control (IoC). The difference is that EJB architecture implements IoC with Dependency Lookup opposed to Spring framework, which implements IoC with Dependency Injection.

- Declarative Transaction Management

This is almost similar to EJBs architecture. The difference is that Spring frameworks provides much finer grained transaction management and much more flexibility in specifying transaction strategy.

The next diagram illustrates a typical Spring framework web application architecture. The Spring architecture is conceptually similar to the “local EJB architecture”. However, it tends to work out very differently in Practice.

Reference:

Example application using Strust and Spring could be found at:

http://www.onjava.com/pub/a/onjava/excerpt/BFLJava_chap8/

Explanation of how Spring handles multithreading at (very good):

http://www.javalobby.org/articles/thread-safe/index.jsp

Example simple application using Strust+Spring+Hibernate(should probably start from here):

http://www.onjava.com/pub/a/onjava/2004/04/07/wiringwebapps.html?page=1

Many useful materials can be found at AppFuse project:

http://raibledesigns.com/wiki/Wiki.jsp?page=AppFuse

Of course, there are lots of materials and examples at www.springframework.org

Test Driven Development TDD

2005-04-17T22:30:00.000-05:00

Impact of Test Driven Development together with test automation on the software design.

by Igor Stoyanov.

I have already mentioned test driven development (TDD) in previous post on this blog. This is often also referred to as test first development.I am TDD practitioner myself, and recommend it highly. Although, some of the other XP techniques can be strongly debatable, I believe that TDD development and continues integrations are mandatory methodologies for the success of a software project today. From being the poor cousin of development, the unit testing - has become welcomed as the heart of the software development.

TDD brings many benefits. For example:

It is the best way of achieving comprehensive test coverage. Retrofitting test to existing code doesn’t produce quality test and it is not exciting at all for developers.

Writing test along with the application code can helps to define the requirements on each class. Theirs is nothing like writing a test first to make think about requirements of a method. What should it do on null argument? Should it return null or throw an exception.

Having comprehensive test suite gives us effective regression testing. We can add new functionality and even more importantly, have the courage to refactor existing code (an essential ability in order to have clean and maintainable code base).

It helps you write the simplest possible code. We don’t need to write complex code until we have a test that proves it’s required. If we continue, we can’t have a test that proves it until we have clear business need for that reflected in the story card requirements. In my opinion, this has the most beneficial reflection over designing clearer, readable and more maintainable code.

TDD means that developers have constant feedback about progress in the form of more and more passing test demonstrating required functionality. This is very satisfying.

In my opinion, using TDD is much more fun and fulfilling.

There are plenty that can be added to this topic. Good resources on this topic is this excellent blog: Exploration Through Example, the book “Test-Driven Development – A Practical Guide” by David Astels as well as very good and important book "J2EE Development withoug EJB" by Rod Johnson.

It is not the most intuitive thing to test something that doesn’t exist. In order to become comfortable with TDD you should start practicing with full believe and fully apply this methodology. Very soon, the benefit would be very obvious. It is simple concept, but I need some working example in order to fully realize the power of TDD approach.

I will briefly summarize the most important steps in TDD development and demonstrate a very simple example.

The most important characteristic in TDD are:

Create a test case to test business driven functionality.

Test against interface methods, which could be generated using IDE support.

Using interfaces is one the best Object Oriented principal and lead to very good overall design. Furthermore, testing against interfaces allows us to use mock object in our unit testing. Using mock objects can improve our unit testing and improve the performance of the test suite. It is very important that the tests run comparably fast because one of their purposes is to give constant and fast feedback to developers. Another advantage of using mock objects is that your test is more accurate and allows truly unit testing.

Always, always make the test to fail first. This is very important and often forgotten in practice. My experience has shown me that many times tests pass even if they are wrong. You should make sure that a test fails first in order to be sure everything is alright when the next time the same test pass.

Write negative test first. This is a test that verifies behavior beside the normal path of execution. Example of that could be data access exception during DAO testing or what happens if a required parameters has not been initialized. Such scenarios are hard to reproduce in a deployment environment, and hence hard to cover in component and integration testing.

Before a demonstration of TDD with a simple test case, I want to mention here that in order for TDD to have the desired effect, developers should create pure unit test. The most popular and sufficient Java framework is JUnit. The design and the decision for technologies that will be used should be strongly impact by the fact whether it is easily testable and in some cases, if it is testable at all. I want to add immediately here that no test can prove full coverage and suffice by itself for good testing approach to a particular application. A TDD development implies that developers are mainly involved in unit testing, also known as white testing. Continues integration testing and rigorous functional testing are invaluable testing approaches but they should not be primarily responsibility of developers.

As a practitioner of TDD I wouldn’t choice working with EJB technology if I have other valuable alternatives. EJB are managed during runtime by EJB container. Its development is not only very complex but it very hard, and in some cases (Entity Beans) it is almost impossible to test, not to mention that TDD is unthinkable. Attempts are made to solve this problem with EJB mocks, which is not very successful and using in container testing frameworks as Cactus. Although, using Cactus is irreplaceable for integration in container testing of EJB, it can not be used for pure unit testing. As I mentioned before, unit testing should provide very fast feedback to a developer in order to be useful. Using cactus as unit testing implies that for testing of even single method, we should wait for deployment and very complex initializing operations of the targeted container before we have a feedback. Not to mention the complexity of setting up Cactus framework on the firs place.

Luckily, lightweight containers come to rescue. Using these lightweight containers involves primarily the use of POJO objects that are the easiest to test and can lead to the best OO design of a given software project. The most popular ones are Spring framework and Pico container. I will talk much more about these containers a little later on.

TDD demonstration:

Requirements:

Business functionality: Withdrawal

Can anticipate: - Refusing access to data storage

- Insufficient Funds in the account

Prerequisites: - There is an existing account (I am really hoping for. J.)

- The account can accept deposits.

- The account knows the current balance.

Targeted test code:

public class BankAccountTest extends TestCase {
private BankAccount account;

protected void setUp() throws Exception {
account = new BankAccount();
}

public void testValidWithdrawalSuccedsWithinLimit() throws DataAccessException,
InsuffiecientFundsException {
account.setBalance(45.0);
account.withdrawal(20.0);
assertEquals(25.0, account.getBalance());
}

public void testNotValidWithdrawalFailOutsideLimit() throws DataAccessException{
try {
account.setBalance(45.0);
account.withdrawal(46.0);
fail("Bank should have rethrown InsufficientFundsExcepiton");
}catch(InsuffiecientFundsException ex) {
assertEquals("Correctly calcualted amount fo unapproved overdraft",1.0, ex.getShorfail());
}
}
}

Agile Development

2005-04-17T20:39:00.000-05:00

Reflecting the use of Agile Methodologies on overall Server Side Design and Development

by Igor Stoyanov.

Agile methodologies are alternative approach to the traditional Engineering Methodologies. The most immediate difference is that they are less document-oriented, usually emphasizing a smaller amount of documentation for a given task. In many ways they are rather code-oriented: following a route that says that the key part of documentation is source code. However, a more precise picture could be drawn with a short excerpt from the article “The New Methodology” written by one of the founders of Agile Manifesto, Martin Fowler:

Agile methods are adaptive rather than predictive. Engineering methods tend to try to plan out a large part of the software process in great detail for a long span of time, this works well until things change. So their nature is to resist change. The agile methods, however, welcome change. They try to be processes that adapt and thrive on change, even to the point of changing themselves.
Agile methods are people-oriented rather than process-oriented. The goal of engineering methods is to define a process that will work well whoever happens to be using it. Agile methods assert that no process will ever make up the skill of the development team, so the role of a process is to support the development team in their work.

The rest of this article is invaluable information about Agile Methodologies and there are many references to related links and materials. I will strongly suggest reading this article if you are interested in Agile Methodologies.

However, my purpose is not to do and overview of Agile Methodologies rather than I will look at only a few aspects of agile that I believe have very strong influence on the overall software design during software development of a business driven project.

Agile development is characterized mainly by using two to four weeks iterations. In the beginning of every iteration, the team commits to deliver some number of story cards. A story card is similar to the user cases defined in traditional software engineering process. The main and very essential difference that story cards has limited set of functionality so that the team can deliver it at the end of the iteration. Furthermore, everything in the story card should be driven by the business requirements.

This methodology influences the software design in the following two very essential ways:

The team is committed to deliver working product at the end of the iteration – avoid 80:20 problem in software development.
Develop functionality that is only business needed driven – doesn’t over design the current systems and produce much cleaner and maintainable code base.

There are many more consequences to add, but all other essential ones are summarized in XP development process, which is a subset of agile methodologies by itself

Continuous Integration

2005-04-15T22:15:00.000-05:00

Unit, Integration and Container testing using JUnit, Cactus and Ant frameworks

by Igor Stoyanov.

A strong principle of eXtreme Programming (XP) is the continuous integration aspect (see the Continuous Integration article by Martin Fowler). The traditional approach to the development of complex software has been to separate the development process into phases of coding, testing and integration. The continuous integration principle is to basically code, test and integrate at the same time, so that at almost any point in time, you have a working product.

In order to achieve this, you need to be able to automatically build your entire application, including running the unit and acceptance tests. Thus, tools such as Ant or Maven are the foundation for continuous integration.

Certainly, you need an actual unit testing framework to start with. I would not surprise anybody by choosing JUnit test framework for my unit testing frame work. It is so widely known and used that everything that I could say would be redundant. The only thing that I would like to add, although it is said by many other is that this framework is really very easy to understand and to use it.

The elegance comes when JUnit is integrated with Ant. This is the first step of achieving continuous integration. Actually, the fundamental benefit of continuous integration is that it removes situations where we are looking for bugs which could not be found during unit testing but would show up when we integrate the different parts in one whole system. These bugs are some of the hardest ones to find and only continuous integration testing could save us from a lot of headache in later phases of a project development. The integration of JUnit with Ant does exactly that.

Furthermore, because my project is a server-side application, I needed some framework for unit testing server-side code. I choose Cactus because it uses JUnit and could be integrated with Ant. The intent of Cactus is to lower the cost of writing tests for server-side code .Cactus implements an in-container strategy. I haven’t work with other testing frameworks for server-side code, but this was the framework that I had the most problems with from all other that I had used in my project until now. There were moments when I was very close to give up on it. I had so many problems only with configuring HSql database on JBoss server, but once I figured it out, I was wondering what took me so long. This wasn’t the case with Cactus framework. After been stacked with it for a long time, finally I succeed to make it work on both Tomcat and JBoss servers. Actually, most of the problems were on JBoss, but Tomcat wasn’t easy either. However, I am hoping for not having any problems with Cactus framework because I have no idea how I made it work. I am taking “once you made it work, just don’t touch anything anymore” approach with Cactus.

One of the problems was that there wasn’t implemented property in “cactus” task for JBoss.4.0.0 yet, so that I had to use the more complicated “runservertest” task. When this task is being used, it should be provided tasks for stopping and starting the current server.

However, I was very satisfied at the end. Although, Cactus framework was very difficult for me to configure and make it run, I found two very important bugs with Cactus testing that I would have very, very hard time finding. One of them was a bug that I had made. It was related to setup test data in the database. I had exchanged the roles for administrator and regular user. The strange thing was that when I tested via a web browser, everything worked fine, but Cactus find this data inconsistency. Otherwise, it would have shown up much later in the development of the project and would have created a lot of troubles for me. Just the thought about how much time I was going to spend on debugging for some minor bug like this, made me appreciate in-container unit testing as well as the continuous integration testing.

The second bug wasn’t mine! While cactus tests worked perfectly well on Tomcat server, they didn’t work on JBoss.4.0.0. It was very strange. The problem was in the statement:

request.isUserInRole(“user”);

JBoss.4.0.0 always returned false. After a lot of debugging and with the argument that this works on Tomcat, I started wondering about JBoss handling of “request.isUserInRole()”. I started searching the web and surprise, JBoss.4.0.0, which was released in this September (2004) has a bug and the statement above always returns false. Fortunately for me, somebody else found this bug, described in this post one week before me. There has been a patch released for this bug that could be found here.

Finding these two bugs convinced me very strongly for the need of in-container unit testing combined with continuous integration testing and for Test Driven Development as a whole. I will try to apply this kind of development very consciously and strictly in this as well as any future projects.

Page Layout

2004-09-18T13:05:00.001-05:00

Using SiteMesh versus Struts Tiles

by Igor Stoyanov

I needed web-application integration system to aid in creating the many pages of the web site for which a consistent look/feel, navigation, and layout scheme is required. Such frameworks are Struts Tiles and SiteMesh. While Struts Tiles are wide spread used and known, SiteMesh is an open source web-page layout and decoration framework and web- application integration framework, which is very powerful but not so popular.

Although, I still have some thought about my decision not to use portlets (at least for now), I am more than sure that choosing SiteMesh rather than Struts Tiles is not the right but the only decision than someone who has worked with both technologies could choose. The reasons are more than many but there are some that were very decisive. Some of the main reason is that SiteMesh is much easier to configure and use (less typing and easier to understand). Using Tiles, you need to have your forwards go to a "tiles page" versus the direct JSP. SiteMesh takes the approach that your page (your JSP) doesn't even know or care that it's being decorated. Using Tiles, each individual page you want to go to has to be associated with a layout - Major pain! Every time you create a new JSP that you want to forward to, you have to create another tiles definition and associate it with a layout and forward to the Tile page (versus the JSP).

The problem with most of the available frameworks for solving this type of business requirement like Struts Tiles is that they are either platform- or framework-specific. When you're using Tiles for decoration, you have to create tiles definition in tiels-defs.xml. Inside of your struts-config.xml while declaring forwards, you will have to use these tiles' definition instead of actual JSP.

The easiest possible solution is to let each of your web applications create plain HTML content without knowing about how it is being decorated, and only then have something else choose and apply appropriate decorators. This is precisely what SiteMesh does.

Although, SiteMesh is ideal for use with J2EE applications, however it can be integrated with server-side web architectures that are not Java based such as CGI (Perl/Python/C/C++/etc), PHP, Cold Fusion, .NET etc...

SiteMesh intercepts requests to any static or dynamically generated HTML page requested through the web-server, parses the page, obtains properties and data from the content and generates an appropriate final page with modifications to the original. This is based upon the well-known GangOfFour Decorator design pattern.

SiteMesh can also include entire HTML pages as a Panel within another page. This is similar to a Server-Side Include, except that the HTML document will be modified to create a visual window (using the document's Meta-data as an aid) within a page. Using this feature, Portal type web sites can be built very quickly and effectively. This is based upon the well-known GangOfFour Composite design pattern.

My experience with SiteMesh was incredibly positive and I would not look for any alternative framework anytime soon. The installation and the set up of SiteMesh includes:.

· SiteMesh is based on Java, J2EE, and XML and depends on filters, a very useful feature introduced in the Servlet specification 2.3, so we have to inform our web application about the SiteMesh filter. Adding the following lines to web.xml can do this:

<filter>

<filter-name>sitemesh</filter-name>

  <filter-class>

com.opensymphony.module.sitemesh.filter.PageFilter

  </filter-class>

</filter>

<filter-mapping>

<filter-name>sitemesh</filter-name>

  <url-pattern>/*</url-pattern>

</filter-mapping>

Here we are telling the container that all requests to this web application
should go through PageFilter. The PageFilter class is part of sitemesh-2.1.jar, which you can download from the SiteMesh downloads page. Once downloaded, you should copy
sitemesh-2.1.jar to the /WEB-INF/lib folder.

· Create a decorators.xml file in the WEB-INF folder like this:

<!—- declare the directory that our decorators will be -->

<decorators defaultdir="/decorators">

    <!—- decorate all page in our web application with “default.jsp” decorator (Decorator Pattern)-->

<decorator name="default" page="default.jsp">

          <pattern>/*</pattern>

</decorator>

    <!—- include a panels in the main window. Create visual windows in a page (Composite Pattern)-->

<decorator name="box" page="box.jsp"/>

</decorators>

The decorators.xml file is used to define decorators for your application. In this file, each < decorator> element defines one decorator. The name attribute is used to define a name for that decorator, while the page attribute defines the JSP page that should be used for decorating. The <pattern> child element defines when a particular decorator should be applied.

The last thing to do in order for SiteMesh to work is to create the decorators. The most important elements in a decorator are:

<?xml version="1.0"?>

<jsp:root xmlns:jsp="http://java.sun.com/JSP/Page"

xmlns:c="http://java.sun.com/jstl/core_rt"

    xmlns:fmt="http://java.sun.com/jstl/fmt_rt"

xmlns:decorator="http://www.opensymphony.com/sitemesh/decorator"

    xmlns:page="http://www.opensymphony.com/sitemesh/page"

version="1.2">

    <jsp:directive.page session="false" language="java" errorPage="/error.jsp" pageEncoding="UTF-8" contentType="text/html; charset=utf-8"/>

<html xmlns="http://www.w3.org/1999/xhtml">

   <head>

<title>

        <fmt:message key="webapp.prefix"/>

<decorator:title default="Structure"/>

     </title>

<decorator:head />

  </head>

<body>

    <!-- Some decorations here( header, page layout and so on)-->

.......................

        <decorator:body />

        .......................

</body>

</html>

Of course, this is just the basic but I think that even this could show
how easy, straightforward and in the same time very powerful is the
SiteMesh framework. I implemented everything that I wanted very easily
for my web interface without any compromises. I just wish that all
tools and frameworks that I use are so simple, easy to use and
intuitive like SiteMesh.

Web Interface

2004-09-15T13:01:00.000-05:00

Discussing the implementation of a user web based interface using JavaScript, CSS or Java Portlets.

by Igor Stoyanov

Note: This blog entry was before AJAX era. It sounds funny reading it today, but I would say it was kind of advance at the time when using javascript was considered evil.

Properly done web page design could be very important for the success of a given web application project. There were a couple of alternatives here which includes different compromises between portability, user satisfaction as well as performance issues. One possible way of implementing user web interface was using a new java specification for building web portals, called portlets. After very basic examination of this technology I decided not to use it for a couple of reason:

1) A request to the server is post every time a user wants to minimize, maximize or do something with the appearance of a particular portlet. This makes the overall performance of the web site to appear not very satisfactory. As an example I could give the new Loyola web based system – LOCUS. I don’t know if the provider, PeopleSoft, had used portlets but the site responding in such a way. Even for very simple operation as expanding of a tree structured menu, which doesn’t need any processing by the server, a request to the server is made and the whole page should be loaded again. This makes their web site looks very slow and a user experience is not very satisfactory, especially if the user uses dial up internet access. Since, I want a user to fill the web site very alive and responsive this was a major issue against using portlets. However, I suspect that later in the project I could encounter some transactional and other issues related to a web portal to be achieved in not so elegant way as if I decided on using portlets on first place. I have to come back to this particular choice later in the project in order more fully to estimate the benefits of using or not using portlets as web interface framework.

2) Another side of using portlet is that as a new technology, there are many different implementations and portable issues related with that.

Because of all stated above, I decided on using CSS and JavaScript as primary user interface technology. Although JavaScript and CSS are not very portable among different browsers and some of the users could disable JavaScript, I decided on using them only for web interface operation related to the feel and look of a web page. This would make the web site look more responsive for the users and overall more users would be satisfied than having problems with the web site.

In the process of implementing the web interface, I succeed in achieving my goals about the look and feel of the web interface. I believe that working with JavaScript together with CSS was sati factorable for me and the combination of these two technologies could enable you with the ability to implement very powerful features for your web site interface.

I used JavaScript and CSS for minimizing, maximizing and deleting of a different box window. Also, the state of the page layout and the different box windows are stored as cookies, so that every time a user go to the web site, he/she we see the same page layout that he/she left before closing of the web browser.

However, I didn’t implement with JavaScript any essential function concerning processing any user information. Even if JavaScript is disabled, a user would be able to use all major functions of the web site; just the experience of using this web application would be a little less satisfactory.

As a conclusion, I would like to say that looking at other very popular web portals like http://my.yahoo.com and Road Runner using JavaScript, CSS and other not so portable technologies like Java applet, Macromedia and so on are very heavily presented in order to make user interaction very fulfilling. Thus, I believe that I made a good choice with JavaScript but I will see how it would integrate with Java Server Faces in the later phases of the project.