Igor's Blog

"... no matter what they tell you, it's always a people problem!"

Monday, January 28, 2008

Observing Behavior

I had to write very simple counting method today. I started with a unit test that pretty much executes a method (which increment a count) and asserted that the count was incremented. This is the test that I wrote:
public void testShouldRecordSystemException(){
exceptionMonitor.recordSystemException();
assertTrue(exceptionMonitor.getCurrentCount());
}

and this is the code that I was intending to write:

public void recordSystemException() {
storeNewTotal(++currentCount);
}

The problems with this test are:
- exposing internal state and breaking encapsulation with getCurrentCount() without anybody using it. (Yes, I know that I can make it protected instead of private, but this is not good enough for me. Changing private methods to protected in order to be tested is a topic, hugely discussed already in the blogsphere and I don’t have too much to add. I share the opinion that exposing private methods to protected just for testing looks like design smell and I try to avoid it. However, from time to time, I could make a method protected just for testing if this is the most pragmatic thing to do).
- This is pure state based testing. Although, sometimes state based testing is the better solution in many other cases verifying behavior is leading to much better design.

As a result, this small test reminded me about one presentation that I gave in our office about a year ago. The presentation is about “Mocking Abuse” but the final point of that presentation is that there is a middle ground between state base testing and interaction base testing (using mocks) that I call “verifying observable behavior”.

What that means is that this test should be changed to observe the behavior instead of verifying the object state:

public void testShouldRecordSystemException(){
assertEqual(1, exceptionMonitor.recordSystemException());
}

What happened here is that the method recordSystemException() changed its return type from void to int:

public int recordSystemException() {
storeNewTotal(++currentCount);
return currentCount;
}

The first question that comes to mind is how to verify that the method executes correctly because you can pass the test above with implementation like:

public int recordSystemException() {
return 1;
}

This is where the thinking about testing state vs verifying behavior should change. In practice, in order to verify behavior, we perform series of steps. Similarly, in mathematics we have so called “Mathematical induction”. Very simplistically put, a given statement is true for all natural numbers if the statement is true for 1, 2 …. n, and n + 1. Similarly, just one verification for verifying observable behavior is not enough:

public void testShouldRecordSystemException(){
for(int n = 1; n< 5; n++){

assertEquals(n, exceptionMonitor.recordSystemException());

}
}

Now, this test should make me implement the right behavior.

That is not a ground braking technique or something not widely known. This is just a change in the way of thinking about design and testing. What is more, in functional languages like Ruby and Python this kind of thinking is encouraged from the fact that every method (function) has always a return value, which allows for observing the behavior of every method.

So, back to the presentation that I mention earlier, it is about something similar. Generally, in our desire to avoid state based testing (which sometimes leads to poorer design, braking encapsulation and so on) we go to the other extreme with interactive testing using mocks (exposing implementation being one of the negative consequences). I already wrote way back about this development behavior in Stubs or Mocks and State or Behavior.

This presentation is building the case on top of those two blog posts that mocks are not the new silver bullet and there is a middle ground between state based testing and interactive based testing that I call verifying observable behavior.


Mock Abuse!



As usual, any feedbacks are welcome!

Labels: , ,

|| Igor, Monday, January 28, 2008

2 Comments:

I would argue that, by changing the return type of recordSystemException(), you've exposed the state of your exceptionMonitor in the same way that a getCurrentCount() method would. How is that better than changing a method from private to protected? Would code that uses the exceptionMonitor object actually care about the return value? Is that the way in which the exceptionMonitor is useful? Maybe all you want is a count of the number of exceptions recorded, but in that case an accessor is exactly what you want to test.

I'm guessing that's not the case, though. Does the exceptionMonitor write an exception to a file? Mock the file object and make sure something relevant gets written to it. Does it use a Log object? Mock the log. Does it write to a database? Mock the database. Does it record to something, but you don't know exactly what? Then you should have an interface on top of the various implementations (file, log, db, etc.) which you can then mock.

I like "verifying observable behavior" as a phrase, but I think that the behavior observed should be natural to how the code is actually used, not an artificial observation that is provided only for testing purposes.
Blogger Brian Reilly, at February 06, 2008 10:11 AM  
Brian, thanks for the good comments you provided to my post.

You raised all valid questions and concerns and I agree with most of them, but the point I was trying to make is that all of them have pros and cons and I am looking for some kind of middle ground. I would try to clarify my point a little bit more here, but I am very happy with your questions, because I have asked them myself many times and still looking for optimal answers.


>>>>

I like "verifying observable behavior" as a phrase, but I think that the behavior observed should be natural to how the code is actually used, not an artificial observation that is provided only for testing purposes.

<<<<


Completely agree with the notion that we should not create additional stuff just for testing. However, when you create your object (domain object for example) you have some idea how it would be used (for example, you know that your object should expose “recordSystemException”) but you don’t know all the details, especially if you start from your domain objects first (bottom to top approach). Almost all the time, I had to go back and redesign or add additional methods to my domain objects based on things that I discovered later (like we need a getter method for the UI that wasn’t specified and so on…). So, I look at testing, not only as a verification whether the code is working or not, but also as a very substantial for the design of my objects. In this particular case, I needed the current count after recording to check whether a service should be blocked or not but this is out of context.

I believe that by having such methods not returning anything, you don’t specify behavior and you leave the caller to figure out the result of calling such a method. To give some analogy - this is similar to a situation when you want to ring a bell in a big building. You get in front of the building and ring the bell. (Basically, you call method ringBell()). Now, there are places that you hear some sound when you ring the bell, but in other places you don’t hear anything and you wonder whether the system is working at all or not. I agree that the fact that you hear(or don’t) a sound doesn’t mean that the people that you are trying to reach hear the bell too, but at least the system is giving you some indication that is working. Obviously, this translate to a method ringBell() with return type of either void or boolean (or some kind of “Sound” type if you want to be fancy).

In both cases (with return value or not), the caller will (or not) achieve the desired result, but I argue strongly that home systems that are design to indicate in some way that the bell is ringing are better designed system than the ones that don’t indicate. (I know that now we use cell phones for that purpose, but the UPS guy is still using the bell :-))

To finish with this point, this topic is very similar to the debate about human interfaces vs minimal interfaces (http://martinfowler.com/bliki/HumaneInterface.html ) or to
fluent interfaces (http://martinfowler.com/bliki/FluentInterface.html).

>>>>>>>>>>>>>>>>>>>>

Does the exceptionMonitor write an exception to a file? Mock the file object and make sure something relevant gets written to it. Does it use a Log object? Mock the log. Does it write to a database? Mock the database. Does it record to something, but you don't know exactly what? Then you should have an interface on top of the various implementations (file, log, db, etc.) which you can then mock.

<<<<<<<<<<<<<<<<<<

Yes, I am not sure to what recordSystemException() is supposed to write (database in my case, but this is out of context again). That’s why I designed the MonitorException object to delegate the storage responsibility to Repository object, which hided all of details about storage. ….and yes, Repository have an interface, …and yes, it is very easy to create a Repository mock and test it with mocks, …and yes, you will test the right behavior without breaking the encapsulation, … and yes, it would be a very good solution.

However, this was the point of the presentation. I would not repeat everything again, but very briefly – there are some negatives together with the positives when using mocks and the biggest negatives are exposing implementation details (which in my opinion is worse than exposing state) and difficulties with refactoring and maintaining tests. I agree, however, that using mocks sometimes is the better solution and this could be just preference, but in that particular case, if I use mock for Repostiory (which is interactive testing), I would expose unnecessary implementation details, because I have to interact with Repository…. and yes, Repository is an interface, but you don’t need to know about it when you test “recordSystemException”. Again, this could be preference based, but I believe that there are consequences of using mocks in this case that I tried to indicate in my presentation.

>>>>>>>>>>>>>>>>>>>>>>>>>>>>>

I would argue that, by changing the return type of recordSystemException(), you've exposed the state of your exceptionMonitor in the same way that a getCurrentCount() method would.

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

Well, here I respectfully disagree. Probably, I should’ve change the return type to “boolean”, or return a “success” in form of “string” instead of “int” in order to demonstrate the idea better, but this is totally not exposing the internal state. I already gave the example with the door bell above that this is not exposing state, just “exposing observable behavior" and the definition for that is that I can change the return type at any point without changing any state on the object.

Again, most of those debates are coming to personal preference and prior experience with design and testing but definitely these comments raise very good points.

Thanks again, Brian.
Blogger Igor, at February 06, 2008 1:31 PM  

Add a comment

Links to this post:

Create a Link