Saturday, January 28, 2012

Understanding the power of isolated object testing

I spent some time last night watching J.B. Rainsberger's excellent InfoQ presentation entitled "Integration Tests are a Scam". I've been recently contemplating why I favor unit testing (or as J.B. calls it, isolated object testing) over integration testing and I found his presentation particularly relevant. I highly recommend watching it. He also has a series of blog entries that support the presentation (Parts 1, 2, and 3). I also recommend reading those. They're truly gems.


Testing to prove basic correctness

I find many developers using integrated tests as a way to prove the basic correctness of the class or system under test. J.B. writes that "While integration tests offer value in other contexts, too many programmers use them to show basic correctness, and when they do that they waste a tremendous amount of time and effort." Integrated testing can be used within a project (I'm personally fond of acceptance testing), but integrated testing should not be used to prove basic correctness of your code. Focused, isolated object tests (aka unit tests using test doubles) should be used for this endeavor. If you discover behavior that a collaborator demonstrates and you have not accounted for in your isolated object tests, you should mimic this behavior in your test doubled collaborator contracts. You want to cover as much of your code with isolated object tests. We'll talk more about contract tests later.

J.B. mentions using an integrated test to learn about how a collaborator might support its contract, but that this integrated test is not included in the basic correctness test suite. I'm wondering if there isn't some other test suite here that we could use to keep integrated tests that support our learning the runtime and external dependencies. This test suite would be run periodically, but is not part of the whole continuous integration process of building a software system. Need to noodle on this more.


Isolated object testing

J.B. states in his aforementioned presentation that he does not use the term "unit testing" and instead favors a more focused term of "isolated object testing". He makes a point to call out the isolated word; these tests isolate the class under test by using test doubles to stub or mock the collaborators of the class under test. These tests focus on a single object and a singular behavior. Any collaborations are realized using test doubles.

I tend to agree that the phrase "unit testing" is a weak phrase describing the type of testing that we use to drive design of a single class and a single method. These tests focus on the responsibility of a single method of a single class. These tests also help you focus on understanding the contracts of collaboration between this class under test and its collaborators. This is really where the design part of the equation comes in. Classes that are hard to test are screaming out that the design is wrong. The tests are saying you have too much responsibility in the class under test or there are too many collaborators. Using your fast isolated object tests, you can engage in a reactive design activity, moving responsibilities out of the class under test, create new collaborators, and other design changes to make .


The collaboration contract

J.B. rightfully makes a big deal about contracts (aka interfaces) to explicitly specify collaboration protocols between objects. He mentions that in Domain-Driven Design, popularized by the Eric Evans book of the same name, three concepts can be used to express a domain-driven model: Values, Entities, and Services. All Services should have contracts and those contracts manifest themselves as interfaces. By specifying interfaces, we explicitly declare the protocol supported by each interface implementation. Constraining ourselves to interface types when specifying collaborations results in looser coupled systems, which is considered a "Good Thing". When engaging in isolated object testing, J.B. details the concept of collaboration tests and the closely associated contract tests. This concept of collaboration tests and contract tests is something new to me and is a big reason the presentation was so valuable in my eyes.


Collaboration tests and contract tests

After watching the presentation, I seem to be doing a pretty good job of using collaboration tests, but I'm not making the association to contract tests. Collaboration tests prove that the client interacts with its collaborators correctly; the client sends the correct messages and message arguments to the collaborator and appropriately handles all outputs from the collaborator. This is traditionally what I have used mock objects for and that seems to be what J.B. is arguing one should do for collaboration testing.

Contract testing, on the other hand, deals with testing that an interface implementation accurately respects the interface it is implementing. Does the implementor support the contract it declares to support? I haven't typically written these types of tests, but I'm going to start. Interestingly, I don't see a lot of this sort of testing in the wild. The part I really dig about contract tests as Rainsberger explains them is that they can be reused across interface implementations. He uses the List interface and two of its implementations, ArrayList and LinkedList, and details how to use implementation inheritance in the tests to DRY up your contract tests.

It really gets interesting when he declares that for every collaboration test that implies that a test double behavior, there better be a corresponding contract test that demonstrates that the interface implementation(s) actually does support that behavior. Same goes for values returned: if a test double returns a value in a collaboration test, there should be a contract test that demonstrates that the real implementation(s) does actually return that value. This is where I think selective, integrated learning tests can help you discover how your classes on the edges of a system may act when integrated to real external dependencies. But again, those integrated tests are not providing basic code correctness semantics. They're in your project to help you learn, but are not part of the isolated object tests suite. Don't lump them with your isolated tests and don't run them as part of your code/update repos/run tests/commit cadence.  This learning test suite should be run periodically during the day, but not part of the CI build process.

The death spiral of integrated testing for proving basic code correctness

Both Rainsberger's tutorial presentation and his blog postings go into great detail to the fallacy of using integrated testing to prove code correctness. I won't rehash what he has to say about it. My conclusions that I draw from his material are:

  • Focus on isolated object tests and isolation techniques and learn how to do collaboration and contract testing.
  • If you need to learn, by all means, write an integrated test. If you need to reproduce a defect, initially write it as an integrated test. However, in all cases, take what you have learned and replicate that in your isolated object test suite.  Replicate the behavior you discovered in the integration test(s) with test doubles.  This will ensure that your isolated object tests stay true to the behavior of the integrated system.
  • Don't get sucked into the downward death spiral of using integrated testing to guide your test-driven design efforts. It will only cause you pain.

J.B.'s series on Integrated Tests are a Scam can be found here.



  1. I absolutely love "death spiral". I wish I'd called it that.

    Thank you for your support of the idea. I'm particularly interested in knowing how it helps you in your work.

  2. Solid advice. Thanks Chris. Using "isolated" to mean "single object and singular behavior" works better for my feeble mind than the word "unit". As I think about it, what we're calling unit tests on my current project might in some cases not be proving basic code correctness. I probably should take one of your TDD classes at DevJam. ;-)

  3. Hmmm... I've seen so-called isolated object tests make so many assumptions about the collaborators it uses that they become a maintenance nightmare, false-redbaring constantly. And I've seen integration tests become so easy to write and so able to survive refactoring that you will take them out of my cold, dead hands to prove the correctness of my code. So I have trouble with this one.

  4. This comment has been removed by the author.

  5. Colonel Nikolai, I respect your opinion. I'm not saying that I don't value integrated tests, I just tend not to use them for code correctness. I've been on more than one project that has allowed integrated testing to get out of hand and turn the entire testing feedback loop on its head. Anything over 2 minute for a full run of the integration test suite kills the feedback loop.

    I love your comment about false-redbarring. Yep, isolation techniques, if not properly controlled, can be a huge mess. You definitely have to understand what the contract is when using test doubles. Thanks for the read and the comments.

  6. Do you know how to keep a good mood. Just remember to pay attention to the fashion trend of Paris Hermes such as Hermes Constance Handbag or the other styles like Kelly Hermes Bag. I promise you will relax yourself. So come on, know more fashion style arround you and make yourself more confidence. You will like it.

  7. Otr214428
    Work from home theory is fast gaining popularity because of the freedom and flexibility that comes with it. Since one is not bound by fixed working hours, they can schedule their work at the time when they feel most productive and convenient to them. Women & Men benefit a lot from this concept of work since they can balance their home and work perfectly. People mostly find that in this situation, their productivity is higher and stress levels lower. Those who like isolation and a tranquil work environment also tend to prefer this way of working. Today, with the kind of communication networks available, millions of people worldwide are considering this option.

    Women & Men who want to be independent but cannot afford to leave their responsibilities at home aside will benefit a lot from this concept of work. It makes it easier to maintain a healthy balance between home and work. The family doesn't get neglected and you can get your work done too. You can thus effectively juggle home responsibilities with your career. Working from home is definitely a viable option but it also needs a lot of hard work and discipline. You have to make a time schedule for yourself and stick to it. There will be a time frame of course for any job you take up and you have to fulfill that project within that time frame.

    There are many things that can be done working from home. A few of them is listed below that will give you a general idea about the benefits of this concept.

    This is the most common and highly preferred job that Women & Men like doing. Since in today's competitive world both the parents have to work they need a secure place to leave behind their children who will take care of them and parents can also relax without being worried all the time. In this job you don't require any degree or qualifications. You only have to know how to take care of children. Parents are happy to pay handsome salary and you can also earn a lot without putting too much of an effort.

    For those who have a garden or an open space at your disposal and are also interested in gardening can go for this method of earning money. If given proper time and efforts nursery business can flourish very well and you will earn handsomely. But just as all jobs establishing it will be a bit difficult but the end results are outstanding.

    Freelance can be in different wings. Either you can be a freelance reporter or a freelance photographer. You can also do designing or be in the advertising field doing project on your own. Being independent and working independently will depend on your field of work and the availability of its worth in the market. If you like doing jewellery designing you can do that at home totally independently. You can also work on freelancing as a marketing executive working from home. Wanna know more, email us on and we will send you information on how you can actually work as a marketing freelancer.

    Internet related work
    This is a very vast field and here sky is the limit. All you need is a computer and Internet facility. Whatever field you are into work at home is perfect match in the software field. You can match your time according to your convenience and complete whatever projects you get. To learn more about how to work from home, contact us today on and our team will get you started on some excellent work from home projects.

    Diet food
    Since now a days Women & Men are more conscious of the food that they eat hence they prefer to have homemade low cal food and if you can start supplying low cal food to various offices then it will be a very good source of income and not too much of efforts. You can hire a few ladies who will help you out and this can be a good business.

    Thus think over this concept and go ahead.

  8. You made some good points there. I did a search on the topic and found most people will agree with your blog...Software Testing Training in Bangalore

  9. It is really a great work and the way in which u r sharing the knowledge is excellent. Java Training in Chennai | Blue prism Training in Chennai

  10. Great post and informative was awesome to read, thanks for sharing this great content to my vision.
    Good discussion.
    Six Sigma Training in Abu Dhabi
    Six Sigma Training in Dammam
    Six Sigma Training in Riyadh

  11. Thank you for this brief explanation and very nice information. Well, got a good knowledge. Java Training in Chennai | Blue prism Training in Chennai

  12. You rock particularly for the high caliber and results-arranged offer assistance. I won't reconsider to embrace your blog entry to anyone who needs and needs bolster about this region.
    nebosh course in chennai

  13. Can you recommend any other blogs/websites/forums that deal with the same subjects? Thanks.
    safety course institute in chennai

  14. Thank you for sharing your article. Great efforts put it to find the list of articles which is very useful to know, Definitely will share the same to other forums.
    Data Science Training in chennai at Credo Systemz | data science course fees in chennai | data science course in chennai quora | data science with python training in chennai

  15. Great informative bog. Thanks for sharing such a valuable information with us.

    Article submission sites

  16. Innovative thinking of you in this blog makes me very useful to learn.
    i need more info to learn so kindly update it.
    Selenium Training in Nungambakkam
    Selenium Training in Vadapalani
    Selenium Training in Kelambakkam
    Selenium Training in Padur

  17. One of the best blogs that I have read till now. Thanks for your contribution in sharing such a useful information. Waiting for your further updates.

    IELTS Course in Mumbai
    IELTS Institute in Mumbai
    Best IELTS Coaching Classes in Mumbai
    IELTS Coaching Center in Mumbai
    Best IELTS Classes in Mumbai
    IELTS Classes near me
    IELTS Courses in Mumbai

  18. Such an excellent and interesting blog, Do post like this more with more information, This was very useful, Thank you.
    Aviation Academy in Chennai
    Aviation Courses in Chennai
    best aviation academy in chennai
    aviation institute in chennai

  19. This is really impressive post, I am inspired with your post, do post more blogs like this, I am waiting for your blogs.
    air hostess training in Bangalore
    air hostess academy Bangalore
    cabin crew training institute in Bangalore
    air hostess training center in Bangalore

  20. Very nice post here and thanks for it .I always like and such a super contents of these post.Excellent and very cool idea and great content of different kinds of the valuable information's.
    rpa training in bangalore
    best rpa training in bangalore
    RPA training in bangalore
    rpa course in bangalore
    rpa training in chennai
    rpa online training

  21. Read all the information that i've given in above article. It'll give u the whole idea about it.
    Devops Training courses
    python Training in chennai
    Devops Training in Bangalore

  22. Congratulations guys, quality information you have given!!!
    Data Science Course In Chennai