Stubs vs Mock Objects

During my presentation on mock objects earlier this week, people started discussing the differences between stubs and mock objects. I wasn’t ready for this question and had used all my Q&A time up already. When I was asked to comment more on this off-line, I decided to discuss it a little here.

I’m not intending to write an essay here. Stubs and mock objects are quite interesting but I only want to give an introduction at this stage. So, with a quick refresher on the differences, here’s what I know…

Introducing a Term

To begin, I’ll need to explain what kind of objects both stubs and mock objects are. For this I’ll need to introduce a term. Since both stubs and mock objects are trying to achieve something similar, they can be grouped together. The phrase I’ve heard most is “Test Double”. The idea is that these objects are similar in some way to production code but aren’t completely identical. Instead, they are like stunt doubles in a movie. Martin Fowler tells us this term comes from Gerard Meszaros, who coined it in his book xUnit Test Patterns.

What is a Stub?

My understanding of stubs has been the same since before I learned about mock objects. The idea is of having a test double capable of satisfying the requirements of testing without the impact of some complex external system. When coded, these stubs normally provide information for consumption by the system under test.

Information within stubs are often hard-coded return values or properties. Their use is intended to satisfy the compilation requirements as well as fulfilling any needs required to test the intended behaviour of the system. The use of stubs is often confined to when production code either isn’t implemented yet or it would impact somehow to use a real object.

Here is a key concept when comparing with mock objects. Stubs are used generally when the real object either isn’t available or isn’t appropriate for testing. In most other situations, the real object is used.

Martin Fowler describes the purpose of stubs as being useful for state-based verification. Given that stubs normally have values and states hard coded, testing takes the approach of ensuring the system functions correctly in certain states.

What are Mock Objects?

I like to describe mock objects as simulations of an object. This description will likely be understood by some people and only work to generate more questions for others. Let me explain more…

Mock objects normally implement an object from end to end. They receive input, perform some process on that input, and generate output. The difference between this test double and a real object is that mock objects have an imaginary processing stage; i.e., it’s processes generate output based on what would happen if the real work had happened.

Let’s look at an example. Suppose I have an interface to determine what sizes a shoe manufacturer supplies as part of each model of shoe. The real object would take the name of the shoe and return all the sizes supplied. It might perform this operation by executing a stored procedure in a database to lookup the record ID of the shoe and match it against a many to many table relationship between shoe model IDs and sizes. It would then format the results in an array and return the results to the calling code.

A mock object would do exactly the same, it’s just that it would pretend that an array given as part of the unit test’s setup was the results of the database’s stored procedure. Let’s take a look at some code:

...
string sizes[] = {"8", "9 1/2", "11D"};
MockDatabase db = new MockDatabase(sizes);

string shoeName = "Puddle Jumper TM";
string shoeSizes[] = db.GetShoeSizes(shoeName);
...
Example 1: Populating a mock object with database information

As you can see, instead of going to a real database, it has some values we have chosen for our test case. Take note, this mock object may be used many times and could use a different set of shoe sizes for each test case. In fact, it could use several sets of shoe sizes within just one test case! This is an important difference from stubs, these values are not hard-coded.

In the background, GetShoeSizes(string) takes the given array and formats it in to the expected representation. On top of this, we could have added features for testing, such as counting the invocations of the method, generating exceptions, as well as ensuring these statistics are all within our expectations. Instead of a silent partner, the mock object becomes co-star, defining and testing our expectations!

Martin Fowler describes mock objects as being used for behaviour-based verification. The input and output are not coded but are instead in part processed (or as I term, an imaginary process), and defines the behaviour of the application.

This point is crucial. It beckons the introduction of contracts within the design of software. As soon as you have test cases defining the expected behaviour, as mock objects do, you are building a contract between the application, the test case, and the developer who is intending to develop the production code. For this reason, test cases should be written to reflect the requirements closely.

Summary

So, hopefully this blog post has helped with the question of what difference there is between stubs and mock objects. We’ve learned that stubs are normally hard-coded substitutes for production code whereas mock objects have an imaginary process which helps to build expectations of how the production system will behave. Also, we’ve learned that stubs work with state-based verification while mock objects work with behaviour-based verification.

Hopefully this has provided an interesting introduction to the area of mock objects. You can read more from Martin Fowler’s essay on this topic at http://martinfowler.com/articles/mocksArentStubs.html, where I did my quick refresh on the differences between these two kinds of objects.

This topic of stubs and mock objects moves in to the space of Design By Contract and Test Driven Design. See my post on Test Driven Design if you’d like my introduction to that space as well.

As usual, keep the discussion going and leave a comment.

Kind Regards
Glenn

Presentation

Well, I did say I was going to be busy. Yes, I’m still here and Quality in Software is still active. Hooray!

These last weeks have seen me feeling like my time has been pretty thinly spread. Nevermind, still, what’s happening?

Well, tomorrow I’m giving a presentation on Testing Components with Complex Associations. It’s a dive in to the world of Mock Objects, patterns, and other things like that. Of course, I’ve held back because my space is only 20 minutes. Quality is such a big topic.

Just lately, I’ve been in a test lead role. It’s been good. I’m surprised just how much I think about my test lead back at Motorola; Chris Lawn. Very dedicated person who I believe has had an impact on how I approach the role of testing.

In the not too distant future? Work, I’m real busy right now, more blogging, and of course one of my favorite professional development activities, reading books. They’re so invaluable!

I’ve been meaning to have a page to share book titles. That will still go ahead. I look forward to sharing book titles with you all as I’m always on the lookout for a great book on quality.

Anyway, for now, it’s back to work. I just got an updated spec I have been waiting for so I’m on the quality trail yet again.

Kind Regards
Glenn

Glenn’s Wedding App – Using Test Driven Design

Here at DWS, I’m “On the Bench”. This essentially means I don’t have any client work to do so I’ve been doing things for DWS internally as well as professional development. With this spare time, I’ve been practicing my skills in XML, .NET 3.5, and TDD (Test Driven Design).

I’m not going to pretend that I know everything about TDD, I don’t. What I will do, however, is tell you about my understanding of this fast way of designing good quality software that has fantastic code coverage as well!

I used to code in .NET 2.0 a lot at home and in TAFE (a technical college where I learned software development): as well as .NET 3.0 as part of my job here at DWS. Having said that, much of my work for the last 9 months or so has been testing, specifically leading UAT at a client’s site, so my coding skills are a little rusty.

I wanted to sharpen these coding skills but first I needed a project. Too easy! My upcoming wedding to my darling fiancee Jenna is a great inspiration for a project! So, I started a project that tracks wedding-related events: you know those events, engagement party, pre-wedding parties, organising the wedding, etc. My project will tell me which events are coming up next as well as how much time there is until the event.

Now, where does Test Driven Design come in to this? Well, it’s a great way of getting software written quickly with high quality, so that’s the method I have chosen to write my program. So, what is Test Driven Design?

Test Driven Design as I understand it is a process where you begin with a single concept. This concept says that any piece of software may either be complete or incomplete. Any defects are just where the software is incomplete. To demonstrate this, a test is written that confirms the software is incomplete and provides a proof for when it is complete. It starts with writing a test case that describes a little of the functionality of the final product.

Once the test case is written, the test case is compiled. At this stage, there is no production code and so the compilation of the test case fails. Let’s take a look at an example:

[TestMethod]
public void DoStuffTest()
{
  Something target = new Something();
  bool expected = true;

  target.DoStuff();
  bool actual = target.StuffDone;

  Assert.AreEqual(expected, actual);
}
Example 1: Initial Test Case

So, there’s a test. The idea here is that Something doesn’t exist, so the compiler fails to compile the test. Let’s add Someting.

public class Something { }
Example 2: Inital Production Code

Ok, so I know what you’re thinking, “He’s left out all the boring stuff.” Not at all, that’s it! There isn’t anything more. Let’s compile… Compilation Failed! Error: DoStuff() is not defined.

That’s right, we have stubbed the class but haven’t filled in the methods yet. Simply put, we didn’t do that because it wasn’t necessary to fix the problem at hand. That’s what TDD is about, fixing what is at hand to the very minimum required to satisfy the tests. As a result, you get very high code coverage, as in greater than 90%, often 100%, and the code is simple and testable.

We continue down this path until we finally get a compiled program: Compilation Successful! Now we run the test…

Test Failed: Expected: true, Actual: false

What happened? Let’s look at the code.

public class Something
{
  public void DoSomething()
  { }

  public bool StuffDone
  {
    get { return false; }
  }
}
Example 3: Production Code - Meeting the minimum requirements of the test case

Ah, this is easy to fix:

...
get {return true;}
...

Now it passes! Fantastic! But now we have only proven that we can know when something has happened, what about before it’s happened? Let’s change our test:

[TestMethod]
public void DoStuffTest()
{
  Something target = new Something();
  bool expected1 = false;
  bool expected2 = true;

  bool actual = target.StuffDone;
  Assert.AreEqual(expected1, actual);

  target.DoStuff();
  actual = target.StuffDone;
  Assert.AreEqual(expected2, actual);
}

Example 4: Updating the Test Case

Now, see the change? We check target.StuffDone before we call DoStuff() and expect it to return false. Of course it should be false, we haven’t “done stuff” yet! Let’s compile: Compilation Successful. Not a problem, it only exercises existing production code. Let’s run the test:

Test Failed: Expected: false, Actual: true.

Ok, so what happened? Remember how we made our code work by statically returning true from the StuffDone property? We’re going to have to be a little smarter now.

How can we get the test to pass with the simplest change to the production code? Really, the best way is to change the result of StuffDone when we call DoStuff(). Because the first Assert statement expects StuffDone to be false, that’s what we’ll initialize it to satisfy that test.

// Something that does stuff.
public class Something
{
  private bool mStuffDone = false;

  // Do something.
  public void DoSomething()
  { mStuffDone = true; }

  // Has something been done? Return true if yes.
  public bool StuffDone
  {
    get { return mStuffDone; }
  }
}
Example 5: The Completed Production Code

Easy! Now, I know what you’re thinking, it’s easy because it’s just some simple example. Well, yes, that’s right but it’s also how TDD works. Everything is broken down in to simple bite-sized pieces. Now, think about what we have here. We have a fully tested piece of production code, a simple design, and a test that proves that we have delivered on the requirement described by the test case: let’s just say, green ticks are hard to argue with.

You’ll likely want to refactor your code at some stage but at least you won’t be refactoring too early and you have test cases to check that your refactoring doesn’t break anything. Try writing test cases around the requirements in your projects. I find it easy and rewarding.

Kind Regards
Glenn

P-Invoke Interop Assistant

Hey

Something good for software quality is good tools. Occasionally, someone comes up with a tool that makes you say, “Hey, that’s pretty good!” That’s what I said to myself when I saw the P-Invoke Interop Assistant on MSDN’s Channel 9.

I won’t go in to too much, you can watch the video and look at the website for that. What I will say is that using native APIs from managed code can be hard. This tool works to make it easier.

If you’re using a lot of native APIs in managed code, another resource is http://www.pinvoke.net .

Kind Regards
Glenn