Update

20 10 2008

Hey all

It’s been a while since I last posted; sorry about that. Recent times have found me quite busy as test lead. Still, I’ve managed to come across a few things that are of interest.

Visual Studio 2010

Microsoft are already pushing the bandwagon out. VS2010 will introduce some exciting new features including requirements management. In their mission to democratize the software engineering landscape, they introduce a host of features around the area of managing requirements, test cases, test runs, and defects. Additional to the VS we all know, there ability to record requirements, perform traceability analysis and coverage analysis. Microsoft say the pricing model will stay largely unchanged. I haven’t played with the preview yet but you can find out more on Channel 9.

comp.RISKS

You know, I like to keep in touch with what’s happening in the world with regards to quality. This means both the successes and the failures. One source I found to be interesting and with a good deal of content is the comp.RISKS on Usenet. Check it out some time. It provides you with the latest on quality done well and quality done badly.

ISTQB Certification

International Software Testing Quality Board (ISTQB) Certification is an internationally recognized certification in software engineering. It’s not just testing but most of it is; the foundations can be studied by anyone who wants to know why we test. I’ve started pursuing this certification in a self-study format as there are no courses offered in Adelaide.

Anyway, I’m on the bench again for a short time, so I should be able to blog a lot more than I have. Again, sorry for the lag, life happens; I’m sure you know how it is.

Kind Regards
Glenn





Hawthorne Wins the Grand Final

29 09 2008

Hey!

I just wanted to mention that Hawthorne, the AFL team who are sponsored by DWS, won the grand final this weekend!

Kind Regards
Glenn





Google Chrome

23 09 2008

As part of the internal work here at DWS, I tried Google Chrome. What I was looking for was another browser that did in-line spell checking; Firefox is the other one I know of and there are likely more. Interestingly, IE doesn’t provide this functionality.

As I looked around this new browser, I went through the comic presentation on the design decisions. It’s quite inspiring and the browser does well for their approach. What I want to talk about today is the sort of thought that went in to this browser.

Threading vs Multi-Process

A stand-out decision was to make each tab a different process. IE, in comparison, treats every tab as a separate thread. If you’re not quite familiar about topics this close to the silicon, keep reading, I’ll expand on how very different these two approaches are!

Threading is all about one process doing multiple things. Threading is often used on the Windows platform. It is a historical decision as much as a technical decision. Some of the key considerations about this are:

  • If the master thread dies, all threads die
  • There are difficulties to be overcome when threads begin talking to each other or using the same memory space
  • Threads have to be monitored and managed
Multi-processing is quite different. It’s about multiple processes all doing their own thing. The multi-process approach is very popular on the Unix platform as well as Unix clones such as Linux. Again, it is a historical decision as much as a technical decision and its key considerations are:
  • Processes are managed by the operating system rather than by a master thread
  • Processes are self-containing, if all the other processes die, the one remaining continues to function unaffected
  • There is no shared memory to manage, although some information can be exchanged using a shared space

Now, what makes one better than the other? For me, it’s all about two competing desires: memory footprint and simplicity. Threads are a smaller memory footprint while multi-processes are simpler to implement. Interestingly, Google went the way of multi-processing and their reasons for doing it appear quite sound. I won’t go in to detail here but take a look at their comic presentation for details.

Focusing on the Tab

Google have also focused greatly on the tab. They think it so important, they put it at the top of the window. Each tab has its own address bar and buttons as well. What’s more, because each tab is a separate process, it’s simple to undock each tab in to a new window.

Speed

People I ask about Google Chrome all say the same thing: it’s fast. A lot of effort has been put in to speeding up the rendering engine as well as Google’s own JScript VM which compiles native code. It all makes a faster browsing experience.

Security

Google sandbox their browser processes. This means that websites can’t change your hosts file to some-very-bad-website.com to steal your credit card number or show inappropriate content. Of course, there are some limitations with plug-ins because they’re not sand-boxed, but the attach surface is greatly reduced.

Summary

So, why do I care about blogging about some new browser? The reason why is because it is another example of people getting together to produce a quality product. It’s not about what makes it so good, it’s about how Google went about creating this product. As a tester/developer, I want to make fantastic cool products that close the gap between people and their tasks.

Kind Regards
Glenn 





Stubs vs Mock Objects

15 09 2008

During my presentation on mock objects earlier this week, people started discussing the differences between stubs and mock objects. I wasn’t ready for this question and had used all my Q&A time up already. When I was asked to comment more on this off-line, I decided to discuss it a little here.

I’m not intending to write an essay here. Stubs and mock objects are quite interesting but I only want to give an introduction at this stage. So, with a quick refresher on the differences, here’s what I know…

Introducing a Term

To begin, I’ll need to explain what kind of objects both stubs and mock objects are. For this I’ll need to introduce a term. Since both stubs and mock objects are trying to achieve something similar, they can be grouped together. The phrase I’ve heard most is “Test Double”. The idea is that these objects are similar in some way to production code but aren’t completely identical. Instead, they are like stunt doubles in a movie. Martin Fowler tells us this term comes from Gerard Meszaros, who coined it in his book xUnit Test Patterns.

What is a Stub?

My understanding of stubs has been the same since before I learned about mock objects. The idea is of having a test double capable of satisfying the requirements of testing without the impact of some complex external system. When coded, these stubs normally provide information for consumption by the system under test.

Information within stubs are often hard-coded return values or properties. Their use is intended to satisfy the compilation requirements as well as fulfilling any needs required to test the intended behaviour of the system. The use of stubs is often confined to when production code either isn’t implemented yet or it would impact somehow to use a real object.

Here is a key concept when comparing with mock objects. Stubs are used generally when the real object either isn’t available or isn’t appropriate for testing. In most other situations, the real object is used.

Martin Fowler describes the purpose of stubs as being useful for state-based verification. Given that stubs normally have values and states hard coded, testing takes the approach of ensuring the system functions correctly in certain states.

What are Mock Objects?

I like to describe mock objects as simulations of an object. This description will likely be understood by some people and only work to generate more questions for others. Let me explain more…

Mock objects normally implement an object from end to end. They receive input, perform some process on that input, and generate output. The difference between this test double and a real object is that mock objects have an imaginary processing stage; i.e., it’s processes generate output based on what would happen if the real work had happened.

Let’s look at an example. Suppose I have an interface to determine what sizes a shoe manufacturer supplies as part of each model of shoe. The real object would take the name of the shoe and return all the sizes supplied. It might perform this operation by executing a stored procedure in a database to lookup the record ID of the shoe and match it against a many to many table relationship between shoe model IDs and sizes. It would then format the results in an array and return the results to the calling code.

A mock object would do exactly the same, it’s just that it would pretend that an array given as part of the unit test’s setup was the results of the database’s stored procedure. Let’s take a look at some code:

...
string sizes[] = {"8", "9 1/2", "11D"};
MockDatabase db = new MockDatabase(sizes);
string shoeName = "Puddle Jumper TM";
string shoeSizes[] = db.GetShoeSizes(shoeName);
...
Example 1: Populating a mock object with database information

As you can see, instead of going to a real database, it has some values we have chosen for our test case. Take note, this mock object may be used many times and could use a different set of shoe sizes for each test case. In fact, it could use several sets of shoe sizes within just one test case! This is an important difference from stubs, these values are not hard-coded.

In the background, GetShoeSizes(string) takes the given array and formats it in to the expected representation. On top of this, we could have added features for testing, such as counting the invocations of the method, generating exceptions, as well as ensuring these statistics are all within our expectations. Instead of a silent partner, the mock object becomes co-star, defining and testing our expectations!

Martin Fowler describes mock objects as being used for behaviour-based verification. The input and output are not coded but are instead in part processed (or as I term, an imaginary process), and defines the behaviour of the application.

This point is crucial. It beckons the introduction of contracts within the design of software. As soon as you have test cases defining the expected behaviour, as mock objects do, you are building a contract between the application, the test case, and the developer who is intending to develop the production code. For this reason, test cases should be written to reflect the requirements closely.

Summary

So, hopefully this blog post has helped with the question of what difference there is between stubs and mock objects. We’ve learned that stubs are normally hard-coded substitutes for production code whereas mock objects have an imaginary process which helps to build expectations of how the production system will behave. Also, we’ve learned that stubs work with state-based verification while mock objects work with behaviour-based verification.

Hopefully this has provided an interesting introduction to the area of mock objects. You can read more from Martin Fowler’s essay on this topic at http://martinfowler.com/articles/mocksArentStubs.html, where I did my quick refresh on the differences between these two kinds of objects.

This topic of stubs and mock objects moves in to the space of Design By Contract and Test Driven Design. See my post on Test Driven Design if you’d like my introduction to that space as well.

As usual, keep the discussion going and leave a comment.

Kind Regards
Glenn





Presentation

8 09 2008

Well, I did say I was going to be busy. Yes, I’m still here and Quality in Software is still active. Hooray!

These last weeks have seen me feeling like my time has been pretty thinly spread. Nevermind, still, what’s happening?

Well, tomorrow I’m giving a presentation on Testing Components with Complex Associations. It’s a dive in to the world of Mock Objects, patterns, and other things like that. Of course, I’ve held back because my space is only 20 minutes. Quality is such a big topic.

Just lately, I’ve been in a test lead role. It’s been good. I’m surprised just how much I think about my test lead back at Motorola; Chris Lawn. Very dedicated person who I believe has had an impact on how I approach the role of testing.

In the not too distant future? Work, I’m real busy right now, more blogging, and of course one of my favorite professional development activities, reading books. They’re so invaluable!

I’ve been meaning to have a page to share book titles. That will still go ahead. I look forward to sharing book titles with you all as I’m always on the lookout for a great book on quality.

Anyway, for now, it’s back to work. I just got an updated spec I have been waiting for so I’m on the quality trail yet again.

Kind Regards
Glenn





Engaged

26 08 2008

Wow, what a week last week was for me! Not only did I begin working on billable work writing system test cases but it was the lead-up to the engagement party for my fiancee Jenna and I.

It went really well and everyone enjoyed themselves. We had a DJ and held it at a great venue that ensured we could just enjoy the night instead of stressing.

Thank you to our families, all those who came and celebrated with us, to the DJ Steve Russel, and to the Village Hotel.

Now, on to business. What does my billable work mean for my blogging? It just means you have a little more time to read each post. I’ll try to keep up a steady flow of posts but it won’t be every two days as I was doing before.

Don’t let that stop you from making this blog great though! Leave comments and share your thoughts and ideas around the topics you’ve read here at Quality in Software! Comments do need to be approved but I haven’t been slack in checking those, so don’t be worried if you don’t see your comment on the site until the next day.

Thank you to all those who have already left comments at Quality in Software, including Rita, Alex, ArielMowan, and Scott.

Anyway, I’ll get back in to the swing of things once I have some more free time on my hands. Engagement parties are quite time consuming to organize. I already have some ideas on what to post next, in addition to what I hinted of in previous posts.

Kind Regards
Glenn





More Than Testing

19 08 2008

One thing I’ve been wanting to write about lately is what types of activities a modern software QA person does these days. As you can probably work out from the name of this post, it’s more than testing.

Now, before I go any further, if you haven’t read my About page yet, I work as a software consultant for a company called DWS. As such, I do development as well but most of what I do is quality related tasks.

Anyway, let’s look at what happens in modern software QA. Now, I’m not going to say this is everything that everyone does, but it is some of what some people do, especially this person. So, let’s dive in!

Testing

Ok, so you already knew that. But what does that involve?

  • Unit Testing
  • System Testing
  • User Acceptance Testing
  • Performance Testing

You probably know at least some of these but I will detail them further in later posts. Essentially, testing is where you get your blue ribbon for everything working or direction on what to fix.

Persona Development

Persona what? Persona development is where we establish different personas of people using software. This generally means we understand who will be using software in a holistic way.

Developing a persona involves identifying the needs, purpose, habits, etc of people to identify behaviours. This of course allows for better performance testing, acceptance testing, and even helps with defining requirements.

Requirement Traceability Analysis

Who ever said that a text editor should make coffee? Traceability analysis looks at requirements and traces them from concept through to the developed property.

Traceability analysis really helps with identifying priorities and stops scope creep. It’s also a great way of giving confidence to managers that a project has addressed all the requirements when used in conjunction with test reports.

Requirements Coverage Analysis

Although all your test cases may pass, it doesn’t mean all your requirements have been tested. Requirements coverage analysis examines requirements for test coverage at each applicable phase of testing. Focusing on requirements that haven’t already been tested can save big on time, money, and help your team look good.

Testware

Testware is all the tools used to test software. These may include load generators for performance testing or an automation framework. The QA function is also about developing testware as well!

Reviews

Code and document reviews are another function of the QA role. Reviews may include using tools such as the Orthogonal Defect Classification, checklists, code analyzers, etc.

Define Requirements

Yes, QA defines requirements as well, not just test them! Where there are issues of technical or organizational acceptance, the QA role can come in and help define requirements to encourage success as well as promote testability.

Advising

The QA role is often called upon for advice as their job often means they are familiar with a lot of the issues that can go wrong. Furthermore, testing efforts later on can be greatly reduced with QA input early on.

Metrics

Quality often involves metrics. Metrics are great at helping us to find where we need to focus our testing attention by keeping track of what has needed our attention in the past. Metrics can also help us with identifying where risk is increased just by metrics, such as where there is more code in a method or class. Testing is often performed in a risk-based approach, focusing on what poses the greatest risk.

Risk Analysis

Risk analysis is a task for many people working as a group. Without a QA presence, many risks may go unnoticed.


So, if you’re thinking of a role in QA or maybe you’re just interested in what the QA role involves, hopefully, I’ve helped you have a clearer understanding. It really isn’t just about testing, it’s about quality.

What sort of activities do you perform in your QA life? I’d be interested to know what other activities people perform to improve quality in software.

Kind Regards
Glenn





Throwing Away – Getting MVP Right

15 08 2008

It is part of programming folk law that we should throw away our first version of any software. Essentially, we are writing software that has never existed before and need to find the best approach. One problem I’ve always had with this is that not all of what we write is that bad on the first run. This may be because we’ve written something similar before or it’s so very simple that we can verify it works properly.

On the other hand, the technique of Refactoring promotes the evolution of software from a throw-away first edition to hardened quality. Refactoring is about improving what already works with each visit.

Yesterday, I realized I needed to choose Throwing Away. If you’ve been following my blog for the past week or so, you’ll know that I am writing a wedding application to track all the events related to my up-coming wedding next year. It uses the Model View Presenter (MVP) pattern as well as being architected via Test Driven Design.

Yesterday, I realized that I had written my Presenter to be the only Presenter in the entire application. This isn’t according to how the MVP works. It should be one presenter instance per view and a lot of the time, it’s a customized presenter built with a single task in mind: tasks such as adding a record, generating a report, etc. My Model suffered the same problem. My MVP had become closer to MVC (Model View Controller) and as a result, I had a monolith Presenter and Model.

Thinking about it deeply first, I decided to throw away my monolith Presenter and add another layer for the Model, abstracting it away from the business logic. I believe I have made the right choice.

If you want to learn more about MVP, there are some great resources. A new one I’m liking is http://www.codeproject.com/KB/architecture/DotNetMVPFramework_Part2.aspx.

Kind Regards
Glenn





MVP Pattern

12 08 2008

Hey!

I haven’t been blogging as much as I’d like to because I’ve been busy writing my Wedding Application. You can find more about it in my previous post but if you’d just like a quick synopsis, in my spare time, I’ve started writing an application around my coming wedding. Being busy with this is not all bad because it’s helped me to revisit a technology I learned about on-site. Let me tell you more…

If you don’t already know about design patterns, they’re essentially descriptions of what works most frequently in programming. They’re based on a lot of the work by Christopher Alexander with his architecture patterns. Now, before you go googling for “software architecture patterns by Christopher Alexander”, you might like to know his patterns are for buildings, suburbs, and other general living spaces: i.e., not software. Although Christopher Alexander’s work wasn’t designed for software, it’s inspired a generation.

Anyway, this post isn’t about all patterns, I don’t have that much spare time; it’s a very large topic. Instead, I want to talk about the pattern I’ve been using for the GUI of my wedding application. This particular pattern is called the MVP Pattern or Model View Presenter.

So, the whole idea behind this model is to decouple the UI controls from the UI logic and from the business model. It doesn’t do this by burying the UI logic with the business model, that would be a step in the wrong direction, but instead, it introduces a new layer. Let me explain more…

The MVP pattern is really about three parts. The first is the Model, the business model. This is where your business logic exists. It’s not unreasonable for the model to be a complete module containing all the non-UI stuff: business logic, persistence logic, etc, etc. It knows how to do stuff, like create records and apply business rules.

The next part of the MVP pattern is the View. This is what you get to see, literally. This is your forms and windows, your dialog boxes and your console screens. Now, what I need to mention here is that it is developed as an implementation of an interface. This is because it decouples so greatly, it allows you to swap and change your views: think of one application with one UI logic yet interchangeably displayed on Windows Forms, WPF, Console, ASP.NET, or Silverlight. The application isn’t changed, the IView (typically) interface is simply implemented differently.

Lastly, the MVP pattern introduces the Presenter. This is where your UI logic exists. It’s actually the driver for the UI and the main conduit between the view and the model. It’s responsible for updating data displays, formatting displays, and notifying the business model of user commands.

What I like most about the MVP pattern is that it makes your UI logic testable. Unlike UI drivers that send window messages to your application, MVP allows you to unit test your UI logic and that’s what I like about it. It’s really early in the development cycle, meaning less cost to fix any defects that are injected in the product at that stage.

You might like to look at more here. You will definitely want to know that Martin Fowler has decided to split the pattern in to two separate patterns called Supervising Presenter and Passive View. Definitely worth the read.

If you haven’t already, I really urge you to get in to design patterns. They really do help with software architecture and enough people know them that you can use them by name in conversation with other software engineers. If you’d like another great resource, try Do Factory, it doesn’t include MVP but it is good none the less and reflects the GoF (Gang of Four) design patterns.

Kind Regards
Glenn





Glenn’s Wedding App – Using Test Driven Design

6 08 2008

Here at DWS, I’m “On the Bench”. This essentially means I don’t have any client work to do so I’ve been doing things for DWS internally as well as professional development. With this spare time, I’ve been practicing my skills in XML, .NET 3.5, and TDD (Test Driven Design).

I’m not going to pretend that I know everything about TDD, I don’t. What I will do, however, is tell you about my understanding of this fast way of designing good quality software that has fantastic code coverage as well!

I used to code in .NET 2.0 a lot at home and in TAFE (a technical college where I learned software development): as well as .NET 3.0 as part of my job here at DWS. Having said that, much of my work for the last 9 months or so has been testing, specifically leading UAT at a client’s site, so my coding skills are a little rusty.

I wanted to sharpen these coding skills but first I needed a project. Too easy! My upcoming wedding to my darling fiancee Jenna is a great inspiration for a project! So, I started a project that tracks wedding-related events: you know those events, engagement party, pre-wedding parties, organising the wedding, etc. My project will tell me which events are coming up next as well as how much time there is until the event.

Now, where does Test Driven Design come in to this? Well, it’s a great way of getting software written quickly with high quality, so that’s the method I have chosen to write my program. So, what is Test Driven Design?

Test Driven Design as I understand it is a process where you begin with a single concept. This concept says that any piece of software may either be complete or incomplete. Any defects are just where the software is incomplete. To demonstrate this, a test is written that confirms the software is incomplete and provides a proof for when it is complete. It starts with writing a test case that describes a little of the functionality of the final product.

Once the test case is written, the test case is compiled. At this stage, there is no production code and so the compilation of the test case fails. Let’s take a look at an example:

[TestMethod]
public void DoStuffTest()
{
  Something target = new Something();
  bool expected = true;

  target.DoStuff();
  bool actual = target.StuffDone;

  Assert.AreEqual(expected, actual);
}
Example 1: Initial Test Case

So, there’s a test. The idea here is that Something doesn’t exist, so the compiler fails to compile the test. Let’s add Someting.

public class Something { }
Example 2: Inital Production Code

Ok, so I know what you’re thinking, “He’s left out all the boring stuff.” Not at all, that’s it! There isn’t anything more. Let’s compile… Compilation Failed! Error: DoStuff() is not defined.

That’s right, we have stubbed the class but haven’t filled in the methods yet. Simply put, we didn’t do that because it wasn’t necessary to fix the problem at hand. That’s what TDD is about, fixing what is at hand to the very minimum required to satisfy the tests. As a result, you get very high code coverage, as in greater than 90%, often 100%, and the code is simple and testable.

We continue down this path until we finally get a compiled program: Compilation Successful! Now we run the test…

Test Failed: Expected: true, Actual: false

What happened? Let’s look at the code.

public class Something
{
  public void DoSomething()
  { }

  public bool StuffDone
  {
    get { return false; }
  }
}
Example 3: Production Code - Meeting the minimum requirements of the test case

Ah, this is easy to fix:

...
get {return true;}
...

Now it passes! Fantastic! But now we have only proven that we can know when something has happened, what about before it’s happened? Let’s change our test:

[TestMethod]
public void DoStuffTest()
{
  Something target = new Something();
  bool expected1 = false;
  bool expected2 = true;

  bool actual = target.StuffDone;
  Assert.AreEqual(expected1, actual);

  target.DoStuff();
  actual = target.StuffDone;
  Assert.AreEqual(expected2, actual);
}

Example 4: Updating the Test Case

Now, see the change? We check target.StuffDone before we call DoStuff() and expect it to return false. Of course it should be false, we haven’t “done stuff” yet! Let’s compile: Compilation Successful. Not a problem, it only exercises existing production code. Let’s run the test:

Test Failed: Expected: false, Actual: true.

Ok, so what happened? Remember how we made our code work by statically returning true from the StuffDone property? We’re going to have to be a little smarter now.

How can we get the test to pass with the simplest change to the production code? Really, the best way is to change the result of StuffDone when we call DoStuff(). Because the first Assert statement expects StuffDone to be false, that’s what we’ll initialize it to satisfy that test.

// Something that does stuff.
public class Something
{
  private bool mStuffDone = false;

  // Do something.
  public void DoSomething()
  { mStuffDone = true; }

  // Has something been done? Return true if yes.
  public bool StuffDone
  {
    get { return mStuffDone; }
  }
}
Example 5: The Completed Production Code

Easy! Now, I know what you’re thinking, it’s easy because it’s just some simple example. Well, yes, that’s right but it’s also how TDD works. Everything is broken down in to simple bite-sized pieces. Now, think about what we have here. We have a fully tested piece of production code, a simple design, and a test that proves that we have delivered on the requirement described by the test case: let’s just say, green ticks are hard to argue with.

You’ll likely want to refactor your code at some stage but at least you won’t be refactoring too early and you have test cases to check that your refactoring doesn’t break anything. Try writing test cases around the requirements in your projects. I find it easy and rewarding.

Kind Regards
Glenn