Automated Testing

Hey there

Lately, I’ve been looking at automated testing. I went through a whole gaggle of tools, mostly FOS (Free/Open Source). Automated testing has been something that interested me for a while.

The project I’m currently working on involves an AJAX web app written in .NET 3.5. As such, I’ve had to contend with the difficulties of background activity after the page has loaded. Today, I want to share with you some of the lessons I’ve learned.

Firstly, you should know I’ve been working a lot on a tool called Selenium IDE. It’s a plugin for FireFox and is free and open source. You can take a look at it here if you’d like.

Hunting Wascally Controls

AJAX introduces a whole new fun. When your page loads, depending on the control, it will be displayed and available immediately or later as things are downloaded and executed in the background. Other times, it may be that a control is unhidden because of a user-selection, or because of some other event. In general, the page loading has no immediate bairing on the availability and visibility of controls! Yay!

Don’t get depressed. Selenium has many commands. Where a command acts on a control, there is likely a “waitFor” command available as well. So, for instance, if I trigger an event, I can “waitForElementPresent” before I inspect it, or “waitForTextPresent”. These commands are specifically intended for an AJAX web app.

Held Up In Traffic

Often, AJAX web apps download so many things in the background, executing scripts and so forth, that the page has not finished loading in the background. Sometimes, this just means a faster user experience not having to wait for things to download but other times it can mean being held up by background traffic.

Selenium often uses the “clickAndWait” command when navigating. This command is generally intended to click, i.e. instigate the navigation, and then wait for the page to load. Selenium, however, is not intending to wait forever and by default will only wait 30 seconds. With the project I’m looking at especially, 30 seconds is no where near enough time. As such, I use the “setTimeout” command to increase the duration that Selenium will wait for; I set it at 5 minutes or 300000 milliseconds when passed to the command. This tells Selenium that things will likely take longer and you get less timeouts.

Starting From The Beginning

Another thing I’ve learned is to always assume the script is not starting in the right place within the application or website. As such, I always have the first few steps navigate back to the beginning or starting point for all scripts. This means I don’t have to worry about where one script ends ready for the next script.

Absolutely!

The last point I’d like to share is to always use an absolute URL for the “open” command. The “open” command simply sends the browser to the correct location to begin testing. By using an absolute URL, it means I can just open a browser window and hit Start. Even if the browser is displaying the Quality in Software blog, the “open” command will take the browser to the right server ready to begin testing.

A Few More Comments

Just to finish, I’d like to make a few comments as to why I like Selenium IDE. Firstly, it is free and open source. There are a lot of arguments both for and against FOS software but the best argument for is that it’s accessible to more people; I like that. The second thing is that it uses a real browser to perform the testing; none of this pretend stuff where it simulates accessing the website or uses its own rendering engine. Although its recording is confined to FireFox, it is possible to execute using all the popular browsers. That makes sense since it means testing the scripting and functionality on the platforms they will be deployed to!

Anyway, I’ll leave it at that. Let me know of your experiences with Selenium or with automated testing in general.

Kind Regards
Glenn

Scope

Hey

One thing important in software is scope. Scope can be the difference between success and failure in some circumstances. Often though, scope can be a little confusing when testing.

In software, we often have two sources of expectations. The first is the explicit requirements as documented. The second is the implicit understanding of usability that we have learned throughout life. Sometimes, these two sources can compete and that’s when decisions need to be made. Today, I want to discuss how I approach navigating this stretch of water.

Scope is defined by the requirements of the software product; that’s a given. Anything outside of those requirements is outside of scope: however, much of the time, requirements are deliberately or otherwise written in such a way as to provide room for our experiences to shape the usability of the product. Herein lies the line between scope creep and usability. Often, these are the little things that would make requirements documents too long if all were documented.

We need to return to the first note we discussed: scope is defined by the explicit requirements of the product. This is the first test as to whether it is in scope or not. Really, there is anything to discuss on this point. If a customer pays 1 million googaflops for an application with red polka dots, then the client gets red polka dots; end of story! So if the requirements say the product behaves in a certain way, that’s the way it must work.

The next thing to consider is whether it is whether there is industry convention or standard that everyone else conforms to. This means you’re applying familiarity to the product as a method of usability. This is the second test.

Now, if you don’t find yourself being motivated by either of those two above tests, you probably are compelled only by your personal preference and it’s not likely to be what the client wants. This is important. We can sometimes feel like we have a vested interest because of how many hours we spend with these products but really, it’s up to the client.

So, next time you find yourself at a loss for what you should do, just think about these two rules: is it in the spec and is it convention. If your answer is No to both of these, then it’s out of scope.

Kind Regards
Glenn

Retaining Quality on an Easy Friday

Ah, it’s Friday, the end of the working week! It’s always fun coming up to a Friday as it means a great opportunity to take my fiancee Jenna out on a date and not have to worry about going home in time to get up the next morning.

Tonight, DWS is having a social party. It’s to celebrate the recent coming together of DWS and SDM. For those of you who don’t know, DWS is the company I work for (http://www.dws.com.au).

Anyway, if you know me, you know that everyday events make me think of software industry topics; quality, good design, concepts, etc. Fridays are no exception.

Most Fridays, I tend to take it easy. It’s casual day at DWS and many other places too. I’ve been working hard all week, trying to get things done and now is a time for me to sit back and relax as I finish off all the things I’ve been doing throughout the week. Many times, however, the tiredness I have felt through the week affects the effort I can put in to things. Although I’m tired, it doesn’t have to affect the quality of my work.

Being aware that I’m tired allows me to take things slower, add more checks, and follow a personal process more rigorously. That way, I ensure that my work remains at a very high level. It’s often handy to look at other things around the office and in our lives that add to our stress levels. For instance, if traffic is pretty bad, you’re likely not in the right frame of mind when you first walk in the door, etc.

It’s often helpful to remember these kinds of things too when in groups. Everyone experiences the world differently and may end up not being at their best. Often, these can be reasonably predicted: it’s Monday morning, it’s Friday afternoon, the company just had a big launch of a product the night before, or someone simply says they’re not at their best. This is when our software engineering processes are most useful.

Consider what your software engineering processes are, not only at a corporate level but also at a personal level. Remember, don’t stress yourself with mindless process just for the sake of having process but make sure it works and adds significant value.

If you already have processes in place, either corporately or personally, take the time today to see if they’re working for you. Things to think about are:

• Is the process more work than its worth?
• Is it flexible enough for my work life?
• Can I do this a better way?
• Could more learning replace this process?
• Are there other areas that I could focus on?
• Has the work I’ve been doing changed since these processes were invented?
• Am I or are we, taking in to account all those times we’re not at our best such as the tired Friday afternoons of our week?

Kind Regards
Glenn

Reviewing Your Documents

Being in the software industry, I often have to write documents. Often, the documents I write are requirements documents or test plans but other times, they’re just for passing on information.

When I was still learning the art and science of software, I read an interesting book called “Introduction to the Personal Software Process”. It introduced me to a whole selection of great ways to increase quality.

Now, I don’t know if you have any formal method of review for your documents. Maybe you do, maybe you don’t. I don’t want to go in to the details of why you should review documents; it’s important to review your documents! Instead, I want to talk about how to improve those reviews.

Back to the book, Introduction to the Personal Software Process: one thing it taught me was to examine my mistakes and look for them in any new work. The idea here is that we’re all different and we all make mistakes but not the same mistakes!

The way it works is that as you do your review, you keep count of each type of mistake you make. Once you’ve finished, just count the number of mistakes of each type. Then, choose two or three types of mistakes and look for those in the next document you write. After a while, you’ll improve and find there are other types of mistakes that you can work on.

So, try it yourself. Take a look at your next document, whether it’s a document under change control or just for informational purposes, and look for your common mistakes. Try building it in to a checklist; I found it helped me!

Anyway, as usual, your millage may vary but have a think and give it a go. Let me know how it works for you, I’d be interested to hear about your experiences.

Kind Regards
Glenn