Friday, May 22, 2009
Domain Based Testing (1) - An Example
When doing test automation for end to end functional test, each scenarios of every story should be included in the
test suite. As the number of stories grows, the complexity of test automation code increases accordingly, thus the
maintainability of test code becomes more important. In this post, a way of organizing code is shown for handling
this problem. We start from an example, gradually refactor code toward a more ‘domain based testing’ style, and then
explain its benefits.
Domain based testing is simply a way of writing tests in terms of domain concept, rather than implementation
details.
Let’s start from an example.
Say we are doing testing automation for Google search engine. At the first step, we want to navigate to the main page
(www.google.com). Using Watir in Ruby, code is as below:
ie = Watir::IE.new
ie.goto "http://www.google.com"
Second, input search keyword, and click ‘Google Search’ button:
ie.text_field(:name, "q").set "Watir"
ie.button(:name, "btnG").click
Then, extract data from entries returned:
title_of_first_entry = ie.cell(:xpath, "XPATH_FOR_TITLE").text
description_of_first_entry = ie.cell(:xpath, "XPATH_FOR_DESC").text
url_of_first_entry = ie.cell(:xpath, "XPATH_FOR_URL").text
title_of_second_entry = ie.cell(:xpath, "XPATH_FOR_TITLE").text
description_of_second_entry = ie.cell(:xpath, "XPATH_FOR_DESC").text
url_of_second_entry = ie.cell(:xpath, "XPATH_FOR_URL").text
…
Finally, assertions can be carried out based on these values:
title_of_first_entry.should == "Watir - Overview"
description_of_first_entry.should == "Watir is an open-source library for automating web browsers."
url_of_first_entry.should == "wtr.rubyforge.org"
…
Now we are going to verify that the ‘site:’ keyword works. (With this keyword, search will be carried out only in
site specified.). To do that, we basically have to open the main page again, type in keywords and click the search
button, then extract the values of each entry and do verification. But should we copy & paste the code? Definitely
not, duplicated code is root of all evils. We can extract a method and reuse:
def search(keywords)
ie.goto http://www.google.com
ie.text_field(:name, "q").set keywords
ie.button(:name, "btnG").click
end
Now we can simply search by just one function call:
search("site:www.amazon.com Watir")
Isn’t this much simpler and readable?
Then how to deal with the code extract data from returned html? Of course, extract a method again:
def extract_data()
title_of_nth_entry = ie.cell(:xpath, "XPATH_FOR_TITLE").text
description_of_nth_entry = ie.cell(:xpath, "XPATH_FOR_DESC").text
url_source_of_nth_entry = ie.cell(:xpath, "XPATH_FOR_URL").text
…
end
Now we can simply call two methods:
search("site:www.amazon.com Watir")
extract_data()
and do assertion against these data.
But, it is still verbose, since every time you search something and do verification, the exactly same two function
calls will be made. To deal with this, we can simply merge them together, calling extract_data inside of the search
function. So now, all we have to do is calling:
search("site:www.amazon.com Watir")Done.
But can it be even simpler? Yes, of course. I won’t ask if the answer is no.
Did you notice a group of ‘three variables’? We have variable title, variable description and variable url for the
first search result entry, second entry, third entry…… Is there any relationship between these three variables? Yes,
they all belong to one result entry. Based on the search keywords, search engine simply returns a list matching
result entries. Thus, we should model this out loudly!
class ResultEntry
attr_accessor :title
attr_accessor :description
attr_accessor :url
end
Now, the search function can return an array of ResultEntry objects, and assertions can be carried out on these
objects:
result_entries = search("site:wtr.rubyforge.org Watir")
result_entries[0].title.should == "Watir - Overview"
result_entries[0].description.should == "Watir is an open-source library for automating web browsers."
result_entries[0].url.should == "wtr.rubyforge.org"
This is domain based testing!
Comparing this to the first style:
ie.goto "http://www.google.com"
ie.text_field(:name, "q").set "Watir"
ie.button(:name, "btnG").click
title_of_first_entry = ie.cell(:xpath, "XPATH_FOR_TITLE").text
description_of_first_entry = ie.cell(:xpath, "XPATH_FOR_DESC").text
url_of_first_entry = ie.cell(:xpath, "XPATH_FOR_URL").text
title_of_second_entry = ie.cell(:xpath, "XPATH_FOR_TITLE").text
description_of_second_entry = ie.cell(:xpath, "XPATH_FOR_DESC").text
url_of_second_entry = ie.cell(:xpath, "XPATH_FOR_URL").text
…
title_of_first_entry.should == "Watir - Overview"
description_of_first_entry.should == "Watir is an open-source library for automating web browsers."
url_of_first_entry.should == "wtr.rubyforge.org"
…
Which one is better in terms of readability and maintainability? The result is evident.