- Posted by liammclennan on May 6, 2010
Tonight the Brisbane Alt.NET group is doing a coding dojo. I am hoping to talk someone into pairing with me to solve the kata in CoffeeScript. CoffeeScript is an awesome language, half javascript, half ruby, that compiles to javascript. To assist with tonight’s dojo I wrote the following micro test framework for CoffeeScript:
<html>
<body>
<div>
<h2>Test Results:</h2>
<p class='results' />
</div>
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.3.2/jquery.min.js" type="text/javascript"></script>
<script type="text/coffeescript">
# super simple test framework
test: {
write: (s) ->
$('.results').append(s + '<br/>')
assert: (b, message...) ->
test.write(if b then "pass" else "fail: " + message)
tests: []
exec: () ->
for t in test.tests
test.write("<br/><b>$t.name</b>")
t.func()
}
# add some tests
test.tests.push {
name: "First Test"
func: () ->
test.assert(true)
}
test.tests.push {
name: "Another Test"
func: () ->
test.assert(false, "You loose")
}
# run them
test.exec(test.tests)
</script>
<script type="text/javascript" src="coffee-script.js"></script>
</body>
</html>
It’s not the prettiest, but as far as I know it is the only CoffeeScript test framework in existence. Of course, I could just use one of the javascript test frameworks but that would be no fun. To get this example to run you need the coffeescript compiler in the same directory as the page.
- Posted by liammclennan on March 24, 2010
Continuing on my series of builders for C# and Ruby here is the solution in Javascript. This is probably the implementation with which I am least happy. There are several parts that did not seem to fit the language.
This time around I didn’t bother with a testing framework, I just append some values to the page with jQuery. Here is the test code:
var initialiseBuilder = function() {
var builder = builderConstructor();
builder.configure({
'Person': function() { return {name: 'Liam', age: 26}},
'Property': function() { return {street: '127 Creek St', manager: builder.a('Person') }}
});
return builder;
};
var print = function(s) {
$('body').append(s + '<br/>');
};
var build = initialiseBuilder();
// get an object
liam = build.a('Person');
print(liam.name + ' is ' + liam.age);
// get a modified object
liam = build.a('Person', function(person) { person.age = 999; });
print(liam.name + ' is ' + liam.age);
home = build.a('Property');
print(home.street + ' manager: ' + home.manager.name);
and the implementation:
var builderConstructor = function() {
var that = {};
var defaults = {};
that.configure = function(d) {
defaults = d;
};
that.a = function(type, modifier) {
var o = defaults[type]();
if (modifier) {
modifier(o);
}
return o;
};
return that;
};
I still like javascript’s syntax for anonymous methods, defaults[type]() is much clearer than the Ruby equivalent @defaults[klass].call(). You can see the striking similarity between Ruby hashes and javascript objects. I also prefer modifier(o) to the equivalent Ruby, yield o.
- Posted by liammclennan on March 23, 2010
My last post was about a class for building test data objects in C#. This post describes the same tool, but implemented in Ruby. The C# version was written first but I originally came up with the solution in my head using Ruby, and then I translated it to C#. The Ruby version was easier to write and is easier to use thanks to Ruby’s dynamic nature making generics unnecessary.
Here are my example domain classes:
class Person
attr_accessor :name, :age
def initialize(name, age)
@name = name
@age = age
end
end
class Property
attr_accessor :street, :manager
def initialize(street, manager)
@street = street
@manager = manager
end
end
and the test class showing what the builder does:
class Test_Builder < Test::Unit::TestCase
def setup
@build = Builder.new
@build.configure({
Property => lambda { Property.new '127 Creek St', @build.a(Person) },
Person => lambda { Person.new 'Liam', 26 }
})
end
def test_create
assert_not_nil @build
end
def test_can_get_a_person
@person = @build.a(Person)
assert_not_nil @person
assert_equal 'Liam', @person.name
assert_equal 26, @person.age
end
def test_can_get_a_modified_person
@person = @build.a Person do |person|
person.age = 999
end
assert_not_nil @person
assert_equal 'Liam', @person.name
assert_equal 999, @person.age
end
def test_can_get_a_different_type_that_depends_on_a_type_that_has_not_been_configured_yet
@my_place = @build.a(Property)
assert_not_nil @my_place
assert_equal '127 Creek St', @my_place.street
assert_equal @build.a(Person).name, @my_place.manager.name
end
end
Finally, the implementation of Builder:
class Builder
# defaults is a hash of Class => creation lambda
def configure defaults
@defaults = defaults
end
def a(klass)
temp = @defaults[klass].call()
yield temp if block_given?
temp
end
end
- Posted by liammclennan on March 23, 2010
When writing tests, other than end-to-end integration tests, we often need to construct test data objects. Of course this can be done using the class’s constructor and manually configuring the object, but to get many objects into a valid state soon becomes a large percentage of the testing effort.
After many years of painstakingly creating builders for each of my domain objects I have finally become lazy enough to bother to write a generic, reusable builder class for .NET. To use it you instantiate a instance of the builder and configure it with a builder method for each class you wish it to be able to build. The builder method should require no parameters and should return a new instance of the type in a default, valid state. In other words the builder method should be a Func<TypeToBeBuilt>. The best way to make this clear is with an example. In my application I have the following domain classes that I want to be able to use in my tests:
public class Person
{
public string Name { get; set; }
public int Age { get; set; }
public bool IsAndroid { get; set; }
}
public class Building
{
public string Street { get; set; }
public Person Manager { get; set; }
}
The builder for this domain is created like so:
build = new Builder();
build.Configure(new Dictionary<Type, Func<object>>
{
{typeof(Building), () => new Building {Street = "Queen St", Manager = build.A<Person>()}},
{typeof(Person), () => new Person {Name = "Eugene", Age = 21}}
});
Note how Building depends on Person, even though the person builder method is not defined yet. Now in a test I can retrieve a valid object from the builder:
var person = build.A<Person>();
If I need a class in a customised state I can supply an Action<TypeToBeBuilt> to mutate the object post construction:
var person = build.A<Person>(p => p.Age = 99);
The power and efficiency of this approach becomes apparent when your tests require larger and more complex objects than Person and Building. When I get some time I intend to implement the same functionality in Javascript and Ruby. Here is the full source of the Builder class:
public class Builder
{
private Dictionary<Type, Func<object>> defaults;
public void Configure(Dictionary<Type, Func<object>> defaults)
{
this.defaults = defaults;
}
public T A<T>()
{
if (!defaults.ContainsKey(typeof(T))) throw new ArgumentException("No object of type " + typeof(T).Name + " has been configured with the builder.");
T o = (T)defaults[typeof(T)]();
return o;
}
public T A<T>(Action<T> customisation)
{
T o = A<T>();
customisation(o);
return o;
}
}
- Posted by liammclennan on March 17, 2010
I have made a change to my code-based BDD style. I start with a scenario such as:
Pre-Editing
* Given I am a book editor
* And some chapters are locked and some are not
* When I view the list of chapters for editing
* Then I should see some chapters are editable and are not locked
* And I should see some chapters are not editable and are locked
and I implement it using a modified SpecUnit base class as:
[Concern("Chapter Editing")]
public class when_pre_editing_a_chapter : BaseSpec
{
private User i;
// other context variables
protected override void Given()
{
i_am_a_book_editor();
some_chapters_are_locked_and_some_are_not();
}
protected override void Do()
{
i_view_the_list_of_chapters_for_editing();
}
private void i_am_a_book_editor()
{
i = new UserBuilder().WithUsername("me").WithRole(UserRole.BookEditor).Build();
}
private void some_chapters_are_locked_and_some_are_not()
{
}
private void i_view_the_list_of_chapters_for_editing()
{
}
[Observation]
public void should_see_some_chapters_are_editable_and_are_not_locked()
{
}
[Observation]
public void should_see_some_chapters_are_not_editable_and_are_locked()
{
}
}
and the output from the specunit report tool is:
Chapter Editing specifications 1 context, 2 specifications
Chapter Editing, when pre editing a chapter 2 specifications
- should see some chapters are editable and are not locked
- should see some chapters are not editable and are locked
The intent is to provide a clear mapping from story –> scenarios –> bdd tests.
- Posted by liammclennan on November 19, 2009
Today I had a twittersation about build servers, and how closely they should match the development and production environments. Damian’s position was that the build server should match the development environment, while I held that the build environment should be as close to production as possible. Martin Fowler’s Continuous Integration article says:
Test in a Clone of the Production Environment
The point of testing is to flush out, under controlled conditions, any problem that the system will have in production. A significant part of this is the environment within which the production system will run. If you test in a different environment, every difference results in a risk that what happens under test won't happen in production.
As a result you want to set up your test environment to be as exact a mimic of your production environment as possible.
I can provide an example of why this is a good idea. I am currently working on a project that is using Asp.Net MVC 2, which is installed on developer’s machines as a standalone MSI. The same article I mentioned before also says, “everything you need to do a build should be in there [repository] including: test scripts, properties files, database schema, install scripts, and third party libraries”. Being fallible, I made a mistake and failed to include one of the required MVC DLLs in the source repository. Because the build environment matched production we quickly detected the error because the build broke. If the build server was similar to the development environments then it would have had the required DLL in the GAC and the bug would have gone undetected.
Here are the things that I consider to be important for a build environment:
- The build server should, as close as possible, match the production environment
- The build server should be fast. Rapid feedback is important
- The same build that runs on the build server should also be runnable on the developer’s machines
- When a build fails, everyone should know
UPDATE: Kyle Baley has some interesting thoughts on this topic including, “plus, the point of a CI server is to mimic a client machine as much as possible and installing Visual Studio in order to run a build process doesn’t meet that criteria.”
- Posted by liammclennan on November 19, 2009
I have written about about WebAii before. It is functional but the API sucks. I have written about NGourd too.
I am currently working on a project that is using the combination of NGourd and WebAiii for automated acceptance testing. We start with a story:
Feature: Search
As a user
I want to search for items
so that I can find data that I am interested in
and then write some scenarios like:
Scenario: Search for a compensation agreement
Given I am at the home page
When I select the Agreements perspective
And I search for 'agreement 1'
Then the search results should be displayed
Within the test project we have the following directory structure:
Search.feature is a text file containing the previously listed feature and scenario definitions. For each scenario step we must have a corresponding step definition. For example the step ‘When I select the Agreements perspective’ matches the following step definition:
[Step(@"search for '([\w\s]+)'")]
public void search_for(string searchTerm)
{
CurrentBrowser.Find.ById("Terms").SetValue("value", searchTerm);
CurrentBrowser.Click(CurrentBrowser.Find.ById("search_submit"));
}
Note the use of regular expressions to parameterise the step. Because this step is an action we put it in the ActionSteps file. Everything that we need to do for our tests falls into one of the three categories: Action, ContentAssertion or Navigation. The goal is to avoid defining the same step twice so that the set of steps form a domain specific language that can be used by business analysts and the like.
NGourd is a Cucumber knockoff, but without many of the features. However, it is surprising how far you can get with just the basics. So far it is working nicely.
- Posted by liammclennan on October 23, 2009
WebAii is a proprietary, but free, functional testing framework from ArtOFTest. It has become more visible since telerik started bundling it with their ASP.NET AJAX UI Controls framework.
The Good News
The Achilles heel of most web testing tools is ajax. The good news is that webaii has more support for testing ajax applications than any of the other frameworks I have tried. Methods are provided to wait for elements, or to wait for the entire page to finish rendering. To integrate webaii into your testing project requires referencing a single dll, which is also very nice.
the bad news
CSS selectors are a well designed DSL for selecting dom elements. One of the reasons why people like jQuery so much is that it uses CSS selectors to select dom elements. Webaii uses a clunky API for dom selection. It is ok for basic selectors (by id or class) but more complex queries require dropping down to a nasty low-level API.
// jQuery: $(‘#main’)
Find.ById(“main”);
// jQuery: $(‘.myclass’)
Find.ByAttributes("class=myclass");
After writing a few smoke tests I added them into our team city continuous integration build, which broke the build. ArtOfTest do provide instructions for getting the tests to run in a build but it is complicated and requires significant changes to our build.
- Posted by liammclennan on August 7, 2009
TestDriven.Net is a Visual Studio add-in to help with testing in general and TDD in particular. It allows the user to run a test or group of tests by right-clicking and selecting 'Run Test(s)' from the context menu.
When I started using TestDriven.Net I used the Shift+F10 keyboard combination to access the context menu so that I did not have to take my hands off the keyboard to run a test. That was until Steve suggested setting up TestDriven.Net keyboard shortcuts in Visual Studio. He suggested two shortcuts:
TestDriven.NET Keyboard Shortcuts
| Shortcut Name |
Shortcut Key Combination |
Description |
| TestDriven.NET.RunTests |
Ctrl+Shift+Alt+T |
This is the equivalend of right-click => Run Test(s). |
| TestDriven.NET.RerunTests |
Ctrl+Shift+Alt+R |
This is a very handy shortcut that runs the last tests run. You can run a test, make a code change, and then rerun the last test without taking the cursor back to it. |
To setup shortcuts in Visual Studio 2008 use the 'Keyboard' settings group in options. Entering 'TestDriven' in the 'Show commands containing' text box will filter the list of shortcuts to just those relating to TestDriven.Net. Then find TestDriven.NET.RerunTests, place your cursor in the 'Press shortcut keys' text box, enter the key combination of your choice and click 'Assign'. Same process for TestDriven.NET.RunTests. Of course there are many other shortcuts you can configure, such as RunWithCoverage, if you're into that.
- Posted by liammclennan on March 24, 2009
The project I am working on is an Asp.Net MVC application. I wanted to add integration tests that call either controller actions or application services and test the integrated application. To do this I needed to:
- automate the process of setting up an integration testing database environment.
- Duplicate my production IoC solution in such a way that it can be used in tests.
- Provide an easy way for integration tests to establish a context for the integration tests. This really just means simulating a logged in user.
Automate the Process of Setting Up an Integration Testing Database Environment
We manage our sql server database by keeping two sql scripts in subversion: a schema script and a test data script. Therefore to setup an integration database environment all I had to do was execute the two scripts. Here is the target I had to add to my NAnt build script:
<target name="integrationTest" depends="test">
<exec program="sqlcmd" commandline="-U ${Integration.DB.Username} -P ${Integration.DB.Password} -S localhost\sql2008 -d ${Integration.DB.Name} -b -i ${database.dir}\schema.sql" />
<exec program="sqlcmd" commandline="-U ${Integration.DB.Username} -P ${Integration.DB.Password} -S localhost\sql2008 -d ${Integration.DB.Name} -b -i ${database.dir}\data.sql" />
</target>
This target builds the integration database back to the desired baseline. The only other tweak required to the build file was to update the connection string to point to the integration database:
<xmlpoke file="${testbindebug.dir}\${testassembly.name}.config"
xpath="/configuration/connectionStrings/add[@name = 'nameofconnstring']/@connectionString"
value="Data Source=localhost\sql2008;Initial Catalog=${Integration.DB.Name};User id=${Integration.DB.Username};password=${Integration.DB.Password};" />
Duplicate the Production IoC Container Setup
Because the application uses dependency injection consistently it is too hard to perform integration tests without an IoC container. Because I already have all of my services configured in the application config file all I have to do is create a singleton that initializes and exposes the container.
public static class DI
{
public static T Resolve<T>()
{
return Container.Resolve<T>();
}
private static WindsorContainer container;
private static WindsorContainer Container
{
get
{
if (container == null) container = new WindsorContainer(new XmlInterpreter(new ConfigResource("castle")));
return container;
}
}
}
Once this is in place I can easily get an instance of any configured service:
var userService = DI.Resolve<IUserService>();
Provide a Way to Establish a Logged In User Context
Every integration test has to be executed in the context of a user. I created a helper to make it easy to establish a user context.
public static class Context
{
public static void Login(string username)
{
var userService = DI.Resolve<IUserService>();
System.Threading.Thread.CurrentPrincipal = userService.RetrieveUserByUsername(username);
}
}
and that is all that is required. I can now run integration tests against any of my controller actions or application services. Here is an example test:
[TestFixture, Category("Integration")]
public class CustomerLogsTests
{
[SetUp]
public void Setup()
{
Context.Login("tim");
}
[Test]
public void Log()
{
var service = DI.Resolve<ICustomerLogsService>();
var config = DI.Resolve<IConfigService>();
var context = DI.Resolve<IOrganisationContext>();
Assert.IsTrue(service.Log("17:08:00",
-29.111, 152.987, 55.0, 66.0, 1, 2, 1, 0, 100000, "T", 7000, 12, 12, 12, 10, 10, 10, 9, 9, 9,
9, 8, 8, 500, 520, 500, 520, 500, 520, 0));
var vehicleService = DI.Resolve<IVehicleService>();
var logs = vehicleService.RetrieveLogs(vehicleCode, DateTime.Now.Date);
Assert.IsTrue(logs.Any(l => l.Latitude.Equals(-29.111)));
}
}
