Silverlight Unit Test Framework: New features & download for RC0
Enhanced interactive results, more log providers, dynamic test methods and attributes, tag expressions: there’s a whole lot of fun new test-dev-geek candy in the latest Silverlight Unit Test Framework download on the MSDN Code Gallery (download now available; breaking changes here).
Here’s a quick look at a few of the top new features in the bits today:
Enhanced interactive results experience: The built-in test log that appears in the web browser has been improved. It now displays a progress bar, numeric test method progress, and average execution duration at the completion of a test run.
All test methods in the test run have a details expander that shows time and duration statistics, exception method details, the ability to retry/re-run a single test method, and also copy result information to the clipboard.
That’s right: you can re-run a single test (passing or failing) without having to restart the debugger, the browser, or do anything other than click the "Retry this test" link – then watch as the result appears again, at the bottom of the test log.
There is a simple log provider that lets you copy all failure results into the clipboard for e-mailing or pasting in bug reports. By clicking the "Copy result details" link, we’ll pop up an alert message box. Just press Ctrl + C (Windows) or select the text and press Command + C (OS X) to grab the details to then open a bug in TFS, e-mail a co-worker to nag them to fix a feature, or whatever.
TagExpressions: Easily select and run sets of tests, organize test classes and methods, and add descriptive metadata to speed up validation time. Originally implemented by expert dev Ted Glaza for the ASP.NET AJAX Toolkit, it was the perfect addition to move beyond the ‘Exclusive’ attribute. More about the expression language will be posted someday soon.
A built-in user interface for working with tags is not yet in today’s drop of bits, but it can be set using parameters or modifying the UnitTestSettings object in the meantime:
/// <summary>
/// The Application_Startup method inside the test project's
/// App.xaml.cs file.
/// </summary>
private void Application_Startup(object sender, StartupEventArgs e)
{
// A tag expression that selects all tests that are NOT
// part of the group tagged with "SlowTests"
string tagExpression = "!SlowTests";
UnitTestSettings settings = UnitTestSystem.CreateDefaultSettings();
settings.TagExpression = tagExpression;
this.RootVisual = UnitTestSystem.CreateTestPage(settings);
}
What are they? Well, for the most part that’s for another post, another day. But tags are attached to methods and classes when the TagAttribute is used (found in the Microsoft.Silverlight.Testing) namespace.
They are also automatically generated at runtime for class names, method names, and full names (namespace + name).
Visual Studio Team Test log provider output (TRX): A built-in, bare-bones log provider built with XLinq. When configured it can output *.trx results that can be opened in the Visual Studio 2008 interface. This is output-only, and not an integration story.
Test Service Provider implementation: A super easy, lightweight implementation of a set of provider interfaces for extending the capabilities of the test harness. Paired with the right environment you can get information about the environment, report results to an external test runner or continuous integration system, and more. The provider framework is simple and extensible, you can add your own providers as you see fit in your test projects and common libraries.
Dynamic test methods: For advanced scenarios and simple test generation, you can implement your own ITestClass that also implements the IProvideDynamicTestMethods interface, located in the Microsoft.Silverlight.Testing.UnitTesting.Harness namespace. The interface defines the contract for a single IEnumerable<ITestMethod> method named "GetDynamicTestMethods."
Dynamic test method attributes: It is now possible to provide attributes dynamically (as if they were attached to the real ITestMethod type, if any). The ITestMethod interface has been expanded to include a GetDynamicAttributes() method that returns IEnumerable<Attribute>.
Custom BugAttribute: The Bug attribute defined in the Microsoft.Silverlight.Testing namespace is similar to the Visual Studio known issue metadata, but it is a custom extension that alters the results of the test run.
The attribute flips the test result, accounting for expected & known bugs. This makes continuous integration testing super easy without the need for extensive analysis. A "Fixed" bool property can be used to keep the metadata once the issue is corrected in a future build.
Upcoming posts
With the test framework download now available, over the next few weeks I’ll make sure to fill out the story:
- Updated test templates for RC0
- Other cool new features for testers
- Step-by-step guide for new test developers
- Understanding tag expressions
- Writing your own simple log provider
- Digging into the psuedo-asynchronous testing story built into the framework
Hope these new features are useful to you, let me know!
- Jeff
The Test Page is blank when I launch the CreateTestPage :
private void Application_Startup(object sender, StartupEventArgs e)
{
string tagExpression = “!SlowTests”;
UnitTestSettings settings = UnitTestSystem.CreateDefaultSettings();
settings.TagExpression = tagExpression;
this.RootVisual = UnitTestSystem.CreateTestPage(settings);
}
There is only a dummy test :
using Microsoft.VisualStudio.TestTools.UnitTesting;
namespace Silver
{
[TestClass]
public class SilverCommandTest
{
[TestMethod]
public void Pasa()
{
Assert.IsTrue(true);
}
}
}
Jeff,
Thanks for the update. Really useful, I like the new features… I’ll try this out tomorrow
@Paulo,
There is an issue present when you host the test project locally (shows up as http:// in the address bar), without having a locally hosted test service. Thanks to Michael Scherotter for assistance debugging this today…
Here’s a temporary workaround. In your App.xaml.cs for the test project, clear out the “TestService” property of the settings before running:
UnitTestSettings settings = UnitTestSystem.CreateDefaultSettings();
settings.TestService = null;
this.RootVisual = UnitTestSystem.CreateTestPage(settings);
FYI, the workaround should not be required on the latest bits at https://code.msdn.microsoft.com/silverlightut/
nemiso의 생각…
Silverlight UnitTest Framework 구축 완료이제 좀 제대로 테스트하며 개발하자….
Thankyou,
now in order to load a Page into the test pane
I suppose that I have to do this:
[TestClass]
public class SilverCommandTest : SilverlightTest
{
private Master Mapa;
[TestInitialize]
public void IniciaTest()
{
Mapa = new Master();
TestPanel.Children.Add(Mapa);
}
}
the Page flashes trying to load and then dissapears.
Nice stuff!
Thanks for the thorough write-up.
I haven’t worked with the test framework yet and want to get started using it. Do you have a “Getting Started” guide, and if not could you put a short tutorial together?
@Paulo, is your Mapa type placed in the TestPanel when you TestMethod’s run? After every TestMethod the TestPanel should be cleared again.
@Ralf, take a look at:
http://www.jeff.wilcox.name/2008/08/16/utbasics/
http://www.jeff.wilcox.name/2008/03/31/silverlight2-unit-testing/
Jeff
Thanks for the great work. I’m looking to use NUnit as my test runner to publish the results on our CI server as individual NUnit test case runs. Will this be possible? Where do I start looking to plug this in?
Thanks
[...] stuff in the WPF and Silverlight BookControls CodePlex project Jeff Wilcox helps us robust up with Silverlight Unit Test Framework: New features & download for RC0 Corrina Barber has started updating her applications; starting with her Red control skin Mike Snow [...]
[...] I can’t think of a good use for it, it’s dang addicting to play with
… very cool, Matthias Silverlight Unit Test Framework: New features & download for RC0 Jeff Wilcox give us the UnitTest download for RC0 and discusses all the new features. Stay in the [...]
[...] highly recommend reading Jeff Willcox’s writeup of his testing tool; don’t let this slip by without taking [...]
[...] highly recommend reading Jeff Willcox’s writeup of his testing tool; don’t let this slip by without taking [...]
[...] Wilcox produces a great article on the Microsoft Silverlight Unit Test Framework which is available at the MSDN Code [...]
Jeff
I’m so looking forward to the remainder of the articles on the features of the testing framework. In an attempt to integrate the test results into our TeamCity CI server, I’ve succeeded in creating a custom logger that logs the test output in the format TeamCity understands. I now need to get the output uploaded onto the TeamCity server for it to be integrated into the build report. I know this should be possible as you have shown the test results integrated into VS itself.
So I did some Reflectoring and it seems like the testing framework invokes a web service to upload the test results outside of the SL app boundary for a consumer (VS/TeamCity) to use. I’ve tried plugging into what I see as the extensibility points to get this working, but I’m not having any success. Any quick pointers on how to get it working would be greatly appreciated.
Thanks
Carel
Jeff,
1. Can you please expand/include examples, of how I can selectively tag the test methods before I assign or set the settings.
2. Also, is there a way we can include silverlight tests in automated builds(using MSbuild) ?
Thanks
Jeff,
I’m also curious as to how to automate this testing – do we have to host the SL test app in a web site and add a web service to upload the trx file from the silverlight test app?
Thanks,
Brant
I, too, am interested in automated testing. I’ve come to the conclusion that I can consume trx files in CruiseControl.NET, but I’m interested in an example producing such a file using the VisualStudioLogProvider.
@Brant B, @Brandon C:
I’m trying to prepare the tools and utilities that we use for this for general release, and will be blogging about it when it is released.
Timeline is the next month to 6 weeks, most likely.