Silverlight 2: Introducing Microsoft.Silverlight.Testing
As announced during the MIX’08 keynote by Scott Guthrie, not only are we making available the source code of the Silverlight controls, but we’re also releasing nearly 2,000 unit tests and a unit test framework for the Silverlight runtime. This is a project I’ve been working on for some time now, so I’m planning on a series of in-depth tutorials about this exciting ability. In the meantime, I’ll send you towards some other links and a quick preview…
It is my hope that this provides the ability for teams to do some level of test-driven development using Silverlight, and so I’m very interested in receiving your feedback on how it works. We’ve had pretty good success using this framework for developing the managed Silverlight controls that were released in the Silverlight 2 Beta 1 SDK at MIX.
Exploring the control unit tests:
Shawn Burke gives an overview of the control source code, unit tests and framework
3/20 Update: I’ve also added a quick video post with a screencast that shows the control tests running.
Using the unit test framework:
Coming soon 
, once I’m back from MIX’08 I’ll get the tutorial posts off to a start – subscribe to my feed!
Microsoft.Silverlight.Testing running DataGrid control tests
Built on the power of the Visual Studio Team Test unit test framework
If you’ve used the unit test framework built into the Visual Studio Team Test editions (now available with Visual Studio 2008 Professional as well), then you know just how easy and powerful the framework is.
You’ll find that pretty much everything in the desktop unit test framework is available here when it comes to metadata and attributes. There’s no data-driven tests or advanced host attribute support, but we’ve also added some helpful attributes such as “Exclusive” attribute (that lives in the Microsoft.Silverlight.Testing namespace), since there’s no Visual Studio integration with the framework.
Hopefully this will also help you to get more familiar with the Visual Studio unit test framework so that you can use it for your desktop apps, since the metadata and assertion class knowledge transfers perfectly.
If you’re new to the unit test framework, check out the MSDN documentation on the Assert, StringAssert, and CollectionAssert classes, plus of course the attribute metadata.
MSDN Library Documentation: Microsoft.VisualStudio.TestTools.UnitTesting
A sample testcase that uses the unit test framework’s attributes; this particular test source code can run using Microsoft.Silverlight.Testing or the desktop Visual Studio Team Test unit test framework, if the references for the project are updated.
Downloading the framework
To get started, download the Silverlight 2 Beta 1 controls source code. Inside the self-extracted Zip file, there is a subdirectory called ‘bin’ that contains the three assemblies that make up the unit test framework:
| Assembly | Purpose |
| Microsoft.Silverlight.Testing | Unit test framework |
| Microsoft.Silverlight.Testing.Framework | Base classes and interfaces for test harnesses |
| Microsoft.VisualStudio.QualityTools.UnitTesting.Silverlight.dll | Visual Studio Team Test metadata built against the Silverlight libraries |
These are typical Silverlight class library assemblies, so you can reference them in your projects like any other library.
Installing the framework
To make it easier to add the test framework assemblies as references’ to your projects, I’d recommend copying the framework assemblies into the client libraries directory where the SDK installs the controls and other bits.
To do this, simply copy the three into “%programfiles%\Microsoft SDKs\Silverlight\v2.0\Libraries\Client”. The next time you use Visual Studio’s Add Reference window from within a Silverlight application project, you’ll see the assemblies present:
Please note that the unit test framework isn’t part of the SDK, but this is a convenient place to store them.
Helpful C# test project templates
Although it’s pretty easy to tie into the framework by adding a new Silverlight application project to your solution, that requires a few hookup details, such as removing Page.xaml.cs, updating the root visual, and some other magic.
It’s much easier to instead use these helpful Visual Studio 2008 project and item templates:
- SilverlightTestProject.zip (project template, adds a test project Silverlight application to the solution)
Copy this into your “%userprofile%\Documents\Visual Studio 2008\Templates\ProjectTemplates” - SilverlightTestClass.zip (item template, adds a test class to your Silverlight test project)
Copy this into your “%userprofile%\Documents\Visual Studio 2008\Templates\ItemTemplates”
Using the C# test project templates
To create a new Silverlight test project, with your Silverlight application or class library open:
- Right-click on the solution
- Select the Add->New Project menu item
- Click on the root “Visual C#” project type node
- Under “My Templates”, select “Silverlight Test Project”, and give your test project a name
There’s a default test.cs file that you can use as a starting point, or to add additional test classes to your test project:
- Right-click on the test project
- Select the Add->New Item menu option
- Click on the root “Visual C#” category
- Select “Silverlight Test Class” and provide a name for your new class
Hope you find this is pretty useful, let me know!
Jeff
Hi Jeff,
I want to thank you, Shawn, Scott G., and Microsoft for sharing all of the control code and unit test stuff!
I belief that this approach will produce the most rapid adoption of Silverlight and the most rapid growth of Silverlight.
Thank you again,
David
Thanks for this – I’m very interested in trying this. I am hoping to apply the MVVM pattern to my Silverlight application (which worked very well for a WPF app I wrote). Clearly unit testing is key for any app using MVC style patterns!
Will keep you posted if I have any issues with the framework. Have you by any chance written any unit test coverage tools for Silverlight?
Incidentally your link to the control source code was wrong, it should be http://www.microsoft.com/downloads/details.aspx?familyid=EA93DD89-3AF2-4ACB-9CF4-BFE01B3F02D4&displaylang=en
Regards
Neil
Neil,
Thanks for the feedback.
I haven’t been able to use any code coverage tools with Silverlight and this runtime, since both the test framework and end-user applications are “untrusted” apps running in the sandbox, there’s no way that I’ve found for instrumentation to really work.
If I do hear of a solution, I’ll of course blog about it!
-Jeff
OK… got my first two tests running and passing!
The first thing I’ll say is this doesn’t work in Firefox, which had me completely flummoxed for half an hour. Now I have green lights (in IE) so all good
Thanks
Neil
Neil,
What’s the error message or issue with Firefox? I haven’t run into this before, and there’s a decent number of folks who’ve been using Firefox, so I’d like to understand your configuration if you have the time to describe the issue.
Thanks!
Hi there. There was no error message in firefox, I just got a static screen with 0 indicating that no unit tests were found / being run. When I get home and have time to test this again, I will email you some screenshots if you want?
I actually just noticed that TestDriven.Net have released support for running tests under the CoreCLR. See the following post: http://weblogs.asp.net/nunitaddin/archive/2007/05/16/test-with-silverlight-coreclr.aspx
Now as I am only looking to test my Model/Presenter/Controllers it may well be possible for me to use this rather than having to use the Silverlight testing DLL. What do you think? Will keep you posted on my findings.
Regards
Neil
[...] You can download the source code from MSDN Downloads. Shawn Burke also talks more about the source right here. Jeff Wilcox talks about the Microsoft.Silverlight.Testing framework in his post "Introducing Microsoft.Silverlight.Testing". [...]
Nice article! We are currently working on our first SL project.
For quick and easy in-browser Silverlight regression/performance testing I can recommend the iMacros Scripting Edition. We use the same tool for our Flash/Flex testing, too.
http://wiki.imacros.net/DirectScreen_Technology
Regards,
Stephan
[...] Jeff Wilcox has a post detailing the Unit Testing framework he built for Silverlight 2. [...]
Hi,
I was very excited to hear about UnitTests for silverlight for I’m developing a large scale application in Silverlight.
My problem is when using UnitTestSystem.RegisterUnitTestProvider.
I tried to register my own provider and had come to get exceptions. After looking a bit in the Reflector I’ve seen that the UnitTestSystem doesn’t really support more than one provides. After commenting out the [TestClass] for the default test class in the project it worked. Are you aware of this issue? Am I missing something?
Thanks in advance,
Eylon.
[...] has a follow up post to the demos from Mix here. The silverlight unit test framework is based on a special build of mstest’s infrastructure, [...]
[...] has a follow up post to the demos from Mix here. The silverlight unit test framework is based on a special build of mstest’s infrastructure, [...]
[...] Jeff Wilcox has a post detailing the Unit Testing framework he built for Silverlight 2. [...]
[...] Silverlight 2: Introducing Microsoft.Silverlight.Testing – Jeff Wilcox (tags: Silverlight unittest lazycoder) Post a comment — Trackback URI RSS 2.0 feed for these comments This entry (permalink) was posted on Monday, March 24, 2008, at 3:19 am by Scott and categorized in Links. [...]
Hi Jeff,
This framework really helps. Thank you for that. But I have a question though. We are trying to use your framework to write some functional test for our new Silverlight application. This framework shows how many Methods passed or failed, but it doesn’t show the page that we are trying to test. Let’s say we want to see that text on the page will change from “Hi” to “Bye” when a button is clicked.
Is it possible to force the test page that we are doing functional test against to appear on the framework testing page next to the Run Results? And if yes, how is it possible to do?
Really appreciate your response.
Thank you
Sasha
Sasha,
What if you have your test class inherit from SilverlightTest, in the Microsoft.Silverlight.Testing namespace, and then use the Silverlight.TestSurface to add your page to the visual tree?
There’s a more advanced set of functionality available for doing user interface simulation testing that might also get there, a tutorial might be helpful, I’ll see what I can go.
hi,
thanks for the information. but i haven’t understand the framework installation procedure.
MAIA,
The installation is admittedly not as nice as I would prefer.
To be clear,
- Download the control source code package from Microsoft (link provided in this article)
- The framework assemblies are located in the .\bin\ subdirectory
- You can reference the assemblies directly
There are also the test project templates that can be downloaded to make “hooking up” the test project a little easier.
Let me know if this is any more clear..
Hi Jeff,
I am getting this Error message “Unable to attach.The Debugger is not properly installed.Can not debug the requested type of code.Run set up to install or repair the debugger.” when trying to run the test project which i created by using the first option ie hooking the frame work manually.Me really bugged of of with this……Please help.
Thanks a lot,
Praveen
[...] a first class application platform. For more information about the test framework see this post fromJeff Wilcox. To start off I choose to use a code sample that had some complexity in it. Brad Abrams just posted [...]
[...] 7.Silverlight 2: Introducing Microsoft.Silverlight.Testing http://www.jeff.wilcox.name/2008/03/07/silverlight-unit-testing/ [...]
[...] Testing for Silverlight… Jeff has a follow up post to the demos from Mix here. The silverlight unit test framework is based on a special build of mstest’s infrastructure, [...]
[...] Unit testing for Microsoft Silverlight Silverlight 2: Introducing Microsoft.Silverlight.Testing [...]
[...] Testing with Silverlight. I showed this briefly. Check out this blog entry introducing it and this updated entry for Beta 2 from Jeff Wilcox for more [...]
Hi there,
Many thanks for the framework, but since I upgraded to VS2008 SP1 I’ve been getting the following error from any project that references the testing dlls
error MSB4018: The “ValidateXaml” task failed unexpectedly.
C:\Program Files (x86)\MSBuild\Microsoft\Silverlight\v2.0\Microsoft.Silverlight.Common.targets(190,9): error MSB4018: System.IO.FileNotFoundException: Could not load file or assembly ‘System.Core, Version=2.0.5.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e’ or one of its dependencies. The system cannot find the file specified.
Any help you could provide would be much appreciated.
Andrew,
Make sure that your project includes a reference to System.Core. This is a beta 2 change I think and won’t affect anything else, other than letting it build.
-Jeff
I’ve followed your setup for the project to “%userprofile%\Documents\Visual Studio 2008\Templates\ProjectTemplates” etc.
ProjectTemplates & ItemTemplates both have subdirectories for VB C# etc. I’ve tried copying to the directory as prescribed as well as the specific directory indicated. No Joy. Do I copy just the .vstemplate files only?
Hi Gary
I stumbled on the same issue.
You probably (like I did) unzipped the file – when I followed the instructions to the letter (simply copy the zip file) to the folder Jeff specified , it worked for me.
Jeff , Thanks for the video and blog posts.
Amit