My Silverlight 2 blog roadmap for Summer ’08
It’s summertime in Seattle, and the warmth throws everything off just a little (and that’s a good thing). I got a little too much sun waiting 7 hours in line at the University Village Apple Store on Friday to upgrade to an iPhone 3G (16gb model, black color)… So I’m hiding from the sun today, and catching up on work.
Tonight I wanted to take some time to share some of the work I’ve been doing in the Silverlight space, a list of some upcoming blog posts and technologies I’m hoping to cover, and then links to some good, current resources on Silverlight.
Earlier today I posted a few quick technical postings:
- Value converters and data binding in the Silverlight data grid
- Fluxcapacity’s FJCore source being available, allowing client side JPEG resizing in Silverlight
What’s in the pipeline for Silverlight test developers
A lot of recent feedback I’ve heard from people around the world has helped me to plan the next set of improvements for the unit test framework. Thanks for all the suggestions and questions!
I don’t have any public timelines for these right now, but do want people to know that I’m working hard to at least get prototype-quality bits prepared that I can write about.
Unit testing fx update
The test framework’s source code is being updated to better conform with design guidelines that Brad and Krzysztof would approve of… this also includes implementing a few oft-requested features:
- Support for grouping & tagging tests, so developers can run specific sets of tests easily.
- Refactoring and renaming parts to make the framework more discoverable and easier to learn.
- Implementing support for a test service that allows out-of-process communication with the test framework, on both Windows and Mac.
Continuous integration support
The test service for the framework enables the use of a console test runner script. As long as your integration system runs in an interactive console, you’ll be able to plug in your rich Silverlight unit tests using CruiseControl.NET, Team Foundation Build, or a MSBuild task.
The console runner is implemented in Python, for a nice cross-platform and cross-browser solution that works in Windows and OS X. The script does not host Silverlight in the console, but instead actually opens and controls browser instances. This allows your tests to work with Silverlight UI elements, the browser, and even integrate your own visual verification system if you had some time to spare.
Code coverage
I’ve been prototyping a code coverage solution for Silverlight, and hope to have something figured out. Stay tuned for details, this is a tricky problem.
Extensible test harness & unit test provider
Inside the test framework for Silverlight, there’s a pretty simple “unit test provider” model that makes it easy to extend or alter the system to meet your needs.
One proof-of-concept project I did was get NUnit’s metadata to work within the Silverlight framework, just by creating a provider that maps NUnit metadata to generic unit test interfaces inside the Microsoft.Silverlight.Testing.dll assembly. I’ll be documenting this unit test provider functionality in a future blog post.
Asynchronous testing guides
The “asynchronous” test support in the Silverlight test framework is designed to allow a test method to queue up a set of conditional, callback, and other work items that run after the method has returned.
Typically any and all test methods that use this functionality are much more than a “unit” test, but I think the capability is powerful, and it’s helped our control developers get even more coverage for their work earlier in a product cycle.
Here’s the quick rundown of what this is about (and in the current test framework assemblies):
The enqueue calls don’t actually run in another thread like an asynchronous delegate in the full .NET Framework, but instead they manage a set of test work items in the framework. This allows the managed code stack to fully unwind, returning control to the web browser to perform other work on the same UI thread.
The framework is then instantiated again-and-again, until all of the work items have been completed on the UI thread (which your tests should not block).
This allows for tests that update the UI of the browser or Silverlight plugin, and doing some other neat things.
I’ll be posting some detailed tutorials and walkthroughs on the topic later this summer. In the meantime, check out:
- Jonas Follesø has written a nice walkthrough of the asynchronous testing capabilities, and provided neat annotations of the code along the way.
- Nikhil Kothari used the [Asynchronous] attribute to test a ViewModel pattern app that integrates with Amazon product search.
What others are writing about Silverlight
A lot of folks at Microsoft have been keeping busy creating excellent content:
- Shawn Burke updated his Silverlight control tutorial sample for Beta 2. He’s using the awesomeness of the VisualStateManager (VSM) in the sample now.
- Tim Heuer has a rundown of what the Visual State Manager is all about.
- David Anson writes about some wild layout system work, and updates his LayoutTransformControl.
- Nikhil talks about search and rich internet apps.
- Kathy Kam shares some photos from TechEd North America 2008.
- It’s a month old now, but I often get e-mails asking about the changes between Beta 1 and Beta 2 of Silverlight 2. Shawn Wildermuth has the rundown on Beta 2.
Have a good start to the week!
Hi Jeff,
Thanks for the link back. Sounds like great stuff. Really looking forward to the next revision of the testing framework. Hopefully you’ll have it out before September. Doing a “Silverlight 2 for developers” talk at TechEd New Zealand and Sydney first week of September. Unit testing and UI patterns will be an important part of the talk, so would be cool to have some fresh bits to show
.
Keep up the good work and enjoy the Seattle summer !
Cheers,
Jonas
Jeff,
Could you consider making InterceptUnhandledExceptions true as default as it is in VSTS and other testing frameworks.
Miguel,
Definitely! Although, I actually think it is true by default right now (if you attach and look at the value, is that the case?).
There are actually a lot of situations in today’s Silverlight bits where the UnhandledException handler for the Application is -not- called. And that’s where I hook up the InterceptUnhandledExceptions property…
The reason for including this property was actually for a set of tests that I have that actually test the unhandled exception property – and so I disable the property before the test, and re-enable it after that.
But my understanding is that it was True by default :-/
-Jeff
I misundestood the use of the property. I tought, than it wouldn’t break if InterceptUnhandledException was true. I looked at the code in reflector and notice the first thing it does is a Debugger.Break call inside the GlobalUnhandledExceptionListener method.
So what happens is that on any exception, even if handled by this Listener, the debugger breaks, this includes exception thrown by asserts.
One option is to run it without debugging, but that means, we cant debug other methods either. For failed tests and excepted exception we dont need the debugger to break. That’s the behavior of other testing frameworks.
By the way, I noticed BrowerUtility also has an InterceptUnhandledException property, but it always returns true. I guess is confusing, specially because it has a set that has no effect.
In your post ‘Unit Testing with Silverlight 2′ you mentioned you were goint to consider releasing the UTFx Source Code. Have this gone any further?
Miguel,
I’ve removed the debugger break statement from the code and will try and release an interim binary release soon.
I’m working to get the framework out there on CodePlex with a MS-PL license, and hope to have some details when it’s available. These things always take a little longer than it feels like they should… sorry I don’t have a known date yet.
Sincerely,
-Jeff
Wow, would be great to be able to get it from codeplex.
Thanks
Jeff, hanging out for a deep-dive into [Asynchronous] testing.
I can see the scenario for testing async services, but would like more info on how it is used in the SL2B1 source (that ships with tests). For example, why is this testing asynchronously:
[Asynchronous]
public override void FontSizeDefaultValue()
{
Button control = new Button();
CreateAsyncTest(control,
() => Assert.AreEqual(11, control.FontSize, Common.HighPrecisionDelta));
}
Taken from ButtonTest.
Cheers
KierenH,
I have a complete draft of the asynchronous post and am just working on a few graphics to help illustrate the system.
To answer your question, most of the “unit tests” that shipped with the controls are borderline “end-to-end” or integration tests: they obviously do interact with the underlying control framework and Silverlight platform.
When working with Silverlight controls and visuals, a lot of the time it’s important for those things to actually be placed within the visual tree for the events and other properties to fire and to be ready to test.
So by using the Asynchronous test system, the CreateAsyncTest can help by giving the runtime the time to place the control in the visual tree.
The asynchronous test isn’t happening in another thread, but rather in the same user interface thread. So, about 5ms after that Create call was run, the callback action to perform the assertion is called. That is a long time in the CLR world, enough time for any pending visual tree updates and native interop to happen.
Let me know if this makes sense or not..
-Jeff
Thanks Jeff. Please correct me if I’m wrong, this is my understanding:
It is a mechanism for ensuring tests that interact with the visual tree run on the UI thread.
The CreateAsyncTest method in the SL2B1 controls source also makes sure the element has loaded first.
Jeff – you said:
“One proof-of-concept project I did was get NUnit’s metadata to work within the Silverlight framework, just by creating a provider that maps NUnit metadata to generic unit test interfaces inside the Microsoft.Silverlight.Testing.dll assembly. I’ll be documenting this unit test provider functionality in a future blog post.”
can you expand a bit on this? Does this mean that you get NUnit defined tests to run against MS Test?
Looking forward to mentioned post.
Also – are there plans to enable a user to run selective tests from the test page?
@Jeff,
Please tell me is it the same way we can test wpf controls as well or we have to do it using Visual studio test framework and how can i test wpf controls asynchronously?
@KierenH: You are correct. The CreateAsyncTest method is a little bit of syntax sugar on top of the same concept. It’s really a psuedo-asynchronous concept.
@Inquisitorjax: The web browser test harness doesn’t use MSTest, since the Silverlight CLR and the execution environment are different enough that we went with a simpler, custom engine for hte time being – while keeping the same Visual Studio Team Test metadata. I should try and finish that post someday, sorry!
@Mike,
I don’t think that there’s an easy way to test WPF controls today in the same manner, but I know I’d love to see something happen there. My team’s heavily involved in WPF as well, so if there isn’t a way to do it today, rest assured we’re going to work and find a solution someday.