Benchmarking analysis accuracy for MVVM code

One of the key motivators for developing XamRight was to have a tool that would support a wide range of MVVM patterns in Xaml development without requiring additional configuration.  You shouldn’t have to adapt your code to suit the tool — the tool should adapt to your code!

To do that, we developed XamRight, with a fast, powerful analysis engine that powers error checking in Xaml, autocompletion, and more.  XamRight analysis is now able to fully understand over 88% of Binding expressions in this real-world testing.  For developers, that means a more efficient Xaml editing process, with fewer deploy/debug cycles to fix Binding bugs.

To measure effectiveness, we’ve taken a set of open source Xamarin apps, and run “Build\Analyze Solution with XamRight” from the Visual Studio IDE.  We make no modifications to the app source code, nor do we provide any hints or configuration.  The numbers reported are shown in the Output window after XamRight analysis is complete, so you can evaluate on your own Xamarin apps.

The Xaml Bindings column counts the total number of “{Binding …}” expressions there are in the Xaml files in the solution.  Bindings Resolved is the number of Binding expressions where XamRight was able to identify the view model.  Effectiveness is the percentage of the total that were resolved, and our goal is to get as close to 100% as possible. Note that for Shared projects, each Xaml file is analyzed once per platform, so a solution with Android, iOS, and UWP targets, the reported number will be three times the number of Binding expressions actually seen in the code (Arc GIS is an example).

These results are with XamRight version 1.690.

Name Source code Xaml Bindings Bindings Resolved Effectiveness
Xamarin CRM https://github.com/xamarin/app-crm 130 119 91.54%
Arc GIS https://github.com/Esri/arcgis-runtime-samples-dotnet 99 99 100.00%
Xamarin Conference https://github.com/xamarinhq/app-conference 263 226 85.93%
Conference Vision https://github.com/Microsoft/ConferenceVision 38 30 78.95%
SmartHotel 360 https://github.com/Microsoft/SmartHotel360-mobile-desktop-apps 212 168 79.25%
Xamarin Sport https://github.com/xamarin/Sport 228 199 87.28%
Coffee Cups https://github.com/jamesmontemagno/app-coffeecups 28 28 100.00%
Hunt https://github.com/rob-derosa/Hunt 80 68 85.00%
My Shoppe https://github.com/xamarinhq/app-myshoppe 84 84 100.00%
Nethereum Wallet
(Uses MvvmCross)
https://github.com/Nethereum/Nethereum.UI.Wallet.Sample 67 67 100.00%
Money Fox
(Uses MvvmCross)
https://github.com/MoneyFox/MoneyFox 335 299 89.25%
Overall   1564 1387 88.68%

We picked these apps because they were some of the best examples we could find of real apps – multiple pages, different kinds of transitions, showing many kinds of data. There are plenty of great examples of apps showing how to use individual features of Xamarin.Forms, but not in a realistic setting, so we haven’t included those. We don’t include some samples we’ve published for exactly that reason; any analysis we do on them won’t be representative of what your experience will be like using XamRight on a real project.

That being said, if there are more open source apps we should include in our testing, please let us know!

Author

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.