Don't Break Your Dynamics CRM Online instance

Yes, Microsoft Dynamics CRM Online is a well-known and solid system. But I thought I should warn you that you can break it, using nothing but the SDK code, if you use it in a certain way.

In a migration job from an On-Premise install to an Online install, I was migrating tracked e-mails with attachments. From how I'd migrated other entities, I figured attachments would follow the same procedure: Import e-mails with their existing Ids, and import attachments, with their existing Ids, referencing the existing e-mail Ids. Boy, was I wrong.

Everything worked, meaning the service did not object to what I was doing to it, and browsing the activites online worked fine. Or so I thought. When looking closer, the attachments were there, but they were "invisible". The file name and files size were missing, and nothing showing up in the e-mail viewer.

So of course, my first action was to delete everything. A batch job to delete all e-mails, failed. That's strange... Well, try to delete the attachments from inside the e-mail viewer, then? The garbage can icon was there, and appeared to do some work when I clicked it. But no, it ended up in an error:

Now I was really getting unsure how to proceed, so I attempted to delete all accounts. Still no joy. Accounts were not getting deleted as long as they had e-mails with attachments connected to them! After a few attempts at this, and realizing there was no backup of the instance available, and worst of all, this was a production instance (but not yet in production), I connected with Microsoft Partner Support. It's now been a couple of weeks, and the problem is still not resolved.

I have since created a sandbox instance, and found out how to migrate e-mails with attachments. So, here's my solution (drumroll):

• Get all e-mails from the source instance
• For each e-mail in the source, create an upsert or an insert for each e-mail
• Then, loop through the attachments, and do a create request for each
• If you're doing this in batches, remember the maximum count for one batch is 1000 entities, so keep track of how many entities you're trying to push

Doing it this way has one downside that I've found, which is that pictures embedded in the e-mail no longer work, because the original guid is embedded in the e-mail source. Maybe someone somewhere will have a solution for this as well.

Hope you read this blog post before you do a failed e-mail migration like myself. If not - best of luck!

Testing an MVC site with integrations - part 1

Just recently I became a Dynamics CRM integration consultant. That's a new product for me, in the past I have done one integration, but it was just a small data syncronization thing, and not a "real" project.

I joined an existing project, where the product is an Orchard (an ASP.NET MVC 5 based CMS system) module that integrates with a Dynamics CRM installation. It provides a self-service point for the client's customers. Sounds good, eh? Orchard was new to me, in addition to Dynamics CRM, but I quickly found my way around. Orchard uses Autofac, log4net and Automapper internally, so it's quick and easy to hook up your work using their conventions.

Anyway, this blog post was intended to be about testing. When I joined the project, there were literally no tests in there. So I started diving in to write some, before doing changes to the code and scratching my head, wondering if things were still working as intended.

It quickly became apparent why there were no tests... At least from my point of view, the only tests that could be written at that point, were end-to-end tests. MVC controllers were calling Orchard and CRM proxy code here and there directly, and a lot of information was being passed to other classes in their constructors, not to mention a lot of work was carried out in these constructors.

As we all know, or should know by now, tight coupling is bad for you. So my first mission became to reduce the coupling between Orchard and our Orchard module to a minimum. Sounds like a strange thing to do? I don't think so, and all you need to do is really to place any Orchard-referencing code in an apt interface. There, now you can mock all your user settings, content and what not, and actually write unit tests on your controllers (or other classes, of course).

And the same applies to the CRM bits of course. Extract interfaces that do the CRM integration bits. In my case, this made sense to put in a separate library. The project actually has several different clients, so gathering the common logic in this library saves time. Now that your controller only references some interfaces that we can mock - we can start writing actual tests!

So let's begin by testing the routes in your application. In my case I found a few routes that lead nowhere, and removed them, I also found a few not working and fixed them. That's value from writing the first ten or so tests... If you're using MVC there's a neat helper function in the MvcContrib.TestHelper package. It let's you write a route test like this:

"sample/12345".Route().ShouldMapTo<SampleController>( x => x.Action("12345"));

If your project is something other than a "vanilla" MVC application, you may be providing routes in some exotic ways, other than adding them to the RouteTable. If so, remember you can still add your routes to the default RouteTable in your test setup method.

In Orchard, you can get the routes from your route provider and use them in tests like so:

var routes = new MyPage.Routes().GetRoutes().ToReadOnlyCollection(); routes.ForEach(s => RouteTable.Routes.Add(s.Route));

Second, I test the validation on the input models. I'm talking about the dataannotation-type validation, with attributes on the model's properties. It does not cover your coded validation of your model. This is important both for your data integrity and security reasons, but also for the users experience when using the application. If validation fails, the user may be left unable to perform whatever task she is up to (worst case: offering you a job). The easiest way I find to test this validation, is by going directly to the ValidationContext that MVC uses internally:

private IList<ValidationResult> ValidateModel(object model) { var validationResults = new List<ValidationResult>(); var ctx = new ValidationContext(model, null, null); Validator.TryValidateObject(model, ctx, validationResults, true); return validationResults; }

You can simply check if the validationResult.Count > 0 if you want to see if the model passed validation. If it is a specific valdiation error you are looking for, don't just check that it is greater than zero - assert that it is exactly the number expected, and with the expected error:

Assert.That(result.ContainsKey("InvalidPostCode"));

Third, we can start testing the controllers. We now know that the routes will resolve to the correct controller and action, time to test whether the action returns the expected result. Again, the MvcContrib.Testhelper library helps us get going:

var builder = new TestControllerBuilder();
var controller = Container.Resolve<YourController>();
builder.InitializeController(controller);

Now our controller variable has all the properties that your code expects to find on it, and you can test TempData, Forms variables, HttpContext etc.

That should be enough for this first part, I hope to write some more soon :)

Turning coding into a 4X game

I am a fan of 4X games, but I don't have time for gaming these days :( So, let's see if we can add a hint of gamification to coding! Get points for doing stuff you should do anyway, and earn new "abilities" on the way.

I've been surprised before at how much a point system or a badge can do (just look at untappd). So why not give this a shot? The hardest part will probably be keeping count of your points.

Worst case if you try, you'll end up with better code - or better knowledge of it.

Explore
This one's a given, I guess. Whenever I join an existing project I spend some time exploring it, taking the tour so to speak. This may be the hardest task, it's not always easy to pick up all quirks or where non-core logic lives. I generally recommend getting an overview from one of the architects or a well seasoned developer on the team, then dig from there.

A little hard to gamify, but: 10 points for completing, not skipping ahead!

Expand
Add to the existing code base. Write tests if they're missing. Implement new specs. Write more tests.

1 point for adding new functionality. After all, that's the fun and easy part. 5 points for each missing test added. 100 points for writing a test that uncovers a bug. In my experience you can get a lot of points here...

You can get new abilities as you progress as well, how about
Level 1: Write unit tests ONLY. Write 10 passing tests.
Level 2: Integration tests allowed. 10 integration tests and 10 unit tests.
...and so on,  let me know where you end up ;)

Exploit
I like to think that I apply SOLID and DRY principles to my code. See if you find parts of the code that may be in violation.

Level 1: Find 5 DRY cases and fix them
Level 2: Find 5 Single Responsibility cases and fix them
Level 3: Find 5 Open Closed cases and fix them
... and so on

Exterminate
The fun part! You get to delete any dead code you encounter, be it commented code, unreferenced functions or entire files.  But here be dragons: Unless you have gained an intimate knowledge of the project, see "Explore", you'll risk deleting code that seems unused, but is actually referenced from some arcane spell.

Another way to "exterminate" may be to do code reviews with your peers (or shall we say, competitors) and give points for finding issues in their pull requests.

1 point per deleted line of comments, 5 points per file or class, 10 points for issues in your peers code.... this may harden the competition. Increase at own risk for further friction in the workplace!

Bonus:
Complete all 4 without breaking anything and get an extra life for you next project!

;-)