A checkin policy to detect that the build is broken in a CI environment
One important aspect of Continuous Integration is getting notification that the build is broken. One common way is via a desktop notification from an application like TfsAlert that listens for events (or polls if there isn’t a TFS-style event notification mechanism).
Ideally, the build or unit test break would be noticed as soon as it happens, so that the problem can be fixed before other checkins pile up. Clark Sell has come up with a checkin policy that detects that the last build failed and produces a policy failure when that happens. That makes it pretty hard to say, “But I didn’t know about the break.”
It’s late, your tired and yet somehow you got stuck fixing the build because junior engineer bob just fired off a check-in and took off. It looks like the build is now broken because junior hasn’t done a get latest in a month. In the meantime 10 other people check-in and just end up compounding the problem.
Unfortunately things like this happen. One day I just got so frustrated, I just wrote a custom TFS check-in policy which gets the last build status and validates a particular build type was actually passing before you check-in.
I’m hoping Clark will post an attachment with the solution in a zip file.
The checkin policy has to make a web service call to find out the state of the last build. The performance hit of multiple evaluation calls could be reduced by putting in a time window check or doing it asynchronously (checkin policies are expected to be really fast). However, if you combine this checkin policy with his TfsAlert project and eliminate the call to the build service by changing the checkin policy to use the last build notification from TfsAlert (assuming TfsAlert has gotten at least one build notification), you get double coverage with better performance.
I think this CI checkin policy is a pretty cool idea.
[Update 10/30/2006] Clark has posted a zip file containing the code in a VS solution, including a setup project.
[Update 3/29/2007] We’ve added this build checkin policy in Orcas!