Tuesday, May 21, 2013

Shine a light on software errors to increase quality

My team develops, tests and operates a fairly complex SaaS product.  We have customers who run their business on our product.  They pay us money and they expect it to work.

I am proud of my team - they deliver a high quality product.  We have a release every 10 weeks with minor point releases in between.  For a complex SaaS, we do a lot of releases.

I believe that the quality of the product you deliver goes up by not hiding errors.  In our application, every time user action generates an error, whether it is visible to the user or not, our application generates an email that goes to the DevOps team and myself.  Every time.  Very few of the errors block or affect user activity and almost all are recoverable, but we know they happening and then we look for patterns, we investigate and we prioritize fixing them.  If we believe a user is blocked or having trouble we reach out them and ask if we can help.

I am a big believer in exposing errors and bugs in a software application.  If your code is silently logging to some log file then you never know they are there, you may ignore them as "expected" and no one ever gets around to fixing them.

Here is a cool side affect of exposing errors - everyone becomes more focused and aware of them.  The QA team makes sure that we test thoroughly and known errors are captured.  The software developers want to make sure their code works - the quality of the entire applications goes up.

If you want to quality of your product to go up, put in mechanisms that expose errors that users are seeing to your technology team and make it clear that minimizing them is a high priority.