Launching Scrum’d: What Went Wrong and What We’re Doing About It

by Robert Dempsey on March 17, 2009

Yesterday’s launch of scrum’d was quite successful. We received a lot of Twitter traffic, and had a lot of users sign up.

However, we ran into some major problems once we put the app into production. The first round of issues made it so that users could not use the application. When you logged in, you received a blank page. When you logged out, you got the same blank page. We fixed this issue, and in so doing, caused another issue – you couldn’t create anything new – releases, sprints, user stories, or tasks. That’s what the application does, so that was a bit of an issue. We squashed that bug late last night, and deployed the update. I am happy to report that you can now sign up, sign in, and use scrum’d.

So what happened?

We have extensive testing in our code, and all tests were passing when we launched. We have a continuous integration server that ensures that all tests are passing. We also did extensive manual testing before launching. What we didn’t do is test the application when it was running in production mode. Ruby on Rails acts differently when run in production, and with this app, it bit us, hard.

So how do we ensure this doesn’t happen again?

We are in the process of setting up another production machine where we can run another round of quality assurance on the updates we do before putting those updates into production. Until then, we are running the app locally in production mode and doing the QA there. These two additional levels of testing should help ensure that something like this doesn’t happen again.

So how does this affect everyone that signed up yesterday?

I appreciate all of the feedback our users posted on our support site while we were dealing with these issues. We love interacting with you all, and are happy to get your feedback. In fact, your feedback is what drives the direction of all of our applications.

For dealing with these issues, we have given all scrum’d and expens’d users an additional three days on their accounts. Why expens’d users also? Scrum’d and expens’d are talking to each other, so one can affect the other. So everyone gets free days!

Thank you

Thanks to the team at Morph Labs for helping us troubleshoot the issues. A big thank you to everyone that signed up for scrum’d yesterday, and thank you for working through these issues with us. We look forward to your feedback on how we can make scrum’d better. We’re listening.

Bookmark and Share

Other Posts That Might Interest You

  1. Scrum’d Updates and Roadmap
  2. Expens’d Shutting Down To Add Focus on Scrum’d
  3. Scrum’d and Expens’d Get Updated Thanks to the Community

Tagged as: ads appsuite, business, scrum'd

  • This indeed proves the value of a real staging server. For business critical solutions it will be worth the effort of maintaining it.
    Nevertheless will be reviewing Scrum'd app shortly as it seems very promising!
  • Thanks Gyuri. We look forward to your feedback and your ideas.
  • For the longest time I had a feeling that production testing was forgotten in the rails circles.
  • I've been a long time believer that there is vast difference between an integration/test server and a staging server. Regardless of what you call each, a production server usually has much a more complicated configuration, with ssl, https, load balancers, richer data, etc. A true staging server has to mimic the production environment exactly in order to reduce production deploy errors. This is true regardless of the language and framework of the web app, but perhaps is even more important with Rails since the rules do change depending on the environments configuration.
blog comments powered by Disqus

Previous post:

Next post: