Warning: This post is the impression based on the feedback of one specific Scrum team. It does not try to enumerate all the properties that a Scrum tool should have, and you would be able to find many more posts from other people about specific benefits of one Scrum tool vs. the other.
We have been working with a Scrum team for the past 2 years, and one of the biggest problems that we were facing was about the usage of a tool. In the context of economy, we first tried to modify the existing tool we were using for Waterfall based development method, and used it for some time for Scrum. However, we used to run into issues within the first Sprint cycle itself, since the team pointed out several items that were not a good mapping for the artifacts and information that they learned about in the training, and during their usage of the tool, a percentage of time was spent in discussions about the tool. We figured out that there was a lot of discomfort related to the tool, and the percentage of time that the team was spending on the tool was more than we could afford, and was not at all productive. And it was getting very frustrating to the team, as even matters such as maintaining the backlog (even though it was was maintained by the Product Owner, the team wanted access to the Backlog to see what features were planned for future Sprints), breaking the stories into tasks and capturing the efforts pending for the task, and a good display of the Burn Down chart, were all either not good or were not there at all. We tried using Macros and other reports to make up for what was not there, but the overall satisfaction level was not there.
Finally, after another round of discussion with the team, we decided to get confirmation from management about buying licenses for the Scrum team members in a professional tool to handle Scrum, and went about the process to evaluate the demo versions of different tools. The evaluation process was intense, too long to be captured in this post, which essentially deals with the standards we used to evaluate these tools and determine whether they were appropriate for our use. These criteria were:
– Ability to capture the backlog items in an unreviewed state by the Product Owners and others, and then for the PO to review and do the prioritization
– Ability of the team members to view the reviewed and prioritized Backlog
– Ability to take items from the prioritized Backlog and assign them to the next incoming Sprint
– Ability to capture more details about these items, including User Stories and other free form text, such that details about these items could be captured to make sure that the items have enough information about them
– Ability to take these items and create tasks for them, including marking whether explanation by the PO and estimation by team members has been done for these tasks
– Ability for team members to update these tasks in an easy manner, so as to update the estimates for the pending tasks
– Ability to see the Burn Down chart in an easy manner
– Additional reports such as being able to calculate the Velocity of the team, and compare across Sprints
– Another additional criteria that would be a good factor was about whether the tool could be installed locally on a server inside the company rather than on a remote server. This helps a lot in terms of speed, but there are issues about maintenance and who will bear this responsibility
(If you folks feel that there are other items that would help, please provide these in comments).