|
by Michael Stebbins Sesame Technology
Recently, I asked around the testing and engineering
community for the top problems relating to defect tracking processes and tools. Quality assurance and development
managers took a moment out of a busy schedule and once engaged, gave responses that conveyed two trends: First,
the interest and knowledge common to each contributor demonstrated that defect tracking is a growing and integral
part of daily product improvement processes at most successful companies. Second, the responses crossed a surprisingly
broad spectrum of best practices and feature-specific issues. For example, some responses were solely focused on
technical features such as integration and escalation, while others focused on solving an intra-departmental behavior
such as problem ownership and assignment.
After reviewing and categorizing the various issues, it was interesting to note that many concerns are already
addressed by existing technology. To their credit, the tracking systems of the 90's did a 'good enough' job for
the 90's, but in the early 2000's, process-oriented managers will clearly benefit by taking a look at the new breed
of defect tracking solutions.
So here are the collated complaints -- the consolidated collection. Here are the top 10 signs that it is time to
look at new technology and improve your defect tracking system.
10. Someone switched a group of issues
to 'won't fix' and you cannot tell who did it.
It might be 'won't fix', 'cannot recreate', 'deferred'.... Someone did something less than optimal and you have
no way to trace the action. How well does your current system log audit trails? Overlooking internal politics and
staff training, episodes like this can be mitigated by a tracking system that logs complete audit trails for each
event that affects an issue. Look for a system that logs all fields and events, and then filters audit views to
preset user security settings. Some modern tracking systems limit or deny record deletions as well.
9. Issues have been assigned to you
that you didn't know about.
This is a good one. Some of the older tracking systems require a user to log on and select a category to see if
any work is assigned. In a process where issues are assigned, perhaps less than once per day (perhaps every hour),
how does a user know to log on if there was no notification? Newer systems typically support home page to-do lists,
email notification, and wireless alerts via cell phone, pager, and PDA. Then again, to keep from getting flooded
with alerts, choose a tracking system that can consolidate multiple events into one notification.
8. Developers have code checked out
of source control and don't know why.
Scenario: Pat Programmer goes to check out build 26r of Project Argus only to find that the project is already
checked out by Elwyn Engineer. Pat calls Elwyn, who cannot remember why the code is still checked out. With the
latest tracking software, a simple lookup for the project would reveal which issues relate to the check out. The
latest defect trackers offer various levels of integration with SCM systems. Integrations include version locking,
updates to related issues on check in/out and uploading of diffs on check in to the defect database. Bonus: Find
a tracking system that uses an open API for SCM integration. This will allow integration updates in the future.
Indeed the open API can be used for integration with much more than the SCM system.
7. Engineering managers continually
get phone calls asking for status.
Those of us who have driven children on a long trip understand this one. When my kids were old enough, I simply
gave them the map. Now it was my turn to ask "Are we there yet?" Take a look at the administration features
on your current tracking system. Does your current system allow simple status updates to all authorized users?
An efficient tracking system with tuned notification will post advances in process to the right groups and individuals.
These notifications may be triggered by a change in status, an update of any field, or a timed escalation. For
those who must constantly ask, a bookmarked query through today's Web-based defect trackers will provide the answer.
6. An issue is moved to another point
in the workflow and then cannot be found.
This happens even with today's tracking systems. To avoid the problem, managers often implement three types of
ownership i.e. 'originator', 'owner', and 'assigned to'. In this case, the originator is usually read-only, and
belongs to the login ID that created the issue. Owner may change in some circumstances, but less often than the
'assigned to' which applies to the person responsible for the current work on the issue. This way, issues may be
quickly located and grouped by owner. Bonus: Be sure to find a tracking system that supports high-performance,
broad keyword searches and optionally searches attachments.
5. Engineers are fixing the same problem
two, three times [also: Boss is on vacation...Work is not assigned].
Does your current system detect product names, modules, and categories and then automatically populate the assigned-to
field? Beyond assignment, the ability to trigger a 'related issues' search on submission can eliminate the 'blind
overlap' problem--two engineers working on the same problem unbeknownst to each other. Be sure to implement a system
that can break problems down to sub cases and assign each sub case to the proper owner.
4. Pick lists are too long.
If you are deleting items from your tracking system to clean up long pick lists and UI clutter, you may be due
for a tracking system that detects user IDs and serves up custom views that are appropriate to the task at hand.
This process of "allowed values" limits filed appearances and list choices to those appropriate for the
current role and task. For example, Engineering may not be concerned with your test battery options and QA doesn't
need to know which compiler Engineering used. Look for a system that groups these items and limits views to a reasonable
working set.
3. Engineering ignores assignments.
Here it is again: The never ending saga of QA influencing Engineering to work on priority issues. But we need to
consider both sides. If your tracker does not support strict rules on Email notification, engineers can be flooded
with inappropriate notifications. In this case, notifications are often ignored or deleted. Consider also that
engineers require access from a multitude of platforms such as Windows, UNIX, Linux and BSD as well as from within
a configuration management system. Ease of access and integration with your configuration management and work assignment
systems is vital to retain the commitment of the engineering team. Bonus: Most new tracking systems employ an escalation
and notification system based on time triggers.
2. Queries take too long.
One of the top reasons that organizations move to a newer tracking system is that queries just take too long. Perhaps
the underlying database cannot handle high volumes of records. Perhaps the algorithms are outdated. Let's face
it--our products usually have more defects and change requests than we originally planned. If your current system
is based on an outdated or proprietary database, you should consider a newer defect tracking systems that supports
ODBC compliant systems such as Oracle and SQL server. This leaves room for scalability in the future. Look for
a fast response time for your general queries, regardless of the number of records in the system. Also, look for
size limit options on query returns so modem-based users can page through reasonable data returns as necessary.
1. Modification requires more engineering
hours than your own products.
Who do you call when you need a workflow change or new product line in the tracking system? If you named one of
your engineers, then it may be time to look at the latest tracking systems. Even homegrown or 'freeware' systems
require programmers to respond to errors or new requirements. Newer commercial tracking systems can reflect a wide
variety of development, test, quality, and support workflows through a click-to-configure interface. A simplified
tracking system will free up your engineers to focus on your core technology.
Runner-ups to the Top 10 List:
These responses were not as common as the others,
but they do warrant mention here.
The system is geographically challenged
[also: No customer-facing option].
Today's trend is to include as many people as possible in your product improvement process. If your defect tracking
system does not implement the security and web-interface necessary to allow partner and customer input to your
product development, test, quality, and support processes, then it may be time to consider systems that can. Do
you need to make your tracking system available to multiple sites? Many of the legacy client-server tracking systems
required extensive security and client-side software to transverse the firewall and join remote teams. Some systems
even maintain separate databases that synchronize every evening! If your system does not leverage the Web as well
as a single scalable database, then it may be time to look at the newer tracking systems on the market today. Secure
logins and role-based, partitioned screens serve up only the appropriate date to each user. Through a Web-based
system, customers and partners may submit requests from the outside while managers and internal staff may perform
custom queries and record edits.
You are still supporting client-side
software.
Are you still supporting client-side defect tracking software? Are you still limited to Windows-based clients?
It may be time to move to a Web-based defect tracker. This alone will provide access to any platform that supports
Netscape or Internet Explorer. Web browser interfaces have come a long way in the last two years. Be aware of "Web-Enabled"
systems that still require client-side software. And lastly, consider using an ASP, online-hosted defect tracking
system that requires no software or maintenance whatsoever.
Out-of-date (or no) API.
Many custom extensions, scripts, and integrations can be made through a good API or Command-line-interface. Should
you change SCM systems or integrate with a marketing database, these interfaces should be simple and accessible
to current scripting or programming languages such as Visual Basic, Perl, Java and C++. If your API requires old
or proprietary languages, or is non-existent, it may be time to check out other options.
Adding new groups or projects is a
pain.
As a company grows, so grows the complexity of representing product lines and corporate structure. Older trackers
require programming or complex administration to add a new project or privacy rule to the system. Some even require
an additional server license. If you find yourself hesitating to add a new team or project to the system, it is
time to consider a system that can offer role based or portal-based views. . These trackers can detect a user's
ID and serve up custom screens, lists and workflow that apply to the task at hand. Scalability without clutter
is the new plan.
Your tracking system was included
in of a suite of tools.
Did your tracking system come as part of a testing or SCM suite? Is it the primary focus of the supplying vendor?
Often, best-of-breed products that are the primary focus of their vendor offer stronger integration options and
a longer implementation life than "freebie" trackers that complete a suite. In the future, you may choose
a new testing tool, or configuration management solution. Consider that you may need to transfer your entire tracking
system as well - especially if your supplier didn't allow for integration with the competitor.
In addition to looking at new technology, give your current vendor a chance to offer the latest update to your
current solution. If an update does not meet your requirements, or requires more effort than implementation of
a better product, then open up your search to new suppliers. Many vendors offer configuration and data migration
services that save you time and keep your focus on your own products. Assessing new technology for your product
improvement process is not without its difficulties, but the effort is clearly worthwhile when engineering and
QA teams are working efficiently, in unison.
Brought to you by ExtraView - the top performing bug tracking system
ExtraView information may be found at www.extraview.com Michael Stebbins is a Strategic Technologist with Sesame
Technology, www.sesame.com in Scotts Valley California. He may be reached at stebbins@sesame.com
 Share your thoughts on Top 10 Signs That You Need a New
Defect Tracker in SQAtester
Group.
 E-mail to a Friend.
|
