|
by Larry Baxter, Sr. QA Engineer.
I recently resigned from a Sr. QA Engineer position
with a company in Dallas, Tx. that develops medical software. Their niche is in patient tracking for emergency
departments.
While working there, the most problematic defect I uncovered came to be call the NightRider 30th patient problem.
Let me explain a little background first.
Basically, the purpose of patient tracking software is to keep and log all the critical time stamps of a patient
encounter as they come into and are discharged from the emergency room. The complexity of this purpose is almost
mind numbing in scope.
Anyway, one day I got the idea to test the system for long term
performance. I gathered a list of the most common tasks which our system included and built an automated routine
(which I named the Night Rider) to perform these tasks over and over with slight variations. The idea was to determine
if the system could stand up to some heavy pounding over time. I built the tool to input 500 patients and to be
run simultaneously from up to 50 different client machines. I used the Automate program by Unisyn software.
The steps I used to set up each client machine were as follows:
1. Connected the workstation client to the NightRider database through the ODBC.
2. Ran the client.
2. Had the user select an available bed through which all the patients would be added.
3. Placed four staff members on status in the system and assigned these staff members to the bed.
The throughput steps then went like this:
1. Logged in as a Registrar staff member and created a new patient in the bed.
2. Logged out then in as a Nurse and moved the patient to an ancillary location (this would vary between Triage,
Radiology and Observations).
3. Logged out then in as a Charge Nurse and moved the patient back to the bed.
4. Logged out then in as a Clerk and placed a lab order on the patient.
5. Logged out then in as a Nurse and forwarded the lab order on the patient to returned.
6. Logged out then in as a Physician, assumed care of the patient, and forwarded the order to reviewed.
7. Logged out then in as a Clerk and advanced the patient to a care complete state (this would vary between Admitting,
Discharging and Transferring).
8. Logged out then in as a Charge Nurse and made the patient leave the ER.
9. Once the patient was gone, the script would repeat the same steps, incrementing the patient name by numbers
1,2,3...etc.
The defect this tool uncovered was the most difficult defect the
developers at this company ever had to face. What would happened without fail was a resource leak would cause the
client to crash at the 30th patient on the final step of making the patient leave the ER. Eight different developers
used everything they had to discover what was the cause of this. They used bounds checker in the developer studio
and found nothing, they used d-bug builds of the product, nothing pointed them to the resource that was leaking.
This went on for about 6 months. Finally, one developer got a hold of a tool developed by some C++ guru that identified
the leak. It seems that a certain list control was not closing out correctly. The developer (and they wouldn't
tell me who the offender was) who built it had forgotten to close the loop on a simple line of code.
Hail to the world of automation! For without it, this defect would
never have been discovered and each of the client systems in our hospitals would have spontaneously crashed by
themselves at least once every week without reason.
 Share your thoughts on Share the Experience by Larry Baxter in SQAtester Group.
 E-mail to a Friend.
|
