| |
| |
Promote Your
Products and Services
SQAtester.com Storefront Program is the best way to help companies Promote their Products and
Services. More...
|
|
|
| |
See Also
Test Descriptions. More...
Best Practices for Software Projects - Estimating. More...
Best Practices for Software Projects - Estimating. More...
Minimizing Software Defects via Inspections.
More...
Testing GUI. More...
Best Practices for Software Projects
- Software Measurements. More...
Key QA Documents. More...
Improving the Quality process by
doing the
Metrics Calculation
Download
Zero-Defect Software Development. More...
|
|
|
|
TestArchitect
solves the biggest problems of software testing: producing
and maintaining vast libraries of manual and automated test scripts. More...
|
|
| |
FREE Try our [ASP Hosted solution] for free for one month, no obligation to pay.
An extremely practical and cost effective Web Based Bug Tracking Software and help desk software ready to be integrated
into company intranet today. More...
|
|
|
| |
Learn more material at a Fraction
of Price!
Get the training that will help you face the challenges
and meet the demands of today's competitive market place.
Programs
Offered: Oracle DBA, Cisco Networking,
Software Testing & Programming. More...
|
|
|
| |
To compete in today's market, you need Access to your Information Quickly and without hassle.
We can make this possible.
FREE 2 week trial
|
|
|
| |
Test your Web Application with
a professional Java™ Load testing Tool.
Proxy Sniffer is a full-featured web load testing tool, designed for professionals More...
|
|
|
| |
Find out which Bug-Track.com account is right
for your software development team, Try our Demo Now!
|
|
|
|
|
As the Web moves to the core of business, testing
of applications and infrastructure is more important than ever. Vendors are supplying the tools.
By Alan Earls
|
| True Test Of The Web |
 |
__________________________________ |
|
Any companies doing business on the World Wide Web
are one click away from disaster. With Web sites playing an increasingly important business role, it's essential
that these systems deliver--whether their job is handling product sales, distributing information, or linking up
with partners' systems. Of course, gaining that assurance has been anything but easy. The task of testing the performance
of a Web application involves more than just running a functional test on the specific application, though that's
important. It's also critical to ensure that all the supporting infrastructure works under varying pressures and
conditions.
A growing number of vendors are addressing that difficult task with testing tools that range from freeware to complex
bundles of products and services that can cost $250,000 and up.
The high stakes of E-business mean it's no longer a question of whether to test but how. Consider the experiences
of those companies that have suffered high-profile site failures. In October 1997, with a market rally in process,
E-Trade Securities Inc., the Palo Alto, Calif., pioneer in online stock trading, suffered a lengthy and embarrassing
service outage when trading volumes got too high for its system to handle. The company survived the crisis and
made several important technical and procedural changes--but not before seriously damaging the confidence of its
customers.
Similarly, a glitch during unscheduled maintenance at Amazon.com forced the site offline for several hours last
year, prompting rumors that the online bookseller had gone out of business. Although Amazon.com wouldn't reveal
the cost of the site shutdown, outside analysts point out that daily business on the Amazon.com site can be as
high as $400,000. And in order to repair its customer relations, the company felt compelled to issue discounts
to those who claimed to have been unable to access the site during the shutdown.
Troubled Debut
Worst of all might have been what happened to Egghead Software Inc. The very weekend that the retailer closed its
last store as part of a much publicized move to Web-based electronic product distribution, its Egghead.com site
went down for two days. Again, the company won't say what the glitch cost but outsiders note that Egghead probably
has about $1 million a week in online sales, making a botched debut a matter of at least a few hundred thousand
dollars in lost revenue.
These three examples of Web-site failures are indicative of what companies can expect as they put their most critical
operations on the Web. Customers have little tolerance for failure. So the questions for these Web-based businesses
are, who should test and how often?
New Venture, Or Old Rules?
Sandy Taylor, director of analyst services at the Software Productivity Group in Natick, Mass., divides these risky
new Web ventures into two categories: companies that are developing a brand-new E-commerce Web site with built-from-the-ground-up
functionality, and companies that are simply building Web functionality on top of their existing business rules
and infrastructures. Taylor says companies in the first group are the ones taking the biggest risks and therefore
are most in need of testing. Companies in the second category also must test their applications, but they at least
have some existing metrics and experience to build upon.
Industry experts agree that Web testing is a good investment. Testing in general has achieved a lot more visibility
lately, aided in part by the focus on the year 2000 remediation efforts. And, given the visibility of Web sites,
it's fairly easy to argue for testing, says Liz Barnett, a VP at Giga Information Group. Barnett is especially
bullish on the need to test sites involved in E-commerce. Although large intranets and extranets can face similar
problems, she says, it's even more difficult for site developers to anticipate load once an E-commerce site is
opened to the general public. Doing business on the Web is like opening a traditional store but not knowing whether
it will be located in the middle of an Iowa cornfield or on New York's Fifth Avenue.
Furthermore, in the world of the Web, functional changes happen much faster and more frequently than in traditional
mainframe environments, which tend to implement functional changes annually Client-server environments tend to
implement functional changes on a monthly basis. But Web sites sometime need to make daily changes, which can impact
back-end infrastructure and system functionality.
Fortunately, the problems faced in the testing of Web applications are not entirely new. Indeed, the history of
most site-testing tools goes back to the traditions of client-server computing. In fact, from a conceptual standpoint,
Web testing is much like client-server testing--but with more potential tiers and greater load variability. With
a Web site, there is no way to predict with certainty how many users will hit a site at any given time. The Web
environment has also introduced extreme heterogeneity to the support equation. Users may be accessing resources
deep in a company's infrastructure, and the IT department can't control what browser they will be using. The potential
interactions between technologies are almost limitless.
That's why some analysts prefer the term Web application testing to the more traditional functional and load testing.
What matters is the application seen by the users, not whether the Web site is having trouble or the server is
broken.
Three Functions Of App Testing
Within application testing there are three functional areas. The first is site testing: a structural test that
looks to see whether there are pages that are inaccessible or whether popular features are unnecessarily hard to
access. The second and third areas deal with whether the application functions as designed, and whether that functionality
survives at various load levels and under a variety of scenarios.
Vendors offer each of these capabilities in a variety of flavors and levels of complexity. Some also offer integrated
suites that provide life-cycle testing.
The Web-testing market is young and fragmented. Prominent vendors include Mercury Interactive and Segue Software,
whose roots are in client-server testing. RSW Software is a new subsidiary of hardware test giant Teradyne and
focuses exclusively on Web testing.
Mercury Interactive is generally regarded as the 800-pound gorilla of the business, with a mature and well-respected
product line that includes LoadRunner and Astra. Segue is an aggressive company offering tools such as SilkPerformer
and a sophisticated scenario testing product called LiveQuality (see story, p. 6A) that permits the generation
of complex interaction with large numbers of virtual users. Other players include Compuware, Mindcraft, Rational
Software, and RadView Software.
Web Watching
All of these products are currently being put through their paces in the real world. E-Trade for example, is using
Mercury Interactive's LoadRunner. After its aforementioned meltdown, E-Trade vowed to never go through that again,
says Gary Kattge, director of quality assurance. E-Trade looks back at that experience as positive because the
company learned a great deal from it. "Since that time, we've had much higher peaks, which we've handled without
any problems," he says.
The meltdown forced E-Trade to adopt a proprietary stateless architecture, which enables the E-Trade Web site to
provide secure, reliable, and almost instantaneous service simultaneously to hundreds of thousands of users.
E-Trade, which rolled out its new and improved Web site this past summer to all 500,000 active accounts, experiences
about 2.2 million page views per day, and expects further traffic increases as more noncustomers visit the site.
During the quarter ended June 30, E-Trade processed an average of $1.5 billion in customer transactions per week.
In preparing for its public launch, E-Trade developed and installed a technology infrastructure able to handle
massive traffic volumes. E-Trade's Stateless Architecture system allows more than 1 million visitors to use the
Web site simultaneously and permits as many as 150,000 customers to place orders at the same time. E-Trade says
the system can be scaled up to handle even heavier loads.
Getting the system to that level of reliability was Kattge's job, and testing was a big part of the project. Several
testing-tool vendors were in the running initially, Kattge says, but based on functionality and extensibility,
E-Trade reduced the field quickly to one: Mercury Interactive.
Mercury Interactive's LoadRunner product lets E-Trade put hooks in the test code. This allows E-Trade personnel
look in-depth at application response time--right down to the SQL database-query level. The testing tool has given
E-Trade the ability to calibrate and verify new system designs and to detect areas that might need additional tuning.
But LoadRunner's most critical role was in helping to measure response times between various tiers, where the potential
for serious errors exists. LoadRunner is used on an hourly basis as part of an alert system to quickly escalate
any performance issues likely to require intervention.
Commonplace Tests
Even in the slightly slower-paced world of mutual funds, testing is becoming commonplace. Mutual funds giant T.
Rowe Price in Baltimore is a strong proponent of Web testing. T. Rowe Price recently implemented its first major
Web site redesign. Time to market and availability were critical to IT manager Tom Dietz, who has been looking
at Web testing issues at T. Rowe Price for more than two years. "We recognized Web-site testing as a very
important area," he says.
After evaluating testing tools, the company settled on Segue's SilkPerformer, due in part to its regression testing,
Web functional testing, and load-testing capabilities, Dietz says.
First Call Corp. in Boston is another organization with a Web site that gets put to the test. First Call keeps
broker notes, research documents, earnings, and buy-sell recommendations flowing to its Wall Street professionals
via its Web site. The company's roots in online services predate the Web, but when it came time to embrace Internet
delivery, they selected Segue's SilkPerformer. "We were interested in testing costs, the ability to simulate
user loads, and memory footprints per virtual user," says First Call IT manager James O'Keefe.
FirstCall also wanted a good scripting language. While it was important to demonstrate site viability through load
and stress testing, it was equally critical that the scripting tool be able to show that real information was being
delivered. SilkPerformer let the First Call team monitor specific deliverables like HTML code, or a copyright symbol
that could verify that the site was delivering as intended.
Of course, not every mission-critical site is seen by the public. Paul Roach, a senior quality assurance specialist
in the Richardson, Texas, office of Northern Telecom Ltd., got his exposure to Web-site testing during the requirements
phase of Nortel's global intranet development project.
Nortel's intranet consists of many applications deployed across a global intranet--an environment that demands
frequent testing of the Web servers and applications. Scalability, robustness, and a rich feature set were on Nortel's
shopping list.
In one instance, Roach helped deploy an employee-satisfaction survey that would be accessible by 70,000 employees
worldwide. Other sites required testing of functions as diverse as financial accounting, sales, and marketing.
Dropping the ball with employees, even on a satisfaction survey, can be just as serious as offending paying customers,
he says.
Cost Conscious
Choosing a product and determining what to spend are perhaps the greatest challenges to the newcomer to Web testing.
CBS.com has opted for a freeware program. Compared with the tasks of supporting real-time stock trading or other
"heavy duty" E-commerce activities, Michael Skorski's job might seem easy. Director of technology for
CBS Net Media, Skorski makes sure www.cbs.com is up and running when the network's viewers come looking for program
information and news. And the old adage about the short shelf life of news applies to Web news sites--specially
when there's scandal in the air. "We did four times our normal traffic and an order of magnitude more hits
and page requests when Kenneth Starr's report hit," says Skorski.
Skorski and CBS have been testing their site extensively, which may account for the fact that it survived the onslaught
of Starr gazers with only minor problems.
That's attributed in part to CBS's use of Webstone, a freeware product now maintained and distributed by Mindcraft.
"There are limitations on what it can do," Skorski says, "but the price is right and the product
is now part of our testing regime."
Commercial testing tools have more bells and whistles, Skorski says, but do not necessarily produce more useful
information. His biggest challenge with testing is to simulate the complex infrastructure of www.cbs.com.
Karl Van Neste, a software system architect at the College Board in Reston, Va., warns buyers to beware. "Test
managers need to continually review the products and new capabilities available on the market." Van Neste
has been using a product recently introduced by a small company called RadView Software. Van Neste says he first
looked at Mercury Interactive's LoadRunner but that RadView's product had what he felt was comparable functionality
for a small fraction of the price.
Indeed, new entrants keep cropping up almost daily, making it likely that increased capability and decreased price
will be the order of the day. IT managers will also see a variety of service offerings for those shops that don't
want to experience the cost and learning curve of Web testing tools.
 Share your thoughts on True Test Of The Web in SQAtester Group.
 E-mail to a Friend.
|
| Books to Read |
|
________________________________________ |
|
|
|
Testing Computer Software, 2nd Edition
The original printing of Testing Computer Software set the standard for the emerging field of test engineering
with a full tour of the state of the art in managing the testing process. The reissued text makes this classic
out-of-print text available once again. Though it relies heavily on older. More...
|
|
|
|
Automated Software Testing
Written for those with some background in software engineering, Automated Software Testing: Introduction, Management,
and Performance delivers a rigorous guide to the state of the art in managing automated testing in a text that
will benefit anyone who tests software for a living. More...
|
| Categories |
|
________________________________ |
| Community |
|
________________________________ |
| Specials |
|
__________________________________ |
| Millions of titles discounted up to 40-90% off. Great
low prices on your favorite books. More... |
|
Find all of your favorite software.
More... |
|
|
