Web Application Stress Tools

This web page contains info on "Web Application Stress Tools" (Web Stress Tools). You use workload tools to provide the work stress on your application and performance tools to locate the bottleneck. I hope this collection of information is helpful to someone searching the internet.

Initial research done on 12/04/00

Email me with any additional notes.

Companies that do stress testing as a service

Neustar
Browser Mob

White Papers

http://www7b.boulder.ibm.com/wsdd/library/techarticles/hvws/manage_perf.html -
A new paper introduces a methodology for managing end-to-end infrastructure performance, and identifies best practices and tools to implement the methodology.

Resources

Organizations
- http://www.cmg.org - Computer Measurement Group (CMG)
  - Quote from their site:
    - "The Computer Measurement Group, commonly called CMG, is a non-profit, worldwide organization of data processing professionals committed to the measurement and management of computer systems. CMG members are primarily concerned with performance evaluation of existing systems to maximize performance (eg. response time, throughput, etc.) and with capacity management where planned enhancements to existing systems or the design of new systems are evaluated to find the necessary resources required to provide adequate performance at a reasonable cost."
- http://www.tpc.org/home.page.html - Transaction Processing Performance Council TPC-C (benchmarks)
Web Sites
- http://www.softwareqatest.com/qatweb1.html#LOAD - A list of Load and Performance Test tools. This is an incredible site!
- A friend emailed me these sites: Look pretty good.
  - http://www.testingfaqs.org/tools.htm
  - http://www.stickyminds.com/tools.asp
Conferences
- http://www.cmg.org/conference/conference.html - Computer Measurement Group's International Conference (usually in Dec).
Books
- "Testing Applications on the Web" (Test Planning for Internet-Based Systems) by Hung Q. Nguyen (pub: Wiley) ISBN: 0-471-39470-X
- "Capacity Planning for Web Performance" (Metrics, Models, & Methods) by Daniel A. Menance & Virgilio Almeida (pub: PHPTR) ISBN: 0-13-693822-1

Workload Tools

These tools provide a simulation of web users accessing a web application. You use workload tools to provide stress on the web application and you use performance monitoring tools to determine the bottleneck.

Pylot (Web Performance Tool)
- www.pylot.org - Performance & Scalability Testing - Web Services
  Open Source...
WAST (Web Application Stress Tool) by Microsoft (formerly called Homer)
- WAST home page:
  http://webtool.rte.microsoft.com/
- Quotes from their site:
  - Microsoft Web Application Stress is a simulation tool that is designed to realistically reproduce multiple browsers requesting pages from a web application.
  - This version covers the most needed features for stress testing three tier personalized Active Server Page web sites running on Microsoft Windows NT server 4.0 and Windows 2000.
  - This is a GUI-based simulation tool designed to realistically reproduce multiple browsers requesting pages from a Web application. The tool works on Windows NT 4.0 and Windows 2000.
- Comparison of Microsoft tools - the above link to WAST has a link called "comparison of tools".
NeoLoad - http://www.neotys.com
- Quote from their site:
  "NeoLoad simulates hundreds of virtual users on your web site, getting performance statistics and revealing errors under stress."
- Quote from an email sent to me from a representative:
  We are a company specialized in Web application load testing. We provide a tool named NeoLoad that is aimed to get a clear and intuitive graphical interface, no scripting, comprehensive reports and test results. It can design complex scenarios to handle real world applications.
  Features include data replacement, data extraction, system monitors, SSL recording, PDF and HTML reporting, IP spoofing, and more. It is Multi-platform: Windows, Linux, Solaris.
  There is a free trial available by download on our website.
SilkPerformer by Segue Software Inc.
- SilkPerformer's home page:
  http://www.segue.com/html/s_solutions/s_performer/s_performer.htm
- Quotes from their site:
  - "... accurately predicts the capacity and constraints of Web apps prior to their launch, optimizes their performance throughout their entire lifecycle, and provides insightful answers to performance challenges."
- http://www.segue.com - Segue Software home page.
- http://www.segue.com/html/s_solutions/silk/s_family.htm - Silk Product Family home page.
LoadRunner by Mercury Interactive
- LoadRunner's home page:
  http://www-heva.mercuryinteractive.com/products/loadrunner/references/ -
- Quotes from their site:
  - "Leading hardware, database and enterprise resource planning (ERP) vendors use LoadRunner, the industry standard for enterprise load testing, in their benchmarking and competency centers to produce and publish standard benchmark results."
- http://www.mercuryinteractive.com - Mercury Interactive's home page.
TestStudio by Rational Software
- http://www.rational.com/products/siteload/index.jsp - Rational SiteLoad.
- Quotes from their site:
  - Rational SiteLoad is a powerful, browser-based performance testing tool for Web applications. This essential testing tool simulates Internet traffic and provides testers with precise, real-time information on site performance. Scalable and easy-to-use, Rational SiteLoad provides real-time resource usage feedback as soon as testing begins.
- http://www.rational.com/products/index.jsp - Rational's Products. Click on the "System Testing" link. (Rational Suite TestStudio)
eMeasure & DynaMeasure by RedHat
- http://www.redhat.com/products/software/performance_mgmt/ - Performance Management homepage for eMeasure and DynaMeasure.
  - eMeasure
  - DynaMeasure
Webload by RadView
- http://www.radview.com/products/webload.htm - Webload's home page.
- Quotes from their site:
  - "WebLoad is the only comprehensive testing and analysis solution to combine performance, scalability and integrity as a single process for unmatched verification of Web applications."
- http://www.radview.com - RadView's home page
eTest Suite (eLoad, eMonitor, eTester) by Empirix (formerly by RSW Software)
- http://www.empirix.com/ - eTest home page
- Quotes from their site:
  - "The Fastest, Most Accurate Way to Test the Quality, Scalability, and Availability of e-Business Applications."
- Products
  - eTest Suite - fully integrated Web testing solution - load, regression, and monitoring. (free download)
  - Bean Test - Functionality and scalability test of EJB applications
  - e-LoadExpert - remotely validate the performance and scalability of your Web site on demand.
- http://www.rswsoftware.com/ - RSW Software's home page.
Webstone by Mindcraft
- http://www.mindcraft.com/webstone - Webstone home page.
- Quotes from their site:
  - "It's an Open Source benchmark. Source and executables are free."
- http://www.mindcraft.com - Mindcraft's home page.

Performance Tools

These tools provide data that can be used to graph a "capacity curve" which determines a saturation point based on where the TPS (Transactions Per Second or throughput) flattens out and the Response Time becomes unacceptable (7 secs, desired is 3-4 secs). You use workload tools to provide stress on the web application and you use performance monitoring tools to determine the bottleneck.

Windows NT Resource Kits by Microsoft
- Netmon - Network Monitor - Sniff networking packets and analyze them from a remote network. Monitor network activity proactively.
- Perfmon - Performance Monitor - Monitor performance of system objects in real time. Log to a file, send administrative alerts on thresholds, and generate reports.
- APIMon - API Profiler - User mode debugging tool that supports attaching to applications remotely to trace API calls.
Vtune™ Performance Analyzer by Intel
- http://developer.intel.com/vtune/analyzer/index.htm - Vtune home page.
- Quotes from their site:
  - "The VTune Performance Analyzer collects, analyzes, and displays software performance data from the system-wide view down to a specific module, function or instruction in your code."
- http://www.intel.com - Intel's home page.
Quantify by Rational Software
- http://www.rational.com/products/pqc/index.jsp - Perfmon home page.
- Quotes from their site:
  - "Rational Purify, Quantify and PureCoverage are a complete set of automated runtime analysis tools for improving application performance and quality for software developers who need to build and deploy resilient, reliable software applications in C/C++, Java and VB."
- http://www.rational.com/products/index.jsp - Rational's Products. Click on the "System Testing" link. (Rational Suite TestStudio)
eGurkha
- www.egurkha.com/technology.htm - is an integrated monitoring application suite that addresses the infrastructure and business monitoring needs of eBusinesses.

Additonal Performance Tools

InetMonitor (http testing) by Microsoft.
- http://www.microsoft.com/SiteServer/site/deployadmin/inetmonitor.htm - INetMonitor home page.
- Quote from site:
  - "This load generation tool allows you to plan and maintain the optimal configuration for your site. The Simulator component of InetMonitor allows you to simulate user load on your site, as well as any type of client or user behavior. The Monitor component allows you to track the behavior of a typical user. Using the information gathered in these tests, you can assess your site's capacity, plan for growth, and track hardware resource utilization."
WCAT - "Windows Capacity Analysis Tool" by Microsoft
- http://msdn.microsoft.com/workshop/server/toolbox/wcat.asp - WCAT download & user guide site.
- Quotes from site:
  - "..runs simulated workloads on client-server configurations. Using WCAT, you can test how your Internet Information Services and network configuration respond to a variety of different client requests for content, data, or Hypertext Markup Language (HTML) pages. The results of these tests can be used to determine the optimal server and network configuration for your computer. WCAT is specially designed to evaluate how Internet servers running Windows 2000 (or Windows NT®) and Internet Information Services respond to various client workload simulations."
Windows DNA Performance Kit by Microsoft
- Web Driver—The Web Driver simulates the actions of many Web clients making requests to your server. This allows you to test the performance of your Web applications in a manner similar to the Microsoft® Web Capacity Analysis Tool (WCAT). See the IIS Resource Kit for more information about WCAT.
- Perfcol—The performance collector is a stand-alone application that you can use to monitor any number of performance counters across a large number of machines. The data is stored in a database to make further analysis easy. The toolkit also includes a way to automate performance collection for experiments run with the toolkit.
LoadSim by Microsoft (to stress test Exchange 5.5 servers)
- http://www.microsoft.com , then search for "LoadSim". Hard to find a product page but you will find white papers.
- Quote from site:
  - You can also use the Microsoft Exchange Server Load Simulator (Loadsim.exe) tool to help you determine the level of performance that's acceptable for your organization's users. Load Simulator can help you determine how many users your Microsoft Exchange Server computer can support. It is designed to provide a realistic load on a Microsoft Exchange Server computer by simulating the behavior or users on one or more Microsoft Outlook computers. For more information on running the Load Simulator tool, see the Microsoft Exchange Server Resource Guide.
Open STA - free tool.
- http://www.opensta.org
NetBench® 6.0 by Ziff-Davis (zdnet) (load generator to stress test file server I/O)
- http://www.zdnet.com/zdbop/netbench/netbench.html
- Quotes from site:
  - NetBench is a portable benchmark program that measures how well a file server handles file I/O requests from 32-bit Windows clients, which pelt the server with requests for network file operations.
BenchWeb
- http://www.netlib.org/benchweb
- Quotes from site:
  - BenchWeb is a starting point for finding information about computer system performance benchmarks, benchmark results, and benchmark code. The site is maintained at the University of Tennessee Computer Science Department by Innovative Computing Labs.
Systest Labs
- http://www.systest.com - SysTest Labs home page. Services - software testing and eBusiness testing
Automated Testing Specialists (ATS)
- http://www.sqa-test.com - ATS. Services - software testing
Web Performance Trainer by Web Performance
- http://www.webperfcenter.com - home page. Trial version available.
- Quotes from site:
  - Web Performance Trainer 2 simulates multiple users hitting your web site so you can find performance bottlenecks, increase performance, or do capacity planning.
  - Runs on Windows NT, Linux, Solaris, and most UNIX variants
  - Supports any kind of back-end process, including Active Server Pages, Applets, servlets, plugins, Active X Components, ISAPI, and cgi-bin
PassMark Performance Test by PassMark Software (PC Benchmarking)
- http://www.passmark.com
- Quotes from site:
  - Passmark PerformanceTest is an award winning PC hardware benchmark utility that allows everybody to quickly assess the performance of their computer and compare it to a number of standard 'baseline' computer systems.

Capacity Sizing Guides

IBM Sizing Tool