See our LinkedIn profile, click on this button:


Monitor Standard Application Scenarios

In recent years, much has been written about the value of use cases and scenarios for capturing functional requirements; by comparison, their usefulness for performance management has received scant attention .  An application scenario defined for performance management purposes:

  • Involves a known fixed workload
  • Runs in the normal production environment
  • Runs against the production databases
  • Is instrumented to record response time

Because standard application scenarios are application/program instances with defined behaviors, their use of computing resources is also (relatively) predictable. In a sense, they are “benchmark” programs, since they perform a similar function. Normally however, performance benchmarks are designed to mimic a particular type of workload on a component or system, and are used to measure system capacity and throughput when processing a typically broad mix of applications.  Standard application scenarios, in contrast, can be designed to measure a system’s responsiveness for a single precisely-defined set of processing needs.

In today’s enterprise computing or Web environments, many components combine to determine the performance of applications in the production environment. These include LANs, WANs, workgroup servers and databases, enterprise servers and databases, internet servers, and the networks that make up the Internet. If an application slows down, which component is responsible? The value of standard application scenarios is that they can be systematically chosen to test the responsiveness of these various components individually.

Do You Need Special Purpose Code?
The applications used for scenarios are usually not purpose built–in fact it’s best if they are not. The ideal approach employs existing production applications, but used in carefully controlled ways. By exploiting the characteristic behaviors of existing applications under different usage conditions, we can create a suite of standard tests each of which comprises a distinct and known level of activity. For example:

  • A minimal workload might involve a request to a remote server that returns without doing any database activity, allowing us to monitor the communication delay.
  • A small workload might retrieve a single database record using an index.
  • A typical workload would be one that matched the expected performance profile for a certain application class.
  • A large workload might do a table scan of a large table, retrieve a large amount of data, or perform extensive computations, depending on the type of application.

Service Level Agreements
When service level agreements (SLAs) exist, the definitions of terms like minimal, small, typical, and large can be tailored to match workload levels defined in the SLAs. Also, the ideal SLA specifies how service levels will be verified in practice; one method is to use standard use cases or scenarios that generate known workloads. I’ve been noting the potential for this kind of synergy between functional and performance specifications for almost 20 years now, so I’m pleased to see Steven Haines also recommending this approach in his 2006 book, Pro JAVA EE 5 performance management and optimization.

Developing and Using Standard Applications
Before applications are deployed, it is common for developers and testers create test cases to validate aspects of their code, or of overall system behavior. With a little more planning, standard application scenarios can be identified at this time, and used during testing and deployment to verify that an application meets its performance goals. Once standard execution scenarios have been defined for an application, test scripts can be written to incorporate them into the suite of tools available for performance monitoring.

Using performance testing tools, we can run a suite of standard application scenarios, measure their response times, and summarize the results. Given a sufficiently comprehensive suite of standard application scenarios, such a report would serve as a system-wide summary of response time performance, to be viewed alongside other performance reports or a performance management dashboard. Individual scenarios can also be executed and measured when investigating specific response time concerns, to isolate the location of performance problems.

This is the second in a series of posts on the subject of “Performance Principles.”  I encourage members of the Apdex community to contribute comments about practical applications of these principles from their own experience.

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>