I’m writing a series of posts about Generalizing Apdex. This is #23. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec documents, Apdex-G and Apdex-R, are enclosed in square brackets and prefixed by the document type, like this: [G 1] and [R 1].
Drafts of the Apdex-G spec were posted as Section  Introduction, Section  Index Overview, Section  Calculation Inputs, Section  Calculating the Index, Section  Reporting, and Section  References.
Apdex-R specifies the Apdex rules for reporting response-time measurements, therefore my first draft draws heavily on the present Apdex spec. My goal is to reformat the present spec’s rules within the context established by Apdex-G. To allow both documents to be viewed together, I show Apdex-G (draft #2, plus a few subsequent amendments) on the left, and Apdex-R (draft #1) on the right.
Differences between Apdex-R and the corresponding sections of the present spec are highlighted, as follows:
text deleted from the present specThis text is covered in Apdex-G, or elsewhere in Apdex-R, or extraneous.
- new text added to Apdex-R (plus a few additions to Apdex-G draft #2)
- new text created for Apdex-G, now being repeated in Apdex-R for readability
- notes, questions, issues to be resolved
While major deletions, additions, and changes to the present spec are marked, many minor wording changes to the present spec’s text, to improve continuity and clarity of meaning, are not marked. I believe those changes do not substantially alter the meaning and intent of the present spec.
Apdex-R, first draft:
Apdex-G, second draft:
See [G 4] for a general introduction to the Apdex index calculation.
The Apdex does not entail new measurements – rather it is a new way to represent existing measurements, calculated by counting the measurement samples in each of the performance zones.
The Apdex does not entail new measurements, rather it is a new way to represent an existing set of measurements, reflecting the degree to which those measurements achieve designated targets.
See [G 4.1] for the Apdex formula, and corresponding input requirements:
The following sections address Apdex-R input requirements:
Measurement data: [R 3.1]
Report group: [R 3.2] and [R 3.3]
Performance zones: [R 2.5], [R 2.5.1], [R 2.5.2], and [R 3.4]
F = 4T
Satisfied_Count = number of satisfied response time samples,
Tolerating_Count = number of tolerating response time samples,
Total_Samples = number of all samples in the report group
The Apdex index is calculated as follows. Given:
Measurement data that meets the requirements of section [G 3.1]
A report group comprising measurement samples, defined as specified in sections [G 3.2] and [G 3.3]
Three performance zones (Satisfied, Tolerating, Frustrated), defined as specified in sections [G 2.5], [G 2.5.1], [G 2.5.2], and [G 3.4]
An allocation process that assigns each sample to a performance zone, and counts all samples, so that:
Total_Samples is the number of all samples in the report group
Satisfied_Count is the number of report group samples in the Satisfied Zone
Tolerating_Count is the number of report group samples in the Tolerating Zone
Then the Apdex index for the report group is:
See [G 4.1.1] for a general overview of the Apdex formula in action.
Note that measurements in the frustrated zone are counted in the number of total user samples in the denominator. To achieve the optimal Apdex value of 1.00, all users must experience satisfactory performance. If some users see tolerating or frustrating performance, then the index rapidly dips below 1.00. For example, if 80% of users are satisfied and 10% are tolerating, while the remaining 10% are frustrated, the index is 0.85.
Note that measurements in the Frustrated zone are counted in Total_Samples, the denominator of the formula, but do not contribute to the numerator.
Another way to describe the Apdex index is as the weighted proportion of satisfactory samples in the report group. Samples in the Satisfied Zone have weights of 1, those in the Tolerating Zone have weights of ½, and those in the Frustrated Zone have weights of 0.
See [G 4.1.2] for the rules for optional alternative scoring. Apdex-R permits tool creators to substitute an alternative scoring function in the tolerating zone, provided that sample values (or value ranges) closest to the satisfied zone receive the greatest weight.
If the characteristics of a particular measurement and reporting domain justify using graduated weights within the Tolerating zone, an Addendum may substitute an alternative scoring function for the factor Tolerating_Count/2 in the standard formula. Any alternative scoring function (such as a sliding scale) must have the property that sample values (or value ranges) closest to the satisfied zone receive the greatest weight, in accordance with the index objectives (see [G 2.1] item 6).
See [G 4.2] for general rules for dealing with exceptions. The following additional notes may apply to measurements of Task (and Task Chain) times.
User aborts are factored into the above equation. A user abort occurs when a user enters a new inquiry before the system responds with the original inquiry. A user-generated abort stops the timing of the Task. Therefore, user aborts can fall into any of the satisfied, tolerating, frustrated zones.
Server failure: If a tool can detect a clear server-generated abort, then it is handled differently. Server aborts (e.g., TCP closes within a Task) are counted as a frustrated sample regardless of the Task time measurement.
Application failure: Some tools may have the optional capability to interpret the application to a greater level of detail than the minimal Task boundary. For example, they may be able to detect user relevant information at the layer of the application logic. If the tool can detect Task errors, then these application errors (e.g. Web page 404 replies) are counted as frustrated samples.
When a Report Group contains samples marked as errors or exceptions, tools performing the Apdex calculation should classify those measurements as follows:
User Error: Exceptions caused by user actions may be classified as Satisfied, Tolerating, or Frustrated in the same way as any other measurement, if the necessary field(s) are present in the sample. If the field(s) required to perform classification are absent, user-generated exceptions should be classified as Satisfied.
Warning: Exceptions indicating abnormal system or process behavior may be classified as Tolerating if the system or process returned immediately to normal operation without requiring abnormal intervention.
Failure: Exceptions indicating a system or process failure that is experienced by the user should be classified as Frustrated.
Measurement Error: Exceptions indicating measurement errors should be discarded from the Report Group.
Unknown: Exceptions not amenable to classification should be discarded from the Report Group. However, tool creators must attempt to minimize the number of error types assigned to this category.
Because all exceptions are specific to a particular measurement domain, an addendum may specify domain-specific refinements to these general guidelines.