I’m writing a series of posts about Generalizing Apdex. The previous post contains my plan of attack. This is #2.
Today, the Apdex specification focuses only on application responsiveness. To create a version of the Apdex standard that will apply in other measurement and reporting domains, we must:
- Generalize any language that refers to application responsiveness
- Identify the core qualities we want to preserve and promote
- Separate core Apdex rules from domain-specific rules
What Are The Core Qualities?
Many of these are already identified in section 2.1 of the current spec, which reads:
2.1 Index Objectives
The fundamental objective of Apdex is to simplify the reporting of application response time measurements by making it possible to represent any such measurement using a common metric. Response time data can describe a wide range of targets, and its magnitude can vary widely. The index is designed to normalize for this variability of time (wide range of seconds) and measurement requirements (many distinct targets), producing a single metric that always has the same meaning. The goals of this metric are:
- To provide a useful summary of an application’s responsiveness
- To make it easy to understand the significance of values produced by the index
- To work for all transactional applications
- To operate within a fixed rage (0-to1) and be unit-neutral
- To indicate application performance directly so that 0 is the worst performance and 1 is the best performance
- To operate in such a way that specific values of the index (e.g., 0.5) report the same user experience across any application, user group, or enterprise
- To operate in such a way that equivalent user experiences observed by different measurement and reporting tools will report the same value
Illustrating the process of generalization, in a previous draft of a more general spec, we proposed replacing the introduction to this section by …
The fundamental objective of Apdex is to simplify the reporting of application quality measurements by making it possible to represent any such measurement using a common metric. Measurement data can describe a wide range of targets, and its magnitude can vary widely. The index is designed to normalize for this variability of measurements producing a single metric that always has the same meaning. The goals of this metric are:
… and we adjusted the first and third bullets to read …
- To provide a useful summary of an application’s quality
- To work for all applications
Note that the proposed language above comes from a first draft, in which we were exploring the generalization process. I’m well aware that “quality” is a notoriously difficult concept to pin down, as anyone who has read Robert Pirsig’s classic, Zen and the Art of Motorcycle Maintenance, knows. Also, it is an open question whether the terminology “application quality” is sufficiently general to apply to all the possible domains where the Apdex method might be applied. I’m hoping that we’ll be able to answer this by looking at enough examples of metrics in other domains where the Apdex method could be useful. So consider “application quality” to be a placeholder for now (and feel free to propose alternatives in the comments, of course).
The list in the current spec is a good one, and any new spec will probably inherit most of it in some shape or form. But I think it can still be be improved. I believe that a core benefit of Apdex is its usefulness as a management indicator (such as a Key Performance Indicator, or KPI). I don’t think the current spec really captures that aspect properly, so I’m going to devote my next post to that subject. If you have other suggestions for improvements, please contribute them in the comments.
Finally, we have not discussed what we will do with the domain-specific language and rules. They do not belong in the core Apdex specification, but they may still be useful, even necessary, when specifying how Apdex is to be applied within a particular domain. That will be a topic for a later post.