LinkedIn

See our LinkedIn profile, click on this button:

APDEX

Apdex-G Section [5] Reporting

I’m writing a series of posts about Generalizing Apdex. This is #19. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

I have been working systematically through the process of generalizing the current Apdex spec. Along the way, I have been skipping over parts of the current spec to work on the paragraphs that presented challenges. I am now going back to fill in those less contentious paragraphs, posting updated drafts of each section of the Apdex-G spec. The first four are Section [1] Introduction, Section [2] Index Overview, Section [3] Calculation Inputs, and Section [4] Calculating the Index; today I continue with Section [5] Reporting.

Continue reading Apdex-G Section [5] Reporting

Apdex-G Section [4] Calculating the Index

I’m writing a series of posts about Generalizing Apdex. This is #18. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

I have been working systematically through the process of generalizing the current Apdex spec. Along the way, I have been skipping over parts of the current spec to work on the paragraphs that presented challenges. I am now going back to fill in those less contentious paragraphs, posting updated drafts of each section of the Apdex-G spec. The first three are Section [1] Introduction, Section [2] Index Overview, and Section [3] Calculation Inputs; today I continue with Section [4] Calculating the Index.

Continue reading Apdex-G Section [4] Calculating the Index

Apdex-G Section [3] Calculation Inputs

I’m writing a series of posts about Generalizing Apdex. This is #17. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

I have been working systematically through the process of generalizing the current Apdex spec. Along the way, I have been skipping over parts of the current spec to work on the paragraphs that presented challenges. I am now going back to fill in those less contentious paragraphs. Over the next week, I plan to post updated drafts of each section of the Apdex-G spec. The first two are Section [1] Introduction and Section [2] Index Overview; today I continue with Section [3] Calculation Inputs.

Continue reading Apdex-G Section [3] Calculation Inputs

Apdex-G Section [2] Index Overview

I’m writing a series of posts about Generalizing Apdex. This is #16. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

I have been working systematically through the process of generalizing the current Apdex spec. Along the way, I have been skipping over parts of the current spec to work on the paragraphs that presented challenges. I am now going back to fill in those less contentious paragraphs. Over the next week, I plan to post updated drafts of each section of the Apdex-G spec. I began yesterday, with Section [1] Introduction; today I continue with Section [2] Index Overview.

Continue reading Apdex-G Section [2] Index Overview

Apdex-G Section [1] Introduction

I’m writing a series of posts about Generalizing Apdex. This is #15. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

I have been working systematically through the process of generalizing the current Apdex spec. Along the way, I have been skipping over parts of the current spec to work on the paragraphs that presented challenges. I am now ready to go back and fill in those less contentious paragraphs. Over the next week, I plan to post updated drafts of each section of the Apdex-G spec, beginning today with Section [1] Introduction.

Because this material includes some drafts posted previously, some of which I have edited, everything in this series of posts is considered the second draft of Apdex-G. After the second draft is posted, I will begin work on Apdex-R, the addendum addressing measurements of response times. Apdex-R will cover all the domain-specific content of the current spec that has been excised from Apdex-G.
Continue reading Apdex-G Section [1] Introduction

Configurable Scoring in Apdex-G

I’m writing a series of posts about Generalizing Apdex. This is #14. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

The current Apdex specification defines an attractively simple scoring rule in which measurements falling within the zones Satisfied, Tolerating, and Frustrated receive scores of 1, ½, and 0 respectively. Enhancements have been proposed that would involve retaining the core approach to scoring while allowing for more than two targets (or thresholds), thereby creating more than three zones, and using finer scoring gradations (but still between 0 and 1) for rating measurements that fall within those zones.

In my previous posts in this series, I have stated that the notion of classifying all measurements into one of three performance zones is a core feature of Apdex that should be retained, because three-category classification schemes are common to many measurement and reporting domains. I have also made the case for allowing more thresholds, and thereby defining each performance zone as the union of one or more distinct performance intervals. For the conclusion of that discussion, see Generalizing the Apdex Thresholds.

In this post I will consider whether the Apdex formula itself should be generalized. In particular, should Apdex-G accommodate other scoring rules within the Tolerating Zone–and if so, how?

Continue reading Configurable Scoring in Apdex-G

Report Groups and Quality Ratings in Apdex-G

I’m writing a series of posts about Generalizing Apdex. This is #13. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

In my previous post in this series, I reviewed the first part of section §5 Reporting in the Apdex-G spec, introducing Interval Notation and proposing a new comma-separated format for the Uniform Output File. For the full story, see Configurable Reporting in Apdex-G. In this post, I cover the remainder of section §5.

First I will explore the question of what data should be made available to an independent reporting tool that queries an Apdex-based data analysis service. The current spec requires tools to report Apdex scores in Uniform Output format, but does not require any identifiers or context to be supplied with those scores. This strikes me as an omission, especially since section §3.2 Report Groups contains mandatory rules for defining report groups, which are “the foundation for an Apdex calculation”.

Continue reading Report Groups and Quality Ratings in Apdex-G

Configurable Reporting in Apdex-G

I’m writing a series of posts about Generalizing Apdex. This is #12. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

In my previous post in this series, I introduced a conceptual model for Apdex-G. Compared to the current Apdex model, the key addition is the Performance Interval, a range of values within the measurement domain that shares a common Satisfaction Level of ‘Satisfied’, ‘Tolerating’, or ‘Frustrated’. Thresholds now form the boundaries of performance intervals, rather than Performance Zones–which are now defined as a collection (union) of one or more performance intervals. For the full story, see Generalizing the Apdex Thresholds.

This generalization allows Apdex-G to accommodate arbitrary orderings of satisfaction levels within the measurement domain, including discontinuous performance zones. But it also complicates Apdex reporting. The current Apdex model, which employs just two thresholds, T and F, makes it easy to report the threshold(s) alongside the value of an Apdex score. And doing so is a mandatory requirement of Apdex:

All Apdex values are calculated with a particular target threshold, T. The value of T must be clearly displayed in association with the Apdex score.

— Apdex spec, section §5 Reporting the Index, introduction

How can we retain this essential Apdex feature in Apdex-G, which must accommodate reporting on arbitrary alignments of performance intervals?

Continue reading Configurable Reporting in Apdex-G

Generalizing the Apdex Thresholds

I’m writing a series of posts about Generalizing Apdex. This is #11. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

In my previous post in this series, I discussed non-monotonic metrics. These are metrics for which improvement in the metric does not always always correspond to an improvement in quality. To accommodate such metrics, the Apdex-G specification must support six distinct orderings of the three Apdex performance zones (Satisfied, Tolerating, and Frustrated). It should also accept discontinuous zone definitions. For the full story, see Configurable Zone Alignment in Apdex-G,

These requirements introduce two complications, both involving the current spec’s use of just two thresholds, T and F, to define the three Apdex zones. In this post, I will review:

  1. The relationship between thresholds and zones
  2. The number of thresholds required

Continue reading Generalizing the Apdex Thresholds

Configurable Zone Alignment in Apdex-G

I’m writing a series of posts about Generalizing Apdex. This is #10.

In my post on Separately Configurable Thresholds in Apdex-G, I began work on a general description of Apdex zones and thresholds. The language I proposed would allow Apdex to be used to report on metrics for which “quality” increases or decreases monotonically with the value of the metric. Response times and VOIP MOS scores are typical monotonic metrics: smaller response times are always better, higher MOS scores are always better. Many metrics have this property. Arthur Schneiderman writes:

Properties of good metrics

The first requirement for a good metric is that it should be a reliable proxy for stake­holder satisfaction. In other words, improvement in the metric should link directly to improved stakeholder satisfac­tion. This linkage should be clear and uncomplicated. It should also be what mathematicians call monotonic—i.e., improvement in the metric should always produce improved stakeholder satisfaction …

–Arthur M. Schneiderman SCHN96

Continue reading Configurable Zone Alignment in Apdex-G