LinkedIn

See our LinkedIn profile, click on this button:

APDEX

Apdex-G Section [5] Reporting

I’m writing a series of posts about Generalizing Apdex. This is #19. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

I have been working systematically through the process of generalizing the current Apdex spec. Along the way, I have been skipping over parts of the current spec to work on the paragraphs that presented challenges. I am now going back to fill in those less contentious paragraphs, posting updated drafts of each section of the Apdex-G spec. The first four are Section [1] Introduction, Section [2] Index Overview, Section [3] Calculation Inputs, and Section [4] Calculating the Index; today I continue with Section [5] Reporting.


Summary

In this section, I have generalized and reorganized the material in Apdex section §5 as follows:

  • The introduction is a rewording of the current spec.
  • Section [5.1] corresponds to Apdex section §5.1. I have numbered the four rules.
  • Section [5.1.1] corresponds to Apdex section §5.3, describing general cases. Placing it here improves the flow.
  • Section [5.2] corresponds to the last part of Apdex section §5.1 on Uniform Output, revised and extended. These extensions were discussed in two previous posts — Configurable Reporting in Apdex-G and Report Groups and Quality Ratings in Apdex-G.
  • Section [5.3] corresponds to Apdex section §5.2. I have generalized the language, and tried to enumerate the possible classes of errors in a general way. This section was discussed briefly with the first draft.
  • Section [5.4] corresponds to Apdex section §5.4. I have generalized the language.

Because this material includes some drafts posted previously, some of which I have edited, everything in this series of posts is considered the second draft of Apdex-G. After the second draft is posted, I will begin work on Apdex-R, the addendum addressing measurements of response times. Apdex-R will cover all the domain-specific content of the current spec that has been excised from Apdex-G.

Apdex, current spec:

Apdex-G, second draft:

§5. Reporting the Index

The index is a decimal value between 0 and 1. The Apdex representation on all reports (screen, print, or otherwise) must adhere to the following rules.

[5] Reporting the Index

In this section, the terms Report(s) and Reporting refer to any representation of an Apdex index, whether in print, on a computer screen, or elsewhere.

§5.1 Displaying the Apdex Value

The index always starts with a 0, followed by a decimal point, followed by the fractional value for the calculation to two decimals, and the value of T is clearly associated with the index presentation as defined below. There is a special case of the value 1.00 that starts with the digit of 1. A 1.00 is presented when the formula produces a result that can be rounded arithmetically to 1.00 (results equal to or greater than 0.995 round to 1.00).

Apdex may not be reported in granularity smaller than one one-hundredth. Decimal values of 3 or more digits are not permitted.

Tools in a computer screen or in printed reports always identify it as an Apdex value. When there are many values of the index presented in tabular form representing many reporting groups, then the Apdex label should appear with the appropriate column or row.

All Apdex values are calculated with a particular target threshold, T. The value of T must be clearly displayed in association with the Apdex score. Furthermore, in order to facilitate exporting Apdex values from a tool and then importing them to other analysis tools, all tools must support at least one uniform output as shown below.

[5.1] Reporting the Apdex Value

Reports must adhere to the following rules:

  1. The index is a decimal value between 0 and 1, rounded to a precision of two decimal places. Values equal to or greater than 0.995 round to 1.00.
  2. Unless its value is 1.00, the index is always reported with a zero in the tens place, followed by a decimal point, followed by no more than two decimal places.
  3. Apdex values must be identified as such. When several Apdex values are presented in tabular form, a single Apdex identifier can be identify a group of values in a row, column, or table.
  4. All Apdex values are calculated based on specified performance zones. The performance zone specification must be clearly displayed in association with the Apdex score.

Current spec:

§5.3 When T is a General Case

General Apdex value discussions that reflect any target of T are written with “T” as shown in the following examples.

Uniform Output: “Everyone should understand that 0.90 [T] is a better value than 0.80 [T].”
Subscripted Output: “Everyone should understand that 0.90T is a better value than 0.80T.”

[5.1.1] Describing General Cases

In general discussions of Apdex values, references to unspecified performance thresholds are written using the notation “[T]”, as shown in the following examples.

“Everyone should understand that 0.90 [T] is a better value than 0.80 [T].”
“Apdex scores in the range 0.85 to 0.93 [T] are rated Good.”

Sections [5.3] and [5.4] also use this notation when referring to threshold reporting.

Uniform Output (Mandatory)

The tool shall display, print, and export to an ASCII file each Apdex value with the following fixed format.

Table 2. Uniform Output Definition

Element Position Definition Example/Range
1 Value whole number digit 0 or 1
2 Decimal point .
3 Value tenths digit 0 through 9
4 Value hundredths digit 0 through 9
5 Space
6 Left bracket designating start of T [
7 First digit of T 0.1 or greater
N Rest of T as defined in section 3.3 Numeric digits
N+1 Right bracket designating end of T ]
N+2 Small group indicator (see 5.2 below) * if present
Table 2. Uniform Output Definition

Examples of the uniform output are: 0.85 [5.5], 1.00 [8.0]*, 0.90 [4.0], and 0.77 [450].

Subscripted Output (Optional)

Tools may display Apdex values with a T shown as a mathematical subscript. For example, an Apdex value of 0.75 that is based upon a T of 4 seconds is shown as 0.754.

When T has a decimal component, as defined in section 3.3, then the exact value of T must be shown (example: 0.754.5). In order to make the index easy to read, tools may drop the decimal portion of T if it is zero (example: 0.754.0 becomes 0.754).

 

[5.2] Uniform Output File (Mandatory)

Separate tools may be used to calculate Apdex index values from measurement data and to report Apdex index values. To support the exchange of index values between Apdex analysis and reporting tools, all Apdex analysis tools must support the Apdex Uniform Output format. A tool shall display, print, and export Apdex-G values to an ASCII file having a comma-separated values (CSV) format. For this purpose, Apdex-G incorporates the IETF specification of CSV published in RFC 4180 (see section [6] References).

A Uniform Output File is composed of a uniform output header record, followed by one or more uniform output data records. The header record contains the same number of comma-separated fields as all data records in the file. Fields in the header record contain fixed literal values, or names that correspond to each field in the data records. The content of a uniform output header and data records are described in Tables 2 and 3 below. In Table 2, each element except the last is followed by a comma.

Element Number Definition Header Record Data Record
Type Content Type Content
1 Apdex data record identifier Literal Apdex Header Literal Apdex
Report Group Metadata Section Header Record Data Record
2 Apdex Report Group section identifier Literal ARG Literal ARG
3 Report Group Name Literal Report Group Name Defined by tool or user
4 Report Group Description Literal Description Text String Defined by tool or user. See note 1 below
5 Measurement Type Literal Type Name G, or addendum type (e.g. R for Apdex-R)
6 Measurement Subtype Literal Subtype Name Values specified in an addendum for this measurement type
7 Application Literal Application Name Defined by tool or user
8 User Group Literal User Group Name Defined by tool or user
9 Time Period Start Literal Start Time Timestamp ISO 8601: [YYYY][MM][DD]T[hh][mm][ss]Z
10 Time Period End Literal End Time Timestamp ISO 8601: [YYYY][MM][DD]T[hh][mm][ss]Z
Input Data Summary Section Header Record Data Record
11 Apdex Data Summary section identifier Literal ADS Literal ADS
12 Sample Count Literal Total Samples Number Integer
13 Satisfied Zone Count Literal Satisfied Count Number Integer
14 Tolerating Zone Count Literal Tolerating Count Number Integer
15 Frustrated Zone Count Literal Frustrated Count Number Integer
16 Earliest Sample Timestamp Literal First Sample Timestamp ISO 8601: [YYYY][MM][DD]T[hh][mm][ss]Z
17 Latest Sample Timestamp Literal Last Sample Timestamp ISO 8601: [YYYY][MM][DD]T[hh][mm][ss]Z
Apdex Index Section Header Record Data Record
18 Apdex IndeX section identifier Literal AIX Literal AIX
19 Apdex index value Literal Apdex Index Number Decimal in range [0.00, 1.00]
20 Satisfied Zone Identifier Literal S Literal S
21 Satisfied Thresholds(s) Group Performance Interval Names Group Interval Group, see Table 3
22 Tolerating Zone Identifier Literal T Literal T
23 Tolerating Thresholds(s) Group Performance Interval Names Group Interval Group, see Table 3
24 Frustrated Zone Identifier Literal F Literal F
25 Frustrated Thresholds(s) Group Performance Interval Names Group Interval Group, see Table 3
26 Small Group Indicator Literal SGI Literal * or NS
Table 2. Layout of a Uniform Output Data Record

Note 1: To avoid ambiguity, tool creators must ensure that Report Group description fields do not contain commas, because commas separate fields in the Uniform Output format.

Performance Interval Names

If a tool has obtained user-specified names for performance intervals, those names should be included in the corresponding fields of the header record (elements 21, 23, and 25 in Table 2). If user-specified names are not available, distinct default names must be supplied for each performance interval field in the header record. See also section [2.5.1] Thresholds and Performance Intervals.

Performance Interval Groups

In Table 2, each Interval Group (elements 21, 23, and 25) is composed of one or more Performance Interval elements, separated by commas. Table 3 shows the structure of a single Performance Interval element, therefore the components shown in Table 3 are not separated by commas.

Component Definition Header Record Data Record
Type Content Type Content
1 Opening backet Name Performance Interval Name Literal ( or [
2 Lower threshold Number Decimal number — see [3.4]
3 Colon Literal : See note 2 below
4 Upper threshold Number Decimal number — see [3.4]
5 Closing backet Literal ) or ]
Table 3. Components of a Performance Interval within a Uniform Output Data Record

Note 2: Interval Notation, described in References [6.8], uses a comma to separate the upper and lower bounds of an interval. To avoid ambiguity Apdex substitutes a colon, because commas separate fields in the Uniform Output format.

Optional Sections

The Uniform Output File format comprises three sections, each having its own identifier (fields 2, 11, and 18). The Report Group Metadata Section and the Apdex Index Section are required, because they contain the necessary data for reporting Apdex scores. The Input Data Summary Section is optional, but tool creators are encouraged to produce this data at the analysis stage, and to pass it to Reporting tools. Reporting tools can use this data to provide additional context for Apdex scores. If the Input Data Summary Section is omitted, fields 11 through 17 must be omitted from the header and data records.

Examples of the uniform output

Examples of the uniform output are:

Apdex Header,ARG,,,,,,,,,AIX,Apdex Index,S,Green,T,Yellow,F,Red,SGI CRLF user labels
Apdex,ARG,,,,,,,,,AIX,0.72,S,[0:4],T,(4:16],F,(16:INF) CRLF
Apdex,ARG,,,,,,,,,AIX,0.85,S,[0:6],T,(6:20],F,(20:INF),* CRLF
Apdex,ARG,,,,,,,,,AIX,0.58,S,[0:3],T,(3:10],F,(10:INF) CRLF
Apdex,ARG,,,,,,,,,AIX,0.91,S,[0:10],T,(10:30],F,(30:INF) CRLF

Apdex Header,ARG,,,,,,,,,AIX,Apdex Index,S,PI3,T,PI2,PI4,F,PI1,PI5,SGI CRLF generic labels
Apdex,ARG,,,,,,,,,AIX,0.88,S,(10:12],T,(6:10],(12:16],F,[0:6],(16:INF) CRLF
Apdex,ARG,,,,,,,,,AIX,0.96,S,(9:15],T,(6:9],(15:18],F,[0:6],(18:INF) CRLF

[ More details and examples will be added here ]

§5.2 Indicating Sample Size

Apdex values are calculated based upon a set of measurements (samples) in the report group. If there are a small number of samples, the tool must still present a result. However, a result for such a small report group must be clearly marked.

A small report group is defined as any number of samples between 0 and 99. Apdex tools will clearly indicate that the result is based upon one of the following scenarios:

No Samples
The Apdex calculation could not be performed because there were no samples (NS) within the report group. Where the calculated Apdex value would normally appear, the tool will show an output of NS. Examples: NS [4.0], NS4
Small Group
When an Apdex value is the output of a small group (1 to 99) calculation, an asterisk (*) must be appended to that value. Examples: 0.80 [4.0]*, 0.804*.

[5.3] Indicating Sample Size

Apdex values are calculated based upon a set of measurements (samples) in the report group. If there are a small number of samples, the tool must still present a result. However, a result for such a small report group must be clearly marked.

A small report group is defined as one having fewer than 100 samples. An addendum may modify this definition to be appropriate for a particular measurement domain. Apdex tools will clearly indicate that the result is based upon one of the following scenarios:

No Samples
The Apdex calculation could not be performed because there were no samples (NS) within the report group. Where the calculated Apdex value would normally appear, the tool will show an output of NS [T], where [T] is the normal threshold display (see section [5.1.1]).
Small Group
When an Apdex value is the output of a small group calculation, an asterisk (*) must be appended to that value, for example: 0.84* [T], where [T] is the normal threshold display (see section [5.1.1]).

§5.4 Additional Reporting Rules

Some tool vendors may wish to add graphical aids to report the Apdex value. This is an optional feature, but, if implemented, it must follow these guidelines.

Two forms of alternative representations are permitted: the rating (a word), and a color indication. The following table shows the fixed set of alternative modes of representing the Apdex. The table shows examples where the target threshold (T) is 4 seconds.

The color indication can be determined by the vendor in line with their existing product set, however a legend must clearly indicate which color represents each Apdex rating.

Table 3 – Apdex Qualitative Reporting Rules
(Examples where T=4)

Apdex Value Range Rating Color Indication
0.944 to 1.004 Excellent4 Determined by vendor (with a 4 plus a color indication)
0.854 to 0.934 Good4 Determined by vendor (with a 4 plus a color indication)
0.704 to 0.844 Fair4 Determined by vendor (with a 4 plus a color indication)
0.504 to 0.694 Poor4 Determined by vendor (with a 4 plus a color indication)
0.004 to 0.494 Unacceptable4 or UNAX4 Determined by vendor (with a 4 plus a color indication)
Low Sample Cases
0.NS4 NoSample4 Determined by vendor (with a 4 plus an NS inside color indication)
0.854* Good4* Determined by vendor (with a 4 plus an * inside color indication)
Table 3. Apdex Qualitative Reporting Rules

Note. In the current specification, the ‘Apdex Value Range’ column of Table 3 contains a typo in the ‘NoSample’ row, which (as shown above) reads ‘0.NS4‘. That cell should read ‘NS4‘.

[5.4] Apdex Quality Ratings

Some tool creators may wish to assign quality ratings to Apdex value ranges, and to present those ratings graphically. This is an optional feature, but, if implemented, it must follow these guidelines.

Two alternative representations are permitted for quality ratings: a rating word or a color indication. Table 6 below lists the value ranges to be used when assigning a rating to an Apdex value. The table shows examples for the target threshold [T], where [T] is the normal threshold display as described in section [5.1.1].

Colors may be selected by the vendor for consistency with other products, or based on user-supplied preferences. However a legend must clearly indicate which color represents each Apdex rating.

Apdex Value Range Rating Word Color Indication
0.94 to 1.00 [T] Excellent [T] Determined by vendor (with [T] plus a color indication)
0.85 to 0.93 [T] Good [T] Determined by vendor (with [T] plus a color indication)
0.70 to 0.84 [T] Fair [T] Determined by vendor (with [T] plus a color indication)
0.50 to 0.69 [T] Poor [T] Determined by vendor (with [T] plus a color indication)
0.00 to 0.49 [T] Unacceptable [T] or UNAX [T] Determined by vendor (with [T] plus a color indication)
Low Sample Cases
NS [T] NoSample [T] Determined by vendor (with [T] plus an NS inside color indication)
0.85* [T] Good* [T] Determined by vendor (with [T] plus an * inside color indication)
Table 6. Rules for Apdex Quality Ratings

Note: An addendum may specify alternative colors [TBD: and/or value ranges?] to be applied when reporting Apdex scores for a particular measurement domain.

Public Comment

As usual, all these proposals are open for public discussion. Please use the comment form below to contribute any comments, suggestions, or questions.

2 comments to Apdex-G Section [5] Reporting

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  

  

  


*