LinkedIn

See our LinkedIn profile, click on this button:

APDEX

Apdex Specification Templates

I’m writing a series of posts about Generalizing Apdex. This is #9.

Previously I analyzed the current Apdex specification, asking the question Which Apdex Features Can Be Generalized? and answering with a list of topics to be worked on. I began by establishing working assumptions about document structure and terminology:

Separate Core Apdex Rules from Domain-Specific Rules
We should separate the core Apdex method into its own specification, and create distinct spec documents that contain any rules associated with domain-specific applications of the Apdex method. I will refer to the domain-specific components of the spec as addenda.

Apdex Document Names
I will refer to the new core specification as Apdex-G. This document will contain definitions and notation for all the core Apdex features we decide to generalize, such as targets, zones, and scoring. If I need to refer to a domain-specific addendum, I will use a suffix appropriate to their particular domain, such as Apdex-R for response time, Apdex-V for VOIP quality, Apdex-S for service quality, Apdex-N for network performance, etc.

As I have begun work on the details (see Generalizing the Apdex Language and Separately Configurable Thresholds in Apdex-G), a mental framework for the structure and content of each document has begun to emerge. This post aims to solidify that framework by laying out draft templates for Apdex-G and the addenda.

Continue reading Apdex Specification Templates

How to Choose an APM Tool: Webinar Archive

The June 30, 2010 Apdex webinar material is available. Download the presentation here.

It is difficult to choose an APM tool that really helps your enterprise manage application performance. With more than 30 application performance management (APM) tool vendors with scores of product offerings, buyers face hundreds of confusing choices. This confusion is compounded by the lack of common APM nomenclature, making cross-vendor comparisons challenging.

In this live webinar Peter Sevcik, Executive Director of the Apdex Alliance, described an APM tools framework to help you define your APM requirements and map them to vendor offerings. The framework is a comprehensive description of application management functions and provides common nomenclature that makes it easy to compare tools. The framework also places Apdex elements in their appropriate contexts.

The APM framework enables you to describe and prioritize the functions appropriate for your environment and helps guide your APM strategy. This video gives you the knowledge to avoid decisions based on vendor hype rather than a good match with the needs of your enterprise. Watch this video before investing in your next performance management tool.

How to Choose an APM Tool from Apdex Alliance on Vimeo.

We thank the 85 people who registered to attend the live event. Their questions prompted interesting discussion that added a lot of value. Please stay tuned for future webinars.

Separately Configurable Thresholds in Apdex-G

I’m writing a series of posts about Generalizing Apdex. This is #8. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

To compute an Apdex score for a set of response-time measurements, you count how many measurements fall into each of three performance zones. Apdex calls these zones Satisfied, Tolerating, and Dissatisfied but, as Neil Gunther observed, the labels … are unimportant. You could just as easily call them Good, Bad and Ugly GUNT09.

Indeed, while investigating other potential applications of Apdex in other measurement domains, I found many other examples of measured outcomes being classified using three categories. Of course, many other labels are being used for those categories. For example:

Continue reading Separately Configurable Thresholds in Apdex-G

Generalizing the Apdex Language

I’m writing a series of posts about Generalizing Apdex. This is #7. To minimize confusion, section numbers in the current spec are accompanied by the section symbol, like this: §1. The corresponding section numbers in the generalized spec, Apdex-G, are enclosed in square brackets, like this: [1].

In my previous post, I reviewed the Apdex specification systematically, asking Which Apdex Features Can Be Generalized? That exercise showed that creating a general version of the spec involves more than generalizing the rules for implementing the features of Apdex; the language describing those features must be generalized too.

The current spec is grounded in a conceptual model of a specific measurement domain. Its language refers to users of an application being productive based on the responsiveness of a human-computer interface. We encounter this as soon as we begin reading Section §1, shown in the left-hand column below. In the new generalized specification documents, that terminology belongs in the domain-specific addendum, Apdex-R, which will deal with how to apply the Apdex method to response time measurements.

In Apdex-G we need abstract language describing only the properties of a metric. If more concrete language is needed to clarify an explanation, if must appear in an illustrative example. The only exception is text in Apdex-G describing the origins of Apdex.

Continue reading Generalizing the Apdex Language

Which Apdex Features Can Be Generalized?

I’m writing a series of posts about Generalizing Apdex. This is #6.

The current Apdex specification is entirely focused on application response time, specifically on the response time of Tasks and Task Chains. However, people have already adopted the Apdex method as a convenient way to report on other metrics, including network “turns”, bandwidth, and VOIP quality scores. These adaptations demonstrate the strength and adaptability of the core Apdex concepts. But because such ad hoc extensions fall outside the standard, they are likely to be implemented in inconsistent ways. By generalizing the Apdex spec, we aim to rectify that situation, bringing a wide class of metrics in a wide range of measurement domains within the scope of the standard.

Which Apdex features are candidates to be generalized? To answer that question, I will review the current Apdex specification. For clarity, I will use the section symbol (§) when referring to specific sections or subsections of the spec. During this pass, my goal is only to identify the major aspects of the current standard to be reviewed and reworked, not to address the actual structure or language of the generalized spec. It will be much easier to produce precise language once the concepts are clear.

Continue reading Which Apdex Features Can Be Generalized?

An Extensible Apdex Glossary

I’m writing a series of posts about Generalizing Apdex. This is #5.

“When I use a word,” Humpty Dumpty said, in rather a scornful tone, “it means just what I choose it to mean — neither more nor less.”
“The question is,” said Alice, “whether you can make words mean so many different things.”
“The question is,” said Humpty Dumpty, “which is to be master — that’s all.”
— Through the Looking Glass, Chapter VI, Lewis Carroll

Specifications need to be precise. Therefore, to create a more general version of the Apdex spec–and before that, to discuss what the spec should contain–we need to define our terminology precisely. This is especially vital in the “Looking Glass” world of information technology, where people like to make words mean so many different things. This post is my attempt to master the meanings of the words I need to use, to make them mean just what I choose them to mean — neither more nor less.

The solution, of course, is to create a glossary. The Apdex spec already contains one, in Section 7, but since we’re working on generalizing the spec, we need to generalize the glossary too. At this stage in the process, I can’t predict all the additional terms we’re going to need, but I can already see some obvious omissions in the current glossary. So I’m going to post an extensible glossary here, and update it whenever necessary. That way, by the time we complete the process of rewriting the spec, we’ll have the new glossary for that spec already written.

Continue reading An Extensible Apdex Glossary

Apdex as a (Key) Performance Indicator

I’m writing a series of posts about Generalizing Apdex. This is #4.

I ended my earlier post on Core Apdex Qualities with this observation: “I believe that a core benefit of Apdex is its usefulness as a management indicator (such as a Key Performance Indicator, or KPI). I don’t think the current spec really captures that aspect properly, so I’m going to devote my next post to that subject”.

Since then I have done extensive research on the subject of performance indicators, resulting in a collection of relevant papers, articles, and blog posts–see my later post on Metrics and Performance Indicators: A Bibliography. I have also been extending my original list of 20 references, appending several newer discoveries of relevant online resources as comments. But much as I enjoy doing research, at some point I have to stop and draw some conclusions, however tentative. So here’s a bit more detail about the role of Apdex as a (key) performance indicator.

Continue reading Apdex as a (Key) Performance Indicator

Webinar: Choosing Application Performance Management (APM) Tools

Date: Wed June 30, 2010
Time: 12:00 noon EDT (1 hour)
Speaker: Peter Sevcik, NetForecast

Now available: Recording and Slides

With over 30 application performance management (APM) tool vendors offering scores of products, buyers face hundreds of confusing choices. Compounding the problem, the lack of a common taxonomy, or standard APM nomenclature, makes cross-vendor product comparisons especially challenging.

To address this challenge, NetForecast has developed an APM tools framework anyone can use to define APM requirements and map them to vendor offerings. The APM framework is a comprehensive description of application management functions and features. Building on essential infrastructure management concepts, it provides a common nomenclature that makes it easy to compare APM tools. The framework also places Apdex elements in their appropriate contexts.

Good performance is delivered with tools that directly support APM processes and best practices. The APM framework lets you describe and prioritize the functions and features appropriate for your environment, and helps guide your APM strategy.

Continue reading Webinar: Choosing Application Performance Management (APM) Tools

Metrics and Performance Indicators: A Bibliography

I’m writing a series of posts about Generalizing Apdex. This is #3.

In my previous post on Core Apdex Qualities, I set out to enumerate the essential characteristics of the Apdex metric. What aspects of the Apdex method, and the metric it produces, would make it useful for reporting measurements in domains other than the application response time, the focus of the Apdex specification today?

The Apdex goals already identified in section 2.1 of the spec are certainly a good starting point, but I felt that to properly assess where a generalized Apdex metric might potentially be applied, I needed to establish a general picture of the use of metrics and performance indicators in management. This is easier said than done. How, for example, can we discuss the applicability of an Apdex metric as a Key Performance Indicator (KPI), when no standard definition of that concept exists?

If you spend a couple of weeks sorting through Google’s ~4 million references to key performance indicators, two things will happen. First, you will see that confusion abounds; there is no generally accepted framework we can use to position Apdex. Second, you will begin to assemble your own short list of useful reading that will eventually reveal some broad conclusions about the field of management metrics. This post records my own experience doing this, with potential applications of a generalized Apdex standard in mind.

Continue reading Metrics and Performance Indicators: A Bibliography

Service Level Management with Apdex: Webinar Archive

The May 12, 2010 Apdex webinar material is available. Service Level Management (SLM) is the art and science of keeping application services running properly once in production. In this live webinar Peter Sevcik, Executive Director of the Apdex Alliance, described how Apdex works and provides a key performance indicator (KPI) that supports each layer of a successful SLM approach.

Download the presentation handout here.

We thank the 110 people who registered to attend the live event. Their questions prompted interesting discussion that added a lot of value. Please stay tuned for future webinars.