LinkedIn

See our LinkedIn profile, click on this button:

APDEX

Apdex Vendor Challenge: Show Me the Data

Apdex is dead simple. It is a standard way to convert many response time measurements into a single numerical value that always stays within the range of 0 to 1 where 0 is a disaster and 1 is perfect performance delivery. Dozens of vendors measure response time, so you would think you could readily get an Apdex report. Alas it is not so.

Last December, the Apdex Alliance asked vendors to submit the names of their products that generate Apdex reports to the Apdex Tools Directory. The directory has entries from the following six vendors: Compuware, Gomez, ip-label.newtest (formerly Auditec), WildPackets, and Xa Systems. However, when you search for information about Apdex at their websites, you get responses like, “No Pages were found containing Apdex”, or “Forbidden: Your client does not have permission to get URL.” WildPackets is the only vendor that has any information about their Apdex features. The WildPackets search on Apdex yields 51 documents with descriptions, FAQs, and tips.

So if you are interested in Apdex, you had better be prepared to roll your own reports. The good news is that you can easily create Apdex reports in a spreadsheet. We also find it useful to use a simple database like Access to organize your measurements. An Access query can pull out the measurements associated with the report you are creating, like all California measurements for yesterday. You can program the Apdex formula directly into queries and show them in a variety of reports.

Regardless of your report generation method, you will need input data. This turns out to be the tricky part. Many measurement tools do not make it easy for you to export measurement samples in a useful format. Most vendors want you to stay within the confines of their product. In fact, they help you import data from other sources but once there, they really don’t want you to export.

The same Apdex Tools Directory lists a dozen vendors that supply data which can be used to generate Apex reports. The most valuable part of the directory is the explanation of how to export the data and what kind of format the data will be when exported. This gives you a guide to match the measurement tool and your report generation tool.

Measurement vendors should make all of this much simpler. For example, they can add a generic report design capability where the user can define what data they want to see and how they want it processed (like the Apdex formula), and then how the report should look. If vendors continue to try and force their tool users of to see only reports that they designed then they must provide a wide range of reports including Apdex.

We are seeing an interesting pattern among enterprises that have created the following solution to the problem. They generate production (ongoing) Apdex reports using their own software using input data from various vendors. They use their own data format and conventions as a way to “normalize” the measurements from the various vendors. This then becomes the common data set which a variety of reports can use.

Enterprises that are serious about Apdex describe three benefits to this approach. First, they do not have to sort out differences in how the vendors generated their unique reports. They simply don’t use the vendor reports. Second, they can integrate data feeds from a variety of vendors into a single report. This would be nearly impossible with any of the vendors that have designed their product thinking they are the only measurement and reporting tool at an enterprise. Their reaction is, “You mean you use my tool and another company’s tool?” Finally, they can now add and remove measurement vendors as needed without changing their reporting system. They call this the “vendor isolation” feature.

If you are a measurement or management vendor you should be worried. This approach reduces your value to the enterprise to commodity status. Vendor isolation also means easy vendor replacement.

We think vendors have two choices. Increase value to your customers with better and more flexible reporting capabilities so your customer can build his own Apdex reports–or slide down the slippery slope to a commodity low-price data feed role. Oh, yes, there is a third choice: add Apdex reports directly into your product.

If you are a vendor that generates Apdex reports or can supply measurement data into a customer’s Apdex reporting engine, we want to know about it. Please post a comment to this blog or send an email to peter@apdex.org.

2 comments to Apdex Vendor Challenge: Show Me the Data

  • Can you please explain the license for this particular measurement?

    I would like to offer the ability to our customers to install an Apdex reporting extension to our reporting but we currently do not plan on becoming a contributing member at this stage. Does the license (where?) restrict such an usage if so then we will not proceed on this plug-in.

    Kind regards,

    William

  • admin

    William has an excellent question. The Apdex methodology is free. There is no licensing arrangement. Licenses apply to software. For example, The GNU General Public License (GPL) is a widely used free software license. However, Apdex is not software. Apdex is a formula: A[T] = (Sat + Tol/2) / AllSamples. That is it. Yes, the Apdex Alliance defined some more things around the formula that are documented in the Apdex Specification in order to create a foundation for consistency among implementations. But just like one does not need to license the use of Poisson distribution we cannot create a license for Apdex.

    Companies may join the Apdex Alliance if they choose to support the work of the Alliance. This is a sponsorship role that provides companies with positive marketing exposure and access to the Apdex community. Contributing members also have a strong voice in the evolution of the Apdex specification. Companies that join the Alliance as contributing members need not have an Apdex reporting product.

    In summary there are three ways a company can participate on Apdex.
    1 – The company can implement Apdex reporting in a product or service (free). We encourage companies to follow the specification in order to use the term Apdex.
    2 – The implementation can be certified by the Apdex Alliance as conforming to the Apdex Specification (US$ 2,000 onetime fee). These products are permitted to use the term “Apdex Certified” in the descriptions of their implementation.
    3 – The company can become a Contributing Member of the Apdex Alliance (US$ 5,000 annual fee with product certification included).

Leave a Reply

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

  

  

  


*