CMG 2007: Day 4
Here’s a summary of the sessions I attended:
Application Performance and How Apdex Makes It Better
This session was about application performance management as a component of Business Service management (Infrastructure Service Management being the other component) and how the Apdex spec can help. The most interesting aspect of Apdex is how simple it is. If you can measure some component from the perspective of the end-user and determine at what point the end user is satisfied, tolerating, and frustrated then you can calculate an index of application peformance. The index goes from 0 to 1 so it is fairly easy to compare results from many applications and determine which ones might need some attention.
In order to measure an application using apdex, you need to determine the point at which end users “feel” the impact of a performance problem. If the application is an average web-based application (e.g. a blog), then most users feel a website is responding well if the page returns in under 2 second. Between 2 seconds and 8 seconds, most users will continue to use the site but may flip to other things while waiting. Beyond 8 seconds, most users will give up and go on to something else.
Of course, this will vary by application. One theory put forth is that the time a user will spend on a given page will determine how quickly they get frustrated. If the page is a login page or some information with a “next” button, then they expect a quicker response time than if it’s a blog post that they will spend several minutes reading.
The formula is: Apdex = Satisfied Count + (Tolerating count / 2) / Total Samples. What it boils down to is that you get “full credit” for satisfied responses, “half credit” for tolerating, and no credit for frustrated. If taken as a percentage, the three values always add up to one (see the Apdex Cases session for a neat use for this fact).
Once you have an Apdex score, you need to determine what is a “good” score and a “bad” score. Goals can be set to improve the Apdex score as part of the Application Performance Management process. Typically, a score of .94 to 1.0 is consider Excellent, .85 to .93 is considered Good, .70 to .84 is Fair, .50 to .69 is Poor, and anything less than .50 is unacceptable.
Apdex Process
This session covered more information on Apdex and how to develop a process for incorporating it into an overall APM (Application Performance Management) strategy. Keep in mind that traditional capacity management (Infrastructure) of the servers, network, and storage is still important. APM simply gives you another angle of attack to the problem. Each user “flows” through the infrastructure in a different way and just looking at individual components of the infrastructure is not enough.
Some of the different kinds of vendor tools which can gather Apdex metrics and/or generate Apdex reports. This includes active monitoring, passive monitoring, and end-user agents. Also discussed was potential pitfalls to using these types of monitoring when new technologies like WAN accelerators are used.
The Best at Better
The Plenary speaker was Dale Dauten, author of several nationally syndicated newspaper columns and several books. He was not the kind of speaker you would expect at a technical conference, but I think this is what made the talk so good. He asked everyone to think to think of someone who they consider to be the best boss they ever worked for and think of what makes that person the best. Through his storytelling, he gave us some really good things that make those kind of people who they are.
The Laughing Warriors: How to Enjoy Killing the Status Quo
This breakout session was also hosted by Dale Dauten. It was in a much smaller room and he was better able to have a more intimate discussion about the concepts he covered in the larger Plenary session. After some “warmup” exercises, Dale asked us to break into groups and think of our favorite place to visit and why. Could we run our businesses or departments like that place? For example, our group picked Six Flags because it’s a fun place to go even when you have to stand in line and you always leave feeling like you didn’t do everything there was to do. Indeed, you want to go back.
Setting Performance Objectives Using Apdex
This session was all about how to determine what “T” should be for an application. This is a critical factor for determine the Apdex score. Peter Sevcik gave us 10 different methods to choose from as possibilities. They ranged from going to the users desk and using a stopwatch while watching their body language for signs of tolerating or frustration to calculating a mathematical inflection point (don’t ask, I didn’t understand this one).
Apdex: Case Studies
Three different case studies were presented of companies implementing Apdex reporting. The first case study was from AOL and covered how they initially began using Apdex to monitor various web properties and the refinements and tweaks they made to the process over time. The second was a case study from Keynote discussing using Apdex with a VoIP provider to monitor call quality with metrics such as call clarity and delay. The final case study was the use of the Barycentric Coordinates system to display lots of scores for applications in a visually appealing and compact way. This topic was also covered in the Barry session I attended earlier in the week. Good stuff.
Blogged with Flock
Tags: cmg, cmg2007, apdex, dale dauten, apm