J2EE Performance Review Software: Software performance analysis of J2EE multi-tiered applications is more complex as the number of hardware and software components that can affect performance increases in many dimensions. This J2EE performance review article describes techniques to use when the number of tunable parameters is too large to examine one variable at a time.

---

by Kingsum Chow, Ph.D., senior performance architect, Managed Runtime Environments group, Intel Corp.

Software performance analysis of multi-tiered applications is more complex as the number of hardware and software components that can affect performance increases in many dimensions. In Part One of the series[1], J2EE Performance Optimization Part 1, we described a top-down closed loop approach to performance tuning. We then described how to apply the approach to a multi-tiered application in a case study in Part Two[2], J2EE Performance Optimization Part 2. While the method described earlier can be applied adequately when you can experiment with one tunable parameter at a time, at times an alternative approach is required, for example, to reduce the time to finish the tuning experiments, or to understand the impact of interaction of factors, which can be done by modifying more than one parameter at a time.

This article describes techniques to use when the number of tunable parameters is too large to examine one variable at a time. We describe how to use the design of experiments (DOE)[3] a well-known statistical approach, to manage for performance analysis. We will also introduce multivariate analysis (MVA)[4] to analyze data that involves multiple variables, and demonstrate how to apply the technique to optimize system configuration for a multi-tiered enterprise application.

A multi-tiered application typically depends on a large number of tunable parameters for improving the performance of the system. Many of the tunable parameters are useful only for programs with certain workload characteristics; a known set of configuration parameters optimized for one workload may not be the best for another. Re-examining all configuration parameters for different workloads is too time-consuming as the number of configuration parameters is large. The selection of proper optimized parameters for a particular workload is controlled by users via database (DB), operating system (OS), managed runtime (MRT) and application server configurations on all the machines involved. Since each configuration can contain many tunable parameters, the total number of parameters is usually large, and each may specify a large range of values. We face the following two problems:

• Selection Problem: select the best set of parameters for a workload; solving this problem provides a powerful tool to obtain the best performance for user programs.
  
• Interaction Problem: determine the interactions between tunable parameters; solving this problem helps both the users and the system developers to understand the effects of the tunable parameters.

The search space for the selection problem is very large. For example, 10 tunable parameters with binary values equals 2[10] =1024 experiments to be studied. Most people rely on educated guesses and often sub-optimal selection is used. Furthermore, the interactions among tunable parameters are often informally described such as "parameters A and B have positive or negative interaction."