Performance Monitoring and Testing are the collective activity of constantly watching peformance metrics to detect anomalies, and testing the system by simulating users and user requests to match a production environment load to ensure that production needs can be met.

It is required that any kind of Performance Testing should be accompanied by adequate data regression to prevent false performance results that could result from cached transactions. From the Oracle Performance Analyzer for Oracle 10G to nMap to analyze network throughput metrics, there are a wide gamut of tools available to both measure and predict the throughput of a system to near-production numbers.

From an Database perspective, it is critical to monitor Transactions per day, transactions per hour and transactions per minute on a daily basis, along with the following metrics by day.

• Processor usage
• Database Locks
• Memory Usage
• Disk Size and Growth
• Database Size and Growth
• Database space usage by store
• Disk Activity, read-write queue length and saturation as pertinent to number of transactions

For windowed applications on Windows, the Windows Management Interface (WMI) provides several performance counters that are indicative of Application Server performance. Key counters include:

• Disk Throughput – Requests Queued, Reads, Writes, Bytes Per Sec and Errors
• Memory – Available Memory, Memory in Use – committed bytes and Page Faults
• Processor – Usage, peak and averages
• Other ASP.NET, SQL Server and application specific counters

On UNIX / LINUX / AIX, topaz and other monitoring utilities will provide the equivalent counters for performance. Simple Network Management Protocol applications can listen for specific SNMP events and threshold deviances.

It is required that a list of performance counters be documented prior to the test and results of the test be published with graphical representations of the performance results in order to be able to analyze and effectively assimilate the results. It is recommended that the Test criteria be determined by interpolating common results and business metrics to match the target criteria. For example, if we knew that each server processed X orders per day with an average of roughly Y orders per hour, 8:00 AM - 5:00 PM being the high-load period, we should be able to measure if the system that currently handles 20 orders is capable of handling 20 orders.

The performance test plan should clearly outline the parallelism of the test, the number of concurrent connections, the number of concurrent transactions and the reasoning behind the load. It is a good practice to increase the threshold after a successful test of the required criteria to determine the breaking point of the system. The disk being the slowest component in any database server is almost always the point of contention when available system memory has been exhausted. This is a direct result of paging. Therefore adequateness of system memory should be targeted and tested.
For Application Servers and Database Servers alike, the Processor is the next point of contention, after memory. SMP Implementations are recommended
Metrics like networking bandwidth, file system performance and application error logs are the last, but not least criteria to consider.