Model Creation and Validation
The model creation and validation process creates an accurate model of your system based on
the data collected on your host system for throughput, CPU utilization, and other workload values.
The key to accurate forecasting is to ensure that your calculated values accurately reflect the
actual values on the host system.
The procedures in this chapter assume:
Forecast Capacity Planner is installed and running on your PC. The product logo is displayed in the Main Program window (this display will be referred to as the Logo screen.)
The data collection process on the host system is complete and the collection file (*.col) is transferred to your PC.
Setting Thresholds
Before loading and validating a new model, review the default forecast and validation thresholds
to determine if they are suitable for your model. In some instances, you may want to set the
threshold CPU utilization limits higher or lower than the default.
Forecast Thresholds
The default CPU utilization values in the Thresholds dialog box are based on typical utilization
limits for MPE/iX and MPE/V CPUs. Generally, system performance is compromised at higher
utilization levels.
To change the CPU utilization threshold:
On the Options menu, click Thresholds.
In the Thresholds dialog box, click the Forecast tab.
Figure 9.1 Thresholds dialog box: Forecast tab
Click in the appropriate text box for your CPU.
Enter the desired CPU value.
Click the Apply button to apply the change to the current session. This change will not be saved until you validate the model.
Validation Thresholds
The validation thresholds limit the variability allowed between your model and the actual data. If
the value predicted by the model, based on measured data values, differs from the actual values
by more than the set percentage, either a warning message will display or the validation process
will fail as described in
Table 9.2.
To view or edit the validation thresholds:
On the Options menu, click Thresholds.
In the Thresholds dialog box, click the Validation tab.
Figure 9.2 Thresholds dialog box: Validation tab
Click in the appropriate box and type the desired values. Each validation threshold is explained in Table 9.1.
Click the Apply button to apply the change to the current session. The change will not be saved until you validate the model.
Table 9.1 Validation thresholds
|
Validation Threshold
|
Description
|
|
Response time
|
The acceptable percentage discrepancy between the response times predicted by the model and the actual response times observed. Validation will fail when this limit is exceeded.
|
|
Throughput
|
The acceptable percentage discrepancy between the throughput predicted by the model and the actual throughput observed. Validation will fail when this limit is exceeded.
|
|
Utilization
|
The acceptable percentage discrepancy between the CPU or disk utilization predicted by the model and the actual utilization observed. Validation will fail when this limit is exceeded.
|
|
Queue length
|
The acceptable percentage discrepancy between the CPU or disk queue lengths predicted by the model and the actual queue lengths observed. Validation will fail when this limit is exceeded.
|
|
Workload calibration
|
This threshold value is used during the workload calibration phase of model validation. A warning message is generated when this limit is exceeded.
|
|
Load calibration
|
When a new model is constructed from a load sample interval, a warning message is generated if the predicted CPU utilization or disk IO rate for any disk differs from the actual values by more than the set Load calibration threshold.
|
|
Session Workload Type
|
Click the appropriate button to select session type or transaction type for interactive workloads:
Session type workloads allow specification and throughput calculation of the user count and think times.
Transaction type workloads allow specification and throughput calculation of the user count.
|
Loading a New Collection File
To load a new model:
On the File menu, click Load/Validate Model. The Open dialog displays.
In the Files of type list, click Collections (*col).
Click once on the collection file to select it.
4.gif)
Figure 9.3 Open dialog box
Click Open to load the collection file.
Validation Messages
As the loading process runs, a list of warnings and informational messages appear in the
Validation Messages dialog box. These messages provide information about workloads that
may violate one or more modeling algorithm assumptions (see
"Queuing Model Algorithm
Assumptions" for information).
8.gif)
Figure 9.4 Validation Messages dialog box
Review the messages presented in the Validation Messages dialog box and decide what, if any,
influence they have on your model. Consider the following examples from the validation
messages listed in
Table 9.2.
Table 9.2 Example validation messages
|
Validation Message
|
Description
|
|
Workload group NETBASE contains multiple process priorities
|
"Multiple process priorities" means that there are processes defined in the NETBASE workload group with different queues. Therefore, the NETBASE workload group may not be homogeneous. (See "Identifying and Characterizing Workload Groups" for information on homogeneous workloads.)
|
|
Workload group LA contains both jobs and sessions
|
The workload group LA contains both jobs (batch processes) and sessions (interactive processes). Batch and interactive processes use system resources differently. (See "Identifying and Characterizing Workload Groups" for information on mixing batch and interactive processes in single workloads.)
|
In both validation message examples, it may or may not be necessary to redefine the workloads
and repeat the collection and reduction processes on the host system. For instance, if the
workload group NETBASE accounts for only a small percentage of total CPU utilization on the
host system, the effect on the model and your forecast may be negligible.
To print the contents of the Validation Messages box:
On the File menu, click Print.
From the Print submenu, click Notes.
Select the desired print options in the Windows Print dialog box. (Refer to your Windows
documentation for information on print options.)
Once you are satisfied that the messages in the Validation Messages dialog box will not affect
the model, click OK to close the box and complete the loading process.
Validating the Model
To validate the baseline model:
On the Forecast menu, click Validated Model.
In the message box, click Yes to begin the validation process. The file name extension of the document changes to the model format (*.mdl).
On the File menu, click Save Model.
In the Save As dialog box, click Save to save the model to the default location on your computer.
Validation Failures
On a rare basis, the validation process fails when the calibrated model and the actual data values
violate the validation thresholds in your model. To understand these assumptions, review the
basic queuing model algorithm assumptions (see
"Queuing Model Algorithm Assumptions").
Making the appropriate modifications will allow you to proceed with the forecast process.
Queuing Model Algorithm Assumptions
The validation process is based on assumptions regarding the workloads and hardware
resources on your host system. If repeated attempts to validate the model fail, call the Lund
Performance Solutions technical support team for help. (See
"Lund Performance Solutions
Technical Support Team".)
Violation of the following assumptions may cause validation to fail.
CPU and disk demands are accurately calculated for each workload transaction
Verify the identity of your CPU and disk types in the model before repeating the validation
process.
Although rarely violated, this assumption is critical to successful validation.
No multi-threading of individual transactions occurs
An individual transaction uses only one resource at a time (either CPU or disk drive). In general,
this assumption is violated in one of two ways:
By applications that perform no-wait terminal I/O or use multiple threads to perform simultaneous activity for one transaction.
By applications that perform a significant number of disk writes. Disk caching and the operating system can each allow a transaction to continue to execute while its disk writes are being performed. During validation, the workload group shows a predicted response time that is higher than actually observed, while the CPU and disk utilization statistics are accurate. The difference between the observed and predicted transaction response times is approximately equal to the number of disk-write requests that the transaction performs, multiplied by the disk access time.
All workload requests for disk service and all CPU requests are satisfied in non-preferential order within each priority queue—requests, once they reach the CPU, are handled on a first-in-first-out (FIFO) basis
The first request in a queue is honored before the second request in that queue.
MPE/iX systems fine tune some of the process priorities within sub queues.
Workloads performing long, CPU-intensive transactions—CPU "hogs"—tend to have a lower priority than other workloads. In this case, the FIFO rule does not apply.
For MPE systems, you can fix the problem by manually setting CPU-intensive workloads to a two-
digit priority code to distinguish them from regular workloads. When the workload group is
identified, assign it a different queue specification within the queue. For example, a CPU-intensive
workload group in the C queue is prioritized as C2. Forecast Capacity Planner will look at that
queue separately. C2 is prioritized higher than D, but lower than C.
HP-UX systems have what is called a nice system call. With nice, a process is able to influence
its own scheduling to a degree. To provide a process with less CPU time, assign a larger nice
value. The following equation explains how priorities are calculated:
priority = ("recent CPU usage" / constant) + (base priority) + (nice value)
For more information regarding the nice system call, read Chapter 6 of Taming UNIX, Volume I,
by Robert A. Lund.
No workload group is spending a significant amount of time waiting for any resource other than the CPU, a disk drive, or a terminal read
Validation failure in this case results in relatively accurate throughput and utilization values, but
significantly negative workload response time variances.
Use a performance monitoring tool (such as SOS Performance Advisor by Lund Performance
Solutions) to determine if workload transactions are spending too much time waiting for such
things as SIRs, file locks, or database locks. Transactions requesting tape I/O or multi-
applications (LANS) that pass all of their processing through a single-threaded resource may
cause this problem.
Transaction requests arrive in a random order
Validation failure in this case results in generally accurate throughput and utilization values, but
significant workload response time variances.
There is no synchronizing effect causing transactions to request CPU and disk resources at the
same time. This could happen when think times are the same. For example, two people doing file
transfers and their think time is always 10 seconds.
Workload Groups
Avoid mixing jobs and sessions in workloads. If you have mixed workloads and your model validation fails, refine your workload definitions and repeat the data collection and reduction processes described in "MPE/iX Host Data" or "HP-UX Host Data".
When possible, break work groups down by user logon rather than program name.
A workload group appears too slow
If a workload group appears too slow, you can isolate that workload group by running another
reduction from the original collection file. Choose the optimum time (the maximum period for that
workload group) and repeat the collection, transfer, loading, and validation processes. If the
validation process fails, review the original workload group definition file.
An actual resource variable is greater/less than the modeled resource variable
To analyze variables other than transaction type, such as average response time, set transactions
per hour as a constant by changing all workloads to the TRANSACTION type. For example,
actual response time may be greater than the modeled response time because of a network delay
for a workload group. Modifying the average delay time for that workload group may eliminate the
problem.
The Trans/Hr variance is greater than +/- 15 percent
Change the average delay for your workloads. Start with the highest priority. Small errors in high-
priority jobs have a greater impact on lower-priority jobs. Change only one workload group at a
time and limit the amount of change to, perhaps, 10 percent. This changes the arrival rate of
transactions and response time.
Changing the Model
Forecast Capacity Planner allows you to change the baseline data in your model to try out
different what-if scenarios. Observe what happens, for example, if you alter the model by:
Replacing the current CPU with a larger model.
Adding growth rates and aging.
Changing the parameters for one or more workload groups.
You can edit, add, insert, or delete items associated with workloads or resource in your validated
model directly from the Main Program window. Some of these menu options are unavailable for
some items.
Editing a Workload Group
To edit a workload group:
Select the workload group from either the global validated workloads or file manger pane in the Main Program window.
On the Edit menu, click Edit or press the Enter key.
Type or select the new workload group parameters in the Edit Workload dialog box.
Figure 9.5 Edit Workload dialog box
Table 9.3 Edit Workload parameters
|
Workload Parameter
|
Instructions to Edit the Workload Parameter
|
|
Workload Group Name
|
Enter the workload group name.
|
|
Description
|
Enter the description used to describe the workload group in forecast graph legends and in the results display.
|
|
Class
|
Select either Transaction, Session, or Job. Session and job workloads are those for which the transaction throughput is calculated based on the number of users and average user think time.
Transaction workloads are those for which the transaction throughput is known in advance.
|
|
Users
|
Enter the number of users in this workload group.
|
|
Avg think time
|
Enter the average user think time in seconds. For job workloads, this parameter should be 0 (zero).
|
|
CPU time per transaction
|
Enter the CPU time in milliseconds required to process one transaction. For job workload groups, this parameter should be the CPU time required to process the entire job.
|
|
Growth rate
|
In the Edit Workload dialog box, click the Growth Rate button.
In the Growth Rate dialog box, choose the monthly, quarterly, or yearly rate of growth for this workload group.
Click OK to save the change and return to the Edit Workload dialog box.
Note: Multiple growth rate periods (split rates) can be entered. You can also set a tiered rate and choose a growth type for the workload group (linear, compound, or tiered).
|
|
Priority
|
Enter the workload group priority. This can be any one- or two-character alphanumeric code.
|
|
Start months
|
If the workload group will be continuously active during the forecast interval, enter 0 (zero).
If the workload group will not activate for a specific time, enter a positive number representing the number of months.
If the workload group is initially active but will terminate after a specific time, enter a negative number that represents the number of months.
|
|
Avg delay time
|
Enter the extra delay time associated with this workload group. This can be used to represent any load-independent, average delay time, such as might be required to represent network delays, or flushing terminal output buffers.
|
|
CM percentage
|
Enter the percent of total CPU time that this workload group spends executing in compatibility mode.
|
|
Throughput
|
Enter the total transaction throughput for this workload group.
|
|
Actual throughput
|
Enter the actual transaction throughput, if known, for this system configuration. This value is used to adjust the predicted model transaction throughput to provide a more accurate forecast result.
To reset the parameter to the unadjusted predicted transaction throughput, enter 0 (zero) or leave the box empty.
|
|
Accept response time
|
Enter the maximum acceptable response time in seconds for this workload group, or 0 (zero) if no limit should be specified.
|
|
Actual response time
|
Enter the actual response time, if known, for this system configuration. This value will be used to adjust the predicted model response time to provide a more accurate forecast result.
To return the field to the unadjusted predicted transaction throughput, enter 0 (zero) or leave the box empty.
|
|
Disk IO per transaction
|
Enter the number of disk accesses on this disk drive required to process one transaction. For job workloads, this should be the number of disk accesses required to process the entire job.
|
Adding, Inserting, and Deleting Workload Groups
Add, insert, or delete workload groups to observe the effects of usage changes on the model.
The Add menu option adds a new workload group to the end of the workload group list.
The Insert menu option inserts a new workload group immediately below the selected workload group on the workload group list.
The Delete menu option removes the selected workload group from the workload group list.
Adding a Workload Group
In the global validated workloads pane or the file manger pane, select any workload group.
On the Edit menu, click Add.
In the Edit Workload dialog box, enter or select the desired values as described in Table 9.3.
Click OK to save your changes and return to the Main Program window.
Inserting a Workload Group
In the global validated workload pane or the file manager pane, select the workload group immediately above the position where you want the new workload group to appear.
On the Edit menu, click Insert.
In the Edit Workload dialog box, enter or select the desired values as described in Table 9.3.
Click OK to save your changes and return to the Main Program window.
