TOCPREVNEXTINDEX

Lund Performance Solutions


Model Creation and Validation
The model creation and validation process creates an accurate model of your system based on the data collected on your host system for throughput, CPU utilization, and other workload values. The key to accurate forecasting is to ensure that your calculated values accurately reflect the actual values on the host system.


The procedures in this chapter assume:
  • Forecast Capacity Planner is installed and running on your PC. The product logo is displayed in the Main Program window (this display will be referred to as the Logo screen.)
  • The data collection process on the host system is complete and the collection file (*.col) is transferred to your PC.
  • Setting Thresholds

    Before loading and validating a new model, review the default forecast and validation thresholds to determine if they are suitable for your model. In some instances, you may want to set the threshold CPU utilization limits higher or lower than the default.

    Forecast Thresholds

    The default CPU utilization values in the Thresholds dialog box are based on typical utilization limits for MPE/iX and MPE/V CPUs. Generally, system performance is compromised at higher utilization levels.


    To change the CPU utilization threshold:
  • On the Options menu, click Thresholds.
  • In the Thresholds dialog box, click the Forecast tab.


  • Figure 9.1 Thresholds dialog box: Forecast tab
  • Click in the appropriate text box for your CPU.
  • Enter the desired CPU value.
  • Click the Apply button to apply the change to the current session. This change will not be saved until you validate the model.
  • Validation Thresholds

    The validation thresholds limit the variability allowed between your model and the actual data. If the value predicted by the model, based on measured data values, differs from the actual values by more than the set percentage, either a warning message will display or the validation process will fail as described in Table 9.2.


    To view or edit the validation thresholds:
  • On the Options menu, click Thresholds.
  • In the Thresholds dialog box, click the Validation tab.


  • Figure 9.2 Thresholds dialog box: Validation tab
  • Click in the appropriate box and type the desired values. Each validation threshold is explained in Table 9.1.
  • Click the Apply button to apply the change to the current session. The change will not be saved until you validate the model.
  • Table 9.1 Validation thresholds
    Validation Threshold
    Description
    Response time
    The acceptable percentage discrepancy between the response times predicted by the model and the actual response times observed. Validation will fail when this limit is exceeded.
    Throughput
    The acceptable percentage discrepancy between the throughput predicted by the model and the actual throughput observed. Validation will fail when this limit is exceeded.
    Utilization
    The acceptable percentage discrepancy between the CPU or disk utilization predicted by the model and the actual utilization observed. Validation will fail when this limit is exceeded.
    Queue length
    The acceptable percentage discrepancy between the CPU or disk queue lengths predicted by the model and the actual queue lengths observed. Validation will fail when this limit is exceeded.
    Workload calibration
    This threshold value is used during the workload calibration phase of model validation. A warning message is generated when this limit is exceeded.
    Load calibration
    When a new model is constructed from a load sample interval, a warning message is generated if the predicted CPU utilization or disk IO rate for any disk differs from the actual values by more than the set Load calibration threshold.
    Session Workload Type
    Click the appropriate button to select session type or transaction type for interactive workloads:
  • Session type workloads allow specification and throughput calculation of the user count and think times.
  • Transaction type workloads allow specification and throughput calculation of the user count.
  • Loading a New Collection File

    To load a new model:
  • On the File menu, click Load/Validate Model. The Open dialog displays.
  • In the Files of type list, click Collections (*col).
  • Click once on the collection file to select it.


  • Figure 9.3 Open dialog box
  • Click Open to load the collection file.
  • Validation Messages

    As the loading process runs, a list of warnings and informational messages appear in the Validation Messages dialog box. These messages provide information about workloads that may violate one or more modeling algorithm assumptions (see "Queuing Model Algorithm Assumptions" for information).


    Figure 9.4 Validation Messages dialog box
    Review the messages presented in the Validation Messages dialog box and decide what, if any, influence they have on your model. Consider the following examples from the validation messages listed in Table 9.2.
    Table 9.2 Example validation messages
    Validation Message
    Description
    Workload group NETBASE contains multiple process priorities
    "Multiple process priorities" means that there are processes defined in the NETBASE workload group with different queues. Therefore, the NETBASE workload group may not be homogeneous. (See "Identifying and Characterizing Workload Groups" for information on homogeneous workloads.)
    Workload group LA contains both jobs and sessions
    The workload group LA contains both jobs (batch processes) and sessions (interactive processes). Batch and interactive processes use system resources differently. (See "Identifying and Characterizing Workload Groups" for information on mixing batch and interactive processes in single workloads.)
    In both validation message examples, it may or may not be necessary to redefine the workloads and repeat the collection and reduction processes on the host system. For instance, if the workload group NETBASE accounts for only a small percentage of total CPU utilization on the host system, the effect on the model and your forecast may be negligible.


    To print the contents of the Validation Messages box:
  • On the File menu, click Print.
  • From the Print submenu, click Notes.
  • Select the desired print options in the Windows Print dialog box. (Refer to your Windows documentation for information on print options.)
    Once you are satisfied that the messages in the Validation Messages dialog box will not affect the model, click OK to close the box and complete the loading process.

    Validating the Model

    To validate the baseline model:
  • On the Forecast menu, click Validated Model.
  • In the message box, click Yes to begin the validation process. The file name extension of the document changes to the model format (*.mdl).
  • On the File menu, click Save Model.
  • In the Save As dialog box, click Save to save the model to the default location on your computer.
  • Validation Failures

    On a rare basis, the validation process fails when the calibrated model and the actual data values violate the validation thresholds in your model. To understand these assumptions, review the basic queuing model algorithm assumptions (see "Queuing Model Algorithm Assumptions").
  • Making the appropriate modifications will allow you to proceed with the forecast process.
  • Queuing Model Algorithm Assumptions

    The validation process is based on assumptions regarding the workloads and hardware resources on your host system. If repeated attempts to validate the model fail, call the Lund Performance Solutions technical support team for help. (See "Lund Performance Solutions Technical Support Team".)


    Violation of the following assumptions may cause validation to fail.
    CPU and disk demands are accurately calculated for each workload transaction
    Verify the identity of your CPU and disk types in the model before repeating the validation process.
    Although rarely violated, this assumption is critical to successful validation.
    No multi-threading of individual transactions occurs
    An individual transaction uses only one resource at a time (either CPU or disk drive). In general, this assumption is violated in one of two ways:
  • By applications that perform no-wait terminal I/O or use multiple threads to perform simultaneous activity for one transaction.
  • By applications that perform a significant number of disk writes. Disk caching and the operating system can each allow a transaction to continue to execute while its disk writes are being performed. During validation, the workload group shows a predicted response time that is higher than actually observed, while the CPU and disk utilization statistics are accurate. The difference between the observed and predicted transaction response times is approximately equal to the number of disk-write requests that the transaction performs, multiplied by the disk access time.
  • All workload requests for disk service and all CPU requests are satisfied in non-preferential order within each priority queue—requests, once they reach the CPU, are handled on a first-in-first-out (FIFO) basis
  • The first request in a queue is honored before the second request in that queue.
  • MPE/iX systems fine tune some of the process priorities within sub queues.
  • Workloads performing long, CPU-intensive transactions—CPU "hogs"—tend to have a lower priority than other workloads. In this case, the FIFO rule does not apply.
  • For MPE systems, you can fix the problem by manually setting CPU-intensive workloads to a two- digit priority code to distinguish them from regular workloads. When the workload group is identified, assign it a different queue specification within the queue. For example, a CPU-intensive workload group in the C queue is prioritized as C2. Forecast Capacity Planner will look at that queue separately. C2 is prioritized higher than D, but lower than C.


    HP-UX systems have what is called a nice system call. With nice, a process is able to influence its own scheduling to a degree. To provide a process with less CPU time, assign a larger nice value. The following equation explains how priorities are calculated:


    priority = ("recent CPU usage" / constant) + (base priority) + (nice value)


    For more information regarding the nice system call, read Chapter 6 of Taming UNIX, Volume I, by Robert A. Lund.
    No workload group is spending a significant amount of time waiting for any resource other than the CPU, a disk drive, or a terminal read
    Validation failure in this case results in relatively accurate throughput and utilization values, but significantly negative workload response time variances.


    Use a performance monitoring tool (such as SOS Performance Advisor by Lund Performance Solutions) to determine if workload transactions are spending too much time waiting for such things as SIRs, file locks, or database locks. Transactions requesting tape I/O or multi- applications (LANS) that pass all of their processing through a single-threaded resource may cause this problem.
    Transaction requests arrive in a random order
    Validation failure in this case results in generally accurate throughput and utilization values, but significant workload response time variances.


    There is no synchronizing effect causing transactions to request CPU and disk resources at the same time. This could happen when think times are the same. For example, two people doing file transfers and their think time is always 10 seconds.

    Workload Groups

    Workload definition is critical to successful validation of your model. Review your workloads for compliance with the following rules. (For more information about workload definitions, see "Identifying and Characterizing Workload Groups".)
  • Avoid mixing jobs and sessions in workloads. If you have mixed workloads and your model validation fails, refine your workload definitions and repeat the data collection and reduction processes described in "MPE/iX Host Data" or "HP-UX Host Data".
  • When possible, break work groups down by user logon rather than program name.
  • A workload group appears too slow
    If a workload group appears too slow, you can isolate that workload group by running another reduction from the original collection file. Choose the optimum time (the maximum period for that workload group) and repeat the collection, transfer, loading, and validation processes. If the validation process fails, review the original workload group definition file.
    An actual resource variable is greater/less than the modeled resource variable
    To analyze variables other than transaction type, such as average response time, set transactions per hour as a constant by changing all workloads to the TRANSACTION type. For example, actual response time may be greater than the modeled response time because of a network delay for a workload group. Modifying the average delay time for that workload group may eliminate the problem.
    The Trans/Hr variance is greater than +/- 15 percent
    Change the average delay for your workloads. Start with the highest priority. Small errors in high- priority jobs have a greater impact on lower-priority jobs. Change only one workload group at a time and limit the amount of change to, perhaps, 10 percent. This changes the arrival rate of transactions and response time.

    Changing the Model

    Forecast Capacity Planner allows you to change the baseline data in your model to try out different what-if scenarios. Observe what happens, for example, if you alter the model by:
  • Replacing the current CPU with a larger model.
  • Adding growth rates and aging.
  • Changing the parameters for one or more workload groups.
  • You can edit, add, insert, or delete items associated with workloads or resource in your validated model directly from the Main Program window. Some of these menu options are unavailable for some items.

    Editing a Workload Group

    To edit a workload group:
  • Select the workload group from either the global validated workloads or file manger pane in the Main Program window.
  • On the Edit menu, click Edit or press the Enter key.
  • Type or select the new workload group parameters in the Edit Workload dialog box.


  • Figure 9.5 Edit Workload dialog box
    Table 9.3 Edit Workload parameters
    Workload Parameter
    Instructions to Edit the Workload Parameter
    Workload Group Name
    Enter the workload group name.
    Description
    Enter the description used to describe the workload group in forecast graph legends and in the results display.
    Class
    Select either Transaction, Session, or Job. Session and job workloads are those for which the transaction throughput is calculated based on the number of users and average user think time.
  • Transaction workloads are those for which the transaction throughput is known in advance.
  • Users
    Enter the number of users in this workload group.
    Avg think time
    Enter the average user think time in seconds. For job workloads, this parameter should be 0 (zero).
    CPU time per transaction
    Enter the CPU time in milliseconds required to process one transaction. For job workload groups, this parameter should be the CPU time required to process the entire job.
    Growth rate
  • In the Edit Workload dialog box, click the Growth Rate button.
  • In the Growth Rate dialog box, choose the monthly, quarterly, or yearly rate of growth for this workload group.
  • Click OK to save the change and return to the Edit Workload dialog box.
  • Note: Multiple growth rate periods (split rates) can be entered. You can also set a tiered rate and choose a growth type for the workload group (linear, compound, or tiered).
    Priority
    Enter the workload group priority. This can be any one- or two-character alphanumeric code.
    Start months
    If the workload group will be continuously active during the forecast interval, enter 0 (zero).
    If the workload group will not activate for a specific time, enter a positive number representing the number of months.
    If the workload group is initially active but will terminate after a specific time, enter a negative number that represents the number of months.
    Avg delay time
    Enter the extra delay time associated with this workload group. This can be used to represent any load-independent, average delay time, such as might be required to represent network delays, or flushing terminal output buffers.
    CM percentage
    Enter the percent of total CPU time that this workload group spends executing in compatibility mode.
    Throughput
    Enter the total transaction throughput for this workload group.
    Actual throughput
  • Enter the actual transaction throughput, if known, for this system configuration. This value is used to adjust the predicted model transaction throughput to provide a more accurate forecast result.
  • To reset the parameter to the unadjusted predicted transaction throughput, enter 0 (zero) or leave the box empty.
  • Accept response time
    Enter the maximum acceptable response time in seconds for this workload group, or 0 (zero) if no limit should be specified.
    Actual response time
  • Enter the actual response time, if known, for this system configuration. This value will be used to adjust the predicted model response time to provide a more accurate forecast result.
  • To return the field to the unadjusted predicted transaction throughput, enter 0 (zero) or leave the box empty.
  • Disk IO per transaction
    Enter the number of disk accesses on this disk drive required to process one transaction. For job workloads, this should be the number of disk accesses required to process the entire job.

    Adding, Inserting, and Deleting Workload Groups

    Add, insert, or delete workload groups to observe the effects of usage changes on the model.
  • The Add menu option adds a new workload group to the end of the workload group list.
  • The Insert menu option inserts a new workload group immediately below the selected workload group on the workload group list.
  • The Delete menu option removes the selected workload group from the workload group list.
  • Adding a Workload Group

  • In the global validated workloads pane or the file manger pane, select any workload group.
  • On the Edit menu, click Add.
  • In the Edit Workload dialog box, enter or select the desired values as described in Table 9.3.
  • Click OK to save your changes and return to the Main Program window.
  • Inserting a Workload Group

  • In the global validated workload pane or the file manager pane, select the workload group immediately above the position where you want the new workload group to appear.
  • On the Edit menu, click Insert.
  • In the Edit Workload dialog box, enter or select the desired values as described in Table 9.3.
  • Click OK to save your changes and return to the Main Program window.
  • Deleting a Workload Group

  • In the global validated workload pane or the file manger pane, select the workload group to delete.
  • On the Edit menu, click Delete. A message box will display and ask you if you want to delete the workload group.
  • Click Yes to delete the object, or No to cancel the process.
  • Editing Resources

    Editing a Disk Drive

  • In the global validated centers pane or the file manger pane, select the disk drive to edit.
  • On the Edit menu, click Edit.


  • Figure 9.6 Disk Edit dialog box
  • In the New disk area of the Disk Edit dialog box, enter or select the new parameters for the disk drive as described in Table 9.4.
  • Table 9.4 Disk Edit parameters
    Disk Parameter
    Instructions to Edit the Disk Parameter
    Title
    Enter a new disk drive name.
    Type
    Select a new disk drive type from the list.
    Access Time
    Enter a new access time for the selected disk drive.
  • Click OK to save your changes and return to the Main Program window.
  • Editing a CPU

  • In the global validated centers pane or the file manger pane, select the CPU to edit.
  • On the Edit menu, click Edit.


  • Figure 9.7 CPU Edit dialog box
  • In the CPU Edit dialog box, enter or select the new parameters for the CPU as described in Table 9.5.
  • Table 9.5 CPU parameters
    CPU Parameter
    Instructions to Edit the CPU Parameter
    CPU
    Enter a new CPU name.
    Type
    Select a new CPU type from the list.
    Adj
    Enter a new CPU adjustment factor.
    CM Adj
    Enter a new compatibility mode adjustment factor.
  • Click OK to save your changes and return to the Main Program window.
  • Adding, Inserting, and Deleting Resources

    Add, insert, or delete disk drives to observe the effects of usage changes on the model. You cannot add or delete the CPU.
  • The Add menu option adds a new disk drive to the end of the list.
  • The Insert menu option inserts a new disk drive immediately below the selected disk in the list.
  • The Delete menu option removes the selected disk drive from the model.
  • Adding a Disk Drive

  • In the global validated centers pane or the file manger pane, select any disk.
  • On the Edit menu, click Add.
  • In the Edit Workload dialog box, enter or select the desired values as described in Table 9.4.
  • Click OK to save your changes and return to the Main Program window.
  • Inserting a Disk Drive

  • In the global validated centers pane or the file manager pane, select the disk drive immediately above the position where you want the new disk to appear.
  • On the Edit menu, click Insert.
  • In the Edit Workload dialog box, enter or select the desired values as described in Table 9.4.
  • Click OK to save your changes and return to the Main Program window.
  • Deleting a Disk Drive

  • In the global validated centers pane or the file manger pane, select the disk drive to delete.
  • On the Edit menu, click Delete. A message box will display and ask you if you want to delete the disk drive.
  • Click Yes to delete the object, or No to cancel the process.
  • Saving and Reusing Desktops

    At any time during the validating and forecasting processes, you can save and rename the current desktop to preserve any changes to the model. This can be useful if you plan to generate multiple-performance projections from your baseline model. The Save Desktop command saves all of your current settings, links to currently-opened files, and window layout options.

    Saving the Active Desktop

    To save the active desktop:
  • On the File menu, click Save Desktop.
  • In the Save As dialog box:
  • Enter the name of the desktop in the File name box.
  • Click the Save button to save the desktop (*.dsk).
  • Reloading a Saved Desktop File

    To reload a saved desktop (*.dsk) file:
  • On the File menu, click Load Desktop.
  • In the Open dialog box:
  • Locate and select the desired desktop file.
  • Click OK to open the desktop file in Forecast Capacity Planner.
  • Restoring the Current Desktop

    To restore the current desktop to the last saved version:
  • On the File menu, click Restore Layout.
  • In the Open dialog box:
  • Locate and select the last saved version of the desktop.
  • Click OK.

  • Lund Performance Solutions
    www.lund.com
    Voice: (541) 812-7600
    Fax: (541) 81207611
    info@lund.com
    TOCPREVNEXTINDEX