Data Collection, Extraction, and Export

Chapter 4 explains the initial steps that must be taken before Forecast can be successfully used for capacity planning. Sections contained in Chapter 4 include:

"Overview."

"Identifying System Resources."

"Identifying and Characterizing Workload Groups."

"MPE/iX Host Data."

"HP-UX Host Data."

Overview

Forecast Capacity Planner creates a model of your system based on the data collection (*.col) files transferred from your MPE/iX system. To accurately predict future performance, it is important to start with an accurate picture of your current system. The usefulness of your forecast depends on the quality and relevance of your baseline data.

Some of the procedures described in this chapter require Lund Performance Solutions’ SOS Performance Advisor software. Refer to your SOS Performance Advisor User’s Guide for further information.

Data Collection Process

Identify host system resources.

Identify and characterize the workload groups on the host system.

Create a workload definition file.

Collect performance data on the host system.

Reduce the performance data on the host system.

Download/export performance data to the PC.

Identifying System Resources

The modeling algorithms used by Forecast Capacity Planner depend on the performance attributes of the hardware. It is important to accurately identify the following components of the host system:

Hardware configuration

CPU type

Disk drive type(s)

NOTE The SOS Performance Advisor programs automatically recognize most CPU types, but not disk drive types. It is important to use the proper syntax to identify the disk drives on your system.

To use Forecast Capacity Planner to review the list of disk drives:

Run Forecast Capacity Planner as described in "Starting Forecast Capacity Planner".

On the Options menu, click Resources.

In the Resources dialog box, click the Disks tab.

Use the horizontal scroll bar to review the list of disk drives.

Identifying and Characterizing Workload Groups

A workload group is a group of similar, identifiable transactions on the host system performed by individual users and programs. Group processes to form a meaningful model of system usage. Identifying and grouping similar components is the key to creating an accurate and useful model. For example, workloads can be grouped by:

Applications

User login

Departmental processes

Disk devices

A workload group may be as simple as one user running one program, or as complex as entire departments running many programs.

Forecast Capacity Planner creates a model of your host system using Mean Value Analysis (MVA) queuing network algorithms. MVA algorithms use average workload groups to generate system data. For that reason, make sure model inputs such as workload groups are homogeneous. A homogeneous workload group consists of processes of a similar type, function, and priority.

Averaging is meaningless for workload groups made up of dissimilar transactions. For example, if an average accounts receivable transaction takes 200 milliseconds of the CPU’s time, while general ledger transactions average 500 milliseconds, taking an average of the two does not provide a meaningful average for either transaction.

Identifying Workload Groups

Input from management and system users is essential in identifying and defining workload groups. Interview managers and users to determine how the system is used, and to identify distinct functions, such as order entry, telemarketing, or accounting. Break down the various departmental functions into essential components, based on your desired result. Identify groupings that will provide you with the needed information. These grouped components make up your workload groups.

While it is important to keep your model as simple as possible and the number of workload groups to a minimum, it is more important to keep all workload groups in the model as homogeneous as possible. Do not sacrifice the accuracy of your model for simplicity.

Characterizing Workload Groups

Once you have identified your workload groups, use the following guidelines to further refine your workload definitions:

Limit the components of any workload group to users or transactions with service demands of comparable magnitude and similar balance across the system. Do not mix heavy-CPU/low-I/O transactions with light-CPU/heavy-I/O transactions.

Do not mix interactive processes and batch processes in the same workload group. System resources, priorities, and think times are different for interactive and batch processes.

Consider the desired outputs of the model when defining workload groups. For example, if response times for database queries are a concern, do not group database queries with other workload components. Create a separate workload group for this task, preferably defined by user logon.

Use separate workload groups for specific divisions, branches, or departments as needed to further refine the model.

Identify workload groups by user logon, if possible.

MPE/iX Host Data

This section describes how to collect, extract, and export your MPE/iX data files. It also contains a helpful sub-section that discusses what methods you may use to establish workloads.

Creating an MPE/iX Workload Definition File

You can use the three pre-defined MPE/iX workload groups or create your own workload definitions file.

Predefined Workload Groups

Forecast Capacity Planner uses three predefined (default) workload groups:

JOB	Includes all batch job streams
SESSION	Includes all interactive (online) terminal sessions
SYSPROCS	Includes all system processes

If you prefer to use more specific workload groups, you may create a workload definitions file (see the next section, "User-defined Workload Groups.").

User-defined Workload Groups

Defining specific workload groups is a good way to track specific processes, such as processes run by a specific department in your business. However, user-defined workload groups are optional—they are not required to run Forecast.

Creating a Workload Definitions File

User-defined workload groups are created in the soswkdef.pub.lps file. Use your editor to create an soswkdef file (quad.util.lps is included on the distribution tape).

The basic format of the soswkdef file requires three items for each workload group:

The name of the group (up to 10 characters).

The type of the processes included in the group (JOB, SESSION, or BOTH).

The user and/or program specifications (a list of one or both of the following):

USER	The account by which MPE knows you (your user or login ID)
PROG	The MPE fully-qualified program file name

Workload Definition File Configuration Rules

A workload group name of up to 10 characters is required.

A workload group type specification is necessary to indicate which types of processes to include or exclude from the workload definition. This makes it possible to create two workload groups for processes that run in both interactive and batch modes. For example:

JOB includes only batch processes.

SESSION includes only interactive (online) terminal sessions.

SYSPROCS includes only system processes.

Workload groups must be separated by one or more blank lines within a definition file.

Comments can be included on any line if preceded by an exclamation point (!).

Either a user or a program specification is required. The specification must be entered one per line and must be one of the following three types:

A program specification (PROG=PROGRAM.GROUP.ACCOUNT).

A user specification (USER=SESSION, USER.ACCOUNT,GROUP). The session name and logon group are optional.

The MPE logical device number (LDEV=nnn) or range of device numbers (LDEV=nnn-nnn).

The symbol "@" can be used as a wildcard for any of the criteria, just as it can with normal MPE/iX rules (partial or full).

NOTE The drive specification means that you can capture activity on a terminal-by-terminal basis, or even within a range of terminals. Use this option carefully!

There is virtually no limit to the number of user, program, and LDEV specifications allowed for each workload group.

The name and type of specification lines are required. All other lines are optional. In order to be considered part of a workload group, a process must satisfy the program, user, and LDEV specifications, if all three are present.

If more than one program specification lines are included, a program needs to satisfy only one of the program specifications to be included in the group.

If no program specifications are entered, all process programs are considered to be in the group, unless the process is somehow disqualified by the user or drive specification.

The user and drive specifications are resolved in the same way. For instance, the lines in the following example should be entered into the soswkdef file to define the workload called "WORKTEST."

	WORKTEST	!Workload name
	SESSION	!Only terminals
	PROG=@.PUB.MFG
	PROG=MONEND.PUB.QTR
	USER=JANE, MGR.MFG
	USER=@,@.QTR
	LDEV=50-60

Figure 4.1 Workload definition file (example)

For a process to be included in the WORKTEST workload group, it must satisfy only one program specification, one user specification, and one LDEV specification. Each is considered to be an "and" condition. For instance, a program, INVEN01.PUB.MFG, run by MGR.QTR at LDEV 56 would be counted in the WORKTEST workload group.

When selecting any of the following system-type processes, the program group and the account must be specified as "@."

Specify command interpreter processes by the program file name "ci" (PROG=ci).

Specify spooler processes by the program file name "sp" (PROG=sp).

All other system processes can be identified by name.

To strip out command interpreters from the catch-all sessions workload group, create a separate workload with the program name "ci" to track the response times for sessions. It will reflect what the users actually experience.

A process can belong to only one workload group. If a process meets the criteria for two or more groups, it will be assigned to the first workload group in the file for which it qualifies.

Collecting MPE/iX Performance Data

Data collection on an MPE/iX system can be accomplished with or without the SOS/3000 Performance Advisor software from Lund Performance Solutions.

Steaming the Data Collector Job with SOS/3000

24-hour Collection

To establish 24-hour continuous data collection, do the following:

From the HP e3000 terminal, type :Stream sosmonj.pub.lps

8-hour Collection

To establish 8-hour continuous data collection, do the following:

From the HP e3000 terminal, type :Stream sosjob.pub.lps

Steaming the Data Collector Job without SOS/3000

24-hour Collection

To establish 24-hour continuous monitoring, do the following:

From the HP e3000 terminal, verify the Duration of job in minutes value is 0 (zero).

Type :Stream fcpcolj.pub.lps

These changes will cause the fcpcolj job to automatically restream itself every 24 hours at midnight. If your backup jobs kill this job, you will need to restream it as part of a daily routine.

8-hour Collection

To establish 8-hour continuous data collection, do the following:

From the HP e3000 terminal, verify the Duration of job in minutes value is 480.

Comment out the Stream fcpcolj line near the bottom of the screen. Do not remove the line.

Type :Stream fcpcolj.pub.lps

Changing the Sampling Interval Length

By default, the performance data sampling interval for all data collection job streams is 300 seconds (five minutes). Every five minutes, an average of all performance indicators for the current cycle is recorded in the log file.

To change the interval:

Locate the Data collection interval in seconds line in the job stream file.

Replace the default value with the desired number of seconds.

Extracting MPE/iX Performance Data

The SOS Performance Advisor SOSLOGX program reduces data from the collected log files, based on the criteria defined by the user. The process consists of two steps:

The program scans the collected data and selects only the records that meet criteria set by the user.

The program creates a data file to download to your PC.

Running SOSLOGX

To run SOSLOGX:

Enter the command Run soslogx.pub.lps at the prompt.

Enter the following information as prompted:

The start date of the initial sample using the format mm/dd/yy. If you do not provide a date, the program will retrieve the earliest sample date recorded by default.

The start time using the format hh:mm. If you do not provide a start date, the program will retrieve the start time of the earliest sample data recorded, by default.

If the SOSLOGX parameters are acceptable, go to the next section, "Creating the SOSLOGX Reduction File".

From the log screen, select OPTIONS (F1). The SOSLOGX MAIN OPTION MENU displays.

	SOSLOGX MAIN OPTION MENU
		1)		Current log file location (LOGFILES.LPS)
		2)		Display process information (Y)
		3)		Use function keys to select reports (Y)
		4)		Process display option menu (SUBMENU)
		5)		Data break configuration menu (SUBMENU)
		6)		Export file configuration menu (SUBMENU)
		7)		Log information exclusions (SUBMENU)
		8)		Zone configuration (SUBMENU)