TOCPREVNEXTINDEX

Lund Performance Solutions


ScopeUtil File Converter
Lund Performance Solutions has created a program, ScopeUtil, for the SOS/3000 Performance Advisor product. ScopeUtil takes Scope extract files as input and creates SL files as output.
  • The SL files are the files that hold SOS/3000 historical data.
  • Scope is a performance data collector sold by Hewlett-Packard.
  • By converting Scope data to SL files, ScopeUtil makes the data available to the historical reporting tools from LUND, including SOSLOGX (in SOS/3000) and Performance Gallery Gold.

    What Is Converted

    Scope and SOS/3000 do not collect the same data. Scope collects some items that SOS/3000 does not, and SOS/3000 collects some items that Scope does not. The purpose of ScopeUtil is to create SL files, which are SOS/3000 data files. Therefore, ScopeUtil calculates SOS/3000 data items from Scope data items. There are some SOS/3000 data items that are impossible to calculate because Scope does not provide adequate data. In such cases, the values for those data items are set to 0.
    The file scopform.pub.lps contains formulas that express SOS/3000 data items in terms of Scope data items. ScopeUtil uses the formulas in this file to do its conversions. You may use this file for reference when you have questions about whether an SOS item is being supplied by ScopeUtil.
    Here is a short extract from that file:
    ! Exact
    QUEUE-LEN=
    GLOB:CPUQUEUE;
    ! Not available from Scope export
    ! SAQ=
    ! Approximate, includes CM from user processes
    CPU-CM%=
    (100*SUM(PROC:CM_SECONDS))/GLOB:INTERVAL;
    A line that begins with an exclamation point is a comment line. This example shows that the SOS data item, QUEUE-LEN, can be calculated exactly from Scope data, and in particular, it is the same as the Scope global data item, CPUQUEUE. The SOS data item, SAQ, is not available from Scope. It will therefore be set to 0 within the SL files. The SOS data item CPU-CM% is calculated from the Scope process data records.

    Getting Started

    The fastest way to start viewing your Scope data in Performance Gallery Gold is to use the scopallj.pub.lps job stream. In this section you will find instructions that should be sufficient to allow you to create an SL file and a Performance Gallery Gold extract file using that job. Additional detail is provided in the sections that follow.
  • Modify these two lines in the scopallj job stream:
  • start <startdate> <starttime>
    stop <stopdate> <stoptime>
    These commands set the start and stop date/time's for the Scope extraction process. They determine the time range of data that will be created. There are comments within the job file to help you modify these commands appropriately.
  • Stream the stream file and wait for completion. If the job runs successfully, you will have a file called outfile.scope.lps, which is the extract file needed by Performance Gallery Gold.
  • Transfer outfile.scope.lps to your PC where Performance Gallery Gold is installed. Use a binary mode of transfer, and give the file an extension of ".pfg" on the PC.
  • Load the file in Performance Gallery Gold. See the Performance Gallery Gold User’s Guide.
  • How ScopeUtil Works

    The steps required to convert Scope data to SOS data are:
  • Create a Scope extract file.
  • This extract file is the input to ScopeUtil. There is a job file, scopextj.pub.lps, supplied with ScopeUtil to create this file. Creating the file is also done by the first job in the file scopallj.pub.lps. These jobs create an extract file named perfextr.scope.lps. Using these jobs is discussed in greater detail in "Creating the Scope Extract File".
  • Create an SL file from the Scope extract files.
  • There is a job file, scoputj.pub.lps, supplied with ScopeUtil, to create the SL file from the extract file. Creating the SL file is also the first activity of the second job in the file scopallj.pub.lps. These jobs place the SL file in the group scope.lps. Using these jobs is discussed in greater detail in "Creating an SL File with ScopeUtil".
    After you perform these two steps, you have an SL file in the group scope.lps. Having an SL file allows you to display the data in SOSLOGX (program file soslogx.pub.lps), and to use SOSLOGX to create a Performance Gallery Gold extract file. SOSLOGX is a program that is part of the SOS/3000 product.
    Typically one will want to display the data with Performance Gallery Gold. This requires two additional steps to prepare the data for use by Performance Gallery Gold:
  • Use SOSLOGX to create the Performance Gallery Gold extract file. This activity is part of the scopallj job. The Performance Gallery Gold extract file is given the name output.scope.lps (this name can be changed in scopallj). The extraction can also be done interactively within SOSLOGX. Refer to the SOS/3000 Performance Advisor User Reference Manual for details on performing this task.
  • Transfer the extract file to your PC where Performance Gallery Gold is installed. Use a binary mode of transfer, and give the file an extension of ".pfg" on the PC.
  • Creating the Scope Extract File

    Before Scopeutil can convert your Scope performance data to an SOS SL data file, the Scope program named Extract must be used to convert the data to a Scope extract file, which holds the Scope data in an ASCII format.
    The Scope extract job is scopextj.pub.lps. The first job in scopallj.pub.lps does the same thing as scopextj.pub.lps. The following comments apply to this part of scopallj also.
    The job is set up to provide the information required by the ScopeUtil conversion program and, with a few exceptions noted here, should not be modified. You might want to make a copy of the job file in another location (group scope.lps for example) and modify the copy.
    Here are acceptable modifications:
  • The job specifies that the Scope collection file is logglob.scope.sys. If this is not the case, this file name needs to be changed.
  • The job logs on as user manager.sys into the scope.sys group. This can be changed, but the user needs to have SM capability.
  • There are also two dates and times that must be supplied. These are the start and stop dates for the data to be extracted. The lines in which to supply them are:
  • start <startdate> <starttime>
    stop <stopdate> <stoptime>
    The items in angular braces need to be replaced with dates and times. For example:
    start 3/1/00 9:00 am
    stop 3/5/00 12:00 pm
  • The Scope extract job writes the extracted data to file perfextr. scope.lps. This file name can be changed. However this file is input to the ScopeUtil conversion activity, so there must be a corresponding change in whatever job performs that activity (scoputj or scopallj). See "Creating an SL File with ScopeUtil".
  • The file equations in the job put a file limit of 100 megabytes on the extract output files. This could be made larger if necessary.
  • For more details on the operation of the Scope Extract program, consult your HP Scope manual, or the online help for the Extract program.

    Creating an SL File with ScopeUtil

    The ScopeUtil conversion job converts the extract file created by the Scope extract job into an SL file that SOSLOGX can read. SOSLOGX is the historical reporting program in SOS/3000.

    The file scoputj.pub.lps contains a job to perform this conversion. The second job in scopallj.pub.lps also has a section that does the same thing as scoputj.pub.lps. The following comments apply to this part of scopallj, also.

    The job reads the extract file created by the Scope extract job. It assumes the file is named perfextr.scope.lps. Altering options used by ScopeUtil can change this. This is discussed below.

    The ScopeUtil conversion program creates an SL file in whatever group it is logged onto. The ScopeUtil conversion job logs on to the group scope.lps, so that is where it will place the SL file. This could be changed if you wish the file to be placed in a different location. Just change the logon specified in the :JOB statement.

    Each run of the ScopeUtil conversion program will create one SL file. The name of the file will be SLYYDDDA, where the letters YY are replaced by a two-digit year, and DDD are replaced by a three-digit day of the year, a number between 1 and 366. The year and day used correspond to the earliest date that appears in the Scope extract file.

    Please note that this behavior makes it difficult to handle ScopeUtil conversion of more than one set of Scope data from a single day. However it would be possible to accomplish such a task using renaming or file equations. The details will not be given here, since we believe that most people will not want to have multiple SL files per day. Talk to the support group at LPS if you need help with such a task.

    Options are provided to the scoputil program in the ScopeUtil conversion job. Here is the section of the job stream in which that is done:

    :scoputil.pub -o
     
    ProcLog
    = Y
    ProcCPUThreshold
    = .0
    ProcOnlyActive
    = Y
    ProcLogSessions
    = Y
    ProcLogJobs
    = Y
    ProcLogSysprocs
    = Y
    ProcLogDead
    = Y
    ProcLogonFilter
    = @.@
    ProcSortOpt
    = 4
    ProcSortAscend
    = Y
    ProcLogLimit
    = 10
    ApplFile
    = perfextr.scope
    ConfFile
    = perfextr.scope
    DiscFile
    = perfextr.scope
    GlobFile
    = perfextr.scope
    ProcFile
    = perfextr.scope
    :eod
     

    The name of each option being set appears to the left of the equal sign. The value for the option appears to the right of the equal sign. The options and appropriate values for them are discussed in "ScopeUtil Command-Line Parameters".

    Running the ScopeUtil Program

    The ScopeUtil program that actually performs the file conversion is scoputil.pub.lps. Its activity is controlled by program options and command-line parameters.
    The program options (or, simply, options) are similar to the options in other SOS/3000 programs such as SOS and SOSLOGX. The ScopeUtil program reads options from the file scopparm in the user's logon group. If the file does not exist, the ScopeUtil program uses default values.
    If the ScopeUtil program is run using the -m command-line parameter, it will present the user with an options menu that can be used to set parameters for the current ScopeUtil run. It also allows the user to save the non-default parameter values into scopparm (in the logon group). Details of using the -m command-line parameter are in the next section.

    ScopeUtil Command-Line Parameters

    When the ScopeUtil program is run, command-line parameters may be specified. There are three command-line parameters: -l, -m and -o.

    scoputil.pub -l

    This parameter causes ScopeUtil to display a "#" for every record written to the SL file.

    scoputil.pub -m

    In a session, this parameter causes ScopeUtil to display the options menu and allows options to be set. This is what is initially displayed:
    SCOPEUTIL MAIN OPTION MENU
    1) Process logging configuration submenu (SUBMENU)
    2) Import files configuration submenu (SUBMENU)
    Which Option:
    Selecting item 1 brings up the process logging configuration submenu, as follows:
    SCOPEUTIL MAIN OPTION MENU
    Process logging configuration submenu
    1) Log processes (Y)
    2) CPU percentage required for process display (.0)
    3) Log only active processes (Y)
    4) Log session processes (Y)
    5) Log job processes (Y)
    6) Log system processes (Y)
    7) Log processes which have died (Y)
    8) Process logon filter (@.@)
    9) Process sort option (4-CPU time)
    10) Log processes sorted in ascending order (N)
    11) Maximum number of processes to be logged per interval (10)
    Which Option:
    These options are discussed in further detail in "Process Logging Configuration Submenu".
    Selecting option 2 from the main menu brings up these options. These specify the names of the Scope extract files that Scopeutil should use as the source for its data.
    SCOPEUTIL MAIN OPTION MENU
    Import files configuration submenu
    1) Application File (perfextr.scope)
    2) Configuration File (perfextr.scope)
    3) Disc File (perfextr.scope)
    4) Global File (perfextr.scope)
    5) Processes File (perfextr.scope)
    Which Option:
    In a job, the -m command-line parameter has no effect. See "scoputil.pub -o" for instructions on how to run ScopeUtil in a job.

    scoputil.pub -o

    In a job, the -o parameter causes ScopeUtil to read the options from $stdin in a parameter=value format. You can see exactly how this is done in the jobs scoputj.pub.lps and scopallj.pub.lps.
    In a session, the -o parameter has no effect. See "scoputil.pub -m" for instructions on how to run ScopeUtil in a session.

    scoputil.pub

    When scoputil is run without command-line options, it proceeds to convert files according to the options in the scopparm file (in the logon group).

    Process Logging Configuration Submenu

    The following table shows the options available in the Process Logging Configuration Submenu.
    Option Name
    Prompt in Option Menu
     
    Type
    Description
    ProcLog
    Log processes
    Y/N
    If "Y", put process data in the log.
    If "N", don't.
    ProcCPUThreshold
    CPU percentage required for process display
    Number,
    nn.n
    CPU usage level that must be exceeded if ProcOnlyActive is "Y."
    ProcOnlyActive
    Log only active processes
    Y/N
    If "Y", only include process whose CPU usage in the most recent interval exceeds the ProcCPUThreshold.
    ProcLogSessions
    Log session processes
    Y/N
    If "Y", log session processes.
    ProcLogJobs
    Log job processes
    Y/N
    If "Y", log job processes.
    ProcLogSysprocs
    Log system processes
    Y/N
    If "Y", log system processes.
    ProcLogDead
    Log processes which have died
    Y/N
    (not used)
    ProcLogonFilter
    Process logon filter
    String
    Filter processes according to the logon filter. The default filter @.@ passes everything. This filter works the same as the logon filter described for SOS in the SOS/3000 Performance Advisor User Reference Manual.
    ProcSortOpt
    Process sort option
    Integer
    Sort processes according to the data item specified here.
    ProcSortAscend
    Log processes sorted in ascending order
    Y/N
    If "Y", sort ascending.
    If "N", sort descending.
    ProcLogLimit
    Maximum number of processes to be logged per interval
    Y/N
    The maximum number of processes to be logged each interval.

    List of Files used by ScopeUtil

    Scopeutil uses the files listed in the next table. All except scopparm are distributed on the production distribution tape.
    Table 29.1 ScopeUtil Files
    ScopeUtil File
    Description
    itemlist.pub.lps
    This file is part of the SOS/3000 product. It describes the items and records that appear in the SL file.
    psitems.pub.lps
    This file describes the items and records of a process record in the SL file.
    scopallj.pub.lps
    The job stream that runs the Scope Extract program to create a Scope extract file, ScopeUtil to create an SL file, and SOSLOGX to create a Performance Gallery extract file.
    scopextj.pub.lps
    The job stream that runs the Scope Extract program to create a Scope extract file.
    scopextl.pub.lps
    The items that the Scope Extract program should extract for use by ScopeUtil. This file should only be modified by LPS.
    scopform.pub.lps
    The set of formulas that defines how ScopeUtil should create SOS data items for the SL files from Scope data items in the Scope extract files. This file should only be modified by LPS.
    scopitem.pub.lps
    The items that can appear in Scope extract files, and the column headings that should be used by the ScopeUtil executable to identify them.
    scopparm.pub.lps
    The scoputil program will write this file when it is run with the -m parameter and non-default options are specified.
    scoputil.pub.lps
    The Scopeutil executable that converts Scope extract files to an SL file.
    scoputj.pub.lps
    The job stream that runs the Scopeutil executable to convert Scope extract files to SL files.
    totdiv.pub.lps
    This file can be thought of as kind of an addendum to the scopform file. It just contains the items that are in totals subblocks within the SL file and need special treatment.

    Lund Performance Solutions
    www.lund.com
    Voice: (541) 812-7600
    Fax: (541) 812-7611
    info@lund.com
    TOCPREVNEXTINDEX