ZMAP Extract Facility Explanation and Examples
zVPS provides a full performance database (PDB) that holds data for as long as an installation desires. The PDB shows system-by-system and application-by-application storage, processor and I/O requirements. Many different reports can be created for specific trending, problem determination and especially capacity planning and hardware consolidation/uplift projects.
- Overview of ESAEXTR
- How to create/run an extract
- Standard User Example
- Top User Example
- User CPU Example
- User Wait Example
- LPAR Utilization Example
- Linux CPU Utilization Example
- Linux Storage Utilization Example
- Linux Application Example
Overview of ESAEXTR Operations
There are over 3,000 variables defined in history keywords that can be extracted from a zVPS history archive (ESAEXTR). The format of the history keywords is "segment.field" where segment defines the source monitor record and the "field" defines the specific field. There are also over 3,000 variables defined in the Performance Data Reference manual that can be used for real time or historical data.
There are three kinds of data items:
- CP Monitor variables defined by IBM and described on one of the many monitor data descriptions. Learning the segment names will help in finding specific data items wanted for extraction. For example, to get the CPU utilization use the "SUBSUM.CPUUTIL" variable name (SUBSUM is the source monitor record and CPUUTIL is the specific field). There are two good resources for finding the variable names:
- The HISTORY KEYWORDS file on the ZMAP code disk (by default VMSYSVPS:ZVPS.CONFIG directory)
- The current zVPS Performance Data Reference (currently 5.2) in the ZMAP column
- Application variables defined by IBM for each application such as TCPIP or SFS.
- zVPS defined fields derived from snmp data collected by ZTCP.
Note that each variable may be a count, rate, or state. For example, one variable "USEACT.VMDTTIME" may be a count of CPU seconds consumed during the interval, but "USEACT.USERPCT" is the rate of consumption, and USEACT.VMDCTPRS is a 'state' value, being the number of pages resident for this user.
There are five files that make up the Performance Data Base (PDB) or "history files":
- ESASYS - system level data, such as channels and saved segments
- ESACPU - CPU related data, reported by the CPU
- ESADEV - DASD and Device data, reported by devices and control units
- ESAUSR - User data, reported by user, userclass and accounting codes
- ESATCP - Network data, Linux data, and distributed server data
How to Create/Run an Extract:
Historical data is extracted via the ESAEXTR exec. On the ZVPS userid on z/VM (by default in the VMSYSVPS:ZVPS.CONFIG directory), there are several files with the file type of ESAEXTR. These can be used as is or as examples to create other desired extracts. Detailed information about running extracts can be found in the ZMAP Installation and Operations Guide. Extract variable names relevant to ESAEXTR can be found in the ZVPS Performance Data Reference.
ESAEXTR: Used to pull data from the performance database. It can create reports or files with raw data for exporting. The HISTORY KEYWORDS file (or the ZMAP column in the Performance Data Reference) can be used to find variable names.
- Logon to a userid that can get a read only link to the following:
- ZMAP SFS directory code disk (default is VMSYSVPS:ZMAP.CODE).
- ZMAP 192 disk for daily history data or ZWRITE 191 for the current day's data.
- Link/Access the sfs/minidsk(s) above as read only.
- Create a file with a file name that relates to the data being extracted and a file type of ESAEXTR. Add the variables desired from the HISTORY KEYWORDS file. If this is to be a report, add column headers as shown in the examples below. CRITERIA= statements are used to limit the amount of data being reported. For instance, if only wanting to see one particular virtual machine, use the criteria statement to get only the information for the specified machine.
- Run the extract via ESAEXTR input (PARM fn - where:
- Input is a filetype from the ZMAP 192 or ZWRITE 191 disk which is in a date format. An * can be used for multiple days/months or an exact date of yyyymmdd can be used for a single day. For example if input is 202501*, it would pull in data from any files that were found from January 2025.
- fn is the file name of the ESAEXTR file created in step 2. (If porting the output to a spreadsheet, the parm csv can be added - (PARM fn csv - to put the data into a csv format). This will create a file called 202501 EXTRACT on the user's 191 disk.
- If desired, an exec (like the top user exec below) can be run. An exec allows a multitude of possibilities to massage the data for reports and/or displays.
- There are many example files (fn ESAEXTR fm) on the code directory (VMSYSVPS:ZMAP.CODE) and the config directory (VMSYSVPS:ZVPS.CONFIG). The directories are the default names, your SFS directories may be different.
Standard User Example
ESAEXTR: For a given class, this will show the CPU utilization, resident storage and virtual I/O per user. This example is for class Linux and prime time hours. This can be changed to any desired class or timeframe. The output below had class Redhat, which is the class for Redhat machines on this system.
EXTRACT: ; First extract system data columns ='10 10 7 8 8 8' TITLE = ' USERID ACCTNBR CPU Pages VIO' TITLE = ' --------- -------- ------- ------ ------' X = 'USERID' Y = 'USRCON.VMDACTNO' Y = 'USEACT.VMDTTIME' ; cpu utilization Y = 'USEACT.VMDCTPVR / 256' ; MB Resident Y = 'USEACT.VMDVDSCT' ; VIO INTERVAL='IN' CRITERIA='USRTYPE = USER' CRITERIA='USRCON.CLASSID = Redhat' CRITERIA='STARTTIME >= 080000' CRITERIA='STOPTIME <= 170000'
Create - Put the extract above into a file named ACCTUSR ESAEXTR on the 191 disk (there is an ACCTUSR example on the
ZMAP code disk that can be copied/updated.)
To run - ESAEXTR yyyymmdd (PARM ACCTUSR
and results in a report that looks like this:
Date Time USERID ACCTNBR CPU Pages VIO
--------- -------- ------- ------ ------
20250830 084900 REDHAT6 REDHAT6 831 36 0
20250830 090000 REDHAT64 REDHAT64 55 19 0
20250830 091500 REDHAT7 REDHAT7 69 16 0
20250830 094500 REDHAT74 REDHAT74 138 60 0
20250830 100000 REDHAT75 REDHAT75 4700 90 0
20250830 104500 REDHAT85 REDHAT85 293 78 0
To put this into csv format for exporting to an Excel-type application, use the same command but with the parm csv added.
To run - ESAEXTR yyyymmdd (PARM ACCTUSR CSV
DATE TIME USERID ACCTNBR CPU Pages VIO "2025/08/30","08:49:00","REDHAT6","REDHAT6","831","36","0" "2025/08/30","09:00:00","REDHAT64","REDHAT64","55","19","0" "2025/08/30","09:15:00","REDHAT7","REDHAT7","69","16","0" "2025/08/30","09:45:00","REDHAT74","REDHAT74","138","60","0" "2025/08/30","10:00:00","REDHAT75","REDHAT75","4700","90","0" "2025/08/30","10:45:00","REDHAT85","REDHAT85","293","78","0"
Top User Example
ESAEXTR: To determine the top users over a month of data would use the following extract (named "TOPUSER ESAEXTR"):
EXTRACT: X = 'STARTTIME' Y = 'USERID' Y = 'USERTYPE' Y = 'USEACT.VMDVTIME' Y = 'USEACT.VMDTTIME' Y = 'useact.VMDVDSCT' ;DASD I/O Y = 'useact.VMDCTPVR' ;Resident Storage INTERVAL='SU' criteria = 'USERTYPE = USER'
Create - Put the extract above into a file named TOPUSER ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM TOPUSER - or use the exec below
EXEC: One method of running this command is a sample exec that calls the extract, processes it, and puts the data to the console. The rexx exec to do this kind of analysis is similar to the following, accepting the history filetype as the argument:
/* Exec to take extracted data, summarize and show top 10 */
Parse arg input .
'ESAEXTR' input '(PARM TOPUSER DATA'
input = STRIP(input,,'*')
'PIPE <' input 'EXTRACT A | sort w4 | stem data.'
curuser = ''
userdata.0 = 0
usercount = 0
DO I = 1 to data.0
Parse var data.i date start stop user type tsecs vsecs .
If curuser <> user
Then Do
usercount = usercount + 1
If curuser <> ''
Then userdata.usercount = user totalcpu
curuser = user
totalcpu = 0
End
totalcpu = totalcpu + tsecs
End
usercount = usercount + 1
If curuser <> ''
Then userdata.usercount = user totalcpu
userdata.0 = usercount
'PIPE stem userdata. | spec w2 1-10 right w1 15',
'| change 1-10 / /0/',
'| sort w1 D | take 10 | cons'
Assuming the EXEC is named TOPUSER EXEC, then issue the following command for April, 2011 data:
TOPUSER 201104*
This results in both a report showing the CPU seconds consumed and the user that consumed them on the 191 disk and it also puts the top ten users to the screen.
0000002473 LNXUWA02 0000001615 LNXUWA11 0000001413 LNXUWA03 0000000441 LNXQWA02 0000000415 LNXQWA01 0000000354 LNXDWA03 0000000309 LNXDWA04 0000000269 LNXPSP03 0000000217 LNXUWA15 0000000215 ESATCP
User CPU Example
This example will extract user CPU data, including number of virtual CPUs, as well as show CPU per virtual cpu. Only data from prime shift is extracted.
EXTRACT: title=' STOP USERID NVCPU USERCPU USER' title=' TIME /CPU' title=' ------ --------- ----- ------ ------' columns = '7 9 5 8 7' X = 'STOPTIME' Y = 'USER' Y = 'USRCON.NVCPU' sig=3 Y = 'USEACT.USERPCT' sig=2 Y = 'USEACT.USERPCT / USRCON.NVCPU' sig=3 criteria='USEACT.USERPCT > 0.1' criteria='starttime > 075900' criteria='stoptime < 170100' interval='IN'
Create - Put the extract above into a file named fn ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM fn
The following is a sample produced by this extract.
*HDR Date Time STOP USERID NVCPU USERCPU USER
* TIME /CPU
* ------ --------- ----- ------ ------
20110415 100000 100100 TCPIP 1 0.11 0.108
20110415 100000 100100 LNXUWA01 1 65.65 65.650
20110415 100000 100100 LNXUWA03 2 22.70 11.349
20110415 100000 100100 LNXUWA11 2 19.23 9.613
20110415 100000 100100 LNXUWA02 2 18.45 9.227
20110415 100000 100100 LNXQWA01 2 11.56 5.778
20110415 100000 100100 LNXDWA04 2 9.88 4.942
20110415 100000 100100 LNXPSP02 1 7.67 7.672
20110415 100000 100100 LNXUWA15 2 6.82 3.410
20110415 100000 100100 LNXDWA02 2 4.40 2.202
User Wait Example
This example will extract user wait state data. When complete, the ESAPLOT process can be run to show how many virtual machines were in each of the different wait states for some period of time.
EXTRACT: INTERVAL = 'INTERVAL' X = 'SYTSCG.SRMCDISP' ; inqueue users Y = 'SYTUWT.CALCPURN' ; Percent in queue VMDBKs running Y = 'SYTUWT.CALCPUWT' ; Percent in queue VMDBKs in CPU Wait Y = 'SYTUWT.CALSIMWT' ; Percent in queue VMDBKs in Simulation Wait Y = 'SYTUWT.CALIOWT' ; Percent in queue VMDBKs in I/O wait Y = 'SYTUWT.CALWTPAG' ; Percent in queue VMDBKs in Page Wait Y = 'SYTUWT.CALTSVM' ; Percent VMDBKs in test idle and SVM wait Y = 'SYTUWT.CALCFWT' ; Percent in queue VMDBKs in Console Function Y = 'SYTUWT.CALOTHR' ; Percent in queue VMDBKs not in above states Y = 'SYTUWT.CALIOACT' ; Percent in queue VMDBKs with asynchronous I/O active
Create - Put the extract above into a file named fn ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM fn or ESAEXTR yyyymm* (PARM fn - First is for one day, second is for all days in that month)
Once complete, use the file produced as input for ESAPLOT.
If desired - rename the file name from yyyymmdd to (something meaningful like) USERWAIT on the 191 disk (the file type
must remain EXTRACT)
To run - ESAPLOT fn (combine
Notes:
The following is a sample produced by this plot.
Plot 1: 2022022 All cross-variables vs. Runnable users 0Base variable: SYTSCG.SRMCDISP Runnable users 0Sym Cross-variable Scale Origin Average Maximum Description --- --------------- ----- ------- ------- ------- ------------------------------------------------------------ * SYTUWT.CALCPURN 1 0.0 3.3 6.3 Percent VMDBKs running + SYTUWT.CALCPUWT 10 0.0 5.2 23.0 Percent VMDBKs in CPU Wait - SYTUWT.CALSIMWT 0.1 0.0 0.3 3.3 Percent VMDBKs in Simulation Wait # SYTUWT.CALWTPAG 1 0.0 0.0 3.8 Percent VMDBKs in Page Wait @ SYTUWT.CALTSVM 1 0.0 2.2 8.7 Percent VMDBKs in test idle and SVM wait > SYTUWT.CALCFWT 0.1 0.0 0.0 1.0 Percent VMDBKs in Console Function Wait = SYTUWT.CALOTHR 1 0.0 1.0 3.1 Percent VMDBKs in other wait state % SYTUWT.CALIOACT 1 0.0 0.8 3.9 Percent in-queue VMDBKs with asynchronous I/O active
LPAR Utilization
To extract CPU utilization by LPAR, use the following extract. This shows the total CPU, virtual CPU and overhead of the lpar. Note the report headings and column settings.
EXTRACT: Interval = 'SU' title = ' Stop LPAR vCPU <--CPU Utilization-->' title = ' time Name count Total Virtual Overhd' title = ' ------ ------- ---- ------ ------- ------' COLUMNS = '07 09 03 08 8 7 ' X = 'STOPTIME '; Y = 'SYTCUP.LCUPNAME '; Y = 'SYTCUP.LCUPCPCT ' sig = 0 Y = 'SYTCUP.LCUCACTM / 10000 / SECONDS ' sig = 2 ;Total Y = 'SYTCUP.LCUCLPTM / 10000 / SECONDS ' ;Logical CPU pct Y = 'SYTCUP.PCTOVHD '; ;Difference CRITERIA = 'SYTCUP.LCUCPUID > 63' ; TOTAL for LPAR */
Create - Put the extract above into a file named fn ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM fn
This results in the following:
*HDR Date Time Stop LPAR vCPU <--CPU Utilization-->
* time Name count Total Virtual Overhd
* ------ ------- ---- ------ ------- ------
20110726 000000 010000 Totals: 5 84.32 84.22 0.10
20110726 000000 010000 VSIVM2 1 0.98 0.97 0.01
20110726 000000 010000 VSIVM1 2 4.89 4.88 0.01
20110726 000000 010000 VSIVM3 1 62.34 62.33 0.00
20110726 000000 010000 VSIVM4 1 16.11 16.03 0.08
20110726 010000 020000 Totals: 5 80.68 80.60 0.09
20110726 010000 020000 VSIVM2 1 1.05 1.05 0.01
20110726 010000 020000 VSIVM1 2 3.14 3.14 0.01
20110726 010000 020000 VSIVM3 1 62.28 62.27 0.00
20110726 010000 020000 VSIVM4 1 14.21 14.14 0.07
Linux CPU Utilization
To extract CPU utilization by Linux NODE, use the following extract. This shows the 'Nice, user, system CPU, as well as I/O time and kernel CPU. Note the report headings and column settings. Note also the CRITERIA allowing only one node to be extracted - this is currently commented out but can be used if desired by removing the ';'.
EXTRACT: ;CRITERIA='NODE=broblx1' title=' STOP NodeIO Kernel' title=' TIME Name Nice user SYST Time CPU ' title=' ------ -------- ------ ------ ------ ------ -------' columns = '7 9 7 7 7 7 7 7' INTERVAL='SU' X = 'STOPTIME' Y ='NODE' Y ='UCDSYS.NICEPCT' Y ='UCDSYS.USERPCT' Y ='UCDSYS.SYSTPCT' Y ='UCDSYS.IOWAIT ' Y ='UCDSYS.KRNLCPU'
Create - Put the extract above into a file named fn ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM fn
An extract would produce the CPU time by server as in the following extract:
*HDR Date Time STOP NodeIO Kernel * TIME Name Nice user SYST Time CPU * ------ -------- ------ ------ ------ ------ ------- 20110726 000000 010000 *TheUsrs 0.42 4.25 5.55 1.25 0.92 20110726 000000 010000 vpnc 0.00 1.87 2.55 0.00 0.00 20110726 000000 010000 linux64 0.08 0.14 0.56 0.64 0.34 20110726 000000 010000 sles11 0.00 0.18 0.28 0.01 0.00 20110726 000000 010000 sles11v2 0.07 0.11 0.16 0.20 0.00 20110726 000000 010000 roblx2 0.22 0.01 0.46 0.03 0.00 20110726 000000 010000 sles11v 0.05 0.07 0.09 0.26 0.00 20110726 000000 010000 suselnx1 0.00 0.40 0.64 0.12 0.59 20110726 000000 010000 vpnz 0.00 1.48 0.81 0.00 0.00 20110726 010000 020000 *TheUsrs 0.41 4.35 5.56 1.25 0.95 20110726 010000 020000 vpnc 0.00 1.87 2.55 0.00 0.00 20110726 010000 020000 linux64 0.08 0.32 0.60 0.63 0.37 20110726 010000 020000 sles11 0.00 0.18 0.28 0.01 0.00 20110726 010000 020000 sles11v2 0.06 0.06 0.16 0.25 0.00 20110726 010000 020000 roblx2 0.22 0.01 0.46 0.02 0.00 20110726 010000 020000 sles11v 0.05 0.06 0.09 0.23 0.00 20110726 010000 020000 suselnx1 0.00 0.41 0.63 0.10 0.58 20110726 010000 020000 vpnz 0.00 1.46 0.79 0.00 0.00
Linux Storage Utilization by Index or by User
To extract information about one filesystem or one specific node. Uncomment the "Criteria='Node=LINUX001' and update the server name for a user exctract. Or leave that line commented (with a ';') and use the "Criteria='HSTME2.INDEX = xx" and update the index number of the desired mounted storage.
EXTRACT: INTERVAL = 'INTERVAL' X = 'NODE' Y = 'HSTME2.INDEX' Y = 'HSTME2.DESCR' Y = 'HSTME2.SIZE' Y = 'HSTME2.USED' ;CRITERIA='NODE=LINUX001' CRITERIA='HSTME2.INDEX = xx'
Create - Put the extract above into a file named fn ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM fn
Linux Application Accounting Example
The following extract provides linux application level data:
EXTRACT: ; First extract system data columns ='10 10 10 8 8 8 8' TITLE = ' STARTTIME STOPTIME NODE APPL PID' TITLEc= ' CPU' TITLE = '---------- --------- --------- ------- ------' TITLEc= ' -------' INTERVAL='SU' CRITERIA='USRTYPE=USER' CRITERIA='USRCON.CLASSID=*Linux' CRITERIA='STARTTIME >= 080000' CRITERIA='STOPTIME <= 170000' CRITERIA='VSINAP.TOTCPU > 0 ' X = 'STARTTIME' Y = 'STOPTIME' Y ='NODE' Y ='VSINAP.NAME' Y ='VSINAP.ID ' Y ='VSINAP.TOTCPU / 100' ;Turn into CPU Seconds
Create - Put the extract above into a file named fn ESAEXTR on the 191 disk
To run - ESAEXTR yyyymmdd (PARM fn or ESAEXTR yyyymm* (PARM fn - First is for one day, second is for all days in that month)
The following is a sample produced by this extract. It shows the process application name and CPU utilization for the process.
HDR Date Time STARTTIME STOPTIME NODE APPL CPU
---------- --------- --------- ------- ------
20250830 000000 000000 240000 REDHAT *Totals 3825
20250830 000000 000000 240000 REDHAT abrt-wa 0
20250830 000000 000000 240000 REDHAT anacron 0
20250830 000000 000000 240000 REDHAT atd 0
20250830 000000 000000 240000 REDHAT auditd 0
20250830 000000 000000 240000 REDHAT avahi-d 3
20250830 000000 000000 240000 REDHAT chronyd 0
20250830 000000 000000 240000 REDHAT conmon 26
20250830 000000 000000 240000 REDHAT cpuplug 124
20250830 000000 000000 240000 REDHAT crond 2052
20250830 000000 000000 240000 REDHAT dbus-br 0
20250830 000000 000000 240000 REDHAT dbus-da 1
20250830 000000 000000 240000 REDHAT gssprox 0
20250830 000000 000000 240000 REDHAT init 2
20250830 000000 000000 240000 REDHAT kernel 242
