Linux Config

Linux Configuration

Configuring Linux - Best Practices

Presentation on Linux configuration: Best Practices for Linux on Z

Helpful configuration settings/information:

Processing configuration:

Giving a Linux guest more virtual CPUs than it can use doesn't necessarily provide more capacity - the scheduler share options determine capacity. Provide just enough for peak capacity.
Giving a linux guest just the amount of virtual CPUs it needs will also help performance by better utilizing cache.
The SHARE setting should be adjusted based on the number of vCPUs. If a guest has one vCPU and a SHARE setting of relative 100 then is given a second vCPU but left at relative 100, the SHARE becomes 50 for each vCPU.
Be careful with the use of cpuplugd. It makes it easy for one server to use more than its share. As vCPUs are parked, the remaining vcpus get larger shares.
See Velocity product zVRM for balancing vCPU usage.
Be careful with the use of priority/nice. It also makes it easy for one server to use more than its share. Linux is not a 'fair share' scheduler.
Linux steal time is not necessarily a bad thing, only if work is not getting done. Steal time is based on share of CPU and how much contention. Steal time is when the virtual machine thought it was dispatched but it wasn't.
Most Linux distrobutions now use DIAG 9C, but if using DIAG 44, there could be performance issues.

Processing configuration additions:

Verify that all linux guests have the Velocity components installed to insure accurate reporting.
Verify that the UDP Queue Limit (found in the NETSTAT CONFIG) is set to NO. When set to YES, it will throttle the information available for reporting.

Storage configuration issues:

Linux was designed for dedicated storage, which is very expensive.
Linux is LRU (Least Recently Used) which competes with VM's storage reference pattern.
When a Linux server logs on, it has a lot of initialization pages that won't be touched again. z/VM still has to back them up.
Linux tends to have timer pops that cause a large amount of data to be cached and then not referenced.
Spin locks can occur when storage that has a lock is paged out and another server is waiting, both having a large amount of storage in general. This can cause the system to start paging excessively.

Storage configuration solutions:

Linux guest size - Do not define more storage than needed. Decrease the size of the machine until swapping occurs.
An example would be: Instead of 128M for a guest machine, define 24M and 100m swap.
If running Oracle (especially), defining HUGE pages is a benefit. Pages are by default 4K. HUGE pages are larger, usually 1M, 2M or 1G.
Linux swapping -

Use vdisk for swap (if using real disks, move to paging disks).
Start with 1 small vdisk (100M) and 1 larger vdisk (2G). Set the priority to favor the smaller vdisk. Set an alert if the larger vdisk is used.
Use $ swapon -s to verify the sizes of the two disks.
Check /etc/fstab to determine the priority of the two disks - the smaller should be set to pri=1.
Unused vdisk is paged out (goodness).
The cost of the swap daemon on Linux is about 10% but z/VM swapping to vdisk is very fast and low cost.
Use DIAG driver instead of FBA. FBA is 512 bytes. DIAG will group those into 4K pages which reduces I/O by a factor of 8.
Adjust the minidisk caching storage to SET MDC STORAGE 128M 128M
Set MDC OFF for minidisks being used for Linux file systems and swap disks to avoid redundant levels of caching.
Set the swappiness setting in linux to zero. Use $ cat /proc/sys/vm/swappiness to check the value.

Helpful ESAMON screens/ESAMAP reports:

ESAUSRC - User Configuration - Shows the virtual machine configuration for all users.
ESALNXC - Linux Process Configuration - Shows the configuration information for the processes being monitored.
ESALNXD - Linux Node List - Shows the mapping between the Node name defined to zTCP, the Network Address defined to zTCP and the Host Name returned by SNMP.

Using zVPS to find information for solving issues with the Linux configuration:

ESAUSRC - Shows the virtual machine configuration for all users.

UserID - This shows the userid of the virtual machine.

SHARE - This shows the SHARE setting of the virtual machine. Relative 100 is the default. The SHARE setting will be divided equally between each vcpu (the next highlighted column). IE - If a guest has one vcpu and a SHARE setting of relative 100 then is given a second vcpu but left at relative 100, the SHARE becomes 50 for each vcpu. Soft limits only limit users if processing capacity is at a maximum. Hard limits set the limit regardless.

CPU - This shows the number of configured and active vcpus assigned to this id. Too many allocated vcpus when not needed can cause spin lock and/or processor cache issues.

VM Size - This shows the amount of storage defined to the virtual machine. Too much storage can cause paging and other performance issues. See above - lower the storage and use swap space.

ESALNXC - Shows the configuration information for the processes being monitored.

Node - This showsthe zTCP node name. This is usually the server name.

Process Name - This shows the names of the processes running (or did run on the report). Children processes are indented under the parent processes. (Process names are truncated to 15 characters.)

Appl Name - This shows the application which the process is associated.

Storage Size/RSS - This shows the process virtual memory size (in kb) and resident set size (memory).

Nice Value - This shows the process nice value.

ESALNXD - Shows the configuration information for the Linux nodes. This can be helpful in general.

Node - This shows zTCP node name. This is usually the server name.

IP Address - This shows the IP address defined in the zTCP node definition.

LPAR - This shows the LPAR the server is running on.

VM User - This shows the VM user id of the server.

Src Typ - This shows the SNMP agent - either VSI for Velocity Software Net-SNMP or HST for the standard Net-SNMP.

MIB Level - This shows the release level of the Velocity Software MIB.

Rel - This shows the release level of ZTCP.

Name - This shows the system name for the server.

Conclusions

Correctly configuring your Linux guests and the z/VM system to support those guests can save a lot of headaches in the long run.