By Elpida Tzortzatos, IBM Distinguished Engineer and Jessie Yu, Developer for IBM z/OS Platform for Apache Spark.
One of the strengths of the z Systems platform and the z/OS operating system is the ability to run multiple workloads simultaneously within one z/OS image or across multiple images while maintaining high system utilization. Such workloads have different, often competing performance completion and resource requirements. These requirements must be balanced to make the best use of system resources while maintaining the optimal throughput and system responsiveness. The function that makes this possible is dynamic workload management, which the Workload Management component of z/OS provides.
With z/OS Workload Management (WLM), you define performance goals and assign a business importance to each goal. You define the goals for work in business terms, and the system decides how much resource, such as CPU or memory, to give the business term to meet the goal. WLM constantly monitors the system and adapts processing to meet the goals. However, WLM’s primary focus is to ensure performance goals are met, and it doesn’t always prevent a workload from overachieving and consuming a large amount of shared system resources.
There are new workloads on z/OS for which having a means of controlling or capping resource consumption could be useful. IBM z/OS Platform for Apache Spark, for example, is known for its aggressive in-memory data caching for performance gains. zCloud is another example that requires hard limits to support its multi-tenancy. The new Metering & Capping support* allows the system capacity planner more granular control over CPU and memory consumption for the workloads.
This new support consists of two major features. Honor Priority by service class allows work, within a WLM service class, exclusion from the system-wide Honor Priority defaults. The capacity planner can now choose to keep specific, specialty-engine-eligible work on just the specialty engines and not overflow to general purpose processors.
The other feature, Memory Limit, puts an upper limit on real memory that can be consumed by address spaces associated with a WLM resource group. When the memory-limited address spaces attempt to consume more than their allotted share, the system takes actions, such as paging or deferring, against these address spaces to ensure that the rest of the system is not impacted by such resource-hungry workloads.
New types of workloads are continuing to be introduced to the z/OS platform. The operating system itself will in term continue to introduce new features to safeguard mission-critical workloads, satisfy all the others, and utilize system resources fully and efficiently.
Apache Spark page: http://www-03.ibm.com/systems/z/os/zos/apache-spark.html
z/OS page: http://www-03.ibm.com/systems/z/os/zos/
* Metering and Capping support consists of the following APARs: OA51171, OA50953, OA50845, PI71118, and OA50760.
Elpida Tzortzatos is a Distinguished Engineer and z Systems Architect working on IBM z/OS Core Design. Her achievements have focused on bringing new workloads and technology to the z Systems platform, contributing to innovative products and enhancements that have delivered significant value in a wide range of areas.
Jessie Yu has 12 years of experience on the z/OS platform, doing design, development, and test. She is currently working as a developer for IBM z/OS Platform for Apache Spark.