K4 Overview Job Scheduling

Parallel computers are operated differently than a normal PC, all users must share the system. Therefore, various operative procedures are in place. Users must understand these concepts and procedures to be able to use the available resources of a system to run a parallel application. A workload manager/job scheduler controls how available hardware resources are distributed among the user requests (jobs).

Users of computing centers typically compete for the expensive HPC resources of cluster systems. HPC resources can be distinguished as

• Shared resources (e.g. a parallel file system that is often shared across all cluster nodes and therefore shared between all users),
• Not-shared resources (e.g. cluster nodes dedicated to a particular parallel program of an individual user).

The configuration of the cluster system matters as well: a cluster node can also be a resource that is shared between several users.

A major aspect of job scheduling is to manage these resources in a way that users are treated fairly. Accounting for users or user groups can additionally support this.

• Comprehend the differences between Batch Systems and Time-Sharing Systems.
• Explain the concepts and procedures for resource allocation and job execution in an HPC environment.
• Run interactive jobs and batch jobs.
• Comprehend and describe the expected behavior of job scripts.
• Change provided job scripts and embed them into shell scripts to run a variety of parallel applications.
• Analyze the output generated from a job scheduler and describe the cause of typically generated errors.
• Comprehend accounting principles (billing for the jobs).
• Comprehend the set of terms for performance criteria like:
  • Resource Utilization.
  • Throughput.
  • Waiting Time.
  • Execution Time.
  • Turnaround Time.
• Comprehend scheduling strategies that increase productivity.
• Comprehend that typical goals of job scheduling are:
  • Maximization of resource utilization.
  • Maximization of throughput.
  • Minimization of waiting time.
  • Minimization of turnaround time.
• Comprehend that there is a variety of scheduling algorithms from rather simple to more complex like:
  • First-Come-First-Served (FCFS).
  • Shortest-Job-First (SJF).
  • Priority-based.
  • Fair-Share.
  • Backfilling.
• Apply advanced scheduling principles (e.g. backfilling) to achieve objectives like minimizing the averaged elapsed program runtimes, and maximizing the utilization of the available HPC resources.
• Discuss sophisticated scheduling principles (e.g. fair share) to achieve objectives like treating the users fair, and maximizing the utilization of the available HPC resources.