Sabbatical Diary: Week 3

Not much to say this week – largely more of the same. Blogged about tree-sitter and (written, to be published shortly) eglot, configuring both for an increasing number of languages.

Also continued trying to land some small funding for continuing Amjad’s excellent work. Following meticulous characterisation of Linux’s process scheduler, his Ph.D. developed an alternative approach that enables a 30% smaller cluster to satisfy the same “serverless” workload.

Serverless computing is a popular style of cloud computing enabling customers to ignore all application deployment details, instead encapsulating their application – most commonly using “container” technology – so the platform can take responsibility for providing the necessary resources for it to run, scaling resources up and down in response to changes in demand. Providers typically provision resources as a shared cluster of (physical or virtual) machines, orchestrated using a cluster orchestrator based on the popular Kubernetes orchestrator. That orchestrator is then responsible for running encapsulated application code, ensuring each job is allocated sufficient resources.

The cluster’s machines typically run Linux, and so it is Linux’s kernel scheduler that is responsible for running a machine’s allocated jobs. That scheduler’s behaviour is taken for granted by the plethora of work that has sought to improve cluster performance. Unfortunately, close examination of cluster performance results showed significant discrepancies when compared to the intended behaviour – current practice masks these discrepancies by over-provisioning the cluster.

Meticulous examination of the results and the Linux kernel scheduler code revealed that the behaviour of a particular core data structure in conjunction with treatment of multi-core processors causes overheads to increase disproportionately as demand increases. This increase in overheads is especially prominent with serverless workloads due to their particular characteristics, and causes the cluster orchestrator makes ever poorer job placement decisions. The net effect is that, in practice, clusters must be significantly over-provisioned to achieve acceptable performance.

This thesis reports how meticulous measurement revealed and unpicked this problem, and describes and evaluates a set of relatively small – just a few hundred lines of code – but critical changes in Linux’s kernel scheduler in mitigation. The final outcome is that these mitigations allow an identical and representative serverless workload to be supported by a cluster comprising 30% fewer machines.

See our arXiv paper, “Mitigating context switching in densely packed Linux clusters with Latency-Aware Group Scheduling” for more details :)