add support for zswap and ksmtuned
This change add a new lib/host-mem file and moves the existing
ksm support to a new configure_ksm function.
Additional support for ksmtuned is added with a new flag
"ENABLE_KSMTUNED" which defaults to true.
This change also adds support for zswap. zswap is disabled
by default. When enabled on ubuntu lz4 will
be used as the default compressor and z3fold as the zpool.
On non debian distros the compressor and zpool are not set.
The default values should result in very low overhead although
the zstd compressor may provide better overall performance in ci
or with slow io due to the higher compression ratio.
Additionally memory and network sysctl tunings are optionally applied
to defer writes, prefer swapping and optimise tcp connection
startup and keepalive. The sysctl tunings are disabled by default
The base devstack job has been modifed to enable zram and sysctl
tuning.
Both ksm and zswap are wrapped by a tune_host function
which is now called very early in devstack to ensure
they are configured before any memory/network intensive
operations are executed.
The ci jobs do not enable this functionality by default.
To use this functionaltiy define
ENABLE_SYSCTL_MEM_TUNING: true
ENABLE_SYSCTL_NET_TUNING: true
ENABLE_ZSWAP: true
in the devstack_localrc section of the job vars.
Change-Id: Ia5202d5a9903492a4c18b50ea8d12bd91cc9f135
diff --git a/lib/host b/lib/host
new file mode 100644
index 0000000..95c5b9b
--- /dev/null
+++ b/lib/host
@@ -0,0 +1,98 @@
+#!/bin/bash
+
+# Kernel Samepage Merging (KSM)
+# -----------------------------
+
+# Processes that mark their memory as mergeable can share identical memory
+# pages if KSM is enabled. This is particularly useful for nova + libvirt
+# backends but any other setup that marks its memory as mergeable can take
+# advantage. The drawback is there is higher cpu load; however, we tend to
+# be memory bound not cpu bound so enable KSM by default but allow people
+# to opt out if the CPU time is more important to them.
+ENABLE_KSM=$(trueorfalse True ENABLE_KSM)
+ENABLE_KSMTUNED=$(trueorfalse True ENABLE_KSMTUNED)
+function configure_ksm {
+ if [[ $ENABLE_KSMTUNED == "True" ]] ; then
+ install_package "ksmtuned"
+ fi
+ if [[ -f /sys/kernel/mm/ksm/run ]] ; then
+ echo $(bool_to_int ENABLE_KSM) | sudo tee /sys/kernel/mm/ksm/run
+ fi
+}
+
+# Compressed swap (ZSWAP)
+#------------------------
+
+# as noted in the kernel docs https://docs.kernel.org/admin-guide/mm/zswap.html
+# Zswap is a lightweight compressed cache for swap pages.
+# It takes pages that are in the process of being swapped out and attempts
+# to compress them into a dynamically allocated RAM-based memory pool.
+# zswap basically trades CPU cycles for potentially reduced swap I/O.
+# This trade-off can also result in a significant performance improvement
+# if reads from the compressed cache are faster than reads from a swap device.
+
+ENABLE_ZSWAP=$(trueorfalse False ENABLE_ZSWAP)
+# lz4 is very fast although it does not have the best compression
+# zstd has much better compression but more latency
+ZSWAP_COMPRESSOR=${ZSWAP_COMPRESSOR:="lz4"}
+ZSWAP_ZPOOL=${ZSWAP_ZPOOL:="z3fold"}
+function configure_zswap {
+ if [[ $ENABLE_KSMTUNED == "True" ]] ; then
+ # Centos 9 stream seems to only support enabling but not run time
+ # tuning so dont try to choose better default on centos
+ if is_ubuntu; then
+ echo ${ZSWAP_COMPRESSOR} | sudo tee /sys/module/zswap/parameters/compressor
+ echo ${ZSWAP_ZPOOL} | sudo tee /sys/module/zswap/parameters/zpool
+ fi
+ echo 1 | sudo tee /sys/module/zswap/parameters/enabled
+ # print curent zswap kernel config
+ sudo grep -R . /sys/module/zswap/parameters || /bin/true
+ fi
+}
+
+ENABLE_SYSCTL_MEM_TUNING=$(trueorfalse False ENABLE_SYSCTL_MEM_TUNING)
+function configure_sysctl_mem_parmaters {
+ if [[ $ENABLE_SYSCTL_MEM_TUNING == "True" ]] ; then
+ # defer write when memory is available
+ sudo sysctl -w vm.dirty_ratio=60
+ sudo sysctl -w vm.dirty_background_ratio=10
+ sudo sysctl -w vm.vfs_cache_pressure=50
+ # assume swap is compressed so on new kernels
+ # give it equal priority as page cache which is
+ # uncompressed. on kernels < 5.8 the max is 100
+ # not 200 so it will strongly prefer swapping.
+ sudo sysctl -w vm.swappiness=100
+ sudo grep -R . /proc/sys/vm/ || /bin/true
+ fi
+}
+
+function configure_host_mem {
+ configure_zswap
+ configure_ksm
+ configure_sysctl_mem_parmaters
+}
+
+ENABLE_SYSCTL_NET_TUNING=$(trueorfalse False ENABLE_SYSCTL_NET_TUNING)
+function configure_sysctl_net_parmaters {
+ if [[ $ENABLE_SYSCTL_NET_TUNING == "True" ]] ; then
+ # detect dead TCP connections after 120 seconds
+ sudo sysctl -w net.ipv4.tcp_keepalive_time=60
+ sudo sysctl -w net.ipv4.tcp_keepalive_intvl=10
+ sudo sysctl -w net.ipv4.tcp_keepalive_probes=6
+ # reudce network latency for new connections
+ sudo sysctl -w net.ipv4.tcp_fastopen=3
+ # print tcp options
+ sudo grep -R . /proc/sys/net/ipv4/tcp* || /bin/true
+ # disable qos by default
+ sudo sysctl -w net.core.default_qdisc=pfifo_fast
+ fi
+}
+
+function configure_host_net {
+ configure_sysctl_net_parmaters
+}
+
+function tune_host {
+ configure_host_mem
+ configure_host_net
+}
\ No newline at end of file