X-Git-Url: https://git.decadent.org.uk/gitweb/?p=kernel-news-talk.git;a=blobdiff_plain;f=index.html;h=31c5d34d1e27a5330b890a9513c964c6e4932350;hp=7d5018d6d296fd2bb04a93905f9ebb4f59a04be9;hb=refs%2Ftags%2Fdebconf2014;hpb=edee31bcd5737cc994db909eb75d0b54535ee3c6
diff --git a/index.html b/index.html
index 7d5018d..31c5d34 100644
--- a/index.html
+++ b/index.html
@@ -4,7 +4,7 @@
-What's new in the Linux kernel - DebConf 2013
+What's new in the Linux kernel - DebConf 2014
@@ -39,7 +39,7 @@
@@ -50,6 +50,7 @@
What's new in the Linux kernel
+
and what's missing in Debian
Ben Hutchings
@@ -58,13 +59,13 @@
Professional software engineer by day, Debian developer by night
+ (or sometimes the other way round)
Regular Linux contributor in both roles since 2008
- Maintaining a net driver in my day job, plus core networking
- and PCI code as necessary
+ Working on various drivers and kernel code in my day job
Debian kernel team member, now doing most of the unstable
@@ -83,10 +84,15 @@
Linux is released about 5 times a year (plus stable updates
every week or two)
+
+
+ ...though some features aren't ready to use when they first
+ appear in a release
+
+
- For 'wheezy' we chose to freeze with Linux 3.2, which was
- getting pretty old by the time of release
+ Since my talk last year, Linus has made 6 releases (3.11-3.16)
Good news: we have lots of new kernel features in testing/unstable
@@ -98,169 +104,306 @@
-
Team device driver [3.3]
+
Recap of last year's features (1)
- Alternative to the bonding driver - simpler, modular, high-level
- control deferred to userland
+ Team device driver: userland package (libteam) was uploaded in
+ October
+
+
+ Transcendent memory: frontswap, zswap and Xen tmem will be
+ enabled in next kernel upload
- Basic configuration can be done with ip, but it really
- needs new tools - teamd, teamnl, etc.
+ New KMS drivers: should all work with current Xorg drivers
- Make it work: see
- http://bugs.debian.org/695850
+ Module signing: still not enabled, but probably will be if we
+ do Secure Boot
-
Transcendent memory [3.0-3.5]
+
Recap of last year's features (2)
- Abstract storage for memory pages, expected to be slower than
- regular memory but faster than disk
+ More support for discard: still not enabled at install time
+ (#690977)
+
+
+ More support for containers: XFS was fixed, and user namespaces
+ have been enabled
+
+
+ bcache: userland package (bcache-tools) still not quite ready
+ (#708132)
- Can provide a second layer of page cache (cleancache and frontswap)
+ ARMv7 multiplatform: d-i works on some platforms but
+ I'm still not sure which. Some progress on GPU drivers, but not
+ in Debian yet.
+
+
+
+
+
+
Unnamed temporary files [3.11]
+
+
+ Open directory with option O_TMPFILE to create an
+ unnamed temporary file on that filesystem
- Pages stored by hypervisor (Xen), compressed local memory
- (zcache) or cluster of machines (RAMster)
+ As with tmpfile(), the file disappears on
+ last close()
- Not yet enabled in Debian kernels, and needs some thought about
- configuration
+ File can be linked into the filesystem using
+ linkat(..., AT_EMPTY_PATH), allowing for 'atomic'
+ creation of file with complete contents and metadata
- Make it work: see
- https://lwn.net/Articles/454795/
- and send proposal to debian-kernel
+ Not supported on all filesystem types, so you will usually need
+ a fallback
-
New KMS drivers [3.3-3.10]
+
Network busy-polling [3.11] (1)
+
A conventional network request/response process looks like:
+
+
+
+ Task calls send(); network stack constructs a
+ packet; driver adds it to hardware Tx queue
+
+
+ Task calls poll() or recv(), which blocks;
+ kernel puts it to sleep and possibly idles the CPU
+
+
+ Network adapter receives response and generates IRQ, waking
+ up CPU
+
+
+ Driver's IRQ handler schedules polling of the hardware Rx
+ queue (NAPI)
+
+
+ Kernel runs the driver's NAPI poll function, which passes
+ the response packet into the network stack
+
+
+ Network stack decodes packet headers and adds packet to
+ the task's socket
+
+
+ Network stack wakes up sleeping task; scheduler switches
+ to it and the socket call returns
+
+
+
+
+
+
+
Network busy-polling [3.11] (2)
- DRM/KMS drivers added for old, new and virtual hardware -
- AST, DisplayLink, Hyper-V, Matrox G200, QEMU Cirrus
+ If driver supports busy-polling, it tags each packet with
+ the receiving NAPI context, and kernel tags sockets
+
+
+ When busy-polling is enabled, poll()
+ and recv() call the driver's busy poll function to
+ check for packets synchronously (up to some time limit)
+
+
+ If the response usually arrives quickly, this reduces overall
+ request/response latency as there are no context switches and
+ power transitions
+
+
+ Time limit set by sysctl (net.busy_poll,
+ net.busy_read) or socket option (SOL_SOCKET,
+ SO_BUSY_POLL); requires tuning
+
+
+
+
+
+
Lustre filesystem [3.12]
+
+
+ A distributed filesystem, popular for cluster computing
+ applications
- Should be more robust than purely user-mode drivers, and
- compatible with Secure Boot
+ Developed out-of-tree since 1999, but now added to Linux staging
+ directory
- Current X drivers don't work with these, so the kernel drivers
- are disabled for now
+ Was included in squeeze but dropped from wheezy as it didn't
+ support Linux 3.2
- Make it work: join the X Strike Force and package the new X
- drivers
+ Userland is now missing from Debian
-
Module signing [3.7]
+
Btrfs offline dedupe [3.12]
- Kernel modules can be signed at build time, and the kernel
- configured to refuse loading unsigned modules
+ Btrfs generally copies and frees blocks, rather than updating
+ in-place
+
+
+ This allows snapshots and file copies to copy-by-reference,
+ deferring the real copying until changes are made
- Necessary but not sufficient to implement Secure Boot -
- we would also need signed kernel images and some other
- restrictions when booted in this mode
+ Filesystems may still end up with multiple copies of the same
+ file content
- Make Secure Boot work: come to the meeting on Tuesday
+ Btrfs doesn't actively merge these duplicates, but userland can
+ tell it to do so
+
+
+ Many file dedupe tools are packaged for Debian, but not one that
+ works with this Btrfs feature, e.g. bedup
-
More support for discard
+
nftables [3.13]
- Flash devices (and thin-provisioned SANs) can be more efficient
- if the filesystem 'discards' unused disk space
+ Linux has several firewall APIs - iptables, ip6tables, arptables
+ and ebtables
+
+
+ All limited to single protocol, and need a kernel module for
+ each match type and each action
- Requires support in hardware, driver, filesystem and any layered
- device drivers - e.g. LVM, RAID (added in 3.7)
+ Kernel's internal netfilter API is more flexible
- Must be explicitly enabled, but d-i doesn't do this by default
+ nftables exposes more of this flexibility, allowing userland
+ to provide firewall code for a specialised VM (similar to BPF)
+ Eventually, old APIs will be removed and old userland
+ tools must be ported to use nftables
-
User namespaces [3.7]
+
User-space lockdep [3.14]
+
+
+ Kernel threads and interrupts all run in same address space,
+ using several different synchronisation mechanisms
+
+
+ Easy to introduce bugs that can result in deadlock, but hard to
+ reproduce them
+
+
+ Kernel's 'lockdep' system dynamically tracks locking operations
+ and detects potential deadlocks
+
+
+ Now available as a userland library! Except we need to package
+ it (build from linux-tools source package)
+
+
+
+
+
+
arm64 and ppc64el ports
- One of the last missing pieces for OpenVZ-like containers
+ 'arm64' architecture was added in Linux 3.7, but was not yet
+ usable, and no real hardware was available at the time
+
+
+ Upstream Linux arm64 kernel, and Debian packages, should now run
+ on emulators and real hardware
- Each user namespace has its own root user with
- privileges over the users and processes in that namespace - but
- not the whole system
+ 'powerpc' architecture has been available for many years,
+ but didn't support kernel running little-endian
- Currently somewhat experimental, and requires filesystem
- changes which haven't been done for XFS
+ Linux 3.13 added little-endian kernel support, along with new
+ userland ELF ABI variant - we call it ppc64el
- Make it work: send patches to upstream XFS developers (this
- one's hard)
+ Both ports now being bootstrapped in unstable and are candidates
+ for jessie release
-
bcache [3.10]
+
File-private locking [3.15]
- Turns a fast block device into a cache for a larger, slower
- device (see also: dm-cache, EnhanceIO)
+ POSIX says that closing a file descriptor removes
+ the process's locks on that file
+
+
+ What if process has multiple file descriptors for the same
+ file? It loses all locks obtained through any descriptor!
+
+
+ Multithreaded processes may require serialisation around
+ file open/close to ensure they open each file exactly once
- Needs its own set of userland tools
+ Hard and symbolic links can hide that two files are really the
+ same
- Make it work:
- see http://bugs.debian.org/708132
- (maybe just needs a sponsor)
+ Linux now provides file-private locks, associated with a
+ specific open file and removed when last descriptor for the
+ open file is closed
-
ARMv7 multiplatform
+
Multiqueue block devices [3.16]
- Until recently, each ARM kernel image could support only a small
- set of different chips
+ Each block device has a command queue (possibly shared with
+ other devices)
- Debian 'armmp' kernel now supports ARMv7 SoCs from Calxeda,
- Freescale and Marvell, and others should be supported soon
+ Queue may be partly implemented by hardware (NCQ) or only
+ in software
- Debian could run on a much larger range of ARM hardware - but we
- need installer and boot loader support to make this easy
+ A single queue means initiation is serialised and completion
+ involves IPI - can be bottleneck for fast devices
- Make it work: join the ARM porters and d-i team
+ High-end SSDs support multiple queues, but kernel needed changes
+ to use them
- Make the GPUs work: join a reverse-engineering project
+ mtip32xx driver now supports multiqueue, but SCSI
+ drivers don't yet - may be backport-able?