HomeFreeBSD

Optionally bind ktls threads to NUMA domains

Description

Optionally bind ktls threads to NUMA domains

When ktls_bind_thread is 2, we pick a ktls worker thread that is
bound to the same domain as the TCP connection associated with
the socket. We use roughly the same code as netinet/tcp_hpts.c to
do this. This allows crypto to run on the same domain as the TCP
connection is associated with. Assuming TCP_REUSPORT_LB_NUMA
(D21636) is in place & in use, this ensures that the crypto source
and destination buffers are local to the same NUMA domain as we're
running crypto on.

This change (when TCP_REUSPORT_LB_NUMA, D21636, is used) reduces
cross-domain traffic from over 37% down to about 13% as measured
by pcm.x on a dual-socket Xeon using nginx and a Netflix workload.

Reviewed by: jhb
Sponsored by: Netflix
Differential Revision: https://reviews.freebsd.org/D21648

Details

Provenance
gallatinAuthored on
Reviewer
jhb
Differential Revision
D21648: ktls: NUMA awareness
Parents
rS368817: libc: Fix most issues reported by mandoc
Branches
Unknown
Tags
Unknown