nbdkit [--tls=off|on|require] [--tls-certificates=/path/to/certificates] [--tls-psk=/path/to/pskfile] [--tls-verify-peer] PLUGIN [...]
TLS (authentication and encryption, sometimes incorrectly called "SSL") is supported if nbdkit was compiled with GnuTLS. This allows the server to verify that the client is allowed access, and to encrypt the contents of the protocol in transit over the network.
TLS can be disabled or enabled by specifying either --tls=off or --tls=on. With --tls=off, if a client tries to use TLS to connect, it will be rejected by the server (in other words, as if the server doesn't support TLS).
--tls=on means that the client may choose to connect either with or without TLS.
--tls=require enables TLS and rejects all non-TLS connection attempts. This prevents downgrade attacks where a malicious proxy pretends not to support TLS in order to force either the client or server to communicate in plaintext.
If certificates have been set up correctly then you should be able to start a TLS server by doing:
nbdkit --tls=require memory 1G
and connect to it by doing:
If certificates are in a non-standard directory and you have libnbd ≥ 1.10:
nbdkit --tls=require --tls-certificates=/certs memory 1G nbdinfo nbds://localhost?tls-certificates=/certs
TLS with X.509 certificates
When nbdkit starts up, it loads TLS certificates from some built-in paths, or from the directory specified by the --tls-certificates option.
In this directory, nbdkit expects to find several files:
The Certificate Authority certificate.
The server certificate.
The server private key.
(Optional) The certificate revocation list.
Setting up the Certificate Authority
This step only needs to be done once per organization. It may be that your organization already has a CA.
$ certtool --generate-privkey > ca-key.pem $ chmod 0600 ca-key.pem
The ca-key.pem file is the CA private key and is extremely sensitive data. With possession of this key, anyone can create certificates pretending to be your organization!
To create the CA certificate file:
$ cat > ca.info <<EOF cn = Name of your organization ca cert_signing_key EOF $ certtool --generate-self-signed \ --load-privkey ca-key.pem \ --template ca.info \ --outfile ca-cert.pem
Issuing a server certificate for the nbdkit server
Each nbdkit server (or host) needs a secret key and certificate.
$ certtool --generate-privkey > server-key.pem $ chmod 0600 server-key.pem
The server key file is sensitive. Setting the mode to
0600 helps to prevent other users on the same machine from reading it.
The common name (
cn below) field must be the fully qualified hostname that the client connects to. However most clients and servers including nbdkit support the Subject Alternative Name extension (RFC 2818) which uses the
ip_address fields and deprecates
$ cat > server.info <<EOF organization = Name of your organization cn = nbd-server.example.com dns_name = nbd-server dns_name = nbd-server.example.com ip_address = 10.0.1.2 ip_address = 2001:24::92 tls_www_server encryption_key signing_key EOF $ certtool --generate-certificate \ --load-ca-certificate ca-cert.pem \ --load-ca-privkey ca-key.pem \ --load-privkey server-key.pem \ --template server.info \ --outfile server-cert.pem
Issuing and checking client certificates
Note: You don't need to create client certificates unless you want to check and limit which clients can connect to nbdkit. nbdkit does not check client certificates unless you specify the --tls-verify-peer option on the command line. There are other methods for limiting access to nbdkit including nbdkit-ip-filter(1).
For each client you should generate a private key and a client certificate:
$ certtool --generate-privkey > client-key.pem $ chmod 0600 client-key.pem
The client key file is sensitive.
The client DNS name (
cn below) is the client's name that nbdkit sees and checks.
$ cat > client.info <<EOF country = US state = New York locality = New York organization = Name of your organization cn = client.example.com tls_www_client encryption_key signing_key EOF $ certtool --generate-certificate \ --load-ca-certificate ca-cert.pem \ --load-ca-privkey ca-key.pem \ --load-privkey client-key.pem \ --template client.info \ --outfile client-cert.pem
Client certificates do not need to be present anywhere on the nbdkit host. You don't need to copy them into nbdkit's TLS certificates directory. The security comes from the fact that the client must present a client certificate signed by the Certificate Authority, and nbdkit can check this because it has the ca-cert.pem file.
To enable checking of client certificates, specify the --tls-verify-peer option on the command line. Clients which don't present a valid certificate (eg. not signed, incorrect signature) are denied. Also denied are clients which present a valid certificate signed by another CA. Also denied are clients with certificates added to the certificate revocation list (ca-crl.pem).
Connecting nbd-client to nbdkit with TLS certificates
With the TLS certificates files generated above in the current directory (
.) you can use:
nbdkit --tls=require --tls-certificates=. --tls-verify-peer memory 1G nbd-client /dev/nbd0 \ -cacertfile ca-cert.pem \ -certfile client-cert.pem \ -keyfile client-key.pem
--tls-verify-peer is only required if you want to check the client certificate. If you want to allow any client to connect then you can omit it.
Default TLS behaviour
If nbdkit was compiled without GnuTLS support, then TLS is disabled and TLS connections will be rejected (as if --tls=off was specified on the command line). Also it is impossible to turn on TLS in this scenario. You can tell if nbdkit was compiled without GnuTLS support because
nbdkit --dump-config will contain
If TLS certificates cannot be loaded either from the built-in path or from the directory specified by --tls-certificates, then TLS defaults to disabled. Turning TLS on will give a warning (--tls=on) or error (--tls=require) about the missing certificates.
If TLS certificates can be loaded from the built-in path or from the --tls-certificates directory, then TLS will by default be enabled (like --tls=on), but it is not required. Clients can choose whether or not to use TLS and whether or not to present certificates.
TLS client certificates are not checked by default unless you specify --tls-verify-peer.
If the --tls-psk option is used then TLS is enabled (but not required). To ensure that all clients are authorized you must use --tls=require.
Each of these defaults is insecure to some extent (including --tls=on which could be subject to a downgrade attack). If you expect TLS then it is best to specify --tls=require, and if you want to check client certificates, additionally use the --tls-verify-peer option.
Controlling TLS fallback to plaintext
When --tls=on is used, the connection can fall back to plaintext. You can use nbdkit-tls-fallback-filter(1) to provide safe fallback content to plaintext connections. With this filter the underlying plugin content is only served on secure connections.
Alternatively a plugin may wish to serve different content depending on whether the client is using TLS. The function
nbdkit_is_tls() can be used during the
.open callback for that purpose.
NBD URIs for TLS
Tools such nbdcopy(1), nbdinfo(1) and nbdsh(1) (from libnbd(3)) allow you to use
nbds+unix:// URIs to connect to nbdkit servers using TLS.
The syntax is fully documented in the NBD URI specification: https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md. This section contains an outline. You can also find further examples in nbd_connect_uri(3).
Connect over TCP with TLS, to
example.comport 10809. If the server does not support TLS then this will fail.
As above, but connect over a Unix domain socket called SOCKET.
As above, but specify the directory DIR containing TLS certificates (used by the client to verify the server, and to present client authentication to the server). Note this requires libnbd ≥ 1.10.
As above, but use TLS with Pre-Shared Keys (PSK), stored in the secrets file FILENAME.
As above, but use
aliceas the username.
Default location of certificates
Without --tls-certificates nbdkit and libnbd look in several locations for certificates.
If nbdkit is started as a non-root user (note this does not include use of the -u or -g options), nbdkit looks in each of these paths in turn:
If nbdkit is started as root:
$sysconfdir is set when nbdkit is compiled, usually /etc. (Use
nbdkit --dump-config and look at the
root_tls_certificates_dir setting to get the actual directory built into the binary.)
In libnbd the paths are different. For non-root:
In nbdkit you can override these directories by using --tls-certificates=/path/to/certificates.
In libnbd you can use nbd_set_tls_certificates(3). In libnbd ≥ 1.10 you can append
&tls-certificates=/path/to/certificates to URIs.
Choice of TLS algorithms
TLS has a bewildering choice of algorithms that can be used. To enable you to choose a default set of algorithms, there is a configure setting --with-tls-priority. This defaults to
NORMAL which, to quote the GnuTLS documentation:
NORMAL means all
secure ciphersuites. The 256-bit ciphers are included as a fallback only. The ciphers are sorted by security margin."
You could also set the TLS priority so that it can be configured from a file at runtime:
means use the policy from /etc/crypto-policies/config.
means use the policy from /etc/crypto-policies/local.d/nbdkit.config and fall back to /etc/crypto-policies/config if the first file does not exist.
More information can be found in gnutls_priority_init(3).
Debugging TLS connections
Encrypted connections makes snooping on network traffic with Wireshark impossible, by design.
The TLS library used by nbdkit, called gnutls, supports the
SSLKEYLOGFILE standard: https://web.archive.org/web/20200118013150/https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS/Key_Log_Format
Before running nbdkit set the environment variable to point to a log file (note it will append to the file if it already exists):
SSLKEYLOGFILE=/tmp/keylog nbdkit [...]
In Wireshark go to Edit → Preferences → Protocols → TLS and set (Pre)-Master-Secret log filename to the log file name. Wireshark should be able to see the unencrypted traffic. For further information read: https://wiki.wireshark.org/TLS#using-the-pre-master-secret
nbdkit(1), nbdkit-luks-filter(1), nbdkit-tls-fallback-filter(1), nbdcopy(1), nbdfuse(1), nbdinfo(1), nbdsh(1), nbd_connect_uri(3), nbd_set_tls(3), nbd_set_tls_certificates(3), gnutls_priority_init(3), psktool(1), https://github.com/NetworkBlockDevice/nbd/blob/master/doc/proto.md, https://github.com/NetworkBlockDevice/nbd/blob/master/doc/uri.md, https://nbd.sourceforge.io/.
Richard W.M. Jones
Copyright Red Hat
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
- Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
- Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
- Neither the name of Red Hat nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
libnbd(3), nbdkit(1), nbdkit-luks-filter(1), nbdkit-nbd-plugin(1), nbdkit-ondemand-plugin(1), nbdkit-plugin(3), nbdkit-tls-fallback-filter(1), nbdkit-tmpdisk-plugin(1).