nbdkit-curl-plugin - Man Page

nbdkit curl plugin (HTTP, FTP and other protocols)

Synopsis

 nbdkit -r curl [url=]http://example.com/disk.img

Description

nbdkit-curl-plugin is a plugin for nbdkit(1) which turns content served over HTTP, FTP, and more, into a Network Block Device.  It uses a library called libcurl (also known as cURL) to read data from URLs. The exact list of protocols that libcurl can handle depends on how it was compiled, but most versions will handle HTTP, HTTPS, FTP, FTPS and more (see: curl -V).

Note: This plugin supports writes.  However for HTTP, you may not want nbdkit to issue PUT requests to the remote server (which probably doesn't understand them).  To force nbdkit to use a readonly connection, pass the -r flag.

Although this plugin can access SFTP (ie. SSH) servers, it is much better to use nbdkit-ssh-plugin(1).

Example

 nbdkit -r curl http://example.com/disk.img

serves the remote disk image as NBD on TCP port 10809 (to control ports and protocols used to serve NBD see nbdkit(1)).

Parameters

cainfo=FILENAME

(nbdkit ≥ 1.18)

Configure CA bundle for libcurl. See CURLOPT_CAINFO(3) for details.

capath=PATH

(nbdkit ≥ 1.18)

Set CA certificates directory location for libcurl. See CURLOPT_CAPATH(3) for more information.

cookie=COOKIE
cookie=+FILENAME
cookie=-
cookie=-FD

(nbdkit ≥ 1.12)

Set a cookie in the request header when connecting to the remote server.

A typical example is:

 cookie='vmware_soap_session="52a01262-bf93-ccce-d379-8dabb3e55560"'

This option can be used at most once.  It only works for HTTP and HTTPS transports.  To set multiple cookies you must concatenate them yourself, eg:

 cookie='name1=content1; name2=content2;'

See CURLOPT_COOKIE(3) for more information about this.

If the cookie is used for authentication then passing it on the command line is not secure on shared machines.  Use the alternate +FILENAME syntax to pass it in a file, - to read the cookie interactively, or -FD to read it from a file descriptor.

password=PASSWORD

Set the password to use when connecting to the remote server.

Note that passing this on the command line is not secure on shared machines.

password=-

Ask for the password (interactively) when nbdkit starts up.

password=+FILENAME

Read the password from the named file.  This is a secure method to supply a password, as long as you set the permissions on the file appropriately.

password=-FD

Read the password from file descriptor number FD, inherited from the parent process when nbdkit starts up.  This is also a secure method to supply a password.

protocols=PROTO,PROTO,...

(nbdkit ≥ 1.12)

Limit the protocols that are allowed in the URL.  Use this option for extra security if the URL comes from an untrusted source and you want to avoid security isues in the more obscure protocols that curl supports.  (See qemu CVE-2013-0249 for an example of a security bug introduced by allowing unrestricted protocols).

For example if you only intend HTTP and HTTPS URLs to be used, then add this parameter: protocols=http,https

The value of this parameter is a comma-separated list of protocols. The following protocols are known: dict, file, ftp, ftps, gopher, http, https, imap, imaps, ldap, ldaps, pop3, pop3s, rtmp, rtmpe, rtmps, rtmpt, rtmpte, rtmpts, rtsp, scp, sftp, smb, smbs, smtp, smtps, telnet, tftp.

The default is to allow any protocol.

proxy=PROXY

(nbdkit ≥ 1.20)

Set the proxy.  See CURLOPT_PROXY(3).

proxy-password=PASSWORD
proxy-password=-
proxy-password=+FILENAME
proxy-password=-FD
proxy-user=USERNAME

(nbdkit ≥ 1.12)

Set the proxy username and password.

sslverify=false

Don't verify the SSL certificate of the remote host.

tcp-keepalive=true

(nbdkit ≥ 1.20)

Enable TCP keepalives.

tcp-nodelay=false

(nbdkit ≥ 1.20)

Disable Nagle’s algorithm.

timeout=SECS

Set the timeout for requests.

timeout=0

Use the default libcurl timeout for requests.

unix-socket-path=PATH

(nbdkit ≥ 1.10)

Instead of using a TCP connection, connect to the server over the named Unix domain socket.  See CURLOPT_UNIX_SOCKET_PATH(3).

[url=]URL

The URL of the remote disk image.  This is passed to libcurl directly via CURLOPT_URL(3).

This parameter is required.

url= is a magic config key and may be omitted in most cases. See “Magic parameters” in nbdkit(1).

user=USERNAME

Set the username to use when connecting to the remote server.  This may also be set in the URL (eg. http://foo@example.com/disk.img)

user-agent=USER-AGENT

(nbdkit ≥ 1.22)

Send user-agent header when using HTTP or HTTPS.  The default is no user-agent header.

Debug Flag

-D curl.verbose=1

This enables very verbose curl debugging.  See CURLOPT_VERBOSE(3). This is mainly useful if you suspect there is a bug inside libcurl itself.

Files

$plugindir/nbdkit-curl-plugin.so

The plugin.

Use nbdkit --dump-config to find the location of $plugindir.

Version

nbdkit-curl-plugin first appeared in nbdkit 1.2.

See Also

curl(1), libcurl(3), CURLOPT_CAINFO(3), CURLOPT_CAPATH(3), CURLOPT_COOKIE(3), CURLOPT_PROXY(3), CURLOPT_TCP_KEEPALIVE(3), CURLOPT_TCP_NODELAY(3), CURLOPT_URL(3), CURLOPT_UNIX_SOCKET_PATH(3), CURLOPT_USERAGENT(3), CURLOPT_VERBOSE(3), nbdkit(1), nbdkit-extentlist-filter(1), nbdkit-readahead-filter(1), nbdkit-retry-filter(1), nbdkit-ssh-plugin(1), nbdkit-torrent-plugin(1), nbdkit-plugin(3), http://curl.haxx.se.

Authors

Richard W.M. Jones

Parts derived from Alexander Graf's “QEMU Block driver for CURL images”.

License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Referenced By

nbdkit(1), nbdkit-extentlist-filter(1), nbdkit-loop(1), nbdkit-plugin(3), nbdkit-readahead-filter(1), nbdkit-release-notes-1.10(1), nbdkit-release-notes-1.12(1), nbdkit-release-notes-1.18(1), nbdkit-release-notes-1.20(1), nbdkit-ssh-plugin(1), nbdkit-tar-plugin(1), nbdkit-torrent-plugin(1), nbdkit-xz-filter(1), virt-v2v-release-notes-1.42(1).

2020-06-29 nbdkit-1.21.13