nbdkit-vddk-plugin - Man Page

nbdkit VMware VDDK plugin

Synopsis

 nbdkit vddk [file=]FILENAME
             [config=FILENAME] [cookie=COOKIE] [libdir=LIBRARY]
             [nfchostport=PORT] [single-link=true]
             [password=PASSWORD | password=- | password=+FILENAME
             | password=-FD]
             [port=PORT] [server=HOSTNAME] [snapshot=MOREF]
             [thumbprint=THUMBPRINT] [transports=MODE:MODE:...]
             [unbuffered=true] [user=USERNAME] [vm=moref=ID]
 nbdkit vddk --dump-plugin

Description

nbdkit-vddk-plugin is an nbdkit(1) plugin that serves files from local VMware VMDK files, VMware ESXi servers, VMware VCenter servers, and other sources.  It requires VMware's proprietary VDDK library that you must download yourself separately.

The plugin can serve read-only (if the -r option is used) or read/write.

Examples

Open a local VMDK file

 nbdkit vddk file=/absolute/path/to/file.vmdk

Note that when opening local files the file= parameter must be an absolute path.

Because VDDK needs to take a lock on this file, the file must be on a writable filesystem (unless you use the -r option).

Open a file on a remote VMware ESXi hypervisor

Connect directly to a VMware ESXi hypervisor and export a particular file:

 nbdkit vddk user=root password=+/tmp/rootpw \
             server=esxi.example.com thumbprint=xx:xx:xx:... \
             vm=moref=2 \
             file="[datastore1] Fedora/Fedora.vmdk"

user and password must be specified.  Use password=+FILENAME to provide the password securely in a file.

server is the hostname of the ESXi server.

thumbprint is the thumb print for validating the SSL certificate. How to find the thumb print of a server is described in “Thumbprints” below.

vm is the Managed Object Reference (“moref”) of the virtual machine.  See “Managed Object Reference” below.

file is the file you want to open, usually in the form "[datastore] vmname/vmname.vmdk".  See “File Parameter” below.

Open a file on a remote VMware vCenter server

Connect via VMware vCenter and export a particular file:

 nbdkit vddk user=root password=vmware \
             server=vcenter.example.com thumbprint=xx:xx:xx:... \
             vm=moref=vm-16 \
             file="[datastore1] Fedora/Fedora.vmdk"

user and password must be specified.  Use password=+FILENAME to provide the password securely in a file.

server is the hostname of the vCenter server.

thumbprint is the thumb print for validating the SSL certificate. How to find the thumb print of a server is described in “Thumbprints” below.

vm is the Managed Object Reference (“moref”) of the virtual machine.  See “Managed Object Reference” below.

file is the file you want to open, usually in the form "[datastore] vmname/vmname.vmdk".  See “File Parameter” below.

Parameters

All parameters are optional except:

file

is required for opening a local VMDK file.

file
server
thumbprint
user
password
vm

When making a remote connection you must supply all of these parameters.

config=FILENAME

The name of the VDDK configuration file.

cookie=COOKIE

Cookie from existing authenticated session on the host.

file=FILENAME
file=[datastore] vmname/vmname.vmdk

Set the name of the VMDK file to serve.

For local files you must supply an absolute path. For remote files see “File Parameter” section below.

If a VM has multiple disks, nbdkit can only serve one at a time.  To serve more than one you must run multiple copies of nbdkit.  (See “Notes” below).

file= is a magic config key and may be omitted in most cases. See “Magic parameters” in nbdkit(1).

libdir=PATHNAME

This sets the path of the VMware VDDK distribution.

VDDK uses this to load its own plugins, if this path is unspecified or wrong then VDDK will work with reduced functionality.

If the parameter is not given, then a hard-coded path determined at compile time is used, see “Dump-Plugin Output” below.

nfchostport=PORT

Port used to establish an NFC connection to ESXi.  Defaults to 902.

password=PASSWORD

Set the password to use when connecting to the remote server.

Note that passing this on the command line is not secure on shared machines.

password=-

Ask for the password (interactively) when nbdkit starts up.

password=+FILENAME

Read the password from the named file.  This is a secure method to supply a password, as long as you set the permissions on the file appropriately.

password=-FD

Read the password from file descriptor number FD, inherited from the parent process when nbdkit starts up.  This is also a secure method to supply a password.

port=PORT

The port on the VCenter/ESXi host.  Defaults to 443.

server=HOSTNAME

The hostname or IP address of VCenter or ESXi host.

single-link=true

(nbdkit ≥ 1.12)

Open the current link, not the entire chain.  This corresponds to the VIXDISKLIB_FLAG_OPEN_SINGLE_LINK flag.

snapshot=MOREF

The Managed Object Reference of the snapshot. See “Managed Object Reference” below.

thumbprint=THUMBPRINT

The SSL (SHA1) thumbprint for validating the SSL certificate.

The format is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx (20 hex digit pairs).

See “Thumbprints” below for how to get this.

transports=MODE:MODE:...

List of one or more transport modes to use.  Possible values include ‘nbd’, ‘nbdssl’, ‘san’, ‘hotadd’, ‘file’ (there may be others).  If not given, VDDK will try to choose the best transport mode.

unbuffered=true

(nbdkit ≥ 1.12)

Disable host caching.  This corresponds to the VIXDISKLIB_FLAG_OPEN_UNBUFFERED flag.

user=USERNAME

The username to connect to the remote server as.

vm=moref=ID

The Managed Object Reference (“moref”) of the virtual machine. See “Managed Object Reference” below.

vimapiver=APIVER

This parameter is ignored for backwards compatibility.

Library and Config File Locations

The VDDK library should not be placed on a system library path such as /usr/lib.  The reason for this is that the VDDK library is shipped with recompiled libraries like libcrypto.so and libstdc++.so that can conflict with system libraries.

You have two choices:

VDDK itself looks in a few default locations for the optional configuration file, usually including /etc/vmware/config and $HOME/.vmware/config, but you can override this using the config parameter.

No need to set LD_LIBRARY_PATH

In nbdkit ≤ 1.16 you had to set the environment variable LD_LIBRARY_PATH when using this plugin.  In nbdkit ≥ 1.18 this is not recommended.

File Parameter

The file parameter can either be a local file, in which case it must be the absolute path.  Or it can refer to a remote file on the VMware server in the format "[datastore] vmname/vmname.vmdk".

For remote files you can find the path using virsh(1).  For ESXi:

 virsh -c 'esx://esxi.example.com?no_verify=1' dumpxml guestname

For vCenter:

 virsh -c 'vpx://vcenter.example.com/Datacenter/esxi.example.com?no_verify=1' \
       dumpxml guestname

Thumbprints

The thumbprint is a 20 byte string containing the SSL (SHA1) fingerprint of the remote VMware server and it is required when making a remote connection.  There are two ways to obtain this.

Extracting thumbprint from ESXi or vCenter server

To extract the thumbprint, log in to the ESXi hypervisor shell and run this command:

 # openssl x509 -in /etc/vmware/ssl/rui.crt -fingerprint -sha1 -noout

For VMware vCenter servers the thumbprint is printed on the text console of the server or is available by logging in to the server and using this command:

 # openssl x509 -in /etc/vmware-vpx/ssl/rui.crt -fingerprint -sha1 -noout

Trick: Get VDDK to tell you the thumbprint

Another (easier) way to get the thumbprint of a server is to connect to the server using a bogus thumbprint with debugging enabled:

 nbdkit -f -v vddk server=esxi.example.com [...] thumbprint=12
 qemu-img info nbd:localhost:10809

The nbdkit process will try to connect (and fail because the thumbprint is wrong).  However in the debug output will be a message such as this:

 nbdkit: debug: VixDiskLibVim: Failed to verify SSL certificate: actual thumbprint=B2:31:BD:DE:9F:DB:9D:E0:78:EF:30:42:8A:41:B0:28:92:93:C8:DD expected=12

This gives you the server’s real thumbprint.  Of course this method is not secure since it allows a Man-in-the-Middle (MITM) attack.

Managed Object Reference

Some parameters require you to pass in the Managed Object Reference (“moref”) of an object on the VMware server.

For VMware ESXi hypervisors, the vm moref is a number (eg. vm=moref=2).  For VMware VCenter it is a string beginning with "vm-") (eg. vm=moref=vm-16).  Across ESXi and vCenter the numbers are different even for the same virtual machine.

If you have libvirt ≥ 3.7, the moref is available in the virsh(1) dumpxml output:

 $ virsh -c 'esx://esxi.example.com?no_verify=1' dumpxml guestname
 ...
 <vmware:moref>2</vmware:moref>
 ...

or:

 $ virsh -c 'vpx://vcenter.example.com/Datacenter/esxi.example.com?no_verify=1' \
       dumpxml guestname
 ...
 <vmware:moref>vm-16</vmware:moref>
 ...

An alternative way to find the moref of a VM is using the moRefFinder.pl script written by William Lam (http://www.virtuallyghetto.com/2011/11/vsphere-moref-managed-object-reference.html https://blogs.vmware.com/vsphere/2012/02/uniquely-identifying-virtual-machines-in-vsphere-and-vcloud-part-2-technical.html).

Dump-Plugin Output

To query more information about the plugin (and whether it is working), use:

 nbdkit vddk --dump-plugin

If the plugin is not present, not working or the library path is wrong you will get an error.

If it works the output will include:

vddk_default_libdir=...

The compiled-in library path.  Use libdir=PATHNAME to override this at runtime.

vddk_has_nfchostport=1

If this is printed then the nfchostport=PORT parameter is supported by this build.

vddk_dll=...

Prints the full path to the VDDK shared library.  Since this requires a glibc extension it may not be available in all builds of the plugin.

Notes

Sector size limitation

The VDDK plugin can only answer read/write requests on whole 512 byte sector boundaries.  This is because the VDDK Read and Write APIs only take sector numbers.  If your client needs finer granularity, you can use nbdkit-blocksize-filter(3) with the setting minblock=512.

Threads

Handling threads in the VDDK API is complex and does not map well to any of the thread models offered by nbdkit (see “THREADS” in nbdkit-plugin(3)).  The plugin uses the nbdkit SERIALIZE_ALL_REQUESTS model, but technically even this is not completely safe.  This is a subject of future work.

Out of memory errors

In the verbose log you may see errors like:

 nbdkit: vddk[3]: error: [NFC ERROR] NfcFssrvrProcessErrorMsg:
 received NFC error 5 from server: Failed to allocate the
 requested 2097176 bytes

This seems especially common when there are multiple parallel connections open to the VMware server.

These can be caused by resource limits set on the VMware server.  You can increase the limit for the NFC service by editing /etc/vmware/hostd/config.xml and adjusting the <maxMemory> setting:

 <nfcsvc>
   <path>libnfcsvc.so</path>
   <enabled>true</enabled>
   <maxMemory>50331648</maxMemory>
   <maxStreamMemory>10485760</maxStreamMemory>
 </nfcsvc>

and restarting the hostd service:

 # /etc/init.d/hostd restart

For more information see https://bugzilla.redhat.com/1614276.

Supported Versions of VDDK

This plugin requires VDDK ≥ 5.5.5, which in turn means that it is only supported on x64-64 platforms.

It has been tested with all versions up to 6.7 (but should work with future versions).

VDDK ≥ 6.0 should be used if possible.  This is the first version which added Flush support which is crucial for data integrity when writing.

Debug Flags

Debugging messages can be very helpful if you have problems connecting to VMware servers, or to find the list of available transport modes, or to diagnose SAN problems:

 nbdkit -f -v vddk file=FILENAME [...]

Additional debug flags are available:

-D vddk.diskinfo=1

Debug disk information returned by GetInfo.

-D vddk.extents=1

Debug extents returned by QueryAllocatedBlocks.

-D vddk.datapath=0

Suppress debugging of datapath calls (Read and Write).

Files

$plugindir/nbdkit-vddk-plugin.so

The plugin.

Use nbdkit --dump-config to find the location of $plugindir.

Version

nbdkit-vddk-plugin first appeared in nbdkit 1.2.

See Also

nbdkit(1), nbdkit-plugin(3), nbdkit-blocksize-filter(1), nbdkit-readahead-filter(1), nbdkit-retry-filter(1), virsh(1), https://www.vmware.com/support/developer/vddk/

Authors

Richard W.M. Jones

License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Referenced By

nbdkit(1), nbdkit-blocksize-filter(1), nbdkit-cacheextents-filter(1), nbdkit-plugin(3), nbdkit-readahead-filter(1), nbdkit-release-notes-1.12(1), nbdkit-release-notes-1.14(1), nbdkit-release-notes-1.16(1), nbdkit-release-notes-1.18(1), nbdkit-release-notes-1.20(1), nbdkit-release-notes-1.6(1), nbdkit-release-notes-1.8(1), virt-v2v(1), virt-v2v-input-vmware(1).

2020-06-04 nbdkit-1.21.7