nbdkit-python-plugin - Man Page

nbdkit python plugin

Synopsis

 nbdkit python /path/to/plugin.py [arguments...]

Description

nbdkit-python-plugin is an embedded Python interpreter for nbdkit(1), allowing you to write nbdkit plugins in Python.

If you have been given an nbdkit Python plugin

Assuming you have a Python script which is an nbdkit plugin, you run it like this:

 nbdkit python /path/to/plugin.py

You may have to add further key=value arguments to the command line.  Read the Python script to see if it requires any.

Writing a Python Nbdkit Plugin

For an example plugin written in Python, see: https://github.com/libguestfs/nbdkit/blob/master/plugins/python/example.py

Broadly speaking, Python nbdkit plugins work like C ones, so you should read nbdkit-plugin(3) first.

To write a Python nbdkit plugin, you create a Python file which contains at least the following required functions (in the top level __main__ module):

 API_VERSION = 2
 def open(readonly):
   # see below
 def get_size(h):
   # see below
 def pread(h, buf, offset, flags):
   # see below

Note that the subroutines must have those literal names (like open), because the C part looks up and calls those functions directly.  You may want to include documentation and globals (eg. for storing global state).  Any other top level statements are run when the script is loaded, just like ordinary Python.

Python versions

In nbdkit ≤ 1.14, either Python 2 or 3 could be used.  It was selected at compile time by either:

 ./configure

which selected the version of Python by looking at the python interpreter found on the $PATH.  Or:

 ./configure PYTHON=/usr/bin/python3

which allowed you to select a different interpreter and hence a different version of Python.

nbdkit ≥ 1.16 drops all support for Python 2, since Python 2 has reached its end of life.

The new behaviour is that ./configure looks for python3 or python (in that order) on the $PATH.  It will fail if the first interpreter it finds is a Python 2 interpreter.  You may also still choose a Python interpreter by setting the PYTHON variable at configure time as above.

If you wish to continue using nbdkit plugins written in Python 2 then you must use nbdkit ≤ 1.14, but we would advise you to update your plugins.

To find out which version the Python plugin was compiled for, use the --dump-plugin option, eg:

 $ nbdkit python --dump-plugin
 ...
 python_version=3.7.0
 python_pep_384_abi_version=3

API versions

The nbdkit API has evolved and new versions are released periodically. To ensure backwards compatibility plugins have to opt in to the new version.  From Python you do this by declaring a constant in your module:

 API_VERSION = 2

(where 2 is the latest version at the time this documentation was written).  All newly written Python modules must have this constant.

Executable script

If you want you can make the script executable and include a “shebang” at the top:

 #!/usr/sbin/nbdkit python

See also “Shebang scripts” in nbdkit(1).

These scripts can also be installed in the $plugindir.  See “WRITING PLUGINS IN OTHER PROGRAMMING LANGUAGES” in nbdkit-plugin(3).

Methods

Your script may use import nbdkit to have access to the following methods in the nbdkit module:

 nbdkit.set_error(err)

Record err as the reason you are about to throw an exception. err should correspond to usual errno values, where it may help to import errno.

Exceptions

Python callbacks should throw exceptions to indicate errors.  Remember to use nbdkit.set_error if you need to control which error is sent back to the client; if omitted, the client will see an error of EIO.

Python callbacks

This just documents the arguments to the callbacks in Python, and any way that they differ from the C callbacks.  In all other respects they work the same way as the C callbacks, so you should go and read nbdkit-plugin(3).

dump_plugin

(Optional)

There are no arguments or return value.

config

(Optional)

 def config(key, value):
   # no return value
config_complete

(Optional)

There are no arguments or return value.

get_ready

(Optional)

There are no arguments or return value.

open

(Required)

 def open(readonly):
   # return handle

You can return any non-NULL Python value as the handle.  It is passed back in subsequent calls.

close

(Optional)

 def close(h):
   # no return value

After close returns, the reference count of the handle is decremented in the C part, which usually means that the handle and its contents will be garbage collected.

get_size

(Required)

 def get_size(h):
   # return the size of the disk
is_rotational

(Optional)

 def is_rotational(h):
   # return a boolean
can_multi_conn

(Optional)

 def can_multi_conn(h):
   # return a boolean
can_write

(Optional)

 def can_write(h):
   # return a boolean
can_flush

(Optional)

 def can_flush(h):
   # return a boolean
can_trim

(Optional)

 def can_trim(h):
   # return a boolean
can_zero

(Optional)

 def can_zero(h):
   # return a boolean
can_fast_zero

(Optional)

 def can_fast_zero(h):
   # return a boolean
can_fua

(Optional)

 def can_fua(h):
   # return nbdkit.FUA_NONE or nbdkit.FUA_EMULATE
   # or nbdkit.FUA_NATIVE
can_cache

(Optional)

 def can_cache(h):
   # return nbdkit.CACHE_NONE or nbdkit.CACHE_EMULATE
   # or nbdkit.CACHE_NATIVE
pread

(Required)

 def pread(h, buf, offset, flags):
   # read into the buffer

The body of your pread function should read exactly len(buf) bytes of data starting at disk offset and write it into the buffer buf.  flags is always 0.

NBD only supports whole reads, so your function should try to read the whole region (perhaps requiring a loop).  If the read fails or is partial, your function should throw an exception, optionally using nbdkit.set_error first.

pwrite

(Optional)

 def pwrite(h, buf, offset, flags):
   length = len (buf)
   # no return value

The body of your pwrite function should write the buffer buf to the disk.  You should write count bytes to the disk starting at offset.  flags may contain nbdkit.FLAG_FUA.

NBD only supports whole writes, so your function should try to write the whole region (perhaps requiring a loop).  If the write fails or is partial, your function should throw an exception,
optionally using nbdkit.set_error first.

flush

(Optional)

 def flush(h, flags):
   # no return value

The body of your flush function should do a sync(2) or fdatasync(2) or equivalent on the backing store. flags is always 0.

If the flush fails, your function should throw an exception, optionally using nbdkit.set_error first.

trim

(Optional)

 def trim(h, count, offset, flags):
   # no return value

The body of your trim function should “punch a hole” in the backing store.  flags may contain nbdkit.FLAG_FUA.  If the trim fails, your function should throw an exception, optionally using nbdkit.set_error first.

zero

(Optional)

 def zero(h, count, offset, flags):
   # no return value

The body of your zero function should ensure that count bytes of the disk, starting at offset, will read back as zero.  flags is a bitmask which may include nbdkit.FLAG_MAY_TRIM, nbdkit.FLAG_FUA, nbdkit.FLAG_FAST_ZERO.

NBD only supports whole writes, so your function should try to write the whole region (perhaps requiring a loop).

If the write fails or is partial, your function should throw an exception, optionally using nbdkit.set_error first.  In particular, if you would like to automatically fall back to pwrite (perhaps because there is nothing to optimize if flags & nbdkit.FLAG_MAY_TRIM is false), use nbdkit.set_error (errno.EOPNOTSUPP).

cache

(Optional)

 def cache(h, count, offset, flags):
   # no return value

The body of your cache function should prefetch data in the indicated range.

If the cache operation fails, your function should throw an exception, optionally using nbdkit.set_error first.

Missing callbacks

Missing: load and unload

These are not needed because you can just use ordinary Python constructs.

Missing: thread_model

See “Threads” below.

Missing: name, version, longname, description, config_help, magic_config_key, can_extents, extents.

These are not yet supported.

Threads

The thread model for Python callbacks currently cannot be set from Python.  It is hard-coded in the C part to NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS.  This may change or be settable in future.

Files

$plugindir/nbdkit-python-plugin.so

The plugin.

Use nbdkit --dump-config to find the location of $plugindir.

Version

nbdkit-python-plugin first appeared in nbdkit 1.2.

See Also

nbdkit(1), nbdkit-plugin(3), python(1).

Authors

Eric Blake

Richard W.M. Jones

Nir Soffer

License

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:

THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Referenced By

nbdkit(1), nbdkit-plugin(3), nbdkit-release-notes-1.12(1), nbdkit-release-notes-1.16(1), nbdkit-release-notes-1.18(1).

2020-05-10 nbdkit-1.21.3