lttng-export-maps - Man Page

Export the counter values of an LTTng recording session as SQL

Synopsis

lttng [GENERAL OPTIONS] export-maps [--session=SESSION] [--format=sqlite]

Description

The lttng export-maps command exports the current counter values of all the map channels (see lttng-add-map-channel(1)) of:

With the --session=SESSION option

The recording session named SESSION.

Without the --session option

The current recording session (see lttng-concepts(7) to learn more about the current recording session).

See lttng-concepts(7) to learn more about recording sessions, map channels, and counters.

Unlike lttng-show-maps(1), which offers a fixed, human-readable presentation with built-in filtering and aggregation, lttng export-maps performs no filtering and no aggregation: it exports every counter of every map channel as a self-contained SQL script for SQLite. Slice, filter, and aggregate the data afterwards with the full power of SQLite.

The default and only output format as of LTTng 2.16.0 is sqlite (see the --format option): an SQL script for the SQLite engine which creates a few tables and a vmap convenience view, and then inserts every counter within a single transaction. Pipe the script into sqlite3 to query the data, for example using an in-memory database:

$ { lttng export-maps; echo 'SELECT * FROM vmap;' } | sqlite3 :memory:

See the “SQL Schema” section below for the schema details and the “Examples” section below for a few query examples.
Important

The lttng export-maps command does NOT provide an atomic snapshot of the counter values: the connected session daemon (see lttng-sessiond(8)) reads the values one by one, and LTTng tracers may concurrently update any of them while the command runs.

Therefore, lttng export-maps does NOT guarantee that its output reflects the state of the counters at a single instant.

To obtain a consistent view of the counter values, LTTng recommends to either:

Stop the recording session with the lttng-stop(1) command before running lttng export-maps.
Disable the recording event rules (see lttng-disable-event(1)) and/or remove the triggers (see lttng-remove-trigger(1)) having an “increment map value” action which targets the map channel to inspect.

Moreover, lttng export-maps doesn’t guarantee the order in which it writes its INSERT INTO statements; don’t rely on it.

SQL Schema

With --format=sqlite (the default), the command writes the following schema, followed by one INSERT INTO statement per row, all wrapped in a single BEGIN/COMMIT transaction:

CREATE TABLE channels (
  id   INTEGER PRIMARY KEY,
  name TEXT NOT NULL,
  type TEXT NOT NULL CHECK (type IN ('kernel', 'user')),
  UNIQUE (name, type)
);

CREATE TABLE groups (
  id          INTEGER PRIMARY KEY,
  channel_id  INTEGER NOT NULL REFERENCES channels(id),
  type        TEXT NOT NULL
              CHECK (type IN ('kernel-global', 'user-per-user',
                              'user-per-process', 'shared')),
  owner_id    INTEGER,
  owner_name  TEXT,
  value_type  TEXT NOT NULL
              CHECK (value_type IN ('signed-int-32', 'signed-int-64')),
  CHECK ((owner_id IS NULL) =
         (type NOT IN ('user-per-user', 'user-per-process')))
);

CREATE TABLE keys (
  id   INTEGER PRIMARY KEY,
  name TEXT NOT NULL UNIQUE
);

CREATE TABLE entries (
  group_id     INTEGER NOT NULL REFERENCES groups(id),
  part_id      INTEGER,
  key_id       INTEGER NOT NULL REFERENCES keys(id),
  value        INTEGER NOT NULL,
  has_overflow INTEGER NOT NULL CHECK (has_overflow IN (0, 1))
);

CREATE VIEW vmap AS
SELECT c.name AS channel_name,
       c.type AS channel_type,
       g.type AS group_type,
       g.owner_id,
       g.owner_name,
       g.value_type,
       e.part_id,
       k.name AS key,
       e.value,
       e.has_overflow
FROM entries e
JOIN groups g   ON e.group_id   = g.id
JOIN channels c ON g.channel_id = c.id
JOIN keys k     ON e.key_id     = k.id;

The numeric id columns are arbitrary surrogate keys which only serve to join the tables; don’t rely on their specific values nor on the row order of the INSERT INTO statements.

Most queries only need the vmap view, of which the columns are:

channel_name

Map channel name.

channel_type

Map channel type: kernel or user.

group_type

Counter group type:

kernel-global: System-wide Linux kernel counters.
user-per-user: User space counters of a single Unix user; owner_id is the Unix user ID.
user-per-process: User space counters of a single process; owner_id is the process ID.
shared: User space map channel-wide shared counters (see lttng-concepts(7)).

owner_id

Unix user ID (user-per-user group type) or process ID (user-per-process group type), or NULL for the kernel-global and shared group types.

owner_name

Human-readable name of the owner: a Unix user name (user-per-user group type) or a process/command name (user-per-process group type).

This column is informational only: LTTng makes no guarantee that the name is set (it may be NULL), that it’s unique amongst the groups of a map channel, or that it remains stable across versions.

Always use owner_id (together with group_type) to identify an owner reliably.

Always NULL for the kernel-global and shared group types.

value_type

Effective value type of the group: signed-int-32 or signed-int-64.

part_id

0-based CPU ID, or NULL for the shared group type (the shared counters have no per-partition decomposition).

key: Counter key (a string as of LTTng 2.16.0).
value: Counter value.

has_overflow

1 if at least one update of the counter wrapped around (modular arithmetic) because the result couldn’t be represented with the effective value type of the map group (see the --value-type option of lttng-add-map-channel(1)), or 0 otherwise.

Note

The schema is stable for a given format name.

Any future incompatible change to the schema would introduce a new --format value so that existing consumers keep working.

Options

See lttng(1) for GENERAL OPTIONS.

Recording target

-s SESSION, --session=SESSION: Export the counter values of the recording session named SESSION instead of the current recording session.

Output

-f FORMAT, --format=FORMAT

Set the output format to FORMAT.

As of LTTng 2.16.0, FORMAT must be sqlite (the default): an SQL script for the SQLite database engine (see the “SQL Schema” section above).
Important

The sqlite format targets SQLite specifically: to quote map channel, owner, and key names, the script relies on SQLite’s string literal syntax, where the backslash is an ordinary character and the only way to write a single quote is to double it. LTTng therefore only guarantees that the script loads correctly with SQLite 3.

You may still load the script with another SQL database engine, provided that you make that engine treat the backslash as an ordinary character within string literals (otherwise a name containing a backslash can break the quoting), for example:

PostgreSQL

Loads the script as is when the standard_conforming_strings setting is on, which is the default since PostgreSQL 9.1.

MySQL and MariaDB

Set the NO_BACKSLASH_ESCAPES SQL mode before loading the script, for example with

SET sql_mode = 'NO_BACKSLASH_ESCAPES';

Even then, LTTng makes no guarantee: a future format dedicated to another engine would use its own FORMAT value.

Program information

-h, --help

Show help.

This option attempts to launch /usr/bin/man to view this manual page. Override the manual pager path with the LTTNG_MAN_BIN_PATH environment variable.

--list-options

List available command options and quit.

Exit Status

0: Success
1: Command error
2: Undefined command
3: Fatal error
4: Command warning (something went wrong during the command)

Environment

LTTNG_ABORT_ON_ERROR

Set to 1 to abort the process after the first error is encountered.

LTTNG_HOME

Path to the LTTng home directory.

Defaults to $HOME.

Useful when the Unix user running the commands has a non-writable home directory.

LTTNG_LIST_LEGACY

Set to 1 to use the legacy output format (LTTng 2.14 and earlier) for the lttng-list(1) command instead of the modern output format.

Note that the legacy output doesn’t show anything related to features introduced after LTTng 2.14.

LTTNG_MAN_BIN_PATH

Absolute path to the manual pager to use to read the LTTng command-line help (with lttng-help(1) or with the --help option) instead of /usr/bin/man.

LTTNG_NO_UTF_8

Set to 1 to NOT emit multi-byte UTF-8 sequences, even if the locale claims to support it.

LTTNG_SESSION_CONFIG_XSD_PATH

Path to the directory containing the session.xsd recording session configuration XML schema.

LTTNG_SESSIOND_PATH

Absolute path to the LTTng session daemon binary (see lttng-sessiond(8)) to spawn from the lttng-create(1) command.

The --sessiond-path general option overrides this environment variable.

LTTNG_TERM_COLOR

Controls when to emit terminal SGR codes in the output.

The NO_COLOR environment variable overrides this.

One of:

auto (default): Only emit SGR codes when the standard output is connected to a color-capable terminal.
always: Always emit SGR codes.
never: Never emit SGR codes.

NO_COLOR

If set and not empty, then it’s equivalent to setting LTTNG_TERM_COLOR to never.

See NO_COLOR <https://no-color.org/> to learn more.

Files

$LTTNG_HOME/.lttngrc

Unix user’s LTTng runtime configuration.

This is where LTTng stores the name of the Unix user’s current recording session between executions of lttng(1). lttng-create(1) and lttng-set-session(1) set the current recording session.

$LTTNG_HOME/lttng-traces

Default output directory of LTTng traces in local and snapshot modes.

Override this path with the --output option of the lttng-create(1) command.

$LTTNG_HOME/.lttng

Unix user’s LTTng runtime and configuration directory.

$LTTNG_HOME/.lttng/sessions

Default directory containing the Unix user’s saved recording session configurations (see lttng-save(1) and lttng-load(1)).

/usr/local/etc/lttng/sessions

Directory containing the system-wide saved recording session configurations (see lttng-save(1) and lttng-load(1)).

Note

$LTTNG_HOME defaults to the value of the HOME environment variable.

Examples

Example 1. Export the maps of the current recording session and open an interactive SQL shell.

The export becomes the initialization script of sqlite3, which then drops to its interactive prompt with the data loaded:

$ sqlite3 -init <(lttng export-maps) :memory:

Example 2. Export the maps of a specific recording session.

See the --session option.

$ lttng export-maps --session=my-session > maps.sql

Example 3. Show the total count per key across all CPUs of a specific map channel.

$ { lttng export-maps; cat << EOF
    SELECT key, SUM(value) AS total FROM vmap
    WHERE channel_name = 'my-counters'
    GROUP BY key
    ORDER BY total DESC;
  EOF
  } | sqlite3 :memory:

Example 4. Show the top 10 “hottest” counters of a specific map channel.

$ { lttng export-maps; cat << EOF
    SELECT key, SUM(value) AS total FROM vmap
    WHERE channel_name = 'my-counters'
    GROUP BY key
    ORDER BY total DESC
    LIMIT 10;
  EOF
  } | sqlite3 :memory:

Example 5. Show the per-CPU breakdown of a specific counter of a specific map channel.

$ { lttng export-maps; cat << EOF
    SELECT part_id, value FROM vmap
    WHERE channel_name = 'my-counters' AND key = 'myapp:my_event'
    ORDER BY part_id;
  EOF
  } | sqlite3 :memory:

Example 6. Show the total count per scheduling event of a specific Linux kernel map channel.

$ { lttng export-maps; cat << EOF
    SELECT key, SUM(value) AS total FROM vmap
    WHERE channel_name = 'my-counters'
      AND group_type = 'kernel-global'
      AND key GLOB 'sched_*'
    GROUP BY key;
  EOF
  } | sqlite3 :memory:

Example 7. Show the top 10 processes by total count, with their command names.

This groups by owner_id (the reliable process identifier) and shows the informational owner_name for display only.

$ { lttng export-maps; cat << EOF
    SELECT owner_id AS pid, owner_name AS command, SUM(value) AS total
    FROM vmap
    WHERE channel_name = 'my-counters' AND group_type = 'user-per-process'
    GROUP BY owner_id
    ORDER BY total DESC
    LIMIT 10;
  EOF
  } | sqlite3 :memory:

Example 8. Show all the overflowed counters.

$ { lttng export-maps;
    echo 'SELECT * FROM vmap WHERE has_overflow = 1;'
  } | sqlite3 :memory:

Resources

LTTng project website <https://lttng.org>
LTTng documentation <https://lttng.org/docs>
LTTng bug tracker <https://bugs.lttng.org>
Git repositories <https://git.lttng.org>
GitHub organization <https://github.com/lttng>
Continuous integration <https://ci.lttng.org/>
Mailing list <https://lists.lttng.org/> for support and development: lttng-dev@lists.lttng.org
IRC channel <irc://irc.oftc.net/lttng>: #lttng on irc.oftc.net
Mastodon <https://mastodon.social/@lttng>