ctdb-statistics - Man Page

CTDB version 1
Current time of statistics  :                Fri Sep 12 13:32:32 2014
Statistics collected since  : (000 01:49:20) Fri Sep 12 11:43:12 2014
 num_clients                        6
 frozen                             0
 recovering                         0
 num_recoveries                     2
 client_packets_sent           281293
 client_packets_recv           296317
 node_packets_sent             452387
 node_packets_recv             182394
 keepalive_packets_sent          3927
 keepalive_packets_recv          3928
 node
     req_call                   48605
     reply_call                     1
     req_dmaster                23404
     reply_dmaster              24917
     reply_error                    0
     req_message                  958
     req_control               197513
     reply_control             153705
 client
     req_call                  130866
     req_message                  770
     req_control               168921
 timeouts
     call                           0
     control                        0
     traverse                       0
 locks
     num_calls                    220
     num_current                    0
     num_pending                    0
     num_failed                     0
 total_calls                   130866
 pending_calls                      0
 childwrite_calls                   1
 pending_childwrite_calls             0
 memory_used                   334490
 max_hop_count                     18
 total_ro_delegations               2
 total_ro_revokes                   2
 hop_count_buckets: 42816 5464 26 1 0 0 0 0 0 0 0 0 0 0 0 0
 lock_buckets: 9 165 14 15 7 2 2 0 0 0 0 0 0 0 0 0
 locks_latency      MIN/AVG/MAX     0​.000685/0​.160302/6​.369342 sec out of 214
 reclock_ctdbd      MIN/AVG/MAX     0​.004940/0​.004969/0​.004998 sec out of 2
 reclock_recd       MIN/AVG/MAX     0​.000000/0​.000000/0​.000000 sec out of 0
 call_latency       MIN/AVG/MAX     0​.000006/0​.000719/4​.562991 sec out of 126626
 childwrite_latency MIN/AVG/MAX     0​.014527/0​.014527/0​.014527 sec out of 1

Version of the ctdb protocol used by the node​.

Time when the statistics are generated​.

This is useful when collecting statistics output periodically for post-processing​.

Time when ctdb was started or the last time statistics was reset​. The output shows the duration and the timestamp​.

Number of processes currently connected to CTDB's unix socket​. This includes recovery daemon, ctdb tool and samba processes (smbd, winbindd)​.

1 if the databases are currently frozen, 0 otherwise​.

1 if recovery is active, 0 otherwise​.

Number of recoveries since the start of ctdb or since the last statistics reset​.

Number of packets sent to client processes via unix domain socket​.

Number of packets received from client processes via unix domain socket​.

Number of packets sent to the other nodes in the cluster via TCP​.

Number of packets received from the other nodes in the cluster via TCP​.

Number of keepalive messages sent to other nodes​.

CTDB periodically sends keepalive messages to other nodes​. See KeepaliveInterval tunable in ctdb-tunables(7) for more details​.

Number of keepalive messages received from other nodes​.

This section lists various types of messages processed which originated from other nodes via TCP​.

req_call

Number of REQ_CALL messages from the other nodes​.

reply_call

Number of REPLY_CALL messages from the other nodes​.

req_dmaster

Number of REQ_DMASTER messages from the other nodes​.

reply_dmaster

Number of REPLY_DMASTER messages from the other nodes​.

reply_error

Number of REPLY_ERROR messages from the other nodes​.

req_message

Number of REQ_MESSAGE messages from the other nodes​.

req_control

Number of REQ_CONTROL messages from the other nodes​.

reply_control

Number of REPLY_CONTROL messages from the other nodes​.

req_tunnel

Number of REQ_TUNNEL messages from the other nodes​.

This section lists various types of messages processed which originated from clients via unix domain socket​.

req_call

Number of REQ_CALL messages from the clients​.

req_message

Number of REQ_MESSAGE messages from the clients​.

req_control

Number of REQ_CONTROL messages from the clients​.

req_tunnel

Number of REQ_TUNNEL messages from the clients​.

This section lists timeouts occurred when sending various messages​.

call

Number of timeouts for REQ_CALL messages​.

control

Number of timeouts for REQ_CONTROL messages​.

traverse

Number of timeouts for database traverse operations​.

This section lists locking statistics​.

num_calls

Number of completed lock calls​. This includes database locks and record locks​.

num_current

Number of scheduled lock calls​. This includes database locks and record locks​.

num_pending

Number of queued lock calls​. This includes database locks and record locks​.

num_failed

Number of failed lock calls​. This includes database locks and record locks​.

Number of req_call messages processed from clients​. This number should be same as client --> req_call​.

Number of req_call messages which are currently being processed​. This number indicates the number of record migrations in flight​.

Number of record update calls​. Record update calls are used to update a record under a transaction​.

Number of record update calls currently active​.

The amount of memory in bytes currently used by CTDB using talloc​. This includes all the memory used for CTDB's internal data structures​. This does not include the memory mapped TDB databases​.

The maximum number of hops required for a record migration request to obtain the record​. High numbers indicate record contention​.

Number of readonly delegations created​.

Number of readonly delegations that were revoked​. The difference between total_ro_revokes and total_ro_delegations gives the number of currently active readonly delegations​.

Distribution of migration requests based on hop counts values​. Buckets are 0, < ​2, < ​4, < ​8, < ​16, < ​32, < ​64, < ​128, < ​256, < ​512, < ​1024, < ​2048, < ​4096, < ​8192, < ​16384, ≥ ​16384​.

Distribution of record lock requests based on time required to obtain locks​. Buckets are < ​1ms, < ​10ms, < ​100ms, < ​1s, < ​2s, < ​4s, < ​8s, < ​16s, < ​32s, < ​64s, ≥ ​64s​.

The minimum, the average and the maximum time (in seconds) required to obtain record locks​.

The minimum, the average and the maximum time (in seconds) required to check if recovery lock is still held by recovery daemon when recovery mode is changed​. This check is done in ctdb daemon​.

The minimum, the average and the maximum time (in seconds) required to check if recovery lock is still held by recovery daemon during recovery​. This check is done in recovery daemon​.

The minimum, the average and the maximum time (in seconds) required to process a REQ_CALL message from client​. This includes the time required to migrate a record from remote node, if the record is not available on the local node​.

Default: 0

The minimum, the average and the maximum time (in seconds) required to update records under a transaction​.

DB Statistics: notify_index​.tdb
 ro_delegations                     0
 ro_revokes                         0
 locks
     total                        131
     failed                         0
     current                        0
     pending                        0
 hop_count_buckets: 9890 5454 26 1 0 0 0 0 0 0 0 0 0 0 0 0
 lock_buckets: 4 117 10 0 0 0 0 0 0 0 0 0 0 0 0 0
 locks_latency      MIN/AVG/MAX     0​.000683/0​.004198/0​.014730 sec out of 131
 Num Hot Keys:     3
     Count:7 Key:2f636c75737465726673
     Count:18 Key:2f636c757374657266732f64617461
     Count:7 Key:2f636c757374657266732f646174612f636c69656e7473

Name of the database​.

Number of readonly delegations created in the database​.

Number of readonly delegations revoked​. The difference in ro_delegations and ro_revokes indicates the currently active readonly delegations​.

This section lists locking statistics​.

total

Number of completed lock calls​. This includes database locks and record locks​.

failed

Number of failed lock calls​. This includes database locks and record locks​.

current

Number of scheduled lock calls​. This includes database locks and record locks​.

pending

Number of queued lock calls​. This includes database locks and record locks​.

Distribution of migration requests based on hop counts values​. Buckets are 0, < ​2, < ​4, < ​8, < ​16, < ​32, < ​64, < ​128, < ​256, < ​512, < ​1024, < ​2048, < ​4096, < ​8192, < ​16384, ≥ ​16384​.

Distribution of record lock requests based on time required to obtain locks​. Buckets are < ​1ms, < ​10ms, < ​100ms, < ​1s, < ​2s, < ​4s, < ​8s, < ​16s, < ​32s, < ​64s, ≥ ​64s​.

The minimum, the average and the maximum time (in seconds) required to obtain record locks​.

Number of contended records determined by hop count​. CTDB keeps track of top 10 hot records and the output shows hex encoded keys for the hot records​.