Skip to main content

Metrics

oasis-node can report a number of metrics to Prometheus server. By default, no metrics are collected and reported. There is one way to enable metrics reporting:

  • Pull mode listens on given address and waits for Prometheus to scrape the metrics.

Configuring oasis-node in Pull Mode

To run oasis-node in pull mode set flag --metrics.mode pull and provide the listen address with --metrics.address. For example

oasis-node --metrics.mode pull --metrics.address localhost:3000

Then, add the following segment to your prometheus.yml and restart Prometheus:

  - job_name : 'oasis-node'

scrape_interval: 5s

static_configs:
- targets: ['localhost:3000']

Metrics Reported by oasis-node

oasis-node reports metrics starting with oasis_.

The following metrics are currently reported:

NameTypeDescriptionLabelsPackage
oasis_abci_db_sizeGaugeTotal size of the ABCI database (MiB).consensus/cometbft/abci
oasis_codec_sizeSummaryCBOR codec message size (bytes).call, modulecommon/cbor
oasis_consensus_proposed_blocksCounterNumber of blocks proposed by the node.backendconsensus/metrics
oasis_consensus_signed_blocksCounterNumber of blocks signed by the node.backendconsensus/metrics
oasis_finalized_roundsCounterNumber of finalized rounds.roothash
oasis_grpc_client_callsCounterNumber of gRPC calls.callcommon/grpc
oasis_grpc_client_latencySummarygRPC call latency (seconds).callcommon/grpc
oasis_grpc_client_stream_writesCounterNumber of gRPC stream writes.callcommon/grpc
oasis_grpc_server_callsCounterNumber of gRPC calls.callcommon/grpc
oasis_grpc_server_latencySummarygRPC call latency (seconds).callcommon/grpc
oasis_grpc_server_stream_writesCounterNumber of gRPC stream writes.callcommon/grpc
oasis_node_cpu_stime_secondsGaugeCPU system time spent by worker as reported by /proc/<PID>/stat (seconds).oasis-node/cmd/common/metrics
oasis_node_cpu_utime_secondsGaugeCPU user time spent by worker as reported by /proc/<PID>/stat (seconds).oasis-node/cmd/common/metrics
oasis_node_disk_read_bytesGaugeRead data from block storage by the worker as reported by /proc/<PID>/io (bytes).oasis-node/cmd/common/metrics
oasis_node_disk_usage_bytesGaugeSize of datadir of the worker (bytes).oasis-node/cmd/common/metrics
oasis_node_disk_written_bytesGaugeWritten data from block storage by the worker as reported by /proc/<PID>/io (bytes)oasis-node/cmd/common/metrics
oasis_node_mem_rss_anon_bytesGaugeSize of resident anonymous memory of worker as reported by /proc/<PID>/status (bytes).oasis-node/cmd/common/metrics
oasis_node_mem_rss_file_bytesGaugeSize of resident file mappings of worker as reported by /proc/<PID>/status (bytes)oasis-node/cmd/common/metrics
oasis_node_mem_rss_shmem_bytesGaugeSize of resident shared memory of worker.oasis-node/cmd/common/metrics
oasis_node_mem_vm_size_bytesGaugeVirtual memory size of worker (bytes).oasis-node/cmd/common/metrics
oasis_node_net_receive_bytes_totalGaugeReceived data for each network device as reported by /proc/net/dev (bytes).deviceoasis-node/cmd/common/metrics
oasis_node_net_receive_packets_totalGaugeReceived data for each network device as reported by /proc/net/dev (packets).deviceoasis-node/cmd/common/metrics
oasis_node_net_transmit_bytes_totalGaugeTransmitted data for each network device as reported by /proc/net/dev (bytes).deviceoasis-node/cmd/common/metrics
oasis_node_net_transmit_packets_totalGaugeTransmitted data for each network device as reported by /proc/net/dev (packets).deviceoasis-node/cmd/common/metrics
oasis_p2p_blocked_peersGaugeNumber of blocked P2P peers.p2p
oasis_p2p_connectionsGaugeNumber of P2P connections.p2p
oasis_p2p_peersGaugeNumber of connected P2P peers.p2p
oasis_p2p_protocolsGaugeNumber of supported P2P protocols.p2p
oasis_p2p_topicsGaugeNumber of supported P2P topics.p2p
oasis_registry_entitiesGaugeNumber of registry entities.registry
oasis_registry_nodesGaugeNumber of registry nodes.registry
oasis_registry_runtimesGaugeNumber of registry runtimes.registry
oasis_rhp_failuresCounterNumber of failed Runtime Host calls.callruntime/host/protocol
oasis_rhp_latencySummaryRuntime Host call latency (seconds).callruntime/host/protocol
oasis_rhp_successesCounterNumber of successful Runtime Host calls.callruntime/host/protocol
oasis_rhp_timeoutsCounterNumber of timed out Runtime Host calls.runtime/host/protocol
oasis_roothash_block_intervalSummaryTime between roothash blocks (seconds).runtimeroothash
oasis_storage_failuresCounterNumber of storage failures.callstorage/api
oasis_storage_latencySummaryStorage call latency (seconds).callstorage/api
oasis_storage_successesCounterNumber of storage successes.callstorage/api
oasis_storage_value_sizeSummaryStorage call value size (bytes).callstorage/api
oasis_tee_attestations_failedCounterNumber of failed TEE attestations.runtimeruntime/host/sgx
oasis_tee_attestations_performedCounterNumber of TEE attestations performed.runtimeruntime/host/sgx
oasis_tee_attestations_successfulCounterNumber of successful TEE attestations.runtimeruntime/host/sgx
oasis_txpool_accepted_transactionsCounterNumber of accepted transactions (passing check tx).runtimeruntime/txpool
oasis_txpool_local_queue_sizeGaugeSize of the local transactions schedulable queue (number of entries).runtimeruntime/txpool
oasis_txpool_pending_check_sizeGaugeSize of the pending to be checked queue (number of entries).runtimeruntime/txpool
oasis_txpool_pending_schedule_sizeGaugeSize of the main schedulable queue (number of entries).runtimeruntime/txpool
oasis_txpool_rejected_transactionsCounterNumber of rejected transactions (failing check tx).runtimeruntime/txpool
oasis_txpool_rim_queue_sizeGaugeSize of the roothash incoming message transactions schedulable queue (number of entries).runtimeruntime/txpool
oasis_upGaugeIs oasis-test-runner active for specific scenario.oasis-node/cmd/common/metrics
oasis_worker_aborted_batch_countCounterNumber of aborted batches.runtimeworker/compute/executor/committee
oasis_worker_batch_processing_timeSummaryTime it takes for a batch to finalize (seconds).runtimeworker/compute/executor/committee
oasis_worker_batch_runtime_processing_timeSummaryTime it takes for a batch to be processed by the runtime (seconds).runtimeworker/compute/executor/committee
oasis_worker_batch_sizeSummaryNumber of transactions in a batch.runtimeworker/compute/executor/committee
oasis_worker_client_lb_healthy_instance_countGaugeNumber of healthy instances in the load balancer.runtimeruntime/host/loadbalance
oasis_worker_client_lb_requestsCounterNumber of requests processed by the given load balancer instance.runtime, lb_instanceruntime/host/loadbalance
oasis_worker_epoch_numberGaugeCurrent epoch number as seen by the worker.runtimeworker/common/committee
oasis_worker_epoch_transition_countCounterNumber of epoch transitions.runtimeworker/common/committee
oasis_worker_execution_discrepancy_detected_countCounterNumber of detected execute discrepancies.runtimeworker/compute/executor/committee
oasis_worker_executor_committee_p2p_peersGaugeNumber of executor committee P2P peers.runtimeworker/common/committee
oasis_worker_executor_is_backup_workerGauge1 if worker is currently an executor backup worker, 0 otherwise.runtimeworker/common/committee
oasis_worker_executor_is_workerGauge1 if worker is currently an executor worker, 0 otherwise.runtimeworker/common/committee
oasis_worker_executor_liveness_live_ratioGaugeRatio between live and total rounds. Reports 1 if node is not in committee.runtimeworker/common/committee
oasis_worker_executor_liveness_live_roundsGaugeNumber of live rounds in last epoch.runtimeworker/common/committee
oasis_worker_executor_liveness_total_roundsGaugeNumber of total rounds in last epoch.runtimeworker/common/committee
oasis_worker_failed_round_countCounterNumber of failed roothash rounds.runtimeworker/common/committee
oasis_worker_keymanager_compute_runtime_countCounterNumber of compute runtimes using the key manager.runtimeworker/keymanager
oasis_worker_keymanager_consensus_ephemeral_secret_epoch_numberGaugeEpoch number of the latest ephemeral secret.runtimeworker/keymanager
oasis_worker_keymanager_consensus_master_secret_generation_numberGaugeGeneration number of the latest master secret.runtimeworker/keymanager
oasis_worker_keymanager_consensus_master_secret_proposal_epoch_numberGaugeEpoch number of the latest master secret proposal.runtimeworker/keymanager
oasis_worker_keymanager_consensus_master_secret_proposal_generation_numberGaugeGeneration number of the latest master secret proposal.runtimeworker/keymanager
oasis_worker_keymanager_consensus_master_secret_rotation_epoch_numberGaugeEpoch number of the latest master secret rotation.runtimeworker/keymanager
oasis_worker_keymanager_enclave_ephemeral_secret_epoch_numberGaugeEpoch number of the latest ephemeral secret loaded into the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_generated_ephemeral_secret_epoch_numberGaugeEpoch number of the latest ephemeral secret generated by the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_generated_master_secret_epoch_numberGaugeEpoch number of the latest master secret generated by the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_generated_master_secret_generation_numberGaugeGeneration number of the latest master secret generated by the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_master_secret_generation_numberGaugeGeneration number of the latest master secret as seen by the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_master_secret_proposal_epoch_numberGaugeEpoch number of the latest master secret proposal loaded into the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_master_secret_proposal_generation_numberGaugeGeneration number of the latest master secret proposal loaded into the enclave.runtimeworker/keymanager
oasis_worker_keymanager_enclave_rpc_countCounterNumber of remote Enclave RPC requests via P2P.methodworker/keymanager/p2p
oasis_worker_keymanager_policy_update_countCounterNumber of key manager policy updates.runtimeworker/keymanager
oasis_worker_node_registeredGaugeIs oasis node registered (binary).worker/registration
oasis_worker_node_registration_eligibleGaugeIs oasis node eligible for registration (binary).worker/registration
oasis_worker_node_status_frozenGaugeIs oasis node frozen (binary).worker/registration
oasis_worker_node_status_runtime_faultsGaugeNumber of runtime faults.runtimeworker/registration
oasis_worker_node_status_runtime_suspendedGaugeRuntime node suspension status (binary).runtimeworker/registration
oasis_worker_processed_block_countCounterNumber of processed roothash blocks.runtimeworker/common/committee
oasis_worker_processed_event_countCounterNumber of processed roothash events.runtimeworker/compute/executor/committee
oasis_worker_storage_commit_latencySummaryLatency of storage commit calls (state + outputs) (seconds).runtimeworker/compute/executor/committee
oasis_worker_storage_full_roundGaugeThe last round that was fully synced and finalized.runtimeworker/storage/committee
oasis_worker_storage_pending_roundGaugeThe last round that is in-flight for syncing.runtimeworker/storage/committee
oasis_worker_storage_round_sync_latencySummaryStorage round sync latency (seconds).runtimeworker/storage/committee
oasis_worker_storage_synced_roundGaugeThe last round that was synced but not yet finalized.runtimeworker/storage/committee

Consensus backends

Metrics Reported by CometBFT

When oasis-node is configured to use CometBFT for BFT consensus, all CometBFT metrics are also reported. Consult CometBFT-core documentation for a list of reported by CometBFT.