VictoriaMetrics

v1.144.0

Released at 2026-05-22

• FEATURE: all VictoriaMetrics components: improve logging for the -memory.allowedBytes flag to warn about excessively low value (less than 1MB). See issue #10935.

• FEATURE: vmagent and vmalert: add basicAuth.usernameFile command-line flags for reading basic auth username from a file, similar to the existing basicAuth.passwordFile. The file is re-read every second. See #9436. Thanks to @kimjune01 for the contribution.

• FEATURE: vminsert in VictoriaMetrics cluster: add clusternative.tls vminsert configuration flags for multi-level cluster setups. See #10958.

• FEATURE: vmsingle, vminsert in VictoriaMetrics cluster and vmagent: add -opentelemetry.labelNameUnderscoreSanitization command-line flag to control whether to enable prepending of key to labels starting with _ when -opentelemetry.usePrometheusNaming is enabled. See OpenTelemetry docs and #9663. Thanks to @andriibeee for the contribution.

• FEATURE: vmui: improve the Top Queries table UI. Duration columns now display human-readable values (e.g. 1.23s) instead of raw seconds, memory column shows human-readable sizes (e.g. 1.23 MB), instant queries are labeled as instant instead of empty string, and column headers now show tooltips with descriptions. See #10790.

• FEATURE: vmagent: drain in-memory remote write queue on shutdown within the 5-second grace period before falling back to persisting blocks to disk. See #9996

• FEATURE: vminsert in VictoriaMetrics cluster: Improve slowness-based rerouting to prevent rerouting storms under high cluster load. Previously, rerouting could cascade across storage nodes when the whole cluster was saturated, making the situation worse. Now rerouting only activates when the cluster p90 saturation is below 60%, and the slowest node is more than 20% slower than p90. See #10876.

• FEATURE: vmauth: add {{.MetricsAccountID}} and {{.MetricsProjectID}} JWT claim placeholders for use in headers and url_prefix config fields. Previously, only the combined {{.MetricsTenant}} (accountID:projectID) JWT placeholder was supported, making it impossible to configure multitenancy via headers. See #10927. Thanks to @Vinayak9769 for the contribution.

• FEATURE: vmui: display null values on Raw Query chart. null values can be actual NaN or null values exposed by the exporter, or stale markers. Before, vmui Raw Query was silently dropping non-numeric values. Displaying such values on the chart could improve the debugging experience. See #10986.

• BUGFIX: stream aggregation: stop emitting stale values for quantiles(...) outputs when a time series has no samples during the current aggregation interval. See #10918. Thanks to @alexei38 for the contribution.

• BUGFIX: stream aggregation: extend delay on aggregation windows flush by the biggest lag among pushed samples. Before, the delay was calculated as 95th percentile across samples, which could underrepresent outliers and reject them from aggregation as "too old". See #10402.

• BUGFIX: vmagent: fix a bug in cardinality limiters where series with different labels, like {a="bc"} and {ab="c"}, could be incorrectly treated as identical and dropped. See #10937.

• BUGFIX: vmagent: hide values passed to -remoteWrite.headers in startup logs, /metrics, and /flags, since they can contain sensitive HTTP headers such as Authorization and API keys.

• BUGFIX: vmagent: hide values passed to -remoteWrite.proxyURL in startup logs, /metrics, and /flags, since they can contain sensitive credentials.

• BUGFIX: vmalert: hide values passed to -remoteWrite.headers,remoteRead.headers, datasource.headers and notifier.headers in startup logs, /metrics, and /flags, since they can contain sensitive HTTP headers such as Authorization and API keys.

• BUGFIX: vminsert in VictoriaMetrics cluster: properly establish mtls connection between vmstorage and vminsert. Regression was introduced in v1.130.0 release for the enterprise version of vmstorage. See #10972.

• BUGFIX: vmrestore: fix a bug where specifying -storageDataPath with a trailing slash could cause vmrestore to panic. See #10823. Thanks to @utafrali for the contribution.

• BUGFIX: vmagent: prevent unintentional rerouting of samples to other sharding targets when one of the -remoteWrite.url targets with -remoteWrite.disableOnDiskQueue becomes blocked. Previously this could break the sharding guarantee by sending samples to wrong targets instead of dropping or retrying them. See #10507.

• BUGFIX: vmagent: return error on startup if -remoteWrite.disableOnDiskQueue is not configured uniformly across all -remoteWrite.url targets when -remoteWrite.shardByURL is enabled. Either all targets must have it enabled or all must have it disabled. See #10507.

• BUGFIX: vmsingle and vmselect in VictoriaMetrics cluster: hide values passed to vmalert.proxyURL in startup logs, /metrics, and /flags, since they can contain sensitive HTTP headers such as Authorization and API keys.

• BUGFIX: vmui: preserve exact series values in graph tooltips instead of rounding them by significant digits. See #10952.

• BUGFIX: all VictoriaMetrics components: fix int64 overflow when parsing timestamp parameters with relative durations. See #10880.

• BUGFIX: vmsingle and vmstorage in VictoriaMetrics cluster: -denyQueriesOutsideRetention now also rejects queries whose end time is beyond -futureRetention. See #10879.

• BUGFIX: vmui: add missing __timestamp__ and __value__ columns to CSV exported from the table view on the Query tab. See #10975.