The Proxmox plugin bug that only crashed half the time

v0.0.25 of pve-nimble-plugin renamed every config option. address became nimble_address, debug became nimble_debug, and so on. If you’re running the plugin already, your storage.cfg still works — but if you have scripts calling pvesm set --debug 1, those need updating. This is why.

The bug

Proxmox loads custom storage plugins from /usr/share/perl5/PVE/Storage/Custom/ at daemon startup. Each plugin registers its config options — the flags for pvesm add — and Proxmox merges them into one global table. If two plugins register the same name, the merge fails. Not a warning, not last-writer-wins. The daemon exits.

I modeled this plugin on the Pure Storage plugin and reused its generic option names: address, check_ssl, token_ttl, debug. Fine on a node running one plugin. On a node with both installed, you hit a name collision.

The annoying part: which plugin loads first is random. Perl hash order changes per process. v0.0.24 added a guard that skips registering a name if something else already claimed it. That works when Pure loads first. When my plugin loads first, it registers the names, Pure loads second and collides, and pvedaemon dies. Restart — maybe it comes up. Reboot — maybe it doesn’t. Roughly half the time on a co-installed node.

The fix

v0.0.24’s release notes said the collision would be fully resolved once the Pure plugin adopted the same guard. That was the wrong read. Pure was there first with those names. The fix was to stop using them.

In v0.0.25, every plugin-specific option has a nimble_ prefix. Proxmox-owned names (username, password, content, nodes) are unchanged. Old spellings in storage.cfg still parse — the plugin maps them internally, and Proxmox rewrites them to the new names the next time the config changes. For legacy configs on nodes where another plugin also uses the old generic names, the plugin only registers the legacy spellings when nothing else has claimed them, using a check against the installed plugin list rather than whatever happened to load first in that process.

Pure needs no changes.

Testing it

A bug that fails half the time on a single run is easy to miss in CI. The verify pipeline now installs a fake plugin that declares the conflicting generic names, then boots the real Proxmox storage stack ten times in fresh processes so both load orders get hit. That runs on bookworm and trixie before a release .deb is built.

If you’re writing a PVE storage plugin: prefix your option names. The property table is global, load order is not stable, and renaming options after release is more painful than getting the names right upfront.

Upgrading

Existing configs work without edits. On a cluster, upgrade the package on all nodes before changing storage config — once Proxmox writes the new nimble_* keys, nodes still on v0.0.24 won’t parse them. See the release notes and README upgrade section. Setup walkthrough: From zero to fully protected Nimble storage on Proxmox.

← All posts Home About Services Contact