Designing a Plugin System That Can Fail Safely

How Multiforum grew from loading extension code into running reliable plugin pipelines

When I first started working on plugins for Multiforum, the basic idea was simple: when something happens in the application, load an external module and call a function.

That was the easy part.

Multiforum plugins can scan uploaded files, apply forum-specific labels, respond through configurable bot profiles, and react to new discussions or comments. Once those plugins were affecting real content, successfully importing the module was no longer enough.

I needed to answer a much larger set of questions.

Where did the code come from? Which version was running? Was it compatible with this server? Had an administrator supplied its required settings and secrets? In what order should several plugins run? What should happen after one failed? If the backend restarted halfway through, how would the system know that the work had stopped? What information could I safely show the uploader without exposing private logs or credentials? And if I introduced a required security check today, what should happen to files uploaded before that policy existed?

Over time, most of the work shifted away from calling the plugin code. The harder part was managing everything around code that lives outside the main application:

registry
   ↓
discover → install a version → configure → enable
                                           ↓
event → pipeline policy → pipeline attempt → plugin jobs
                                           ↓
                            status, diagnostics, retries

The design kept coming back to a few distinctions:

Discovered is not installed. Installed is not configured. Configured is not enabled. A pipeline definition is not a pipeline attempt. Internal logs are not public diagnostics. “Not required” is not “passed.”

Those distinctions gave the application more states to handle, but they also made failures easier to explain and recover from.

Installation and enablement are different decisions

One of my first decisions was to separate installation from enablement.

Installing a plugin means selecting a particular version, verifying its artifact, reading its manifest, and recording it as available on the server. Enabling it means allowing that installed version to participate in runtime pipelines.

Combining those actions would save a step, but it would also let newly downloaded code start receiving events before an administrator had reviewed it or finished configuring it. A plugin that needs an API key, webhook URL, model name, or forum-specific profile should not become active just because its package downloaded successfully.

The separate states also make the system easier to operate. I can install a new version while the current version remains in use, inspect what changed, configure any new requirements, and decide when the replacement is ready. I can disable a broken plugin without deleting its package or settings.

This is more complicated than one enabled checkbox, but each state answers a different question:

• Discovered: Does a configured registry advertise this plugin?
• Installed: Has this exact artifact been verified and recorded locally?
• Configured: Are its required settings and secrets present and valid enough to run?
• Enabled: May it participate in event pipelines?

The frontend shows those states separately. An administrator can see the installed version, missing configuration, enablement state, available updates, release information, settings, secrets, manifest, and README.

The manifest became the contract

Every plugin includes a plugin.json manifest. It started as package metadata, but it eventually became the shared contract between the registry, installer, backend, and frontend.

The manifest describes the plugin's identity and version, entry point, supported events, compatibility requirements, settings defaults, required secrets, documentation, and the schema used to build its configuration forms.

I did not want to add a custom Vue page every time I created a plugin with a new setting. Instead, the plugin declares fields such as text inputs, numbers, toggles, selections, and secrets. It can also specify required values, ranges, patterns, choices, defaults, labels, and descriptions. The frontend turns those declarations into a form using shared components.

The backend reads the same schema. Client-side validation gives administrators quick feedback, but the browser cannot have the final say. The GraphQL API checks the types and allowed values again, along with required settings and secrets, before it allows a plugin to be enabled.

The tradeoff is that a generated form cannot support every interaction that I could build by hand. In return, new plugins get consistent forms, validation, accessibility, and dark-mode support without adding plugin-specific components to the main frontend.

It also means that changing a manifest can affect saved data. It is not only a documentation change.

A plugin upgrade is a configuration migration

Suppose version 1 of a plugin declares a setting called model, and an administrator selects a value. Version 2 might keep the setting, remove it, change its allowed values, introduce a new default, or replace it with a differently typed field. It might also require a new secret or stop using an old one.

Copying everything could give the new version settings it no longer understands. Discarding everything would make upgrades unnecessarily destructive.

I added reconciliation logic that compares the saved settings with the new manifest and classifies each value:

• Carried over: the field still exists and the value remains compatible.
• Reset: the field exists, but its old value no longer passes the new schema.
• Removed: the new version no longer declares or defaults the field.
• New default: the new version introduces a value the administrator has not previously set.

Before an upgrade, the frontend shows that report and lets the administrator carry compatible settings or start fresh. It also shows whether the new version's required secrets are already present.

Secrets need different treatment from ordinary settings. The administration interface is write-only: after saving a value, the browser can see its status but cannot retrieve the plaintext. If a new version stops declaring a stored secret, the UI marks it as unused instead of silently deleting it. The administrator can then remove it deliberately.

Removing a password field from a form does not remove the saved credential. Without this cleanup step, old secrets could remain in storage indefinitely.

A registry is part of the supply chain

Multiforum can discover releases from several registries, including ordinary registry documents and GitHub releases. Supporting multiple sources introduced another problem: two registries could claim to offer the same plugin version while pointing to different packages.

The registry merger reports that as a conflict instead of quietly choosing whichever source loaded last.

During installation, the backend downloads the selected tarball and verifies its SHA-256 hash against the registry record. It then opens the package and checks that the embedded manifest's plugin ID and version match what the administrator requested. Compatibility metadata can reject a release that requires a newer server or a different plugin API version.

These checks protect against mismatched releases, corrupted downloads, and accidental substitution. The installed record also keeps the registry, source repository, source commit, release notes, hash, and exact version so I can trace where it came from.

They do not make arbitrary code safe.

Plugins currently run as dynamically imported JavaScript inside the backend's Node process. They are not isolated in a container, subprocess, worker, or restricted virtual machine. A matching hash proves that the package is the one listed by the registry. It does not prove that the code is trustworthy. This is a trusted-extension model: the server administrator must trust the registries and plugins they enable.

Runtime isolation would be a valuable future improvement, but it would also make the plugin API more complicated. File access, network requests, secrets, logging, and application operations would all need to cross that boundary. I started with the simpler in-process model and made the trust assumption explicit.

From event handlers to pipelines

The simplest runtime would loop over every enabled plugin. That stops being sufficient once order and failure behavior matter.

Imagine an uploaded file that needs a malware scan and then a metadata extractor. Should the extractor run if the security scan fails? Should a notification plugin run only after a failure? If the second step fails, should a later cleanup step still run?

I introduced configurable pipelines for events such as file creation, discussion submission, and comment creation. Each pipeline has ordered steps. A step can always run, run only after success, or run only after failure. It can allow later steps to continue after it fails, while the pipeline can also stop after the first unhandled failure.

Server administrators configure server-wide events such as uploaded-file processing. Forum administrators configure forum-scoped events using only plugins that the server has already installed and allowed. Defaults, server settings, and forum overrides are merged into the context passed to the plugin.

The frontend offers both a visual editor and a YAML editor for the same pipeline. The visual editor makes ordering and conditions easier to understand. YAML is more convenient for inspecting, copying, and reviewing larger configurations. Keeping the two formats in sync takes extra work, but administrators can use whichever representation fits the task.

For now, pipelines run sequentially. This keeps ordering and previous-step conditions predictable. The cost is speed: a five-step pipeline waits for every earlier step, even when two plugins could run at the same time. Parallel branches could improve throughput, but they would make configuration, status calculation, retries, and the UI much more complicated.

A pipeline definition is not an execution

At first, individual plugin runs shared a pipeline ID. That grouped related records, but there was no record for the pipeline as a whole. I could not easily answer questions such as:

• Who or what started it?
• Was this the first attempt or a retry?
• Which policy and plugin versions did it use?
• Did the pipeline fail, time out, or finish with skipped jobs?
• If the current configuration has changed, what configuration explains this historical result?

I added a PluginPipelineRun record for every attempt. It stores what triggered the run, who started it, which content it applies to, whether it is a retry, its overall status, its timing, and a snapshot of the exact configuration it used.

Each expected step also gets its own PluginRun record. I create all of those records as pending before execution starts. The interface can then show the complete plan immediately, including jobs that may later be skipped because an earlier condition was not met.

This is similar to a CI service. A workflow says what should happen. A workflow run records one attempt. Jobs record the individual steps. Editing the workflow tomorrow should not change the meaning of yesterday's run.

A retry creates a new attempt instead of changing the failed one. It links back to the previous attempt and records whether the uploader, a moderator, an administrator, or an automatic process started it. The history still shows the original failure and what happened next.

A running badge needs a recovery story

Saving a RUNNING status creates a new problem. If the backend crashes after writing it, the database can keep saying that the work is active forever.

I implemented execution leases and heartbeats.

A lease works a little like checking out a library book with a very short due date. A worker claims a pending job and receives a unique lease ID. While the plugin runs, the worker periodically extends the deadline. Only the worker holding that lease can complete the job. This also keeps two workers from claiming the same job and writing conflicting results.

A watchdog looks for pending or running jobs whose deadlines have expired. It marks them as timed out, recalculates the pipeline's status, and can notify the uploader. After a process restart, abandoned work becomes an explicit timeout instead of an endless spinner.

The lease detects a lost worker, but it is not a hard execution limit. A worker that stays alive and keeps sending heartbeats can still run a plugin indefinitely. A plugin that blocks Node's event loop can affect the rest of the backend. Enforcing CPU, memory, and wall-clock limits would require moving plugin execution outside the main process.

The current system can recover from a lost worker. It does not fully contain a badly behaved plugin.

Public logs need different rules

Once pipeline history was durable, I wanted uploaders to see why their checks failed and moderators to help resolve them. Returning the internal logs would have been simple, but unsafe.

Internal records can contain payloads, stack traces, storage URLs, prompts, credentials, provider responses, and server configuration. That can help an administrator debug a problem, but it does not belong on a public status page.

I created a separate format for public diagnostics. A plugin can publish a level, stable code, readable message, optional details, and a help link. The backend limits the number and size of those entries and redacts secret values, bearer tokens, sensitive object keys, and credential-like URL parameters.

The public GraphQL API returns only those cleaned diagnostics, and only when the download itself is visible. Authorized administrators can still access the full internal execution record.

The frontend turns that data into a checks page. It shows which policies apply, previous attempts, each step's status and duration, safe diagnostics, and links to individual attempts. It checks for updates while work is active and stops when the attempt finishes. Uploaders and authorized forum moderators can start missing checks or retry failed pipelines.

The result is more useful than a generic failure message without treating every user like a server administrator.

A new policy cannot change the past

The attachment scanner created one more difficult question. If I make a security pipeline required today, what status should the system show for an older file that has never been scanned?

Calling it “passed” would be false. Calling it “failed” would also be false. Immediately blocking every historical file might be safe in one deployment and unacceptably disruptive in another.

I modeled applicability as part of the pipeline policy:

1. New files only: enforce the pipeline for new and replaced files.
2. Gradual rollout: enforce it for new files now and process historical files over time.
3. Immediate rollout: require it for all files and hold older files until they pass.

An older file excluded by the policy is shown as not required, not passed. That is the most accurate statement the system can make.

For policies that include historical files, an administrator can preview a campaign before starting it. The preview reports how many files are affected, how many are still accessible, how many cannot be processed, and how many external provider runs the campaign is expected to require.

Campaigns have concurrency and per-minute rate limits. They can be paused and resumed, and each failure links back to the exact pipeline attempt. This matters when plugins call paid or rate-limited services. Saving a policy is instant, but applying it to years of files may be expensive and slow.

I kept the policy separate from the campaign for that reason. Deciding what should be true is not the same action as launching the work needed to update old data.

What I would carry into the next plugin system

The plugin system now does much more than the dynamic loader I started with. These are the lessons I would reuse:

1. Treat installation, configuration, and enablement as different decisions. Code should not begin receiving production events merely because its package downloaded successfully.
2. Make the manifest executable as a contract. Use the same declarations to drive installation, compatibility, forms, validation, defaults, and upgrade behavior.
3. Treat upgrades as data migrations. Settings and secrets have a lifecycle that continues across versions.
4. Record attempts, not just logs. A durable execution needs an initiator, configuration snapshot, jobs, timing, outcome, and retry history.
5. Plan for recovery as soon as work can be marked running. Without leases or another ownership mechanism, a crash can leave work stuck in progress forever.
6. Separate public explanations from private diagnostics. Give people enough information to act without exposing sensitive internal details.
7. Represent uncertainty honestly. Not checked, not required, skipped, failed, and passed are different claims.
8. Make the trust model explicit. Integrity verification is valuable, but it is not a sandbox.

I began with a way to call extension code. Most of the design work ended up around that call: how the code enters the system, which version runs, what it needs, who can activate it, how several plugins work together, what happens after a crash, what users can see, and how a new rule applies to old data.

That lifecycle is the real plugin system. The function call is only one part of it.