TwinStor

Hyperconverged storage for two XCP-ng hosts. TwinStor turns the local disks of a 2-host pool into fully redundant, self-healing shared storage: live migration, automatic VM restart (HA), and Rolling Pool Updates all work, with no SAN, no witness node, and no third host.

Tech Preview

TwinStor is currently distributed as a Tech Preview: a pre-release test build meant for lab pools, not production. Run it on hardware you can afford to break, and keep backing up your VMs (with Xen Orchestra or the tool of your choice) whether you use TwinStor or not. To get the current build and the guided track, join the Tech Preview or go straight to the TwinStor deploy page. Feedback and questions go to the TwinStor thread on the XCP-ng forum.

The problem it solves

A 2-node XCP-ng pool with local disks cannot do live migration, HA, or anything else that requires shared storage. The traditional answers all add cost and complexity:

• Hardware SAN / NAS: a third box to buy, power, and maintain, and a new single point of failure.
• Other 2-node storage solutions: a witness appliance or quorum device that must run somewhere else, plus a licensing layer.
• Scale-out storage clusters (including XOSTOR): a third node as arbiter, which is more infrastructure than a small office, remote site, or edge deployment can justify.
• No shared storage at all: no live migration, no HA, manual disaster recovery.

TwinStor removes the dilemma: install it on both hosts, point it at a free local disk on each, and you get a fully functional shared Storage Repository. Nothing besides the two hosts.

How it works, from a user's point of view

Both hosts hold an identical, continuously synchronized copy of every byte, and both serve storage at all times. There is no passive node and no failover delay, because there is no failover.

Both hosts active · one replicated storagesynchronous replicationlocal I/O

• Reads are local. Every VM reads from its own host's disk: zero network hop, full NVMe/SSD performance.
• Writes are safe. A write is acknowledged to the VM only once it is stored on both hosts (synchronous replication). Lose a server at any instant: not a single acknowledged write is lost.
• No tiebreaker infrastructure. Where competitors need a witness node, TwinStor uses the network gateway you already have.
• Zero patches to XCP-ng. The storage appears as a standard shared SR, so live migration, snapshots, backups, XO and the whole existing stack work unmodified, with any XCP-ng version. Your support path stays standard.
• Safe on commodity drives. TwinStor detects volatile write caches on consumer SSDs and disables them (re-applied at every boot), so an acknowledged write is on persistent media before the ack. Enterprise drives with power-loss protection keep their full cache performance.

Under the hood, TwinStor is a single small service in dom0 orchestrating battle-tested standard components (DRBD for replication, iSCSI with multipath for the data path, XHA for fencing). You never interact with those components directly: one command installs, one command sets up, one command tears down.

The data promise

Every failure behavior in TwinStor reduces to four commitments:

1. A write your VM was told is safe is never silently destroyed. If the two copies ever diverge, TwinStor only ever discards writes that no VM received a completion for. The losing side of a partition freezes instead of serving, so diverged history never accumulates.
2. When the evidence is ambiguous, it stops and asks. No verdict means no action: both copies are kept, an alert names the decision that is yours, and waiting costs nothing. TwinStor never silently picks a side it cannot justify.
3. Decisions are made on evidence, not a single glance. A winner verdict requires the pool's answer to be stable across a 45-second sampling window, and a host that is actively running your VMs is never discarded automatically, whatever the control plane claims.
4. Everything is accountable. Every incident is a numbered episode in the logs from open to close, every resolution names the VMs it touched, and every state that waits for a human is listed in one table (see Failure behavior).

The trade-off underneath: the CAP theorem

Any system that keeps two live, synchronized copies of your data runs into the CAP theorem. The name stands for Consistency, Availability, and Partition tolerance, and the theorem says you can hold at most two of the three at once. A partition (the network between the two copies breaking) is the one you cannot design away, so the real question is what happens during one: either every node stays available and keeps serving, or the two copies stay perfectly consistent, but not both at the same time. No replicated storage gets around this. Products differ only in how they choose to spend the trade-off.

TwinStor spends it in two stages.

Most partitions are made to cost nothing. When a node loses contact with its peer, it first works out whether it is the side that got cut off. If it is, it stops serving and freezes rather than hand out data that might be stale. In the usual case only one side keeps running, the two copies never diverge, and there is nothing to reconcile afterwards. And because replication is synchronous (a write is acknowledged only once it is safely on both copies), the great majority of partitions cost you no acknowledged data at all.

A genuine split is the one case that has to cost something. If both sides really did keep writing while they could not see each other, CAP guarantees that one side's most recent work has to go. TwinStor's choice here is to bring your VMs back up and heal on its own rather than freeze the whole pool until you step in. The price is a rollback of the last few seconds the losing side had acknowledged on its own, in the short moment before it stepped aside. That rollback is always bounded to those last seconds, it never reaches back into older data, and it is never silent: a priority-1 alert names the VMs it touched.

Why a direct link (arbitrated mode) helps

You only pay the CAP price during a partition, and on a two-node pool the usual cause of a "partition" is fairly mundane: the switch between the hosts having a bad moment, a reload, a firmware update, a flapping port.

Arbitrated mode gives replication its own back-to-back cable between the two hosts. A switch problem then stops being a storage partition at all, because replication keeps flowing over the cable, so the most common failure never reaches the CAP trade-off. To get a real partition you would now have to lose the switch path and the cable at the same instant, which is far less likely. None of this repeals CAP, since nothing can, but you meet the trade-off far less often and the window is shorter when you do. That is why, on any network where the hosts and the gateway can fail independently, the direct link is a safety requirement rather than a nicety.

How unlikely is a real split, in practice?

The rollback above is the worst case, and it only happens when a very specific chain of failures lands at the same moment. TwinStor stacks several independent checks in the way, and a real split has to get past all of them at once:

1. The link has to be genuinely cut, not just flicker. A brief drop (a cable bounce, a switch reconvergence) is ridden out and reconnects with nothing lost. The cut has to persist for seconds to count at all.
2. On a redundant network, the cable and the switched path have to fail together. Lose only one and replication simply keeps flowing over the other, with no split.
3. Both hosts have to still reach the gateway while they cannot reach each other. This is the only situation in which neither host can tell that it is the isolated one. On a single-switch setup it essentially cannot happen, because the peer and the gateway sit on the same wire: lose one and you lose the other, and the cut-off host steps aside on its own.
4. Both hosts have to already be actively serving. A host that is booting or rejoining is held back until it can confirm the other side, so it can never wake up into a split.
5. All of that has to happen inside the short window before HA notices the split and fences one side for you.

Only when every one of those lines up at the same instant does a rollback occur, and even then it is at most the last few seconds on the losing side, only for a VM that was being written to on that side, and always announced by an alert. On a normal single-switch deployment the chain cannot complete at all. On a redundant network it takes a genuine double failure arriving in the same moment. That is why the event stays rare and always bounded, and it is the main reason the direct link is worth the extra cable.

See Pick your topology to choose the mode that matches your network.

When not to use TwinStor

• You need more than 2 nodes. TwinStor is strictly a 2-node product. For scale-out, use XOSTOR or a SAN.
• You need more than one level of redundancy. TwinStor keeps 2 copies of your data. One failed disk means running without redundancy until it is replaced and resynced; if you need 3-way replication, use XOSTOR.
• You need multiple storage tiers. TwinStor exposes a single SR. There is no fast-SSD tier plus slow-HDD tier.
• Your network is a single unreliable switch and you cannot add a direct link between the hosts. See the topology discussion in Installation.

In this section

• Installation: prerequisites, topology choice, install and setup wizard, verification, uninstall.
• Administration: day-to-day usage, CLI, monitoring and alerts, planned maintenance.
• Failure behavior: what heals itself, what fails closed, and the few states that wait for your decision.
• Updates: updating TwinStor and the hosts under it.
• Troubleshooting and feedback: logs, support bundles, and how to report what you find.