The docs on devops tools and kubernetes was moved
from the Vates VMS doc to the Xen Orchestra doc.
As a result, one link to the kubernetes page broke in the XO doc.
This commit fixes that broken link.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* add more details in task log
* fix an issue with VM with more than 10 snapshot, add unit test for this parsing
* fix the real bug: vmware do not update the vmdk description file before the next VM reboot. It means that the uid of the snapshot and vm are the same, thus finishing the transfer with " nothing to transfer" . We are now using the path on source as key + add unit test + add compatibiliy with legacy VM migrated with the uid.
* add e2e test for v2v
Customer imported large VM from VMWare (> 3 To), import switched to Qcow2 automatically but the process was stuck while building the qcow2 L1/L2 address tables, where the NbdDisk's hasBlock is called for each l2 table entry which iterated over the disk dataMap which contained ~113,528 ranges. This froze the process for more than 18 minutes after which it was either killed or died due to OOM error.
This fix improves the NbdDisk by:
Updating processDatamap to throw on overlapping ranges and merge touching extents.
Override default getBlockIndexesCount with optimized Nbd one.
Improve hasBlock with a built in cursor to improve performance.
Test file added to cover the new function and updated code.
from ticket 59505
node-tar >= 7 returns a Minipass stream: it has `pipe()` but is not a `node:stream`
instance. xo-proxy serves this stream through Koa, which pipes only bodies passing
instanceof Stream` and JSON-serializes anything else — so a file restore through a
proxy answered a JSON dump of the tar object instead of the archive. Wrapping it in a
real node stream also propagates tar errors to the consumer and aborts the tar job
when the client disconnects.
The load balancer did not take into account that hosts may be in disabled state, which means it's impossible to migrate a VM to it.
This PR fixes that by ignoring these hosts instead.
Fixes ticket 62077
* feat(xo-server): serve doc locally
* the doc is already distributed from the mono repo
* add a build phase to the global yanr build that build a doc with the
right prefix
* add a http mount doc => embed build
* doc(backups): add a space calculator
* estimate the space needed for backups
* compute the duration protected by immutability
introduced by #10058
the current method assumes that source block are full but they can
be sparse. In the worst case getBlockIndexesCount can be the same as
the source
the offset must be relative to the start of THIS large block, not to the
start of the disk — otherwise every large block after index 0 lands past
the end of destinationBlock and copy() silently writes nothing (zeroed block).
affect qcow2 incremental backup with NBD disable
NBD + qcow2 and Vhd backups don't go through this code path
The previous static/js/matomo.js snippet never loaded the tracker
script, so nothing was ever recorded; it also pointed at site 5, the
XO website. Replace it with the client module from the Vates VMS docs
on the new dedicated docs site (23): cookieless page-view tracking that
follows SPA navigations, and instrumentation of the lunr search box
reporting each settled query with its suggestion count, so zero-result
searches surface missing content. The feedback widget mirrors votes and
thumbs-down reasons as doc_feedback events.
Also fills in the Formbricks IDs for the "XO Docs page feedback"
survey, enabling the feedback widget.
Swizzles DocItem/Footer to show a 'Was this page helpful?' widget under
every doc article, ported from the Vates VMS docs. Votes post to the
self-hosted Formbricks instance via its public client API; a thumbs-down
asks why (outdated, unclear, missing info, inaccurate) plus optional
free text. The vote is recorded on click so it survives the visitor
leaving before the follow-up. Page path rides along as a hidden field
for per-page filtering.
* ensure cache is generated completely at the end of the backup job
* ensure a broken cache is regenerated even without merge during the cleanup phase
* ensure a broken cache is regenerated on request
add new option and script to launch a load test :
number of vm is configurable
concurrency is configurable
number of backup run is configurable
will do multiple run, then a restore
- Show only traffic rules of VMs. Don't include `vm-snapshot` or `vm-template`
- For VM selection, display all VMs and disable those without a VIF, with a
message displayed
A VM backup log status is calculated depending on its subtask status. Previously, we made that it would be a failure if at least one of its subtask is a failure.
However, in case of successful retries, the intended behaviour is to have failed subtasks, which should not affect the parent task status. The first commit of this PR fixes that.
The second commit simplifies the getStatus function and its usage.
---------
Co-authored-by: Florent BEAUCHAMP <florent.beauchamp@vates.tech>
A server left enabled but without a Xapi instance was never retried by anything in
xo-server. Three paths lead to that state:
a failed explicit connect: xo-server boots while a pool is down, a user clicks
"connect" during an outage, or the host-eject path exhausts its 5 attempts;
a SESSION_INVALID teardown after a pool master restart (_sessionCall calls
disconnect(), and nothing listens to server:disconnected);
fixing a server's credentials/address while it is disconnected (server.set never
triggers a connection).
Once there, the server stays disconnected forever, showing an error that can be
minutes or hours old. Support ends up chasing false "pool is dead" reports, typically
right after a rolling pool update, which is exactly when hosts reboot by design.
* feat(xo-server-openmetrics): estimate per-VM power consumption from CPU
usage
* refactor(xo-server-openmetrics): move hostPowerWatts undefined guard to
caller
* fix(xo-server-openmetrics): log IPMI power read failure as info
* feat(xen-api): reconnect to a surviving pool member when the master is unreachable
XO follows a pool master change only through HOST_IS_SLAVE redirection, which
needs the contacted host to be up and answering. That covers a graceful master
hand-over (the old master stays up as a slave and redirects us), but not the
case where the master host itself becomes unreachable while another host takes
over as master: HA automatic failover, or a manual emergency transition. The
stored URL then points at a host that no longer responds, login fails with a
transport error instead of HOST_IS_SLAVE, the existing retry never redirects,
and the connection stays stuck on the dead master.
Maintain a set of pool-member addresses (seeded from the configured target and
optional opts.candidateHostnames, refreshed from host.get_all_records on every
session open) and, when the current target is unreachable at the transport
level, probe the other members in parallel. The first to answer wins; if it is
a slave, its login redirects us to the current master through the existing
HOST_IS_SLAVE retry. XapiError responses (including auth failures) are not
failed over, so wrong credentials still fail fast.
* feat(xo-server): persist pool-member addresses so master failover survives a restart
The xen-api failover relies on an in-memory set of pool-member addresses, which
is lost when xo-server restarts: it then only knows the configured host. That is
exactly the address that is dead when XO runs on the very pool whose master just
failed and was HA-restarted onto a survivor, leaving XO unable to reconnect.
Persist the member addresses on the server record (Redis, JSON-encoded like the
existing error field), refresh them whenever the pool object is seen via the
xen-api candidateHostnames getter, and feed them back as opts.candidateHostnames
when (re)building the connection. After a restart the connection can then probe
the surviving members and reconnect to the new master.
* chore(changelog): reconnect to a surviving pool member on master failure
Add the user-facing changelog entry and list xen-api in the packages to
release for the master-failover reconnect feature.
* fix(xen-api): prune ejected hosts from pool-member failover candidates
host.get_all_records is authoritative current membership, so replace the
candidate set on refresh instead of unioning into it. An ejected host is
typically still alive as its own standalone master with the same
credentials, so leaving it in the set could make a failover probe
silently connect to the wrong pool. The currently-connected target is
preserved since host.address may differ from the address XO used to
reach it.
* fix(xen-api): harden surviving-member failover per review
- reject failover probes on sessionId-based connections (no reusable
credentials to open a session on another host)
- bracket bare IPv6 addresses (host.address / HOST_IS_SLAVE) before using
them as URL hostnames
- log the probe session out even when the host answers after the probe
timeout, to avoid leaking a session
- abort the mid-session failover loop when the connection was torn down or
replaced while waiting (e.g. user disconnected the server)
- xo-server: drop persisted pool-member addresses when the connection
identity (host/credentials) changes, so a stale slave list can't send a
failover to the wrong pool
See #10016
* fix(fs): compute the output stream precisely when encrypted
the current encryption add 12 byte of initialization vector at
the start + 16 bytes of authentication tag at the end
thus, the destination stream can be bigger than the source stream,
and this length is used to compute the optimal chunk size for s3/
azure
from https://xcp-ng.org/forum/topic/12328/error-mirroring-full-backups-to-backblaze-b2
---------
Co-authored-by: Pierre Brunet <pierre.brunet@vates.tech>
* docs(xo): overhaul navbar to create a unified doc ecosystem
- Clean up the navbar by dropping the redundant GitHub link (retained in footer).
- Relocate "Blog" to the "What's new in XO 6" sidebar.
- Introduce direct navbar links for XCP-ng and Vates VMS.
- Rename "Documentation" to "Xen Orchestra".
This aligns the navigation layout across our three documentation sites, allowing readers to smoothly transition between ecosystems with a fully unified experience.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* docs(xo): move home link from header to footer
To make the navbar cleaner and make space
for the links to the other documentation websites
We're moving the Home link from the header to the footer
and renaming the link to 'About Xen Orchestra'.
Also, we're displaying the 'What's new in XO 6' section again
which now contains a link to the XO blog
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Recently we've introduced new sections for the xo6 category of the XO doc.
These categories were meant to be filled later with new pages.
The pages were created as placeholders, but didn't actually contain any text, just titles.
These made readers confused, so this commit
Hides the sections from the sidebar until we update them
With actual content.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Introduced by #9765
When a backup merge fails on a VHD file, the footer might be missing. This caused a footer1 !== footer2 error and triggered a full backup while not deleting the previous chain and resulted in all further backup jobs to be displayed as failures even though the new backup chain was working properly.
The fix allows the cleanVm to resume the merge where it stops even if the VHD file footer is missing.
when hrp get a 404, the stream stay ready to be consumed, for example to get
more information on the error
but most of the usage won't handle this case, and will keep the stream in memory
(for example a failing XC patches check) leaking memory
This fixes the calls from xo, and also identifies and fixes the point where we
need body content on 4xx code
relevant code in hrp is here:
https://github.com/JsCommunity/http-request-plus/blob/master/index.js#L149
setting undefined is a no ops, we need to use null to clean a value
this don't have any visible or functionnal impact, but complexify
the build of a filter to look for any non completely migrated VM
* fix(configuration.md): remove trailing spaces
* feat(sdn-controller.md): explain traffic rules migration path: from OpenFlow Protocol to XAPI Plugin
- add sdn-migration.py python script
- in configuration.md, refer to sdn_controller.md for changing `useDirectChannel`
- in sdn_controller.md, provide documentation on migration path
XCPNG-3260
report an error if we detect that the runtime configuration doesn't match the of-rules format stored in xapi database.
the admin is expected to ran migration script by hand to fix it.
This commit creates a new, refreshed tree structure
for the XO 6 documentation, which we intend
to be more helpful for readers and XO 6 users.
New sections are created, to be completed later.
Additionnally, the ACLv2/RBAC page has been
moved from the Support section to the Security section.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Adds rest api endpoints testing for the sdn controller.
Don't forget to set the required SDN_CONTROLLER_VM_ID to a vm that has both a vif and a network.
Run @xen-orchestra/qa-test yarn qa:sdn-controller for testing.
Add a message if you try to leave the page during XOA deployment.
Unfortunately, it is not possible to have a custom info box for the
`beforeUnload` event. So for homogeneity, `onBeforeRouteLeave` is
`window.confirm` too.
Use XO Tasks in the load balancer so users can see on the task page what the load balancer does. Now, it creates a task (with no progress) for each VM migration.
The task has a description explaining the reason of the migration, which can be accessed by opening the task log.
XO-992
Reduce memory consumption up to 700MB / TB of data ( 300-350Mb per call of getBlockIndexes().length , called twice).
Ticket: 59102
Tested in client environment. Works for a 7TB import
this enable the test to run using from and to shared storage ( nfs/samba/s3/azure)
the local path is only used for export tests
tested with a remote garage (object storage) provider, full suite is passing
this is an async bug leading to the full size not really sumed, thus
depending on the async and concurrency leading to a size 0 even if another
disk has data
* fix(backups): tolerate ENOENT when deleting missing backups
* fix(xo-server): propagate content-length in VM import
* fix(qa-test): retry REST API calls on transient 401
* fix(qa-test): harden cleanup against stale cache files
* fix(qa-test): fail fast on misconfigured test environment
Assert that source and mirror repositories are distinct before running
any backup steps, giving an actionable error when both env vars resolve
to the same remote ID. Also improve the mkdir failure message in the
export test to include the exact path and the remediation command.
* feat(qa-test): make source vm runs to ensure there is content to transfer
* fix: harden job deletion
* fix(qa-test) don't use a deprecated route for authent
* fixup! feat(qa-test): make source vm runs to ensure there is content to transfer
* fixup! fix(backups): tolerate ENOENT when deleting missing backups
* fixup! fix(backups): tolerate ENOENT when deleting missing backups
Implements XO-2155. The OpenMetrics plugin now ships XOSTOR-specific metrics so operators can build a Grafana dashboard for XCP-ng Hyperconverged Infrastructure.
All metrics use the xcp_xostor_* prefix:
Cluster status: xcp_xostor_up, xcp_xostor_node_status (master / satellite roles + raw state from linstor-manager.healthCheck), xcp_xostor_resource_total.
Storage pool health: xcp_xostor_resource_state_count — replica counts per disk-state (UpToDate, Inconsistent, Outdated, Diskless, Unknown), aggregated per cluster.
Performance filter: a new sr_type label on every SR-tagged metric (xcp_host_disk_iops_*, xcp_host_disk_throughput_*, latency, xcp_sr_*, xcp_vdi_*, xcp_vm_disk_*). Dashboards can filter sr_type="linstor" without changing existing queries.
Active alarms: xcp_xostor_alarms_count aggregated by alarm_name (ALARM, BOND_STATUS_CHANGED, MULTIPATH_PERIODIC_ALERT) and target_type (sr / host).
Disk SMART: xcp_xostor_disk_smart_status per (host, device) with the overall-health string in a status label, sourced from smartctl.py.
Pending updates: xcp_xostor_package_update_available for xcp-ng-linstor, linstor-satellite, linstor-controller, xcp-ng-release-linstor, and xcp-ng-xapi-plugins.
Each collector caches at its own TTL (60 s for cluster and alarms, 5 min for SMART, 1 h for updates) and exposes an xcp_xostor_<feature>_up gauge so silent collection failures still show up on dashboards. A TtlCache<T> helper encapsulates the cache-plus-coalescing pattern shared by the three caching collectors.
Replace the hard-coded CONFIRM_PATTERNS list in the MCP server with a per-endpoint exposure decision carried by an x-mcp-exposure OpenAPI vendor extension. Each REST endpoint now declares its MCP exposure (allow / confirm / deny) with @Extension('x-mcp-exposure', ...) next to its HTTP method decorator. A new local ESLint rule mcp/require-mcp-expose makes the decision mandatory: any new @Get/@Post/@Put/@Patch/@Delete method without an exposure tag fails lint.
The MCP parseSwagger now reads the extension and applies a default-deny policy:
'allow' → exposed as a query/action tool
'confirm' → exposed only when XO_MCP_ENABLE_ACTIONS=1, with confirm-token round-trip
'deny' / absent → never exposed
Defense in depth is preserved: route-filter regex (binaries, streams, stats) is applied before the exposure check, so a binary endpoint mistakenly tagged 'allow' still stays out of the MCP surface. The confirm_token UUID + TTL machinery in risk-config.mts is untouched.
All 248 endpoints across 34 controllers have been annotated with conservative defaults (GET → allow, destructive POST/DELETE/PUT/PATCH → confirm, binary exports → deny).
the number of snapshots will be as high as the retention selected, but this is not really
an issue since those VM will be run only when a disaster occurs. In this case, degraded
performance is an accepted tradeoff
Introduced by 7a8ca
When creating an HVM VM from a template that already has a disk attached, the boot order was being incorrectly modified to put the network first, resulting in the VM attempting a PXE boot before eventually falling through to the disk.
The boot order was modified whenever vdis and existingVdis were both empty, since no new disk needs to be provisioned. This caused network to be put first in the boot order regardless of whether a network install was actually requested.
Boot order modification now depends exclusively on installMethod:
Network: network (n) moved to the front of the string, this is unchanged.
CD: DVD-drive (d) moved to the front of the string, this is a fix to another issue where HVM + CD installs did not update the boot order string at all.
Everything else: String is unchanged, preserving the template's original order
Also, previously creating a diskless HVM VM with no install repository would get the network moved to the front of the boot order, that implicit behavior is gone, now diskless VMs now inherit the template's boot order.
This means that if in the template boot order, the network is last, the BIOS will try to find the disk and the cd first before trying the network. This is acceptable as these steps are fairly short.
following the symmetrical backup PR
reuse any snapshot in common between the source and destination if
* using incremental replication
* only one target
* a common snapshot exists between source and destination , even from another job, based on the CONTENT_KEY
The same rule of reuse the existing VM target / create a new one of symmetrical backups applies
if the destination don't have any data between the snapshot and the active disk
update an existing VM
else create a new VM but chain disks
When a Redis entry has an ID but no corresponding data key, undefined was returned which caused errors when new data was added.
Error case was the configuration of a plugin failing to save.
Move ipmi sensor discovery to it's own plugin and removed old logic, ships with a default configuration with the same behaviour as the old system.
The plugin can be configured via the UI, accepts an exact match for the vendor and regexes for the ipmi matching.
Tested via unit tests.
Tested with R620-L1 host on xo-lab by adding and removing Dell from configured vendors
- Add `getRequiredEnv()`: replaces all `process.env.X || 'default'` fallbacks
with a hard throw so a missing or typo'd .env key fails immediately with a
diagnostic message instead of silently operating on wrong defaults
- Add `logSetup.js`: dual-transport logging — console filtered by $DEBUG,
plus a full debug-level file in /tmp for post-mortem inspection
- Add `assertRepositoryMatchesConfig()`: fails fast when a named repository
in XO points at a different path than the one declared in .env
- Make `getAllowedCleanupPaths()` lazy so env vars are read after .env is
loaded, not at ESM module evaluation time
- Pass full error objects (not `.message`) to all structured log calls
- Add retry logic on flaky operations and surface more detail on failures
- Fix: only purge non-empty backup repositories
- Fix: backup jobs no longer skipped under some conditions
- Fix: missing `backuprepository.delete` function
- Add `.USAGE.md` and update `README.md`
add xo-server-ssh executable that ease a connection to the hypervisors
Without parameters it list the pool uuid/master ip
With a pool uuid parameter it opens a sshg connection to the pool master
With a pool uuid and ip, it will reuse the pool credentials to connect to the slave host
* feat(mcp): support HTTP_PROXY, HTTPS_PROXY, NO_PROXY env vars
* chore(changelog): mention MCP proxy env var support
* refactor(mcp): delegate proxy env handling to undici
the cleanVm script was a monolith in charge of cleaning the backup,
detecting any corrupted data, deleting the irrecoverable one, removing
and merging older backups
this PR do a complete rewrite, using the RemoteDisk abstraction and add
tests
The transport-nagios plugin lets users monitors backup jobs from the Nagios interface.
This was previously documented only in a Xen Orchestra blog post.
This commit completes the existing Backup reports page, by explaining how
to get backup reports in Nagios, using that plugin.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
This PR contains several commits to do some code cleanup in xo-server-sdn-controller.
They doesn't belong to any specific tasks, but it was done while working in the code.
They will be better upstreamed than in my local worktree.
unify createLogger inside xo-server-sdn-controller package : it simplifies filter usage in configuration to have debug output of the sdn-controller
in openflow-plugin.js, avoid using log.error() for a not fatal error but use log.warn(). reorder a bit the throw code to make it more readable (the error is throw only if no plugin call successed)
Fixes:
* better support of localized systems
* types
* add a retry on chattr error, selinux and / or xfs seems to need a little more time before locking it
* force a complete check on startup to remove immutables files that should not be froma previous run
* don't try to lock deleted files
docs(sdn_controller.md): fix error in code sample
As reported by Jon Sands, the previous version of the code sample showed the script executing line by line. It works as expected after removing the -xe
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
docs(aclv2): fix broken links
The ACLv2 page was created and merged at roughly the same time we split the documentation pages into two sections (XO6 and XO5), so the change in file structure broke some links to and from that ACLv2 page.
This commit fixes the links so that the Docusaurus build can complete without errors.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
XO6 has become the default interface for Xen Orchestra,
but not all features have been ported to XO5 yet
and they don't work the same way.
To make the documentation easier to maintain
and prevent confusion,
we're moving the current pages to a new section called 'XO 5'
and create a new section for XO6-specific documentation. The XO 6 section will be completed gradually.
On top of that, this commit updates internal links and redirections
to adapt to this new doc architecture.
Finally, most of the pages in the Support category
have been moved to a new section called General support,
since these pages apply the same way to XO 5 and XO 6.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
- ancestor (blocks 1, 3) merged into child (blocks 1, 2) -> 3 blocks
- cleanAncestor merged into clean -> 0 blocks
grandchild is not merged so its size is not counted here
* fix(backups): record actual incremental VHD size
* fix(backups): record merged VHD size from merge result
* fix(backups): only rewrite metadata after actual merge
* fix(backups): record actual incremental VHD size
it will be used by xapi to know if the OpenFlow channel is used and so if port 6653 needs to be opened or not.
it might also be used later to know how to interprete of-rules in the migration path from channel to xapi-plugin, if needed.
XCPNG-2470
XO-2282
It could happens if xapi database is out-of-sync with filesystem.
As the purpose of this block is to decide if the certificate should be installed,
assume that on CERTIFICATE_DOES_NOT_EXIST you need to install the new certificate.
For strong passwords, you need a good source of entropy (true randomness).
lodash.sample's randomness relies on the JavaScript runtime's PRNG, which might
not have enough entropy.
on topology changes (VM started/stopped/migrated/...), network rules needs to be
refreshed as they are installed on each port of virtual switch.
add the same logic than VIF rules to refresh them.
Enable the usage of `cookie` parameter exposed by sdn-controller xapi plugin
- https://github.com/xcp-ng/xcp-ng-xapi-plugins/pull/58
It fixes several cases where OF rules where leaked and left behind.
Upgrade notes (apply to Network Rules only, VIF Rules are not impacted):
- a new xapi-plugin with old XO is compatible (and no behaviour change)
- an old xapi-plugin with new XO will fail to communicate
XO will try to use `cookie` but xapi-plugin will complain about invalid parameter
- a new xapi-plugin with new XO will use new behaviour
and existing OF rules will migrate smoothly from old to new behaviour
When VM_destroy is called on a default template, it is first converted to a simple template (and then to a VM).
After converting a default template to a simple template, the object received after the xapiObjectToXo transformation has a different ID (the default template's ID = pool.uuid + template.uuid, and non default template ID = template.uuid).
Since this object does not already exist in the XO server cache(because the ID is new), XO treats it as a new object and not as an update, so the old object with ID pool.uuid + template.uuid stay in the cache
* fix(backups): use a delta transfer even if the VM is modified
* test(backup): fix date test without timezone
* refactor(backups): extract method that check which snapshot can be reused for replica target
* reset before launching the replication
xo-server uses two libraries for its web server:
http-server-plus — a multi-protocol server facade that multiplexes HTTP and HTTPS listeners behind a single EventEmitter interface ( introduced here JsCommunity/http-server-plus@ec63f3f )
stoppable — wraps the server to track open connections for graceful shutdown, using an internal reqsPerSocket Map (server._pendingSockets) (introduced here : 782505b )
Fix
@vates/http-server-plus (fork of JsCommunity/http-server-plus): add 'secureConnection' to the forwardedEvents list so that webServer.on('secureConnection', handler) is propagated to underlying https.Server instances at listen() time.
xo-server/src/index.mjs: register the missing handler that stoppable was supposed to add but never did:
Change the backups so they stop using their own task system, and use the same system as XO Tasks.
For the moment, it shouldn't affect the users, but when the previous backup logs will be old enough to be erased, this will allow us to have shorter loading times for backup logs.
XO-52
* test(replication): test the new snapshot based replication
* fix(tests): use mergeBackupsSynchronously to ensure the backups don't fails with lock already heled error
* fix: ensure the repository that will be purged start empty
* test(backup): add test for full mirror backups
* fix compression not used on front whe copying a VM
* ensure the first bytes are ready before starting to import the VM
---------
Co-authored-by: Florent BEAUCHAMP <florent.beauchamp@vates.tech>
Fixes#7773
See Zammad#53857
On Netbox >= 4.4 (which introduced platform hierarchies):
- If the major version number is available: create a versioned child platform (e.g. "Debian 12") with the distro-only platform (e.g. "Debian") as its parent, and assign the child to the VM
- Otherwise: keep previous behaviour and assign the VM to the distro-only platform
* chore: upgrade dependencies with known security vulnerabilities
* Update Node.js version in CI workflow
* chore: upgrade additional security dependencies
* chore: stabilize yarn.lock path-to-regexp entry
* chore: bump minimum version ranges for security-patched dependencies
* chore: update CHANGELOG.unreleased.md with security upgrades
* chore: revert vite upgrade to 6.4.1 (6.4.2 breaks vue-tsc)
* chore: fix vite upgrade and pin vue to 3.5.28
* feat(mcp): add list_srs and get_sr_details tools for storage
repositories
* docs(mcp): add list_srs and get_sr_details to documentation
* fix(mcp): extract expected tool names into a constant in tests
* docs(mcp): add link to live filter syntax documentation for all filter
parameters
* feat(disk-cli): implement disk cli tooling
* changelog
* fix test path
* fix following review
* fix following rebase
* ci
* fix following rebase
* more tests
* fix(backups): use the snapshot of the source
introduced by 9524
use only the vdi from the source VM as the source on backup
this bug could only occurs if the replication was intra pool, if
not the different xapi was filtering the results
When editing VM RAM, the high-level `memory` field dispatches to `memoryMax` or
`_memory` depending on whether the VM uses DMC. Both paths need `limitName:
'memory'` on their spec so the resource set quota counter is correctly updated,
but `_memory` was missing it, causing memory resizes on non-DMC VMs to silently
create a stale `_memory` counter instead of updating the actual memory quota.
Fixes out of range error during merge that was happening on delta backups when the child disk was bigger than the parent disk but no blocks where written in the "new" space.
Also added a test for this error.
Fix npx @xen-orchestra/mcp failing with "Permission denied".
tsc does not preserve the executable bit on compiled files, so bin pointing to ./dist/index.mjs (mode 644) caused the error. This adds a bin/xo-mcp.mjs wrapper tracked by git with mode 755, following the same pattern as other packages (@xen-orchestra/xapi, @xen-orchestra/proxy, etc.).
Add optional transient parameter (boolean) to host.disable API endpoint.
When transient=true, passes auto_enable=false to XAPI, keeping the host disabled across reboots and toolstack restarts (requires XAPI >= 25.31.0). Default behavior unchanged (backward compatible).
XAPI 25.31.0 introduced a new auto_enable parameter to Host.disable (xen-api#6652). When set to false, the host stays disabled regardless of toolstack restarts, host reboots, or consider_enabling_host calls. Only an explicit Host.enable can re-activate it.
This PR adds the new @xen-orchestra/mcp package — an MCP (Model Context Protocol) server that enables AI assistants (Claude Desktop, Claude Code, or any MCP-compatible client) to query a Xen Orchestra infrastructure in read-only mode.
The size is computed from the blocksize * nb allocated blocks during the backups
which is not aligned with the real size on disk
Checking the file on disk is too costly for vhd directory
This will be eventually reimplemented by the next cleanVm rework
This pull request documents the manual steps that will be needed for the SDN controller, with the incoming March 2026 update of XCP-ng that will include OpenSSL3.
OpenSSL-3 (on xcp-ng) requires CA set to true in basicConstraints.
"All CAs should have the CA flag set to true."
For context, here are additional resources:
- https://docs.openssl.org/3.0/man1/openssl-verification-options/#basic-constraints
- https://docs.openssl.org/1.0.2/man1/x509v3_config/#basic-constraints
- https://datatracker.ietf.org/doc/html/rfc5280#section-4.2.1.9
To connect to OpenSSL 3, certificates should be re-generated if they
already exist (just remove xo-server/data/sdn-controller).
Renewed certs were tested on XCP-ng host on "OpenSSL 1.0.2k-fips"
without regression, so it is safe to assume those ext are designed to
be backward compatible.
Observed issue:
openssl verify \
-CAfile .../xo-server/data/sdn-controller/ca-cert.pem \
.../xo-server/data/sdn-controller/client-cert.pem
C=XX, L=Default City, O=Default Company LTD
error 79 at 1 depth lookup: invalid CA certificate
error .../xo-server/data/sdn-controller/client-cert.pem: \
verification failed
The expected output is now:
.../xo-server/data/sdn-controller/client-cert.pem: OK
The expected output is now:
.../xo-server/data/sdn-controller/client-cert.pem: OK
It was tested along ovsdb-server (Open vSwitch) 2.17.7 :
openssl s_client \
-connect 10.1.75.62:6640 \
-cert .../xo-server/data/sdn-controller/client-cert.pem \
-key .../xo-server/data/sdn-controller/client-key.pem
(...)
CONNECTED(00000003)
Can't use SSL_get_servername
(...)
verify error:num=18:self-signed certificate
(...)
SSL handshake has read 2713 bytes and written 3799 bytes
Verification error: self-signed certificate
(...)
New, TLSv1.2, Cipher is AES128-GCM-SHA256
(...)
{"id":"echo","method":"echo","params":[]}
Extra info at:
- https://xapi-project.github.io/new-docs/design/pool-certificates/
Origin: https://github.com/vatesfr/xen-orchestra/pull/9507
Relate-to: XCPNG-2710
Relate-to: XO-1986
Signed-off-by: Philippe Coval <philippe.coval@vates.tech>
permit to have working traffic rules when a pool is in a 'degraded' mode (for
example, one host temporary down or unreachable).
currently, exceptions occurs and nothing happens (no rules added or deleted).
with it, unreachable hosts are ignored and reachable hosts properly updated.
The deletion of old backups is done globally
For Full backups/replication:
* retention is computed on the total of the target
* choose the storage with the most free size ( if possible, since some backup repositories don't have any size used/ free )
* one vm is written on the same target ( xva, index, checksum )
* healthcheck ( if enabled ) will run only on the target that received the data
* the cleanup of old backups/replication is done on the right target ( but considering the full lenght across all the targets )
For incremental:
* delta chain are continued on the same target
For mirror :
if a vm has multiple backups to transfer during the same run, then
they will all go to teh same remote
UX :
* show a toggle on remote/sr selectors
* toggle is disabled if 1 or less targets are selected
* for backup and mirror backup forms
* show the details in the job list
More details on the PR page
this merge code leverage the new classes introduced to handle qcow2, improving our flexibility, and future uses.
this also improve the performance on "block mode" backups (s3/ azure and other storage with the option), since it reliably updates the block allocation table only at the end of the merge instead of each time a new block is detected
when the connection to the host break between the snapshot() and
the update of other config
this could leave snapshot that will never be cleaned by any backup job
The fix is that the snapshot will be named with a specific name, it
will cause an issue if a user reuse this name
* fix(backup-reports): prevent succesfull backups being sometimes reported as interrupted
* increase delay after user feedback
* change comment according to delay change
---------
Co-authored-by: Florent BEAUCHAMP <florent.beauchamp@vates.tech>
* feat(xo-server-openmetrics): add SR capacity metrics
* fix(xo-server-openmetrics): refactor SR types and add host labels for
local SRs
---------
Co-authored-by: Florent BEAUCHAMP <florent.beauchamp@vates.tech>
In some cases, when converting an xapi object into an xo object, the xapi.is_a_snapshot param may be set to false even tho the xapi.snapshotOf refers to another xapi object. Handle this case better.
Add Affinity rule option to the load balancer plugin.
This options ensures some VMs are positioned on the same host, when it's possible.
It's important to be careful while configuring the Affinity tags and Anti-affinity tags fields, otherwise it may lead to conflicts between the two behaviours. In such case, affinity takes precedence over anti-affinity.
This commit renames and updates the **Docker management** section in the Xen Orchestra documentation. We explain a but better how containers are supported in Xen Orchestra, and remove links and references to the now-removed Docker support section, which was triggering a Docusaurus warning at build time.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Add two new API methods to download XCP-ng/XenServer system status (bug reports) via xo-cli:
host.getSystemStatus - Download system status from a single host
pool.getSystemStatuses - Download system status from all hosts in a pool in parallel
---------
Co-authored-by: Mathieu Piton <mathieu.piton@vates.tech>
* feat(xo6): added loader in treeview when search is triggered
* update changelog file
* fix(xo6): code cleanup following PR feedback
* fix order packages in changelog file
* feat(xo6): added new links in the user menu and customized it
* udpate Changelog file
* fix(xo6): code cleanup following PR feedback
* fix order packages in changelog file
docs: upgrade docusaurus from version 3.8 to 3.9
The onBrokenMarkdownLinks option is deprecated and has been replaced by siteConfig.hooks.onBrokenMarkdownLinks. However, since the option was configured to 'warn' and is the default behavior for this option, deleting the onBrokenMarkdownLinks seems like easier
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
- State to use a single XOA instance with SDN Controller plugin enabled
- Cleanup existing OpenFlow documentation
- Document XAPI plugin usage, including xo-cli commands
- Add configuration to toggle between direct channel and XAPI plugin
* feat(xo6): add loader on input search treeview when is triggered
* udpate Changelog file
* fix(xo6): removed condition on props icon in UiInput
* fix(xo6): removed debounced on clear
Several sections of the Xen Orchestra Appliance documentation mention modifying the config file, without specifying the actual file path. This commit updates those references to explicitly include /etc/xo-server/config.toml, which helps users locate and edit the correct configuration file more easily.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Changing the value of a `Size` component by editing the field and pressing
'Enter' was executing twice the `_save()` function in parallel (which calls the
`onChange` function).
In some cases like editing a running VM memory limit, this caused errors (trying
to open two modals at the same time, and trying to restart the VM twice)
* docs(xo): change link to the xcp-ng docs, add some infos in HA section
* Update docs/docs/manage_infrastructure.md
Co-authored-by: Thomas Moraine <thomas.moraine@vates.tech>
* Update docs/docs/manage_infrastructure.md
Co-authored-by: Thomas Moraine <thomas.moraine@vates.tech>
* Update docs/docs/manage_infrastructure.md
Co-authored-by: Thomas Moraine <thomas.moraine@vates.tech>
---------
Co-authored-by: Thomas Moraine <thomas.moraine@vates.tech>
docs(manage_infrastructure.md): remove content related to xscontainer, which is deprecated
Since XSContainer is no longer supported on XCP-ng 8.3 or XenServer and
we currently have no alternative, instructions related to XScontainer in
the XO doc are removed
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Updating the `Toggle` component to avoid throwing an error coming from a
`confirm` modal, which ends up appearing in the console.
This also changes a bit the behaviour of the `Toggle` component, which now
awaits its `onChange` function.
See Zammad#45442
Wrap "Copy to clipboard" button with a span to make the hoverable zone
continuous between the copiable element and the button. Without this, the 1ex
gap between the 2 was sometimes enough to make the button disappear before the
user had a chance to click it.
Adds a page to the Xen Orchestra documentation to list the compatible S3-based
storage support providers for Xen orchestra, with the different support tiers
and appropriate caveats.
* feat(sdn-controller): use XCP plugin to manage open switch
try to instantiate the plugin and fall back to direct query
if it fails
The plugin allow to define and target multiple bridges in parallel
* feat(sdn-controller): implement network wide rules
this introduce 2 new API method sdnController.addNetworkRule
and sdnController.deleteNetworkRule
Theses methods will only work with the plugin, and can't work with
the legacy ( direct channel) approach
Since the 5.110 release of Xen Orchestra, the Vmware to Vates (V2V) process has been updated, with a whole new backend. This commit documents this new process, in the XO documentation
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Pass the license object to isTrialRunning instead of passing the parent object.
See Zammad#44878 Zammad#44755 Zammad#44431 Zammad#43526 Zammad#44556 ...
Introduced by #8557
See #8523 → #8551 → #8557
The modal to activate HA used to suggest SRs from all pools, but selecting a SR from another pool would result to an error. This change makes that modal only suggest SRs from the correct pool.
* fix(nbd-client): better handling of error during getMap
* fix(xo-server): esxi check errors are objects instead of text
* fix(xo-web): V2v/check improve alignment
* feat(xo-web): add button to install nbdinfo/nbdkit if needed
Added a new section to the Incremental backups documentation to explain why some deltas can be unexpectedly large. Covers common causes and includes tips for investigation and mitigation.
This helps clarify expected behavior and avoids confusion when backup sizes seem too large.
add tips for 2tb+ disks and qcow disks
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
---------
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* docs(xo): explain the usage reports plugin more explicitly
Previously, it wasn't clear what the toggle called All did. This commit makes the documentation more clear about how it works.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* Update docs/docs/manage_infrastructure.md
Co-authored-by: Pierre Donias <pierre.donias@gmail.com>
---------
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Co-authored-by: Pierre Donias <pierre.donias@gmail.com>
Make the load balancing documentation clearer, better structured, with more illustrations, so that users can follow the process of creating or editing a load balancing plan without friction or hassle
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
The README now specifies that Xen Orchestra documentation updates are deployed automatically once per day. This makes it clearer for users that the online docs are always up to date without requiring manual intervention.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Network bonding was absent from the Xen Orchestra documentation. Information about this feature was previously only available in a blog article on XCP-ng.
This commit updates the XO doc so that it properly documents this feature, and remains the single source of truth in this topic. XO users won't have to dig through blog posts to find the relevant information, making the documentation more useful and trustworthy.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* feat(lite/release): check that version matches build version
1. Let Vite's build get the last commit hash instead of passing an env variable
2. Emit a build file containing the current commit hash and package version
3. Before releasing a new version, compare hash and version with current git commit and version passed by user
* readJson
In the REST API, we generally work on authenticated requests. getCurrentUser can therefore be a bit tedious, as it can return undefined | XoUser. So, whenever we need the current user in an authenticated request, we need to add a check:
if user === undefined throw for typescript to be happy
So now: getCurrentUser returns XoUser (throw if no user) and getCurrentUser({throwUnauthenticated: false}) returns undefined | XoUser
Enhancements:
Fixed @vates/types
Added limitAndFilterArray in utils.helper to easily handle user filter/limit everywhere
Enhance getObject type in rest-api
makeObjectMapper can take a function to define the path (for path based on the object type)
add getVmVdis in vmService to get VDIs from a VM
Endpoints:
/vms/:id/vdis
/vm-snapshots/:id/vdis
/vm-controllers/:id/vdis
/vm-templates/:id/vdis
* docs(xo): deduplicate acls content
The ACLs page in the XO documentation wasn't listed in the navigation bar and had content that duplicated the ACLs section of the Users page.
This commit removes the standalone page and updates the valid section with minor improvements (presentation, some missing info from the lone ACLs page) so that it becomes the only source of information on the topic.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* docs(xo): remove obsolete administration page
The Administration page duplicates content in the VMs section of the Infrastructure management page.
Although the Administration page seemingly refers to itself as a 'section', it contains no subsections or extra content, so this might as been a draft for a bigger page, and was never elaborated upon.
In any case, this page isn't accessible from the navigation bar and doesn't seem to add much value to the doc, so I suggest we remove it from the tree view.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* docs(xo): deduplicate backups and disaster recovery content
The 'Backups and Disaster Recovery' page in the XO documentation wasn't listed in the navigation bar and had content that was already present in other pages in the doc.
The last section of that page dealt with scheduled snapshots and completes the existing content on 'rolling snapshots'. That content has been added to the rolling snapshots section, so that everything can be grouped in a single location.
Finally, since the Backups and disaster recovery page is now obsolete, it has been deleted.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* Update docs/docs/users.md
Co-authored-by: Laure S. <laure.souai@vates.tech>
---------
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Co-authored-by: Laure S. <laure.souai@vates.tech>
The Xen Orchestra documentation moved to a new domain name a while ago. This commit updates the last remaining links in the XO doc and app that pointed to the old domain name
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* docs: deduplicate delta_backups page
Previously, the content of the Incremental Backups page (filename: delta_backups.md) was duplicated by the page titled "Incremental Backups (formerly: Continuous Delta backups)", which was not referenced in the sidebar and had the filename incremental_backups.md.
We don't want duplicate content, as it is troublesome to maintain and may cause confusion for the reader.
Also, since Delta backup is outdated terminology and we now talk about Incremental backups, the lone incremental backups.md page has been deleted, and the current "Incremental Backups" page with the filename delta_backups.md has been renamed as the new incremental_backups.md
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* docs: deduplicate full_replication page
Previously, the content of the Full Replication page (filename: disaster_recovery.md) was duplicated by the page titled "Full Replication (formerly: Disaster recovery)", which was not referenced in the sidebar and had the filename full_replication.md.
We don't want duplicate content, as it is troublesome to maintain and may cause confusion for the reader.
Also, since Disaster recovery is outdated terminology and we now talk about Full Replication, the lone full_replication.md page has been deleted, and the current "Full Replication" page with the filename disaster_recovery.md has been renamed as the new full_replication.md
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
---------
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* fix(nbd-client): handle unreachable VM through nbd
when the user select nbd but don't have any way to reach
the storage/host of the VM (for example no NBD enabled network,
or a NBD enabled network unreachable by xo/proxy it was timing out
and blocking the start of the backup
This error was blocking the fallback to export stream mechanism
* fix(xapi): respect the config file setting for preferNbd
* fix(xapi): don't close export stream if nbd failed to connect
* fix(xapi):respect vhd/qcow2 when faling back to a full if delta exports fails
doc(configuration) document let's encrypt support in xoa
Add documentation explaining how to configure Let's Encrypt in Xen Orchestra Appliance (XOA). This feature was mentioned in the XO 5.73 release blog post but was not included in the technical documentation for Xen Orchestra.
* feat(xo6/lite): update side panel behavior for mobile view
* feedback & improvement
* feedback
* feedback
* fix issue with sticky side panel when datatable has a lot of data
* feedback and fix issue
* add changelog
* update icon
* review feedback
* update vts copy button icon
This PR implement a limited support of generating and reading qcow2 . The support is limited since it only handles qcow2 with 64KB cluster size, with this structure :
header
refcount table
refcount blocks
L1 table
L1 tables
data clusters
generated by a9cd5bc639/scripts/qcow2-to-stdout.py
Add safety functions in abstract instead of s3 and azure. A new function "withTimeout" allows to handle timeout and success/failure after timeout.
It should help for better debug and it can be used in several situations.
XO6 is translated into languages, not existing for XO5. And as it is not
possible to select language in XO6 yet (is inherited from XO5 settings), interim
solution is to add these new languages into XO5 user settings, to make them
actually accessible in XO6. It will not affect XO5, because it will simply
fallback into English in case of not existing language.
Add a step-by-step guide for creating VM templates with cloudbase-init
support, including master VM creation, template branching, customization,
reseal and VM instantiation.
Include best practices, tips and visual aids for each step.
Signed-off-by: Tu Dinh <ngoc-tu.dinh@vates.tech>
remove a value that was only used for backup reports, and was no longer used since we use the template system
avoid the backup report content to differ when the backup job has reportWhen = false, which also caused differences between regular backup reports and backup reports generated with the "test plugin" button. To make sure the users can still have the previous behavior, we add parameter in the backup job configuration that allows short failure reports by not displaying all successful VMs in error reports.
rewording "successful VMs" to "successful items" so it also makes sense for metadata backups
Introduced by #6684
If sub-properties are required but the parent property is not, then omitting
the parent property passes the validation, which is not what we want.
The Viridian setting in Xen Orchestra was mentioned in a release blog (XO 5.79) but not in the XO technical documentation.
This improvement explain what Viridian is and how to enable it for VMs and templates, from within XO.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
The documentation for mutliple plugins was updated or moved outside the general Plugins page over time.
As a result, the links contained in various plugin's Readme or Usage files got obsolete, as they all point to the general Plugins page, instead of the specific plugin documentations.
This commit updates those links with the relevant location.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Previously, the Readme and Usage files related to the server-usage-report plugin contained links pointing to a general page on plugins.
The server-usage-report plugin has since been documented in the Management → Manage infrastructure management page of the Xen Orchestra documentation. As a result, these links have become obsolete.
This commit make the obsolete links point to the actual, new documentation on the server-usage-report plugin.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
See xoa#234
Prevent importing hub templates that are actually not templates.
Importing an incorrect template previously created a VM without any way to know it.
The Alerts page was not listed in the sidebar and duplicated content from the Alerts section in the Advanced page. This commit removes the standalone Alerts page and redirects all links pointing to it to the Alerts section of the Advanced page. This change also fixes a broken anchor warning in Docusaurus, which could prevent new documentation builds from succeeding.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* doc(XO): Explain VIFs and Network locking mode
VIFs and the Network locking mode were previously missing in the Xen Orchestra technical documentation, with details scattered across XO blog posts, CLI references in the XCP-ng doc, and other sources.
This commit adds information on VIFs and the network working modes to the XO documentation, making it a more complete and reliable reference for users.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
* Update manage_infrastructure.md
---------
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Co-authored-by: Julien Fontanet <julien.fontanet@vates.tech>
Fixes Zammad#38602
Introduced by #7247
Avoid immediately falling back to a hard shutdown when calling `VM.stop`,
especially in cases where a clean shutdown was requested explicitly by the user
like in `VM.setAndRestart`.
Rolling pool update (RPU) scheduling was introduced in Xen Orchestra 5.102 and mentioned in the corresponding blog post, but it was not included in the XO technical documentation.
This commit explains how to schedule RPUs, so users no longer need to refer to blog posts for instructions.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
As mentioned in the XO 5.104 release blog post, we're gradually documenting the Xen Orchestra REST API using Swagger.
The REST API page in our technical documentation already has a 'Future Improvements' section, which seems like a good place to mention Swagger, even though the documentation process is still ongoing.
This way, users won't need to check the XO blog to learn about this feature.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
The GitHub link in the footer was the default one used by the Docusaurus template.
This commit changes the link so that it points to the Xen Orchestra GitHub repository.
Correct a malformed link to the Support -> XOA Support -> XOA check section in the XO documentation, ensuring Docusaurus builds the doc properly.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
The General Troubleshooting page is not listed in the sidebar and contains minimal content, making it suitable to be merged into the existing Troubleshooting page, which is already accessible. Consequently, the standalone General Troubleshooting page is no longer needed and is being removed.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
While Software RAID is provided as-is, Xen Orchestra allows users to monitor the health of their software RAID arrays.
This includes a dedicated status interface and alert messages for any issues.
This information was previously documented in the Xen Orchestra 5.102 release blog post, and is now integrated into the XO technical documentation.
This update ensures users have a single reference point for technical information, eliminating the need to search through blog posts.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Differential restores were mentioned in the XO blog (release 5.89)
but not in the XO technical documentation.
This commit explains in the technical doc what differential restores are
and how to perform one from the XO interface.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Fixes Zammad#37604
Introduced by 36b94f745d
Since version 4, Redis' Node library doesn't support passing multiple flat
arguments to its methods to perform an action on multiple entries. It expects
an array instead.
we must compute a relevant partSize when uploading a file bigger than 50G
bigger partSize means more memory consumption, but we are limited to
upload 10K chunk of partSize bytes
this commit will ensure there is an estimate of the size from encrpted remote
and pass this estimate to the FullRemoteWriter
from ticket 38031
The usage-reports plugin was not covered at all in the Xen Orchestra technical documentation.
This improvement explains how to get usage reports from your resources, by enabling and setting up the plugin.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
The XCP-ng doc has been recently updated with more details on how to migrate from VMware to a Vates environment,
with a choice of methods including V2V within Xen Orchestra.
As a result, a mention of V2V is added to the XO doc, with a link to the relevant section in the XCP-ng doc.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
from ticket 37120
an EPERM error during a rmtree , called by the cleanup at the end
of a metdatadat backup, will trigger a ENOTDIR error
this can be caused by a misconfigured/failing immutable backup
remote without any VM ( either empty remote of config only remote)
are flooding the logs with errors like
Error: ENOENT: no such file or directory, scandir '/run/xo-server/mounts/f0952d37-294a-4d5e-8ba5-565e9932b4a7/xo-vm-backups'
I have checked the source code, and it uses a "Contains" and not some kind of "StartsWith" to check whether the disk name should be excluded.
I think it is important to clarify that it is possible to place the "[NOBAK]" label after the disk name, which might help some to maintain their naming convention.
- Updated VM creation code to match the TS typing and remove errors in `type-check`
- Refactored code
- Simplified and improved business logic
- Improved naming
Introduced by #3774
Introduced by #8412
ACLs feature is only available for XOA plans Enterprise and Premium. The
feature is properly disabled on the main ACL config page but was still
available from a VM's Advanced page and from the VM creation form.
Implement the mecanism to expose non XAPI object endpoints:
Added a new type, NonXapiXoRecord, which represents all XO objects that are not XAPI objects (e.g., users, servers, groups, etc.).
Split XapiXoController to create BaseController, allowing code reuse between XapiXoController and the new XoController.
BaseController exposes sendObjects and createAction (unchanged) and enforces subclasses to define the getObject and getObjects methods.
XoController requires subclasses to implement getAllCollectionObjects and getCollectionObject, as these are used in its getObjects and getObject methods.
filter and limit are handled by xo-server for XAPI objects. XoController implements its own filter and limit mechanisms for all non XAPI object in getObjects
And finally expose /rest/v0/servers and /rest/v0/servers/<server-id>🥳
Fix `Unsupported header 'x-amz-checksum-mode' received for this API call.`
on backblaze/cloudlfare object storage
from version 3.729, aws sdk send headers that aren't supported yet by backblaze
it happens when the source disk is not aligned with the vhd block size ( 2MB)
this PR will read more data into the grain directory / grain tables, but
this data won't be used since we use the virtual size to compute which
grain to read
from ticket 33018
doc(XO): Add Amazon S3 to list of supported remotes
Amazon S3 remotes have been supported for a while,
but there was no section mentioning it in the Backups → Remotes doc.
This is now fixed
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Also, replace the screenshot of the file restore feature with an example that uses Linux instead of a Microsoft/NTFS resource
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Introduced by 4eed730a09
Due to an issue with bump-pkg script, some packages were previously published on
NPM before their dependencies' versions were updated. This commit simply bumps
the versions of those packages to publish the correct list of dependencies.
@vates/types Added an export to the root of the lib because TSOA requires importing the type from the root library to generate the OpenAPI specification
@vates/types Tuples ([number, number]) throw errors when generating the OpenAPI specification. They have been updated to number[]
@vates/types Created a XapiXoRecord to be able to differentiate between XO types that come from xapi-object-to-xo and XO types that come from XO collections
xo-server Added getObjectsByType to avoid iterating over all XAPI objects when we know what type of object we want
Branded is not a valid type for the OpenAPI spec. I created a Unbrand type, that convert all Branded to string . (Use Unbrand only for OpenAPI specification definition to maintain Branded type safety)
Added another error middleware to catch TSOA errors and throw XO errors
Added a README.md to have consistency of order in the use of decorators
Document how to restore a file from a VHD with fuse-vhd CLI.
This will allow users to restore by themselves large files that make the restore fail when done from the UI.
Configure dependency injection + configure authentication middleware for openapi (copy of what actually exists in the current REST API. Will be improved later when implementing ACL/resource set)
A forum user suggested changes for better understanding of VM templates.
A new page has been created just for templates,
with Create, Read, Update, Delete (CRUD) sections.
Also, the Cloud-init/Cloudbase-init section of the Advanced page has been moved
to this new page.
This should make it easier for users to get all useful information on templates
in a single document.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Update the Xen Orchestra documentation to guide users
on transitioning from AES-256-GCM to ChaCha20-Poly1305 encryption,
solving an issue where users could encounter warnings due to mixed
encryption algorithms in remote storage.
Fix [this]((https://xcp-ng.org/forum/topic/10453/remote-encryption-algorithm)
Some users weren't able to sync all LDAP groups with XO.
Adding what to check to troubleshoot the issue.
Also, refreshing the style and presentation of the LDAP section in the Users page
XCP-ng 8.0 and 7.6 are references in multiple sections.
These versions are obsolete, nobody should be using SDN Controller on anything older than XCP-ng 8.2.
As a result, references to XCP-ng 8.0 and older can be removed safely.
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Sometime, the name does not provide enough information about the VM, adding more with Description, we will know more which VM has issue with backup.
Co-authored-by: b-Nollet <150058367+b-Nollet@users.noreply.github.com>
Since XOSAN is now legacy and XOSTOR has taken its place, we are removing XOSAN from the Xen Orchestra doc.
Links to XOSAN docs are replaced with links to XOSTOR (from the XCP-ng doc).
Signed-off-by: Thomas Moraine <thomas.moraine@vates.tech>
Remember server's pool name and description, and display the pool name as server default label. The server label can be deleted (since 81e7e07) to get back to the default pool name value.
We did multiple commits to keep the original changes around, but I would advise not to review by commit.
XO-437
* feat(backups): don't use checksum if not needed
encrypted backups are already authenticated, this guarantee that
a readble file has not been modified
thus, no need to compute a relatively slow hash is encryption is already done
In some failure cases, backup reports were not sent because of an error during handlebars compilation.
The `formatDate` function was incorrectly passed from partial to partial causing the `executeFunction` helper to execute `undefined` as a function (the `vmFailure` helper used default `formatDate=./formatDate` instead of `formatDate=../formatDate`, the `..` being necessary because of the `#each` loop)
Introduced by aa9c49ce6a
Xen Orchestra now supports Cloudbase-init (for Windows VMs),
on top of the already-supported Cloud-init (for Linux VMs).
The Cloud-init section is therefore expanded to mention Cloudbase-init,
and we explain how to create a template for Windows VMs using Cloudbase-init.
@@ -16,7 +16,7 @@ Returns a promise wich rejects as soon as a call to `iteratee` throws or a promi
-`concurrency`: a number which indicates the maximum number of parallel call to `iteratee`, defaults to `10`. The value `0` means no concurrency limit.
-`signal`: an abort signal to stop the iteration
-`stopOnError`: wether to stop iteration of first error, or wait for all calls to finish and throw an `AggregateError`, defaults to `true`
-`stopOnError`: whether to stop iteration of first error, or wait for all calls to finish and throw an `AggregateError`, defaults to `true`
@@ -34,7 +34,7 @@ Returns a promise wich rejects as soon as a call to `iteratee` throws or a promi
-`concurrency`: a number which indicates the maximum number of parallel call to `iteratee`, defaults to `10`. The value `0` means no concurrency limit.
-`signal`: an abort signal to stop the iteration
-`stopOnError`: wether to stop iteration of first error, or wait for all calls to finish and throw an `AggregateError`, defaults to `true`
-`stopOnError`: whether to stop iteration of first error, or wait for all calls to finish and throw an `AggregateError`, defaults to `true`
* @param {Array|Buffer|string} data1 - The first data array or buffer to compare.
* @param {T} data2 - The second data array or buffer to compare.
* @param {(index: number, diff: T) => void} [cb] - The callback function to invoke for each difference. If not provided, an array of differences will be returned.
* @returns {Array<number|T>|undefined} - An array of differences if no callback is provided, otherwise undefined.
* @returns {Array<number|T>|undefined} - An array of differences if no callback is provided; otherwise, undefined.
@@ -28,7 +28,7 @@ fs.stat('autoexec.bat', function (e, stats) {
## API
-`fs = fatfs.createFileSystem(vol, [opts], [cb])` — Simply pass in a block driver (see below) mapped to a FAT partition somewhere, and get back the API documented [here](http://nodejs.org/api/fs.html). An options dictionary can be provided, details are in the next section. You may also optionally provide a callback `cb(err)` which will be automatically registered for the on `'ready'` or `'error'` event.
-`fs = fatfs.createFileSystem(vol, [opts], [cb])` — Simply pass in a block driver (see below) mapped to a FAT partition somewhere, and get back an instance of an [`fs`](http://nodejs.org/api/fs.html). An options dictionary can be provided, details are in the next section. You may also optionally provide a callback `cb(err)` which will be automatically registered for the on `'ready'` or `'error'` event.
-`'ready'` event — fired on `fs` when initial volume information has been determined and the API is ready to use. It is safe to call other `fs` methods before this fires **only if** you are sure the first sector will be readable and represents a valid FAT volume.
-`'error'` event — fired if initialization fails for whatever reason.
@@ -36,7 +36,7 @@ fs.stat('autoexec.bat', function (e, stats) {
The `opts` dictionary you pass to `fatfs.createFileSystem` can contain any of the following options:
-`ro` — Enables readonly mode if `true`. It defaults to `false`, but if your volume driver does not provide a `writeSectors` method it will be overriden to `true`.
-`ro` — Enables readonly mode if `true`. It defaults to `false`, but if your volume driver does not provide a `writeSectors` method it will be overridden to `true`.
-`noatime` — The FAT filesystem can track the last access time (just a date, actually) but this means every read operation would also incur some write overhead. Defaults to `true`, meaning by default access times will **not** be updated on reads. Set this to `false` to track access times.
-`modmode` — chooses how `fs.chmod` (and the mode field from `fs.stat`–family calls) should map FAT attributes to POSIX permissions. Set to the number `0111` to map the readonly flag to the user's write bit being unset, and the archive/system/hidden flags to the user/group/other executable bits respectively. Set to the number `07000` to map the readonly flag to _all_ write bits being unset, and the archive/system/hidden flags to the sticky/setgid/setuid bits respectively. Set to `null` for readonly mapping. Defaults to `0111`.
-`umask` — any bits _set_ in this octal number will be _unset_ in the 'mode' field from `fs.stat`–family calls. It does not affect anything else. Defaults to `process.umask()`, or `0022` if that is unavailable.
@@ -53,7 +53,7 @@ Well, sort of…
### Temporary
- **BETA** **BETA****BETA**. Seriously, this is a _brand new_, _from scratch_, _completely unproven_ filesystem implementation. It does not have full automated test coverage, and it has not been manually tested very much either. Please please please **make sure you have a backup** of any important drive/image/card you unleash this upon.
- **BETA** **BETA****BETA**. Seriously, this is a _brand new_, _from scratch_, _completely unproven_ filesystem implementation. It does not have full automated test coverage, and it has not been manually tested very much either. Please, please, please,**make sure you have a backup** of any important drive/image/card you unleash this upon.
- A few methods are not quite implemented, either: `fs.rename`, `fs.unlink` and `fs.rmdir`, as well as `fs.watchFile`/`fs.unwatchFile` and `fs.watch`. These are Coming Soon™.
- There are several internal housekeeping items (redundant FAT tables, extra FAT32 information, etc.) that are not done. These do not seem to affect interop, but you may see warnings when repairing a filesystem written by this module.
- Oh, and not to scare you, but if an IO error happens while writing, the library usually just bails — bubbling an error up to your callback as if it were a hot potato. Although some attempt has been made to do separate writes in the safest order (e.g. allocating an additional file cluster, then appending data into it, and then finally updating the file's size), but this behavior has not been thoroughly audited for all operations. There's certainly no attempt to retry/cleanup/rollback if a multi-step change runs into trouble partway through.
remoteUrl can be found by using cli in `@xen-orchestra/fs` , for example a local remote will have a url like `file:///path/to/remote/root`
The `remoteUrl` can be found using the following command from the XOA CLI, and then picking the right `url` property:
`xo-server-db ls remote`
The `vhdPathInRemote` follows a file structure described in [VM backups](https://github.com/vatesfr/xen-orchestra/blob/master/%40xen-orchestra/backups/docs/VM%20backups/README.md).
## Restore a file from a VHD using `fuse-vhd` CLI
1. Mount a VHD to the Filesystem (see **Usage**).
2. Verify the VHD is Correctly Mounted:
`mount | grep fuse`
3. Get the `START` and `SIZE` of the disk whose file you want to restore. Multiply `START` by 512 (block size) to get the offset value. (`mountedVhdPath` is the `mountPoint` value from earlier, followed by `/vhd0`, for example). Depending on the partition type, you may need to do some additional setup to discover the partitions.
4. Mount the disk. Depending on the partition type, you may need to add some additional options. The `norecovery` option is used for ext3/ext4/xfs file systems; otherwise, remove this option.
remoteUrl can be found by using cli in `@xen-orchestra/fs` , for example a local remote will have a url like `file:///path/to/remote/root`
The `remoteUrl` can be found using the following command from the XOA CLI, and then picking the right `url` property:
`xo-server-db ls remote`
The `vhdPathInRemote` follows a file structure described in [VM backups](https://github.com/vatesfr/xen-orchestra/blob/master/%40xen-orchestra/backups/docs/VM%20backups/README.md).
## Restore a file from a VHD using `fuse-vhd` CLI
1. Mount a VHD to the Filesystem (see **Usage**).
2. Verify the VHD is Correctly Mounted:
`mount | grep fuse`
3. Get the `START` and `SIZE` of the disk whose file you want to restore. Multiply `START` by 512 (block size) to get the offset value. (`mountedVhdPath` is the `mountPoint` value from earlier, followed by `/vhd0`, for example). Depending on the partition type, you may need to do some additional setup to discover the partitions.
4. Mount the disk. Depending on the partition type, you may need to add some additional options. The `norecovery` option is used for ext3/ext4/xfs file systems; otherwise, remove this option.
wrap a source async generator to have it respect a max speed ( in bytes per seconds).
speed is either a strictly positive number or a function returning a strictly positive number. A speed change will be used for the next emitted packet.
The source generator must yield object with a length property.
Optimized for small yields regarding to the speed, since it won't split incoming packet.
If the generator reached the max speed it will be paused, limiting memory consumption.
wrap a source async generator to have it respect a max speed ( in bytes per seconds).
speed is either a strictly positive number or a function returning a strictly positive number. A speed change will be used for the next emitted packet.
The source generator must yield object with a length property.
Optimized for small yields regarding to the speed, since it won't split incoming packet.
If the generator reached the max speed it will be paused, limiting memory consumption.
Creates a new server instance. `opts` may contain `createServer` and `createSecureServer` factories (defaults to `http.createServer` and `https.createServer`). `requestListener` is registered as a `request` event listener.
```js
import{create}from'@vates/http-server-plus'
constserver=create(functiononRequest(req,res){
res.end('Hello')
})
```
### `server.listen(opts)`
Starts listening. Returns a promise that resolves to a human-readable address string, or rejects with `error.niceAddress` set.
```js
// HTTP on a specific port
awaitserver.listen({port:8080})
// HTTPS — detected automatically when cert+key are provided
awaitserver.listen({
port:443,
cert:fs.readFileSync('certificate.pem'),
key:fs.readFileSync('private_key.pem'),
})
// Unix socket
awaitserver.listen({socket:'/var/run/http.sock'})
// File descriptor
awaitserver.listen({fd:3})
// Systemd socket activation
awaitserver.listen({systemdSocket:0})
```
> As a convenience, if `hostname` is `'localhost'`, it will listen on both `127.0.0.1` and `::1`, similar to what Node does when no hostname is provided.
Multiple endpoints can be started on the same instance:
> Augmented `http.Server`, HTTP/HTTPS/HTTP2 and multiple ports on the same instance
## Install
Installation of the [npm package](https://npmjs.org/package/@vates/http-server-plus):
```sh
npm install --save @vates/http-server-plus
```
## Usage
### `create([opts], [requestListener])`
Creates a new server instance. `opts` may contain `createServer` and `createSecureServer` factories (defaults to `http.createServer` and `https.createServer`). `requestListener` is registered as a `request` event listener.
```js
import{create}from'@vates/http-server-plus'
constserver=create(functiononRequest(req,res){
res.end('Hello')
})
```
### `server.listen(opts)`
Starts listening. Returns a promise that resolves to a human-readable address string, or rejects with `error.niceAddress` set.
```js
// HTTP on a specific port
awaitserver.listen({port:8080})
// HTTPS — detected automatically when cert+key are provided
awaitserver.listen({
port:443,
cert:fs.readFileSync('certificate.pem'),
key:fs.readFileSync('private_key.pem'),
})
// Unix socket
awaitserver.listen({socket:'/var/run/http.sock'})
// File descriptor
awaitserver.listen({fd:3})
// Systemd socket activation
awaitserver.listen({systemdSocket:0})
```
> As a convenience, if `hostname` is `'localhost'`, it will listen on both `127.0.0.1` and `::1`, similar to what Node does when no hostname is provided.
Multiple endpoints can be started on the same instance:
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.