Uses the ZFS utils to detect, import and mount zpools. These are
available as a new Bucket type 'zpool'.
Requires some minor changes to the existing disk and partiton detection
code, so the ZFS-specific part can use the information gathered in the
previous pass to associate drive names with their 'drive-xxxN.img.fidx'
node.
For detecting size, the zpool has to be imported. This is only done with
pools containing 5 or less disks, as anything else might take too long
(and should be seldomly found within VMs).
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
This reverts commit 75f9f40922, which is
no longer needed now that we use tokio >= 1.6 which contains the proper
fix.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
While the issue with vsock packets starving kernel memory is mostly
worked around by the '64k -> 4k buffer' patch in
'proxmox-backup-restore-image', let's be safe and also limit the number
of concurrent transfers. 8 downloads per VM seems like a fair value.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
The extract API call may be active for more than the watchdog timeout,
so a simple ping is not enough.
This adds an "inhibit" API, which will stop the watchdog from completing
as long as at least one WatchdogInhibitor instance is alive. Keep one in
the download task, so it will be dropped once it completes (or errors).
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
See this PR for more info: https://github.com/tokio-rs/tokio/pull/3756
As a workaround use a pair of connected unix sockets - this obviously
incurs some overhead, albeit not measureable on my machine. Once tokio
includes the fix we can go back to a DuplexStream for performance and
simplicity.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Used to specify a filesystem placed directly on a disk, without a
partition table inbetween. Detected by simply attempting to mount the
disk itself.
A helper "make_dev_node" is extracted to avoid code duplication.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
A bucket might contain multiple (or 0) layers of components in its path
specification, so allow a mapping between bucket type strings and
expected component depth. For partitions, this is 1, as there is only
the partition number layer below the "part" node.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
otherwise, the kernel driver exposes file names as iso 8859-1,
but we want to have them as utf8.
This mapping should always work, since UTF16 can be cleanly converted
to UTF8.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
This allows mounting XFS partitons with 'dirty' states, like from a
running VM. Otherwise XFS tries to write recovery information, which
fails on a read-only mount.
Tested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Drive serials have a character limit of 20, longer names like
"drive-virtio0.img.fidx" or "drive-efidisk0.img.fidx" would get cut off.
Fix this by removing the suffix, it is not necessary to uniquely
identify an image.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Read image sizes (.pxar.fidx/.img.didx) from manifest and partition
sizes from /sys/...
Requires a change to ArchiveEntry, as DirEntryAttribute::Directory
does not have a size associated with it (and that's probably good).
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
For the actual partitions and blockdevices in a backup, which the
user sees like folders in the file-restore ui
Encoded as "None", to avoid cluttering DirEntryAttribute, where it
wouldn't make any sense to have.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The data on the restore daemon is either encoded into a pxar archive, to
provide the most accurate data for local restore, or encoded directly
into a zip file (or written out unprocessed for files), depending on the
'pxar' argument to the 'extract' API call.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Allows listing files and directories on a block device snapshot.
Hierarchy displayed is:
/archive.img.fidx/bucket/component/<path>
e.g.
/drive-scsi0.img.fidx/part/2/etc/passwd
(corresponding to /etc/passwd on the second partition of drive-scsi0)
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Includes functionality for scanning and referring to partitions on
attached disks (i.e. snapshot images).
Fairly modular structure, so adding ZFS/LVM/etc... support in the future
should be easy.
The path is encoded as "/disk/bucket/component/path/to/file", e.g.
"/drive-scsi0/part/0/etc/passwd". See the comments for further
explanations on the design.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Add a watchdog that will automatically shut down the VM after 10
minutes, if no API call is received.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Implements the base of a small daemon to run within a file-restore VM.
The binary spawns an API server on a virtio-vsock socket, listening for
connections from the host. This happens mostly manually via the standard
Unix socket API, since tokio/hyper do not have support for vsock built
in. Once we have the accept'ed file descriptor, we can create a
UnixStream and use our tower service implementation for that.
The binary is deliberately not installed in the usual $PATH location,
since it shouldn't be executed on the host by a user anyway.
For now, only the API calls 'status' and 'stop' are implemented, to
demonstrate and test proxmox::api functionality.
Authorization is provided via a custom ApiAuth only checking a header
value against a static /ticket file.
Since the REST server implementation uses the log!() macro, we can
redirect its output to stdout by registering env_logger as the logging
target. env_logger is already in our dependency tree via zstd/bindgen.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
