Commit Graph

5160 Commits

Author SHA1 Message Date
Dietmar Maurer 164ad7b706 sgutils2: use thiserror to derive Error 2021-04-12 13:27:34 +02:00
Thomas Lamprecht a5322f3c50 buildsys: fix restore package names
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:52:19 +02:00
Thomas Lamprecht fa29d7eb49 ui: improve code-readability s/tapestore/tapeStore/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:34:26 +02:00
Thomas Lamprecht a21f9852fd enable tape backup by default
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:31:56 +02:00
Thomas Lamprecht 79e2473c63 d/control: file restore: only recommend proxmox-backup-restore-image
should not be a hard dependency, as one can use the file-restore tool
for pxar archives without it too

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 07:56:31 +02:00
Thomas Lamprecht 375b1f6150 d/control: rename proxmox-file-restore to proxmox-backup-file-restore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 07:56:31 +02:00
Dietmar Maurer 109ccd300f cleanup: move tape SCSI code to src/tape/drive/lto/sg_tape/ 2021-04-09 11:34:45 +02:00
Thomas Lamprecht c287b28725 buildsys: dinstall: only install server/client debs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 10:09:15 +02:00
Dietmar Maurer c560cfddca tape: read_drive_status - ignore media changed sense info 2021-04-09 09:46:19 +02:00
Dietmar Maurer 44f6bb019c sgutils2: implement scsi_request_sense() 2021-04-09 09:46:19 +02:00
Thomas Lamprecht d6d42702d1 bump version to 1.0.14-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:59:07 +02:00
Thomas Lamprecht 3fafd0e2a1 d/postinst: check for old tape.cfg
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:59:07 +02:00
Thomas Lamprecht 59648eac3d avoid extra separate upload to pbs repo
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:45:30 +02:00
Thomas Lamprecht 5b6b5bba68 upload file restore package only to PVE for now
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:45:30 +02:00
Stefan Reiter b13089cdf5 file-restore: add 'extract' command for VM file restore
The data on the restore daemon is either encoded into a pxar archive, to
provide the most accurate data for local restore, or encoded directly
into a zip file (or written out unprocessed for files), depending on the
'pxar' argument to the 'extract' API call.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:43:41 +02:00
Stefan Reiter 1f03196c0b tools/zip: add zip_directory helper
Encodes an entire local directory into an AsyncWrite recursively.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 14:32:03 +02:00
Stefan Reiter edf0940649 pxar/extract: add sequential variant of extract_sub_dir
extract_sub_dir_seq, together with seq_files_extractor, allow extracting
files from a pxar Decoder, along with the existing option for an
Accessor. To facilitate code re-use, some helper functions are extracted
in the process.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:24:23 +02:00
Stefan Reiter 801ec1dbf9 file-restore(-daemon): implement list API
Allows listing files and directories on a block device snapshot.
Hierarchy displayed is:

/archive.img.fidx/bucket/component/<path>
e.g.
/drive-scsi0.img.fidx/part/2/etc/passwd
(corresponding to /etc/passwd on the second partition of drive-scsi0)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 14:24:14 +02:00
Stefan Reiter 34ac5cd889 debian/client: add postinst hook to rebuild file-restore initramfs
This will be triggered on updating proxmox-file-restore (via configure,
necessary since the daemon binary might change) and
proxmox-backup-restore-image (via 'activate-noawait', necessary since
the base image might change).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:20:05 +02:00
Stefan Reiter 58421ec112 file-restore: add basic VM/block device support
Includes methods to start, stop and list QEMU file-restore VMs, as well
as CLI commands do the latter two (start is implicit).

The implementation is abstracted behind the concept of a
"BlockRestoreDriver", so other methods can be implemented later (e.g.
mapping directly to loop devices on the host, using other hypervisors
then QEMU, etc...).

Starting VMs is currently unused but will be needed for further changes.

The design for the QEMU driver uses a locked 'map' file
(/run/proxmox-backup/$UID/restore-vm-map.json) containing a JSON
encoding of currently running VMs. VMs are addressed by a 'name', which
is a systemd-unit encoded combination of repository and snapshot string,
thus uniquely identifying it.

Note that currently you need to run proxmox-file-restore as root to use
this method of restoring.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:11:02 +02:00
Stefan Reiter a5bdc987dc add tools/cpio encoding module
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:10:45 +02:00
Stefan Reiter d32a8652bd file-restore-daemon: add disk module
Includes functionality for scanning and referring to partitions on
attached disks (i.e. snapshot images).

Fairly modular structure, so adding ZFS/LVM/etc... support in the future
should be easy.

The path is encoded as "/disk/bucket/component/path/to/file", e.g.
"/drive-scsi0/part/0/etc/passwd". See the comments for further
explanations on the design.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:03:54 +02:00
Stefan Reiter a26ebad5f9 file-restore-daemon: add watchdog module
Add a watchdog that will automatically shut down the VM after 10
minutes, if no API call is received.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:58:29 +02:00
Stefan Reiter dd9cef56fc file-restore-daemon: add binary with virtio-vsock API server
Implements the base of a small daemon to run within a file-restore VM.

The binary spawns an API server on a virtio-vsock socket, listening for
connections from the host. This happens mostly manually via the standard
Unix socket API, since tokio/hyper do not have support for vsock built
in. Once we have the accept'ed file descriptor, we can create a
UnixStream and use our tower service implementation for that.

The binary is deliberately not installed in the usual $PATH location,
since it shouldn't be executed on the host by a user anyway.

For now, only the API calls 'status' and 'stop' are implemented, to
demonstrate and test proxmox::api functionality.

Authorization is provided via a custom ApiAuth only checking a header
value against a static /ticket file.

Since the REST server implementation uses the log!() macro, we can
redirect its output to stdout by registering env_logger as the logging
target. env_logger is already in our dependency tree via zstd/bindgen.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 13:57:57 +02:00
Stefan Reiter 26858dba84 server/rest: add ApiAuth trait to make user auth generic
This allows switching the base user identification/authentication method
in the rest server. Will initially be used for single file restore VMs,
where authentication is based on a ticket file, not the PBS user
backend (PAM/local).

To avoid putting generic types into the RestServer type for this, we
merge the two calls "extract_auth_data" and "check_auth" into a single
one, which can use whatever type it wants internally.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Stefan Reiter 9fe3358ce6 file-restore: allow specifying output-format
Makes CLI use more comfortable by not just printing JSON to the
terminal.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Dominik Csapak 76425d84b3 file-restore: add binary and basic commands
For now it only supports 'list' and 'extract' commands for 'pxar.didx'
files. This should be the foundation for a general file-restore
interface that is shared with block-level snapshots.

This is packaged as a seperate .deb file, since for block level restore
it will need to depend on pve-qemu-kvm, which we want to seperate from
proxmox-backup-client.

[original code for proxmox-file-restore.rs]
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

[code cleanups/clippy, use helpers::list_dir_content/ArchiveEntry, no
/block subdir for .fidx files, seperate binary and package]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Thomas Lamprecht 42355b11a4 update d/control
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 13:57:57 +02:00
Dominik Csapak 511e4f6987 ui: tape/DriveStatus: improve status grid a bit
by using format_boolean for compression/write protect,
combining file/block posiition into one (saves a line)

and adding the missing alert-flags

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak 3f0e344bc1 ui: tape/ChangerStatus: hide selector for single drives in barcode-label
it is rather pointless to let the user select something were there
is no choice. We have to keep the window though, since the user may
want to choose a pool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak a316178768 ui: tape/ChangerStatus: shortcut Inventory for single drives
like 'load-media'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak dff8ea92aa ui: tape/ChangerStatus: shortcut 'load-media' for single drive
if a changer only has a single drive, there is no point in showing
a window with a DriveSelector, just do want the user wanted.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak 88e1f7997c ui: tape/ChangerStatus: rework EraseWindow
to make it more like a 'dangerous' remove window
also works in the singleDrive logic to hide/show the driveselector

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak 4c3eabeaf3 ui: tape/ChangerStatus: save assigned drives
so that we can shortcut later if we only have one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak 4c7be5f59d ui: tape/ChangerStatus: add missing property
it will actually not fail, but we declare it nonetheless to indicate
that it exists

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak 6d4fbbc3ea ui: dashobard/DataStoreStatistics: add 'Available' column
for some storages, it is valuable information, e.g. if one has datastores
on separate datasets of the same zpool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:27:22 +02:00
Dietmar Maurer 1a23132262 tape: add TapeDensity::Unknown 2021-04-08 12:23:54 +02:00
Dietmar Maurer 48c4193f7c ui: update tape DriveStatus for new driver 2021-04-08 12:04:14 +02:00
Dietmar Maurer 8204d9b095 tape: avoid unneccessary SCSI request in Drop 2021-04-08 11:26:08 +02:00
Dietmar Maurer fad95a334a tape: clear encryption key after backup (for security reasons) 2021-04-08 10:37:49 +02:00
Dietmar Maurer 973e985d73 cleanup: remove unused linux tape driver code 2021-04-08 10:15:52 +02:00
Dominik Csapak e5a13382b2 ui: tape/TapeRestore: use correct value check for store & mapping
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 10:02:31 +02:00
Dominik Csapak 81c0b90447 ui: tape/TapeRestore: fix restoring without mapping
we have to delete the 'mapping' variable in any case since it's not
a valid api parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 10:02:17 +02:00
Dietmar Maurer ee9fa953de docs: Mention our new user space tape driver, adopt device path names 2021-04-08 09:50:09 +02:00
Dietmar Maurer 09acf0a70d do not depend on mt-st and mtx
We now use our own driver, so those tools are no longer required.
2021-04-08 09:48:47 +02:00
Dietmar Maurer 15d1435789 tape: add vendor, product and revision to LtoDriveAndMediaStatus 2021-04-08 08:34:46 +02:00
Dietmar Maurer 80ea23e1b9 tape: pmt - implement options command 2021-04-08 08:34:45 +02:00
Dietmar Maurer 5d6379f8db tape: implement locate_file without LOCATE(10) 2021-04-08 08:34:45 +02:00
Dietmar Maurer 566b946f9b tape: pmt - re-implement lock/unlock command 2021-04-08 07:28:30 +02:00
Dietmar Maurer 7f7459677d tape: pmt - re-implement fsr/bsr 2021-04-08 07:28:30 +02:00