Commit Graph

4958 Commits

Author SHA1 Message Date
Thomas Lamprecht
fa29d7eb49 ui: improve code-readability s/tapestore/tapeStore/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:34:26 +02:00
Thomas Lamprecht
a21f9852fd enable tape backup by default
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:31:56 +02:00
Thomas Lamprecht
79e2473c63 d/control: file restore: only recommend proxmox-backup-restore-image
should not be a hard dependency, as one can use the file-restore tool
for pxar archives without it too

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 07:56:31 +02:00
Thomas Lamprecht
375b1f6150 d/control: rename proxmox-file-restore to proxmox-backup-file-restore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 07:56:31 +02:00
Dietmar Maurer
109ccd300f cleanup: move tape SCSI code to src/tape/drive/lto/sg_tape/ 2021-04-09 11:34:45 +02:00
Thomas Lamprecht
c287b28725 buildsys: dinstall: only install server/client debs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-09 10:09:15 +02:00
Dietmar Maurer
c560cfddca tape: read_drive_status - ignore media changed sense info 2021-04-09 09:46:19 +02:00
Dietmar Maurer
44f6bb019c sgutils2: implement scsi_request_sense() 2021-04-09 09:46:19 +02:00
Thomas Lamprecht
d6d42702d1 bump version to 1.0.14-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:59:07 +02:00
Thomas Lamprecht
3fafd0e2a1 d/postinst: check for old tape.cfg
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:59:07 +02:00
Thomas Lamprecht
59648eac3d avoid extra separate upload to pbs repo
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:45:30 +02:00
Thomas Lamprecht
5b6b5bba68 upload file restore package only to PVE for now
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 18:45:30 +02:00
Stefan Reiter
b13089cdf5 file-restore: add 'extract' command for VM file restore
The data on the restore daemon is either encoded into a pxar archive, to
provide the most accurate data for local restore, or encoded directly
into a zip file (or written out unprocessed for files), depending on the
'pxar' argument to the 'extract' API call.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:43:41 +02:00
Stefan Reiter
1f03196c0b tools/zip: add zip_directory helper
Encodes an entire local directory into an AsyncWrite recursively.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 14:32:03 +02:00
Stefan Reiter
edf0940649 pxar/extract: add sequential variant of extract_sub_dir
extract_sub_dir_seq, together with seq_files_extractor, allow extracting
files from a pxar Decoder, along with the existing option for an
Accessor. To facilitate code re-use, some helper functions are extracted
in the process.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:24:23 +02:00
Stefan Reiter
801ec1dbf9 file-restore(-daemon): implement list API
Allows listing files and directories on a block device snapshot.
Hierarchy displayed is:

/archive.img.fidx/bucket/component/<path>
e.g.
/drive-scsi0.img.fidx/part/2/etc/passwd
(corresponding to /etc/passwd on the second partition of drive-scsi0)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 14:24:14 +02:00
Stefan Reiter
34ac5cd889 debian/client: add postinst hook to rebuild file-restore initramfs
This will be triggered on updating proxmox-file-restore (via configure,
necessary since the daemon binary might change) and
proxmox-backup-restore-image (via 'activate-noawait', necessary since
the base image might change).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:20:05 +02:00
Stefan Reiter
58421ec112 file-restore: add basic VM/block device support
Includes methods to start, stop and list QEMU file-restore VMs, as well
as CLI commands do the latter two (start is implicit).

The implementation is abstracted behind the concept of a
"BlockRestoreDriver", so other methods can be implemented later (e.g.
mapping directly to loop devices on the host, using other hypervisors
then QEMU, etc...).

Starting VMs is currently unused but will be needed for further changes.

The design for the QEMU driver uses a locked 'map' file
(/run/proxmox-backup/$UID/restore-vm-map.json) containing a JSON
encoding of currently running VMs. VMs are addressed by a 'name', which
is a systemd-unit encoded combination of repository and snapshot string,
thus uniquely identifying it.

Note that currently you need to run proxmox-file-restore as root to use
this method of restoring.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:11:02 +02:00
Stefan Reiter
a5bdc987dc add tools/cpio encoding module
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:10:45 +02:00
Stefan Reiter
d32a8652bd file-restore-daemon: add disk module
Includes functionality for scanning and referring to partitions on
attached disks (i.e. snapshot images).

Fairly modular structure, so adding ZFS/LVM/etc... support in the future
should be easy.

The path is encoded as "/disk/bucket/component/path/to/file", e.g.
"/drive-scsi0/part/0/etc/passwd". See the comments for further
explanations on the design.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:03:54 +02:00
Stefan Reiter
a26ebad5f9 file-restore-daemon: add watchdog module
Add a watchdog that will automatically shut down the VM after 10
minutes, if no API call is received.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:58:29 +02:00
Stefan Reiter
dd9cef56fc file-restore-daemon: add binary with virtio-vsock API server
Implements the base of a small daemon to run within a file-restore VM.

The binary spawns an API server on a virtio-vsock socket, listening for
connections from the host. This happens mostly manually via the standard
Unix socket API, since tokio/hyper do not have support for vsock built
in. Once we have the accept'ed file descriptor, we can create a
UnixStream and use our tower service implementation for that.

The binary is deliberately not installed in the usual $PATH location,
since it shouldn't be executed on the host by a user anyway.

For now, only the API calls 'status' and 'stop' are implemented, to
demonstrate and test proxmox::api functionality.

Authorization is provided via a custom ApiAuth only checking a header
value against a static /ticket file.

Since the REST server implementation uses the log!() macro, we can
redirect its output to stdout by registering env_logger as the logging
target. env_logger is already in our dependency tree via zstd/bindgen.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 13:57:57 +02:00
Stefan Reiter
26858dba84 server/rest: add ApiAuth trait to make user auth generic
This allows switching the base user identification/authentication method
in the rest server. Will initially be used for single file restore VMs,
where authentication is based on a ticket file, not the PBS user
backend (PAM/local).

To avoid putting generic types into the RestServer type for this, we
merge the two calls "extract_auth_data" and "check_auth" into a single
one, which can use whatever type it wants internally.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Stefan Reiter
9fe3358ce6 file-restore: allow specifying output-format
Makes CLI use more comfortable by not just printing JSON to the
terminal.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Dominik Csapak
76425d84b3 file-restore: add binary and basic commands
For now it only supports 'list' and 'extract' commands for 'pxar.didx'
files. This should be the foundation for a general file-restore
interface that is shared with block-level snapshots.

This is packaged as a seperate .deb file, since for block level restore
it will need to depend on pve-qemu-kvm, which we want to seperate from
proxmox-backup-client.

[original code for proxmox-file-restore.rs]
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

[code cleanups/clippy, use helpers::list_dir_content/ArchiveEntry, no
/block subdir for .fidx files, seperate binary and package]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Thomas Lamprecht
42355b11a4 update d/control
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 13:57:57 +02:00
Dominik Csapak
511e4f6987 ui: tape/DriveStatus: improve status grid a bit
by using format_boolean for compression/write protect,
combining file/block posiition into one (saves a line)

and adding the missing alert-flags

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
3f0e344bc1 ui: tape/ChangerStatus: hide selector for single drives in barcode-label
it is rather pointless to let the user select something were there
is no choice. We have to keep the window though, since the user may
want to choose a pool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
a316178768 ui: tape/ChangerStatus: shortcut Inventory for single drives
like 'load-media'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
dff8ea92aa ui: tape/ChangerStatus: shortcut 'load-media' for single drive
if a changer only has a single drive, there is no point in showing
a window with a DriveSelector, just do want the user wanted.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
88e1f7997c ui: tape/ChangerStatus: rework EraseWindow
to make it more like a 'dangerous' remove window
also works in the singleDrive logic to hide/show the driveselector

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
4c3eabeaf3 ui: tape/ChangerStatus: save assigned drives
so that we can shortcut later if we only have one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
4c7be5f59d ui: tape/ChangerStatus: add missing property
it will actually not fail, but we declare it nonetheless to indicate
that it exists

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:56:46 +02:00
Dominik Csapak
6d4fbbc3ea ui: dashobard/DataStoreStatistics: add 'Available' column
for some storages, it is valuable information, e.g. if one has datastores
on separate datasets of the same zpool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 13:27:22 +02:00
Dietmar Maurer
1a23132262 tape: add TapeDensity::Unknown 2021-04-08 12:23:54 +02:00
Dietmar Maurer
48c4193f7c ui: update tape DriveStatus for new driver 2021-04-08 12:04:14 +02:00
Dietmar Maurer
8204d9b095 tape: avoid unneccessary SCSI request in Drop 2021-04-08 11:26:08 +02:00
Dietmar Maurer
fad95a334a tape: clear encryption key after backup (for security reasons) 2021-04-08 10:37:49 +02:00
Dietmar Maurer
973e985d73 cleanup: remove unused linux tape driver code 2021-04-08 10:15:52 +02:00
Dominik Csapak
e5a13382b2 ui: tape/TapeRestore: use correct value check for store & mapping
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 10:02:31 +02:00
Dominik Csapak
81c0b90447 ui: tape/TapeRestore: fix restoring without mapping
we have to delete the 'mapping' variable in any case since it's not
a valid api parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-08 10:02:17 +02:00
Dietmar Maurer
ee9fa953de docs: Mention our new user space tape driver, adopt device path names 2021-04-08 09:50:09 +02:00
Dietmar Maurer
09acf0a70d do not depend on mt-st and mtx
We now use our own driver, so those tools are no longer required.
2021-04-08 09:48:47 +02:00
Dietmar Maurer
15d1435789 tape: add vendor, product and revision to LtoDriveAndMediaStatus 2021-04-08 08:34:46 +02:00
Dietmar Maurer
80ea23e1b9 tape: pmt - implement options command 2021-04-08 08:34:45 +02:00
Dietmar Maurer
5d6379f8db tape: implement locate_file without LOCATE(10) 2021-04-08 08:34:45 +02:00
Dietmar Maurer
566b946f9b tape: pmt - re-implement lock/unlock command 2021-04-08 07:28:30 +02:00
Dietmar Maurer
7f7459677d tape: pmt - re-implement fsr/bsr 2021-04-08 07:28:30 +02:00
Dietmar Maurer
0892a512bc tape: correctly set/display drive option 2021-04-08 07:28:30 +02:00
Dietmar Maurer
b717871d2a sgutils2: add scsi_mode_sense helper 2021-04-08 07:28:30 +02:00