Commit Graph

5494 Commits

Author SHA1 Message Date
Dominik Csapak 82a4bb5e80 api2/tape/restore: return backup manifest in try_restore_snapshot_archive
we'll use that for partial snapshot restore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-07 12:25:30 +02:00
Fabian Grünbichler 94bc7957c1 progress: shorter format
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-07 12:14:37 +02:00
Fabian Grünbichler c9e6b07145 progress: add current group to output
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-07 12:14:23 +02:00
Thomas Lamprecht 3c06eba17a docs: online help info: surpress warnings during scan
We get lots of warnings due to sphinx complaining about missing
includes for generated synopsis. We do not reference to any of those
for now, so we can ignore that now and supress all standard and
warning output.

Note: Errors are still reported.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-07 11:51:30 +02:00
Fabian Grünbichler 8081e4aa7b fix #3331: improve progress for last snapshot in group
especially for the last group, without this the progress would report:

"percentage done: 100.00% (1 of 2 groups, 1 of 1 group snapshots)"

instead of the more logical

"percentage done: 100.00% (2 of 2 groups)"

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-07 11:20:17 +02:00
Dietmar Maurer d8769d659e use build.rs to pass REPOID to rustc-env 2021-05-07 10:11:39 +02:00
Stefan Reiter 572cd0381b file-restore: add debug mode with serial access
Set PBS_QEMU_DEBUG=1 on a command that starts a VM and then connect to
the debug root shell via:
  minicom -D \unix#/run/proxmox-backup/file-restore-serial-10.sock
or similar.

Note that this requires 'proxmox-backup-restore-image-debug' to work,
the postinst script is updated to also generate the corresponding image.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-07 10:00:12 +02:00
Thomas Lamprecht 5e91b40087 d/control: update for cargo manifest update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-07 09:26:41 +02:00
Stefan Reiter 936eceda61 file-restore: support more drives
A PCI bus can only support up to 32 devices, so excluding built-in
devices that left us with a maximum of about 25 drives. By adding a new
PCI bridge every 32 devices (starting at bridge ID 2 to avoid conflicts
with automatic bridges), we can theoretically support up to 8096 drives.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-07 09:03:17 +02:00
Stefan Reiter 61c4087041 file-restore: add more RAM for VMs with many drives or debug
The guest kernel requires more memory depending on how many disks are
attached. 256 seems to be enough for basically any reasonable and
unreasonable amount of disks though.

For debug instance, make it 1G, as these are never started automatically
anyway, and need at least 512MB since the initramfs (especially when
including a debug build of the daemon) is substantially bigger.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-07 09:03:17 +02:00
Stefan Reiter 7d39e47182 file-restore: try to kill VM when stale
Helps to clean up a VM that has crashed, is not responding to vsock API
calls, but still has a running QEMU instance.

We always check the process commandline to ensure we don't kill a random
process that took over the PID.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-07 09:03:17 +02:00
Wolfgang Bumiller c4e1af3069 make sure URI paths start with a slash
Otherwise we get an empty error message.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-07 08:46:47 +02:00
Dietmar Maurer 3e234af16e tape: improve inline docs for READ POSITION LONG 2021-05-06 11:45:40 +02:00
Dietmar Maurer bbbf662d20 tape: use LOCATE(16) SCSI command
Turns out this works on LTO4 and newer.
2021-05-06 10:51:59 +02:00
Wolfgang Bumiller 25d78b1068 client: use build_authority in build_uri
so we don't need to also duplicate the IPv6 bracket logic

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-06 10:27:40 +02:00
Dietmar Maurer 78bf292343 call create_run_dir() at daemon startup 2021-05-06 10:23:54 +02:00
Dietmar Maurer e5ef69ecf7 cleanup: split SimpleHttp client into extra file 2021-05-06 10:22:24 +02:00
Dominik Csapak b7b9a57425 api2/tape/restore: remove unnecessary params from (try_)restore_snapshot_archive
we do not need them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-06 08:02:14 +02:00
Dominik Csapak c4a04b7c62 api2/tape/restore: factor out check_datastore_privs
so that we can reuse it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-06 08:01:31 +02:00
Dominik Csapak 2e41dbe828 tape/media_catalog: add helpers to look for snapshot/chunk files
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-06 07:58:03 +02:00
Dominik Csapak 56d36ca439 tape/drive: add 'move_to_file' to TapeDriver trait
so that we can directly move to a specified file on the tape

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-06 07:55:08 +02:00
Dietmar Maurer e0ba5553be http proxy: add necessary brackets for IPv6 proxy 2021-05-05 11:57:04 +02:00
Dominik Csapak 8d6fb677c1 proxmox_restore_daemon: mount ntfs with 'utf8' option
otherwise, the kernel driver exposes file names as iso 8859-1,
but we want to have them as utf8.

This mapping should always work, since UTF16 can be cleanly converted
to UTF8.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-05 11:07:31 +02:00
Dietmar Maurer a2daecc25d client/http_client: add necessary brackets
if we are given a 'naked' ipv6 without square brackets around it,
we need to add them ourselves, since the address is ambigious otherwise
when we add the port.

e.g. giving 'fe80::1' as address we arrive at the url (with the default port)
'https://fe80::1:8007/'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-05-05 10:30:39 +02:00
Wolfgang Bumiller ee0c5c8e01 use api_string_type macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-05-05 08:24:37 +02:00
Dominik Csapak ae5b1e188f docs: tape: clarify LTO-4/5 support
some features we need (e.g. READ POSITION long form) are only officially
available with LTO-5, but work on many LTO-4 drives, so move LTO-4 to
'best-effort' support.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-04 13:21:44 +02:00
Dominik Csapak 49f9aca627 tape/restore: optimize chunk restore behaviour
by checking the 'checked_chunks' before trying to write to disk
and by doing the existance check in the parallel handler. This way,
we do not have to check the existance of a chunk multiple times
(if multiple source datastores gets restored to the same target
datastore) and also we do not have to wait on the stat before reading
the next chunk.

We have to change the &WorkerTask to an Arc though, otherwise we
cannot log to the worker from the parallel handler

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-04 13:06:31 +02:00
Dietmar Maurer 4cba875379 bump version to 1.1.6-2 2021-05-04 12:25:16 +02:00
Dietmar Maurer 7ab4382476 update debian/control 2021-05-04 12:24:16 +02:00
Wolfgang Bumiller eaef6c8d00 Revert "temporarily disable broken test"
This reverts commit 888d89e2dd.

The code this depends on should now be available.
2021-05-04 12:11:35 +02:00
Wolfgang Bumiller 95f3692545 fix permissions set in create_run_dir
This directory needs to be owned by the backup user instead
of root.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 12:11:35 +02:00
Dietmar Maurer 686173dc2a bump version to 1.1.6-1 2021-05-04 12:09:56 +02:00
Dietmar Maurer 39c5db7f0f move basic ACME types into src/api2/types/acme.rs
And rename AccountName into AcmeAccountName.
2021-05-04 11:32:18 +02:00
Dietmar Maurer 603aa09d54 tape restore: do not verify restored files
Because this is too slow and causes the tape motor to stop. Instead,
remove the verify_state from the manifest.
2021-05-04 11:05:32 +02:00
Dietmar Maurer 88aa3076f0 tape restore: add restore speed to logs 2021-05-04 11:05:32 +02:00
Dietmar Maurer 5400fe171c tape restore: write datastore in separate thread 2021-05-04 11:05:32 +02:00
Dietmar Maurer 87bf9f569f tape restore: split restore_chunk_archive
Split out a separate function scan_chunk_archive() for catalog restores.

Note: Required, because we need to optimize restore_chunk_archive() to
write datastore in separate threads (else thape drive will stop during restore)
2021-05-04 11:05:32 +02:00
Wolfgang Bumiller 8fb24a2c0a daily-update: check acme certificates
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:43:50 +02:00
Wolfgang Bumiller 4b5d9b6e64 ui: add certificate & acme view
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:42:50 +02:00
Wolfgang Bumiller 72bd8293e3 add acme commands to proxmox-backup-manager
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:39:16 +02:00
Wolfgang Bumiller 09989d9963 add node/{node}/config api path
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:32:42 +02:00
Wolfgang Bumiller 4088d5bc62 add node/{node}/certificates api call
API like in PVE:

GET    .../info             => current cert information
POST   .../custom           => upload custom certificate
DELETE .../custom           => delete custom certificate
POST   .../acme/certificate => order acme certificate
PUT    .../acme/certificate => renew expiring acme cert

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:31:30 +02:00
Wolfgang Bumiller d4b84c1dec add config/acme api path
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:30:49 +02:00
Dietmar Maurer 426847e1ce node config cleanups 2021-05-04 09:29:31 +02:00
Wolfgang Bumiller 79b902d512 add node config
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 09:29:31 +02:00
Dietmar Maurer 73c607497e cleanup acme client 2021-05-04 09:28:53 +02:00
Wolfgang Bumiller f2f526b61d add acme client
This is the highlevel part using proxmox-acme-rs to create
requests and our hyper code to issue them to the acme
server.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 07:56:52 +02:00
Wolfgang Bumiller cb67ecaddb add acme config
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-04 07:43:43 +02:00
Dominik Csapak 5bf9b0b0bb docs: user-management: add note about untrusted certificates for webauthn
Since currently it works fine with untrusted certs, but that may change
anytime.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-03 12:01:23 +02:00
Thomas Lamprecht 7a61f89e5a tape backup job: fix typo in error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-03 12:01:23 +02:00