2ab2ca9c24
tape: add MediaPool::lock_unassigned_media_pool() helper
2021-03-19 10:13:38 +01:00
54fcb7f5d8
api2/tape/backup: wait indefinitely for lock in scheduled backup jobs
...
so that a user can schedule multiple backup jobs onto a single
media pool without having to consider timing them apart
this makes sense since we can backup multiple datastores onto
the same media-set but can only specify one datastore per backup job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 09:04:32 +01:00
4abd4dbe38
api2/tape/backup: include a summary on notification e-mails
...
for now only contains the list of included snapshots (if any),
as well as the backup duration
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 09:03:52 +01:00
eac1beef3c
tape: cleanup PoolWriter - factor out common code
2021-03-19 08:56:14 +01:00
166a48f903
tape: cleanup - split PoolWriter into several files
2021-03-19 08:19:13 +01:00
82775c4764
tape: make sure we only commit/write valid catalogs
2021-03-19 07:50:32 +01:00
88bc9635aa
tape: store media_uuid in PoolWriterState
...
This is mainly a cleanup, avoiding to access the catalog_set to get the uuid.
2021-03-19 07:33:59 +01:00
1037f2bc2d
tape: cleanup - rename CatalogBuilder to CatalogSet
2021-03-19 07:22:54 +01:00
f24cbee77d
server/email_notifications: do not double html escape
...
the default escape handler is handlebars::html_escape, but this are
plain text emails and we manually escape them for the html part, so
set the default escape handler to 'no_escape'
this avoids double html escape for the characters: '&"<>' in emails
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 07:02:49 +01:00
25b4d52dce
server/email_notifications: do not panic on template registration
...
instead print an error and continue, the rendering functions will error
out if one of the templates could not be registered
if we `.unwrap()` here, it can lead to problems if the templates are
not correct, i.e. we could panic while holding a lock, if something holds
a mutex while this is called for the first time
add a test to catch registration issues during package build
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 07:02:17 +01:00
2729d134bd
tools/systemd/time: implement some Traits for TimeSpan
...
namely
* From<Duration> (to convert easily from duration to timespan)
* Display (for better formatting)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 07:00:55 +01:00
32b75d36a8
tape: backup media catalogs
2021-03-19 06:58:46 +01:00
237314ad0d
tape: improve catalog consistency checks
...
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.
Note: this changes the catalog format again.
2021-03-18 08:43:55 +01:00
caf76ec592
tools/subscription: ignore ENOENT for apt auth config removal
...
deleting a nonexistant file is hardly an error worth mentioning
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-17 20:12:58 +01:00
6ee3035523
tape: define magic number for catalog archives
2021-03-17 13:35:23 +01:00
b627ebbf40
tape: improve catalog parser
2021-03-17 11:29:23 +01:00
ef4bdf6b8b
tape: proxmox-tape media content - add 'store' attribute
2021-03-17 11:17:54 +01:00
54722acada
tape: store datastore name in tape archives and media catalog
...
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
2021-03-17 11:08:51 +01:00
0e2bf3aa1d
SnapshotReader: add self.datastore_name() helper
2021-03-17 10:16:34 +01:00
365126efa9
tape: PoolWriter - remove unnecessary move_to_eom
2021-03-17 10:16:34 +01:00
7914e62b10
tools/zip: only add zip64 field when necessary
...
if neither offset nor size exceeds 32bit, do not add the
zip64 extension field
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 09:13:39 +01:00
e36135031d
ui: tape/Restore: let the user choose an owner
...
so that the tape backup can be restored as any user, given
the current logged in user has the correct permission.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 08:55:42 +01:00
5a5ee0326e
proxmox-tape: add missing notify-user to 'proxmox-tape restore'
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 08:54:38 +01:00
776dabfb2e
tape: use MB/s for backup speed (to match drive speed specification)
2021-03-16 08:51:49 +01:00
5c4755ad08
tape: speedup backup by doing read/write in parallel
2021-03-16 08:51:49 +01:00
7c1666289d
tools/zip: add missing start_disk field for zip64 extension
...
it is not optional, even though we give the size explicitely
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-15 12:36:40 +01:00
cded320e92
backup info: run rustfmt
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-14 19:18:35 +01:00
b31cdec225
update to pxar 0.10
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:48:09 +01:00
591b120d35
fix feature flag logic in pxar create
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:17:51 +01:00
e8913fea12
tape: write_chunk_archive - do not consume partially written chunk at EOT
...
So that it is re-written to the next tape.
2021-03-12 07:14:50 +01:00
ef0ea4ba05
server/worker_task: improve endtime for unknown tasks
...
instead of always using the starttime, use the last timestamp from the log
this way, one can see when the task was aborted without having to read
the log
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-11 09:56:12 +01:00
2892624783
tape/send_load_media_email: move to server/email_notifications
...
and reuse 'send_job_status_mail' there so that we get consistent
formatted mails from pbs (e.g. html part and author)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-11 09:56:12 +01:00
2c10410b0d
tape: improve backup task log
2021-03-11 08:43:13 +01:00
d1d74c4367
typo fixes all over the place
...
found and semi-manually replaced by using:
codespell -L mut -L crate -i 3 -w
Mostly in comments, but also email notification and two occurrences
of misspelled 'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 16:39:57 +01:00
5ef4c7bcd3
tape: fix scsi volume_statistics and cartridge_memory for quantum drives
2021-03-10 14:13:48 +01:00
6396bace3d
tape: improve backup task log (show percentage)
2021-03-10 10:59:13 +01:00
713a128adf
tape: improve backup task log format
2021-03-10 09:54:51 +01:00
affc224aca
tape: read_tape_mam - pass correct allocation len
2021-03-10 09:24:38 +01:00
6f82d32977
tape: cleanup - remove wrong inline comment
2021-03-10 08:11:51 +01:00
2a06e08618
api2/tape/backup: continue on vanishing snapshots
...
when we do a prune during a tape backup, do not cancel the tape backup,
but continue with a warning
the task still fails and prompts the user to check the log
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-09 10:20:54 +01:00
1057b1f5a5
tape: lock artificial "__UNASSIGNED__" pool to avoid races
2021-03-09 10:00:26 +01:00
af76234112
tape: improve MediaPool allocation by sorting tapes by ctime and label_text
2021-03-09 08:33:21 +01:00
1d7fcbece8
tape/linux_tape: remove debug prints
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:19 +01:00
8703a68a31
tape/backup: add success/error notifications
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:06 +01:00
9ce2481a69
tape: clean - fix if cleaning tape is already loaded
2021-03-08 09:08:55 +01:00
d95c74c6e7
tape: test tape alert flags after drive clean
2021-03-08 09:08:55 +01:00
5ca5f8daf3
pmt: add tape-alert-flags command
2021-03-08 07:37:05 +01:00
c9793d47f9
api2/tape: add notify_user to backup(-jobs) and restore api calls
...
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-05 17:20:37 +01:00
9152a0077f
config/tape_job: add notify-user config to tape jobs
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-03-05 12:51:25 +01:00
0b90c67fb4
tape: avoid Updatable for now
2021-03-05 12:51:25 +01:00
b4975d3102
tape: finish api permission checks
2021-03-05 11:49:07 +01:00
ee33795b72
acl: use /tape/device instead of /tape/drive and /tape/changer
2021-03-05 10:06:19 +01:00
90e16be3ae
tape: improve PoolWriter logging
...
Log reason why we allocate a new media set.
2021-03-05 09:59:38 +01:00
cf90a369e2
cleanup: rename token_user into auth_id_filter
2021-03-05 08:36:18 +01:00
cd5d6103ea
tape: fix media_expire_time
2021-03-04 16:58:24 +01:00
1bed3aedc8
tape: fix wrong media set expire time
2021-03-04 12:38:28 +01:00
ab77d660cc
tape: improve media status in list_media
2021-03-04 10:17:10 +01:00
78593b5b5c
tape/drive: improve error and email handling for requesting a tape load
...
Try once first to load the correct tape before sending an email to
insert the correct one. This way, the admin does not get a mail
if the correct tape is already inserted.
Also include the error we got that prompted the email to insert the
tape. This means that if the admin gets prompted to insert e.g.
"FOO" but inserts "BAR", he'll get an email that the wrong
tape is inserted.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
831c43c91b
config/tfa: webauthn: disallow registering a token twice
...
by adding the existing credential id to the 'excludeCredentials' list
this prevents the browser from registering a token twice, which
lets authentication fail on some browser/token combinations
(e.g. onlykey/solokey+chromium)
while is seems this is currently a bug in chromium, in a future spec
update the underlying behaviour should be better defined, making this
an authenticator bug
also explicitly catch registering errors and show appropriate error messages
0: https://bugs.chromium.org/p/chromium/issues/detail?id=1087642
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 14:05:05 +01:00
7f37cacfac
config/tfa: set UserVerificationPolicy to Discouraged
...
the current default is 'Preferred', which is not really useful, as the
(web) client can simply change this to discouraged, since the
webauthn_rs crate does not verify the 'user_verified' bit of the
response in that case
setting this to 'Required' is not really useful either at the moment,
since a user can have a mix of different authenticators that may or
may not support user verification
there is ongoing discussion in the crate how to handle that[0]
we could probably expose this setting(discouraged/required) to the user/admin
and save it to the credential and allow only registering credentials
of the same type or filter them out on login (i.e. if there is an
authenticator that can handle userVerification, require it)
in any case, the current default is not helpful for security, but
makes loggin in harder, since the key will by default want to verify
the user
0: https://github.com/kanidm/webauthn-rs/pull/49
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 14:05:05 +01:00
3bb7e62e88
zpool parsers: small code cleanup
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
3b060167f6
zpool parsers: fixup test value
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
8a76e71129
tape: impl access permission for media content list
2021-03-03 13:16:15 +01:00
396fd747a6
tape: impl access permissions for tape jobs
2021-03-03 12:48:06 +01:00
16bd08b297
tape: add acl path /tape/job/{id}
2021-03-03 12:48:06 +01:00
ccdf327ac8
tape: impl access permissions for encryption key config
2021-03-03 12:21:55 +01:00
8cd63df0dc
tape: impl api permissions for drive/changer/pool configuration
2021-03-03 12:10:00 +01:00
b90cb34fd6
complete_acl_path: add more paths
2021-03-03 11:09:54 +01:00
d6c1e12c06
add roles to implement tape backup permissions
2021-03-03 11:09:02 +01:00
40bf636b47
ui: enable experimental tape UI if tape.cfg exists
2021-03-03 09:02:02 +01:00
347cde827b
dashboard: fix datastore full estimate for not changing usage
...
If the usage of a datastore did not change, we did not
return an estimate. The ui interpreted this as 'not enough data', but
it should actually be 'never'.
Fixing this by always setting the estimate first to 0 and overwriting
if we successfully calculated one, and checking for 'undefined' in the ui.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 17:51:13 +01:00
8bf5769382
api2/tape/drive: add state to DriveListEntry
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:51:19 +01:00
db04d10d14
tape: add comment to media pool config
2021-03-02 11:42:58 +01:00
1d14c31658
Revert "tape: also abort backup/restore on server shutdown"
...
This reverts commit 9bd81bb384
2021-03-02 08:00:10 +01:00
9bd81bb384
tape: also abort backup/restore on server shutdown
2021-03-02 07:19:31 +01:00
d64226efee
disks/zfs: fix test input and enforce check for config key
...
we test for the config key in the API so it makes sense to have as
test here too. Actually it would be better if we'd have a expect
Value defined here and enforce that it matches, but better than
nothing.
Fix the input for test 1, where tabs got replaced by spaces, as else
it fails
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-01 17:36:16 +01:00
2440eaa2df
disks/zfs: pull common test code out in helper
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-01 14:48:35 +01:00
1552d9699c
tape: add 'Restore Key' button
2021-03-01 11:11:38 +01:00
16f9ea6708
tape: restore - do not check if pool is configured
2021-03-01 09:43:23 +01:00
d984a9acf0
tape: add Inventory.media_list() helper
2021-03-01 09:30:45 +01:00
955f4aefcd
tape: show media inventory/content, even if there is no pool configuration
2021-03-01 09:21:11 +01:00
3832911d50
docs: remove backup protocol api dumps
...
We now have the api-viewer for that.
2021-02-26 19:00:08 +01:00
08ec39be0c
tape: add api to set media status
2021-02-26 10:44:07 +01:00
25350f3370
tape: set media status if we detect damaged medium at start of backup
2021-02-26 09:00:50 +01:00
0023cfa385
tape: fix 'proxmox-tape backup' cli command
2021-02-26 08:51:24 +01:00
5830e5620d
tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig
2021-02-25 09:44:31 +01:00
3554fe6480
update to schema changes in proxmox
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 14:50:08 +01:00
0dadf66dc7
tape: implement backup using latest-only flag
2021-02-24 13:51:53 +01:00
a941bbd0c9
client: raise HTTP_TIMEOUT to 120s
...
As 20s is really not that high, especially for loaded setups one is
connected to through a spotty network (looking at you ÖBB railnet)
and gets latency spikes of 5 - 10s for some minutes at a time..
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-24 11:57:21 +01:00
21e3ed3449
tape: add 'latest-only' property to backup job config
2021-02-24 11:19:12 +01:00
52d8db7925
api2/config/tape_backup_job: fix duplicate id parameter
...
since the PUT api call is using the 'Updater', the 'id' parameter is
already encoded in there, tripping up the api verify tests with
'Duplicate keys found in AllOf schema: id'
"fixing" it by removing the explicit id from the api call and
taking it from the Updater (and failing if it does not exists there;
even though that should never happen)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 17:23:47 +01:00
875d375d7a
api2/types: ArchiveEntry: fix inverse 'leaf' value
...
a 'leaf' node is every file *except* directories, so we have
to reverse the logtic here
this fixes the pxar.didx browser in the web ui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 17:23:47 +01:00
cba167b874
pxar/extract: if possible create files sparesly
...
instead of filling them with zeroes
this fixes an issue where we could not restore a container with large
sparse files in the backup (e.g. a 10GiB sparse file in a container
with a 8GiB disk)
if the last operation of the copy was a seek, we need to truncate
the file to the correct size (seek beyond filesize does not change it)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 14:44:59 +01:00
4961404c7c
api2/config/tape_backup_job: enable update api call
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 13:17:49 +01:00
3fbf2311e4
tape: improve backup task logging
2021-02-23 12:58:44 +01:00
043018cfbe
doc: fix wrong api method description
2021-02-22 12:10:34 +01:00
2037d9af03
api-viewer: show permissions
2021-02-22 12:01:24 +01:00
7f07991035
docs: improve MEDIA_SET_NAMING_TEMPLATE_SCHEMA description
2021-02-22 09:50:16 +01:00
18ce01caff
tape: use correct schema for changer path (SCSI_CHANGER_PATH_SCHEMA)
2021-02-22 09:43:13 +01:00
5bc8e80a99
cleanup: cleanup use clause (avoid *)
2021-02-22 09:33:29 +01:00
6252df4c18
docs: fix EXPORT_SLOT_LIST_SCHEMA description
2021-02-22 09:05:07 +01:00
451856d21d
api-viewer: fix path for upgraded HTTP/2 proptocls
2021-02-22 08:38:27 +01:00
aa30663ca5
docgen: improve regex format
2021-02-21 16:14:11 +01:00
8616a4afe5
docgen: dump string format (property strings)
2021-02-21 15:54:14 +01:00
bc2358319b
docgen: improve api schema dump
2021-02-21 12:00:06 +01:00
0bf4b81370
docs: api-viewer - add backup and restore APIs
2021-02-21 10:38:33 +01:00
fee0fe5422
docgen: implement api schema generation (for api-viewer)
...
Just a start - not complete jet.
2021-02-21 09:06:58 +01:00
86d9f4e733
tape: extend MediaChange trait to return MtxStatus
2021-02-20 10:23:16 +01:00
3f16f1b006
tape: update changer status inside ScsiMediaChange implementation
2021-02-20 09:56:27 +01:00
cbd9899389
tape: update changer status cache after load/unload
2021-02-20 09:06:17 +01:00
cd44fb8d84
tape: create tmp dirs early at server startup
2021-02-19 18:10:02 +01:00
4188fd59a0
tape: cache changer state
2021-02-19 16:48:19 +01:00
926d05ef0b
api2/tape/{backup, restore}, proxmox-tape: set device state
...
set the drive device state everywhere we lock it, so that we
know what it currently does
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:16:30 +01:00
8be48ddfc7
api2/tape/changer: add drive state to changer status output
...
if we can find the drive in the config and it has a state
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:15:15 +01:00
41e66bfaf6
api2/tape/drive: wrap some api calls in run_drive_blocking_task
...
those calls could also block, so we have to run them in a blocking
tokio task, as to not block the current thread
nice side effect is that we now also update the state for that
drive in those instances
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:14:08 +01:00
47a7241410
api2/tape/drive: use run_drive_blocking_task where possible
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:13:36 +01:00
54c77b3d62
api2/tape/drive: add wrapper for tokio::task::spawn_blocking
...
similar to the worker wrapper, lock, write status, run code, unset status
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:13:21 +01:00
a1c5575308
api2/tape/drive: use 'run_drive_worker' where possible
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:11:04 +01:00
a44c934b5d
api2/tape/drive: add run_drive_worker wrapper
...
a wrapper for locking, starting the worker and writing the state
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:08:27 +01:00
546d2653ee
tape/drive: add get/set status functions
...
simply writes into/reads from a file in /run, we will use this
for writing the upid (or potential other states) per drive
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:05:09 +01:00
33c06b3388
tape/drive: add test_device_path_lock
...
this makes it possible to detect if the drive was locked in a
non-blocking manner
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:03:30 +01:00
1917ea3ce1
cleanup: use serde(flatten) for VerificationJobStatus, improve code reuse
2021-02-19 09:50:25 +01:00
70842b9ef2
cleanup: use serde(flatten) for SyncJobStatus, improve code reuse
2021-02-19 09:36:39 +01:00
7690a8e7bd
api: list tape backup jobs with scheduling status
2021-02-19 09:02:13 +01:00
1860208560
add helper to compute job scheduling state
2021-02-19 08:58:50 +01:00
1689296d46
api: add new type JobScheduleStatus
2021-02-19 08:23:41 +01:00
9bbd83b1f2
tape: media_list API - allow to update online status for a single changer
2021-02-18 10:59:33 +01:00
d0647e5a02
tape: use worker tasks for media load/unload
2021-02-18 09:04:51 +01:00
f1a83e9759
client: extract common functions to proxmox_client_tools module
...
...including common schemata, connect(), extract_*() and completion
functions.
For later use with proxmox-file-restore binary.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 10:13:10 +01:00
6afb60abf5
asyncify pxar create_archive
...
...to take advantage of the aio::Encoder from the pxar create.
Rather straightforward conversion, but does require getting rid of
references in the Archiver struct, and thus has to be given the Mutex
for the catalog directly. The callback is boxed.
archive_dir_contents can call itself recursively, and thus needs to
return a boxed future.
Users are adjusted, namely PxarBackupStream is converted to use an
Abortable future instead of a thread so it supports async in its handler
function, and the pxar bin create_archive is converted to an async API
function. One test case is made to just use 'block_on'.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:24:20 +01:00
a42212fc1e
pxar/extract: add extract_sub_dir
...
to extract some subdirectory of a pxar into a given target
this will be used in the client
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
2e21948156
api2/admin/datastore: refactor create_zip into pxar/extract
...
we will reuse that code in the client, so we need to move it to
where we can access it from the client
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[clippy fixes]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
5279ee745f
api2/admin/datastore: accept "/" as path for root
...
makes more sense than sending "root'"
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
227501c063
api2/admin/datastore: refactor list_dir_content in catalog_reader
...
we will reuse that later in the client, so we need it somewhere
we can use from there
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[add strongly typed ArchiveEntry and put api code into helpers.rs]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
89d25b1931
client: add VsockClient to connect to virtio-vsock VMs
...
Currently useful only for single file restore, but kept generic enough
to use any compatible API endpoint over a virtio-vsock[0,1] interface.
VsockClient is adapted and slimmed down from HttpClient.
A tower-compatible VsockConnector is implemented, using a wrapped
UnixStream as transfer. The UnixStream has to be wrapped in a custom
struct to implement 'Connection', Async{Read,Write} are simply forwarded
directly to the underlying stream.
[0] https://www.man7.org/linux/man-pages/man7/vsock.7.html
[1] https://wiki.qemu.org/Features/VirtioVsock
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:15:43 +01:00
b57c0dbe30
rest: implement tower service for UnixStream
...
This allows anything that can be represented as a UnixStream to be used
as transport for an API server (e.g. virtio sockets).
A tower service expects an IP address as it's peer, which we can't
reliably provide for unix socket based transports, so just fake one.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 07:50:35 +01:00
8b910bb6bc
tape: add regression test for compute_media_state
2021-02-17 06:57:29 +01:00
3cdd1a3424
tape: erase_media - automatically load media (with tape libraries).
2021-02-17 06:55:53 +01:00
be61c56c21
api2/tape/drive: add optional label-text to erase
...
if given, erases the tape only iff the inserted tape contains that label
used to safeguard tape erasing from ui for standalone drives
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
dbaef7d04d
tape: add regression test for alloc_writable_media
2021-02-17 06:55:49 +01:00
159100b944
tape: ui - avoid changer status query in backup content panel
2021-02-15 12:17:36 +01:00
41a8db3576
tape: implement API to run tape backup jobs
2021-02-15 11:09:30 +01:00
fe291ab794
cleanup: tape_backup_job.rs - do not use #[allow(non_camel_case_types)]
2021-02-15 10:41:04 +01:00
adb65b9889
tape: fix proxmox-tape pool config command
2021-02-15 08:58:05 +01:00
8513626b9f
proxmox-backup-proxy: schedule tape backup jobs
2021-02-15 08:30:42 +01:00
7ca0ba4515
tape: add tape-job.cfg manual page
2021-02-15 07:05:39 +01:00
42200c405a
fix api sort order
2021-02-15 06:35:45 +01:00
be327dbccd
tape: add tape backup job configuration
2021-02-14 10:35:51 +01:00
7b1bf4c098
tape: erase_media - automatically erase catalog and inventory
2021-02-12 11:30:34 +01:00
7c9fb570cc
api2/types/tape/media_location: fix deserialization of MediaLocation::Vault
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:23:54 +01:00
56d22c66c0
api2/tape/media: add 'move' api call
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:23:27 +01:00
85cdc4f371
proxmox-tape: change lookup_drive_name to extract_drive_name
...
in most uses, we want to remove the drive from the param afterwards
where we don't, we already overwrite it with the result of this function
this fixes some commands (like 'proxmox-tape read-label --drive foo')
that failed with:
parameter 'drive': duplicate parameter.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:22:50 +01:00
4a874665eb
proxmox-backup-manger verify-job CLI
...
Add missing command line interface to manage verification jobs.
2021-02-11 14:06:44 +01:00
6f6b69946e
docs: use type_text for config::acl::Role
...
The list of roles is simply too long, so we use type_text to generate
reasonable docs.
2021-02-11 13:47:01 +01:00
5b7f44555e
docs: add manual page for verification.cfg
2021-02-11 13:40:11 +01:00
2ca396c015
docs: add manual page for acl.cfg
2021-02-11 13:13:34 +01:00
8f02db04f9
docs: use new type_text option for calendar events
2021-02-11 11:51:22 +01:00
9f35e44681
fix proxmox_backup_manager remote list: s/userid/auth-id/
2021-02-11 11:31:20 +01:00
6279b8f5a5
docs: add manukal page for remote.cfg
2021-02-11 11:25:15 +01:00
3084232cb5
depend on proxmox 0.10.5
2021-02-11 10:49:16 +01:00
2322a980d0
docs: add utility binary to generate docs
2021-02-10 11:05:02 +01:00
7f9d8438ab
tfa: derive WebauthnConfigUpdater via api macro
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-10 10:47:47 +01:00
befd95a90a
pxar: improve error messages
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-09 13:09:00 +01:00
ab6cd4229b
pxar: skip write_attr_fd if they are equal
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-09 13:09:00 +01:00
9213744ecb
fix #3026 : pxar: metadata: apply flags _after_ updating mtime
...
otherwise we'll fail to update the mtime of immutable files
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-09 13:08:49 +01:00
afcf8b3ed6
api2/tape/drive: add missing locks for some drive actions
...
else a backup/restore can easily be disrupted
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-09 08:46:03 +01:00
15d2c7786e
client: move test output files to target/testout
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-08 09:52:16 +01:00
fbf8779388
doc: improve pmt/pmtx manual pages
2021-02-06 16:50:56 +01:00
ced7838de4
tape: fix path for test files
2021-02-06 09:32:50 +01:00
2f26b8668a
client: track key source, print when used
...
to avoid confusing messages about using encryption keys when restoring
plaintext backups, or about loading master keys when they are not
actually used for the current operation.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:07:31 +01:00
9432838914
client: refactor crypto_parameter handling
...
pull out the crypt-mode to logically group arms and make the whole mess
a bit more "human-parsable".
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:06:28 +01:00
1a89a7794e
client: extend tests for master key handling
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:05:01 +01:00
c0a87c12fb
client: allow passing specific master key
...
it's needed for PVE's LXC integration, and might be interesting for
other more special usage scenarios as well.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:04:42 +01:00
c6a7ea0a2f
client: refactor keyfile_parameters
...
no semantic changes intended
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:03:03 +01:00
5bb057e5a2
client: add test for keyfile_parameters
...
this will get more complex soon, so add test to document current
behaviour.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:58:15 +01:00
2924b37d6d
key: rustfmt module
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:56:19 +01:00
42c0f784e2
key: add show-master-pubkey command
...
and print public key when generating/importing..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:55:21 +01:00
05f17d1ec4
key: make 'default' master key explicit
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:52:11 +01:00
f197c286d5
tape/inventory: fix missing parameter in error message
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:12:18 +01:00
4c209d6b10
install pmt binary
2021-02-05 12:42:20 +01:00
25aa55b5f5
tape: correctly lock the tape device
2021-02-05 10:50:21 +01:00
cdf39e62b3
tape: MediaPool - replace use_offline_media with changer_name
...
This way, we can improve location_is_available, because we only
consider media from that changer as available.
2021-02-04 10:15:18 +01:00
b81e37f6ab
tape: improve code reuse
2021-02-04 09:39:16 +01:00
ddebbb52fd
tape: fix tests for BlockedReader
2021-02-04 08:54:54 +01:00
983e929e25
tape: add multi volume reader/writer implementations
...
We currently do not use it. Added anaways, to show the possibility.
2021-02-04 08:36:35 +01:00
f47e035721
tape: cleanup - move tape file readers/writers into src/tape/file_formats folder
2021-02-04 07:59:37 +01:00
a80d72f999
tape: allow to abort restore tasks
2021-02-04 07:05:43 +01:00
8de9a9917f
cleanup: use task_log macro
2021-02-04 06:55:18 +01:00
fa016c1697
HttpsConnector: use hostname instead of URL again
...
fixes connecting to hosts with valid certificates without a
pinned fingerprint
this was accidentally changed in the tokio-1.0 updates
apparently
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Fixes: 0f860f712f
2021-02-03 15:18:18 +01:00
7d2c156eb1
tape: BlockedReader - always consume EOF
2021-02-03 13:25:59 +01:00
b22e8c3632
tape: add media pool regression tests
2021-02-03 10:23:04 +01:00
7929292618
tape: add regresion test for media state
2021-02-03 09:34:31 +01:00
0d4e4cae7f
tape: improve pmt command line completion
2021-02-03 08:54:12 +01:00
50a4797fb1
api2/types/tape/media: add media_set_ctime to MediaContentEntry
...
to be able to better sort in the ui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:45:54 +01:00
cc2a0b12f8
test: define tape tests as submodule
2021-02-02 14:38:15 +01:00
988e8de122
tape: set correct ownership on lock file
2021-02-02 14:18:57 +01:00
2f8809c6bc
test: src/tape/inventory.rs - avoid chown when running tests
2021-02-02 13:43:16 +01:00
8937c65951
tape: add pmt stoptions/stsethoptions/stclearoptions
2021-02-02 08:58:02 +01:00
6c6ad82d90
tape: add pmt setblk
2021-02-02 07:19:54 +01:00
d0f11b66f7
thape: add read_tapedev_options, display driver options with status command
2021-02-02 06:40:40 +01:00
ca953d831f
cleanup: remove MT_ST_ prefix from SetDrvBufferOptions
2021-02-01 17:54:53 +01:00
01c023d50f
paperkey: rustfmt
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 17:05:40 +01:00
c2113a405e
paperkey: simplify block generation
...
the chunk-iterator already does exactly what we want here..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 17:05:32 +01:00
5dae81d199
paperkey: allow RSA keys without passphrase
...
some users might want to store the plain version of their master key for
long-term storage and rely on physical security instead of a passphrase
to protect the paper key.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 17:05:22 +01:00
6248e51797
change half-ticket time range from -120..240 to -60..600
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-01 15:13:11 +01:00
19e4a36c70
tape: do not use drive.open() within pmt
...
Do not fail if no media is loaded. Inportant for load command.
2021-02-01 12:39:50 +01:00
90769e5694
tape: add pmt lock/unlock
2021-02-01 12:18:55 +01:00
b8cbe5d65b
tape: fix tape alert flag decoding
2021-02-01 12:18:55 +01:00
d690d14568
tape: add pmt bsr/fsr
2021-02-01 10:39:04 +01:00
85ef624440
tape: add pmt asf
2021-02-01 10:32:21 +01:00
e995996290
tape: pmt - fix count parameter schema
2021-02-01 10:21:25 +01:00
8e6ad4301d
tape: add pmt fsfm/bsfm, pass count as arg_param
2021-02-01 10:18:18 +01:00
1399c592d1
garbage_collection: only ignore 'missing chunk' errors
...
with the fix for #2909 (improving handling missing chunks), we
changed from bailing to warning during a garbage collection when
updating the atime of a chunk.
but, updating the atime can not only fail when the chunk is missing,
but also on other occasions, e.g. no permissions or more importantly,
no space left on the device. in that case, the atime of a valid and used
chunk cannot be updated, and the second sweep of the gc will remove that chunk.
[0] is a real world example of that happening.
instead, only warn on really missin chunks, and bail on all other
errors.
0: https://forum.proxmox.com/threads/pbs-server-full-two-days-later-almost-empty.83274/
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-01 09:18:59 +01:00
9883b54cba
tape: remove drive from pool config
2021-02-01 09:14:28 +01:00
83b8949a98
tape: add pmt weof
2021-01-31 17:33:07 +01:00
28f60e5291
cleanup: avoid compiler warnings
2021-01-31 17:02:55 +01:00
1f31d06f48
tape: add pmt bsf
2021-01-31 17:00:15 +01:00
2f2e83c890
tape: add pmt fsf
2021-01-31 16:54:16 +01:00
b22c618734
tape: add pmt erase
2021-01-31 16:34:10 +01:00
1e041082bb
tape: add pmt command line tool
...
Experimental, not installed by now.
2021-01-31 16:19:53 +01:00
b5b99a52cd
tape: API type cleanup, use serde flatten to derive types
2021-01-30 09:36:54 +01:00
9586ce2f46
tape: move scan_drives API code to correct file
2021-01-30 08:03:17 +01:00
c297835b01
tape: proxmox-tape - use API instead of direct functions calls
2021-01-29 11:49:11 +01:00
e68269fcaf
tape: proxmox-tape inventory: call API
2021-01-29 11:21:57 +01:00
5243df4712
tape: proxmox-tape - use API instead of direct functions calls
2021-01-29 10:50:11 +01:00
4470eba551
cleanup: factor out common client code to view task log/result
2021-01-29 10:10:04 +01:00
1f2c4713ef
tape: improve backup task abort behaviour
2021-01-29 09:23:39 +01:00
a6c16894ff
worker_task: log something when we receive an abort request
2021-01-29 09:22:37 +01:00
271764deb9
tape: make it possible to abort tape backup tasks (check_abort)
...
Also use task_log makro instead of worker.log.
2021-01-29 09:07:55 +01:00
52f7a73009
display_task_log: make it possible to abort tasks with CTRL-C
2021-01-29 09:06:15 +01:00
bdb6e6b83f
api2/reader: asyncify the reader worker task
...
this way, the code is much more readable
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 06:59:25 +01:00
41dacd5d3d
tape: use worker task for eject-media api
2021-01-28 16:49:08 +01:00
eb1dfb02b5
tape: proxmox-tape - use api for erase-media and rewind
2021-01-28 16:36:10 +01:00
bdb62b20a3
tape: media_pool config api - set protected flags where required
2021-01-28 15:42:32 +01:00
f2ca03d7d0
cleanup: avoid compiler warning
2021-01-28 15:32:21 +01:00
00ac86c31b
tape/drive/linux_tape: fix and refactor usage of sg-tape-cmd
...
when executing this code as non-root, we use sg-tape-cmd (a setuid binary)
to execute various ioctls on the tape device
we give the command the open tape device fd as stdin, but did not
dup it, so the std::process:Stdio handle closed it on drop,
which let subsequent operation on that file fail (since it was closed)
fix it by dup'ing it before giving it to the command, and also refactor
the calling code, so that we do not forget to do this
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:24:32 +01:00
627d000098
tape: change changer-drive-id to changer-drivenum
...
because it changed in the config
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:11:22 +01:00
4be4736603
tape/changer: refactor marking of import/export slots from config
...
we did this for 'mtx', but missed it for the sg_pt_changer code
refactor it into the MtxStatus strut, and call it from both
code paths
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:10:55 +01:00
2da7aca8e8
tape/changer: add vendor/model to DriveStatus
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:10:31 +01:00
b03ec281bf
api2/config/{drive, changer}: prevent adding same device multiple times
...
this check is not perfect since there are often multiple device
nodes per drive/changer, but from the scan api we should return always
the same, so for an api user this should be enough
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:03:56 +01:00
cef4654ff4
api2/tape/drive: change methods of some api calls from put to get
...
makes more sense to have retrieving api calls as get instead of put
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:02:52 +01:00
f45dceeb73
api2/tape/drive: add load_media as api call
...
code was already there, just add it as api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:02:13 +01:00
18262a88c9
api2/tape/changer: add changer filter to list_drives api call
...
so that an api user can get the drives belonging to a changer
without having to parse the config listing themselves
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:01:41 +01:00
87f4be7998
tape: use api to run proxmox-tape backup
2021-01-28 14:56:42 +01:00
d737adc6be
tape: rename changer_drive_id to changer_drivenum
2021-01-28 11:29:59 +01:00
5fdaecf6f4
api2/tape/drive: reorganize drive api
...
similar to the changers, create a listing at /tape/drive and put
the specific api calls below that
move the scan api call up one level
remove the status info from the config listing
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 11:15:17 +01:00
d8792b88ef
api2/types/tape/drive: add changer_drivenum
...
so that an api user can see which drive belongs to which drivenum of a changer
for ones with multiple drives
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 11:14:28 +01:00
651a61f559
pmtx: implement scan command
2021-01-27 12:40:51 +01:00
89ccb125d1
tape: use 36 byte Inquiry (recommended size)
2021-01-27 12:35:28 +01:00
887f1cb90c
cleanup: move scan changers API implementation
2021-01-27 09:58:16 +01:00
16b4d78400
tape: rename retry_command to execute_scsi_command, make retry a flag
2021-01-27 09:34:24 +01:00
ec8d9c6b80
tape: repeat changer scsi command until successful
2021-01-27 08:59:10 +01:00
49c2d1dcad
sgutils2: use sg_get_asc_ascq_str to produce error messages
2021-01-27 06:56:11 +01:00
d0f51651f9
sgutils2: add ASC codes from tandeberg docs
2021-01-26 18:54:08 +01:00
481ccf16a5
sgutils2: further improve error messages
2021-01-26 15:19:43 +01:00
a223458753
sgutils2: support RequestSense Descriptor format
2021-01-26 13:38:16 +01:00
e1740f3f01
tape/changer/mtx: add mtx parser test
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:51:26 +01:00
740dc9d1d4
api2/tape/changer: reorganize api
...
add a changer listing here (copied from api2/config/changer)
and put the status and transfer api calls below that
puts the changer scan into the top level tape api
and removes the (now redundant) info from the config api path
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:47:34 +01:00
bbf01b644c
tape: fix typos
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:39:54 +01:00
c81c46c336
sgutils2: improve error messages
2021-01-26 12:24:58 +01:00
c3747b93c8
tape: add new command line tool "pmtx"
...
Also improve sgutil2 error reporting
2021-01-26 11:57:15 +01:00
6864fd0149
server/worker_task: improve newline handling in upid_read_status
...
improves upid_read_status with:
* ignore multiple newlines at the end
* remove all code that could panic (array index access)
the one place where we access with '[pos+1..]' is ok since
we explicitely test the len of the vector, this is done to
let rust optimize away the range checks, so it cannot panic
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 10:48:15 +01:00
340c0bf9e3
pxar: don't clone patterns unnecessarily
...
The options struct has no Drop handler and is passed by-move
so we can partially move out of it.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-26 10:24:18 +01:00
4d104cd4d8
clippy: more misc fixes
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:55 +01:00
367c0ff7c6
clippy: allow api functions with many arguments
...
some of those can be reduced/cleaned up when we have updater support in
the api macro.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:52 +01:00
9c26a3d61a
verify: factor out common parameters
...
all the verify methods pass along the following:
- task worker
- datastore
- corrupt and verified chunks
might as well pull that out into a common type, with the added bonus of
now having a single point for construction instead of copying the
default capacaties in three different modules..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:49 +01:00
93e3581ce7
derive/impl and use Default for some structs
...
and revamp HttpClientOptions with two constructors for the common use
cases
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:45 +01:00
f4e52bb27d
authid: make Tokenname(Ref) derive Eq
...
it's needed to derive Hash, and we always compare Authids or their
Userid components, never just the Tokenname part anyway..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:40 +01:00
72064fd0df
pxar: extract PxarExtractOptions
...
same as PxarCreateOptions, but for extraction/restore rather than
create.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:36 +01:00
77486a608e
pxar: factor out PxarCreateOptions
...
containing the CLI parameters that are mostly passed-through from the
client to our pxar archive creation wrapper in pxar::create
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:32 +01:00
e97025ab02
pxar: typedef on_error as ErrorHandler
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:26 +01:00
e43b9175c0
client: factor out UploadOptions
...
to reduce function signature complexity.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:18 +01:00
9cc1415ef5
systemd/time: extract Time/DateSpec structs
...
could be pulled up into CalendarEvent if desired..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:13 +01:00
bd215dc0e4
async index reader: typedef ReadFuture
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:58 +01:00
12e874cef0
allow complex Futures in tower_service impl
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:55 +01:00
6d233161b0
client: refactor catalog upload spawning
...
by pulling out Result type into separate struct
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:51 +01:00
905a570489
broadcast_future: refactor broadcast/future binding
...
into its own, private struct.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:48 +01:00
432fe44187
report: type-alias function call tuple
...
to make clippy happy.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:43 +01:00
51b938496d
tools::sgutils2: name fixup
...
it's not a box anymore
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 15:05:52 +01:00
b7f9b25e4d
tools::sgutils2: use NonNull
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:56:10 +01:00
fe61280b6b
tools::sgutils2: extern 'C' and import ordering
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:54:25 +01:00
68c087d578
tools::sgutils2: don't transmute to a Box
...
Otherwise we run the drop handler for the scsi pt object AND
the box itself, which shouldn't even work as it should be
doing a double-free (unless the library does some kind of
reference counting in which case this should simply crash
later on?)
anyway, let's make a wrapper simply called `SgPt` containing
the pointer from `construct_scsi_pt_obj()`
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:48:27 +01:00
d6bf87cab7
tools::sgutils2: const correctness
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:33:45 +01:00
2b96a43879
tape: cleanup - use ScsiMediaChange trait instead of mtx_status()
2021-01-25 13:25:22 +01:00
697c41c584
tape: add/use rust scsi changer implementation using libsgutil2
2021-01-25 13:14:07 +01:00
a2379996e6
sgutils2: add scsi_inquiry command
2021-01-25 13:14:07 +01:00
29077d95db
http-client: further clippy cleanups
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:54 +01:00
dbd00a57b0
http-client: fix typoed ticket cache condition
...
which was even copy-pasted once without noticing.
found with clippy.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:51 +01:00
d08cff51a4
rework GC traversal error handling
...
the error message don't make sense with an empty default
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:48 +01:00
3e461dec1c
apt: let api handle optional bool with default
...
one less FIXME :)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:46 +01:00
4d08e25913
clippy: rewrite ifs with identical return values
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:43 +01:00
43313c2ee7
clippy: rewrite comparison chains
...
chunk_stream one can be collapsed, since split == split_to with at set
to buffer.len() anyway.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:39 +01:00
81b2a87232
clippy: fix Mutex with unused value
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:36 +01:00
3d8cd0ced7
clippy: add is_empty() when len() is implemented
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:32 +01:00
7c78d54231
sgutils: allow command which does not transfer any data
2021-01-24 15:19:43 +01:00
f9d71e8b17
sgutils2: allow to set custom timeouts
2021-01-24 14:54:30 +01:00
0107fd323c
cleanup: avoid compiler warnings
2021-01-23 17:34:26 +01:00
979dccc7ec
tape: avoid error when clearing encryption key
...
Simply ignore clear request when sg_spin_data_encryption_caps fails.
Assume those are tapes without hardware encryption support.
2021-01-23 10:20:43 +01:00
bf78f70885
improve code docs in api2
...
Note: API methos should be declared pub, so that they show up in the generated docu.
2021-01-22 15:57:42 +01:00
0d916ac531
tape: add media pool config code docs
2021-01-22 12:01:46 +01:00
d4ab407045
tape: add drive config code docs
2021-01-22 11:51:36 +01:00
45212a8c78
fix mtx parser bug: s/strip_suffix/strip_prefix/
2021-01-22 11:00:56 +01:00
64b83c3d70
tape: implement paperkey command for tape encryption keys
2021-01-22 09:56:14 +01:00
639a6782bd
paperkey: move code to src/tools/paperkey.rs
2021-01-22 09:42:59 +01:00
5f34d69bcc
tape: add volume-statistics api/command
2021-01-22 08:45:35 +01:00
337ff5a3cc
tape: add estimated medium wearout to status
2021-01-22 08:06:25 +01:00
8e6459a818
tape: set encryption key on restore
2021-01-22 07:26:42 +01:00
aff3e16194
tape: add code docs to src/config/tape_encryption_keys.rs
2021-01-21 18:23:07 +01:00
9372c0787d
renamed src/tape/sgutils2.rs -> src/tools/sgutils2.rs
2021-01-21 17:57:17 +01:00
83fb2da53e
tape: move MediaCatalog magic number into struct (doc cleanup)
2021-01-21 17:48:07 +01:00
645a044bf6
tape: further hierarchy improvements
2021-01-21 17:25:32 +01:00
37796ff73f
tape: change code hierarchy to improve docs
2021-01-21 17:12:01 +01:00
e1fdcb1678
tape: do not export/doc low level libsgutils2 bindings
2021-01-21 16:38:24 +01:00
edda5039d4
tape: improve code docs
2021-01-21 13:19:07 +01:00
1c86893d95
cleanup: always compute fingerprint in KeyConfig constructors
2021-01-21 11:56:54 +01:00
d543587d34
Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup
2021-01-21 10:56:52 +01:00
780bc4cad2
tape: try to set encryption key with read-label command
2021-01-21 10:31:49 +01:00
18bd6ba13d
tape: restore_key - always update key, even if there is already an entry
2021-01-21 10:31:49 +01:00
4dafc513cc
tape: fix file permissions for tape encryptiuon keys
2021-01-21 10:31:49 +01:00
7acd5c5659
cleanup: remove missleading wording from code docs
2021-01-21 10:31:49 +01:00
8428063d9e
cleanup: KeyConfig::decrypt - show password hint on error
2021-01-21 10:31:49 +01:00
f490dda05a
tape: use type Uuid instead of String
2021-01-21 10:31:49 +01:00
2b191385ea
tape: use specialized encryption key per media-set
2021-01-21 10:31:49 +01:00
bc228e5eaf
api: add types for UUIDs
2021-01-20 17:16:46 +01:00
8be65e34de
clippy: replace transmute with &*
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:41:02 +01:00
d967d8f1a7
clippy: remove drop(&..)
...
it does nothing.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:41:02 +01:00
50deb0d3f8
clippy: use is_null to check for null pointers
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:41:02 +01:00
1d928b25fe
clippy: remove some unnecessary reference taking
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
f2f81791d1
clippy: fix for_kv_map
...
and allow it in the one case where the entry loop is intended, but the
code is not yet implemented fully.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
382f10a0cc
clippy: fix/allow needless_range_loop
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
0d2133db98
clippy: use while let loops
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
09faa9ee95
clippy: pass &str/&[..] instead of &String/&Vec
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
ccec086e25
clippy: remove unnecessary &mut
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
05725ac9a4
clippy: remove unnecessary `let` binding
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
96b7483138
clippy: remove/replace needless explicit lifetimes
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
81281d04a4
clippy: fix/allow identity_op
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
e062ebbc29
clippy: us *_or_else with function calls
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
b92cad0938
clippy: convert single match to if let
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
ea368a06cd
clippy: misc. fixes
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3f48cdb380
clippy: don't pass along unit value
...
make it explicit. this whole section should probably be re-written with
select!
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
17c7b46a69
clippy: use unwrap_or_default
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a375df6f4c
clippy: use copied/cloned instead of map
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a3775bb4e8
clippy: shorten assignments
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
1e0c6194b5
clippy: fix option_as_ref_deref
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a6bd669854
clippy: use matches!
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
6334bdc1c5
clippy: collapse nested ifs
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3b82f3eea5
clippy: avoid useless format!
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
38556bf60d
clippy: remove explicit returns
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
d8d8af9826
clippy: use chars / byte string literals
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3984a5fd77
clippy: is_some/none/ok/err/empty
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
397356096a
clippy: remove needless bool literals
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:52 +01:00
365915da9a
clippy: use strip_prefix instead of manual stripping
...
it's less error-prone (off-by-one!)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
87152fbac6
clippy: drop redundant 'static lifetime
...
those declarations are already const/static..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
22a9189ee0
clippy: remove unnecessary closures
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
4428818412
clippy: remove unnecessary clones
...
and from::<T>(T)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
47ea98e0e3
clippy: collapse/rework nested ifs
...
no semantic changes (intended).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
6dd0513546
tape: allocate new media set when pool encryption key changes
2021-01-20 15:43:39 +01:00
8abe51b71d
improve code docs
2021-01-20 15:43:19 +01:00
69b8bc3bfa
tape: implemenmt show key
...
Moved API types Kdf and KeyInfo to src/api2/types/mod.rs.
2021-01-20 15:43:19 +01:00
301b8aa0a5
tape: implement change-passphrase for tape encryption keys
2021-01-20 15:43:19 +01:00
e5b6c93323
tape: add --kdf parameter to create key api
2021-01-20 15:43:19 +01:00
9a045790ed
cleanup KeyConfig
2021-01-20 15:43:19 +01:00
82a103c8f9
add "password hint" to KeyConfig
2021-01-20 15:43:19 +01:00
feb1645f37
tape: generate random encryptions keys and store key_config on media
2021-01-19 11:20:07 +01:00
8ca37d6a65
cleanup: factor out decrypt_key_config
2021-01-19 11:20:07 +01:00
9b6bddb24c
tfa: remove/empty description for recovery keys
...
While the user chosen description is not allowed to be
empty, we do leave it empty for recovery keys, as a "dummy
description" makes little sense...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 15:20:39 +01:00
ad5cee1d22
tfa: add 'created' timestamp to entries
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 14:06:12 +01:00
ca1060862e
tfa: remember recovery indices
...
and tell the client which keys are still available rather
than just yes/no/low
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 13:51:23 +01:00
8a0046f519
tape: implement encrypted backup - simple version
...
This is just a proof of concept, only storing the encryption key fingerprint
inside the media-set label.
2021-01-18 13:38:22 +01:00
84cbdb35c4
implement FromStr for Fingerprint
2021-01-18 13:38:22 +01:00
1e93fbb5c1
tape: add encrypt property to media pool configuration
2021-01-18 13:38:22 +01:00
619554af2b
tape: clear encryption key before writing labels
...
We always write labels unencrypted.
2021-01-18 13:38:22 +01:00
d5a48b5ce4
tape: add hardware encryption key managenent api
2021-01-18 13:38:22 +01:00
44de5bcc00
pull: add error context for initial group list call
...
otherwise the user is confronted with a generic error like "permission
check failed" with no indication that it refers to a request made to the
remote PBS instance..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-18 06:51:05 +01:00
e2956c605d
pull: rustfmt
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-18 06:50:23 +01:00
b22b6c2299
tape: encryption scsi command cleanup
2021-01-16 18:24:04 +01:00
90950c9c20
tape: add scsi commands to control drive hardware encryption
2021-01-16 15:59:05 +01:00
0c5b9e7820
tape: sgutils2.rs - add do_out_command()
...
Make it possible to run commands that writes data.
2021-01-16 15:59:05 +01:00
a670b99db1
tfa: add webauthn configuration API entry points
...
Currently there's not yet a node config and the WA config is
somewhat "tightly coupled" to the user entries in that
changing it can lock them all out, so for now I opted for
fewer reorganization and just use a digest of the
canonicalized config here, and keep it all in the tfa.json
file.
Experimentally using the flatten feature on the methods with
an`Updater` struct similar to what the api macro is supposed
to be able to derive on its own in the future.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
aefd74197a
bakckup::manifest: use tools::json for canonical representation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
9ff747ef50
add tools::json for canonical json generation
...
moving this from backup::manifest, no functional changes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
a08a198577
tape: do not abort backup if tape drive does not support tape-alert-flags
2021-01-15 11:43:17 +01:00
6bbe49aa14
access: restrict password changes on @pam realm to superuser
...
for behavior consistency with `update_user`
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-15 08:49:22 +01:00
5aa1019010
access: limit editing pam credentials to superuser
...
modifying @pam users credentials should be only possible for root@pam,
otherwise it can have unintended consequences.
also enforce the same limit on user creation (except self_service check,
since it makes no sense during user creation)
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-15 08:49:22 +01:00
29a59b380c
proxmox 0.10: adapt to moved ParameterSchema
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0bfcea6a11
cleanup: remove unnecessary 'mut' and '.clone()'
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
89e9134a3f
hyper: use new hyper::upgrade
...
the old Body::on_upgrade method is no more
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
b5a202acb6
tokio 1.0: update to new Signal interface
...
Signal does not yet re-implement Stream (and is not yet wrapped in
tokio-stream either).
see https://github.com/tokio-rs/tokio/pull/3383
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0f860f712f
tokio 1.0: update to new tokio-openssl interface
...
connect/accept are now happening on pinned SslStreams
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
7c66701366
tokio 1.0: use ReceiverStream from tokio-stream
...
to wrap a Receiver in a Stream. this will likely move back into tokio
proper once we have a std Stream..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
585e90c0de
tokio: adapt to 1.0 process:Child changes
...
Child itself is no longer a Future, but it has a new wait() async fn
that does the same thing
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
5c852d5b82
tokio: adapt to 1.0 runtime changes
...
enter() now returns a guard, and the builder got revamped to make the
choice between MT and current thread explicit.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
484172b5f8
tokio 1.0: AsyncRead/Seek with ReadBuf
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
d148958b67
proxmox 0.10: use tokio::time::timeout directly
...
TimeoutFutureExt is no more
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0a8d773ad0
tokio 1.0: delay -> sleep
...
almost the same thing, new name(s), no longer Unpin
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
427d90e6c1
update to tokio 1.0
...
and various related crates
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
1a0b410554
manager: user/token list: fix rendering 0 (never) expire date
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-14 13:59:08 +01:00
2d50a6192f
tape: sg-tape-cmd - add more ways to specify devices
2021-01-14 13:05:26 +01:00
781da7f6f0
tape: add --inventorize flag to read-label API/CLI
2021-01-14 11:51:23 +01:00
25e464c5ce
tape: MediaPool - allow to allocate free tapes
2021-01-13 14:25:51 +01:00
8446fbca85
tape: rename changer_id to label_text
2021-01-13 13:26:59 +01:00
0bce2118e7
tape: improve docu
2021-01-12 16:37:23 +01:00
6543214dde
tape: MediaListEntry - add ctime
2021-01-12 12:01:21 +01:00
a484c9cf96
tape: automatically reload tapes inside autoloader
...
We always automatically unload tapes to free library slots,
so it should not happen that an ejected tape resides inside the drive.
This is just a safe guard to handle the situation in case it happens ...
You can manually produce the situation by ejecting a tape without unloading:
mt -f /dev/nst0 eject
Note: Our "proxmox-tape eject" does automatic unload
2021-01-12 09:49:05 +01:00
5654d8ceba
tape: make eject/export more reliable, improve logging
2021-01-12 09:16:16 +01:00
31cf625af5
tape: improve backup logs
2021-01-11 13:23:12 +01:00
93be18ffd2
tape: fix tape alert flag values
2021-01-11 13:23:12 +01:00
ad0ed40a59
api: return "invalid" as CSRF token for partial tickets
...
So that old clients don't `unwrap` a `None` value.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
7ad33e8052
tfa: use UNAUTHORIZED http status in password check
...
to trigger our 3s delay in the rest handler
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
abfe0c0e70
tfa: fixup for challenge file split
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
f22dfb5ece
tfa: remove tfa user when a user is deleted
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:10 +01:00
4bda51688b
tfa: improve user existence check
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
eab25e2f33
tfa: allow deletion of entries of non-existent users
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
94bd11bae2
typo fixups
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
759af9f00c
tfa api: return types and 'pub' structs/methods
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
f58e5132aa
tfa: entry access/iteration cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
d831846706
tfa: r#type parameter name
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
1fc9ac0433
tfa: _entry api method name suffix consistency
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
7f066a9b21
proxy: expose qrcodejs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
027ef213aa
api: tfa management and login
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
dc1fdd6267
config: add tfa configuration
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
96918252e5
buildcfg: add rundir helper macro
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
014dc5f9d7
tools: add create_run_dir helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
59e94227af
add tools::serde_filter submodule
...
can be used to perform filtering at parse time
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
6638c034d2
tape: remove unused eject_on_unload method
2021-01-10 16:20:18 +01:00
04df41cec1
tape: more MediaChange cleanups
...
Try to provide generic implementation for complex operations:
- unload_to_free_slot
- load_media
- export media
- clean drive
- online_media_changer_ids
2021-01-10 15:32:52 +01:00
483da89d03
tape: improve export media to directly export from drive, add CLI
2021-01-10 13:44:44 +01:00
c92e3832bf
tape: cleanup: s/transfer/transfer_media/, avoid compiler warnings
2021-01-10 12:18:30 +01:00
edb90f6afa
tape: backup - implement export-media-set option
2021-01-10 11:59:55 +01:00
0057f0e580
tape: MediaChange - add transfer, implement export
2021-01-10 11:51:09 +01:00
e6217b8b36
tape: renamed src/tape/changer/linux_tape.rs -> src/tape/changer/mtx.rs
2021-01-10 10:07:40 +01:00
6fe16039b9
tape: simplify media changer implementation - new struct MtxMediaChanger
2021-01-10 10:02:01 +01:00
42967bf185
tape: backup - implement --eject-media option
2021-01-09 15:17:03 +01:00
5843268c47
tape: abort backup when we detect critical tape alert flags
2021-01-09 12:34:00 +01:00
7273ba3de2
tape: change default media set naming template to "%c"
2021-01-09 10:51:51 +01:00
0bf1c314da
tape: show catalog status in media list
2021-01-09 10:24:48 +01:00
c7926d8e8c
tape: split MediaSet into extra file
2021-01-09 08:54:58 +01:00
df69a4fc59
tape: implement drive clean
2021-01-08 11:32:56 +01:00
25d3965769
tape: correctly skip cleaning tapes (not regular tapes)
2021-01-08 09:16:42 +01:00
879569d73f
tape: changer transfer - make name parameter optional
2021-01-07 17:09:47 +01:00
b63f833d36
tape: fix paramater name - s/slot/source-slot/
2021-01-07 15:39:25 +01:00
482c6e33dd
tape: changer status command: make changer name optional
2021-01-07 15:12:19 +01:00
46a1863f88
tape: improve MediaChange trait
...
We expose the whole MtxStatus, and we can load/store from/to
specified slot numbers.
2021-01-07 14:26:43 +01:00
d5035c5600
tape: mtx_status - consider new export-slots property
2021-01-06 11:53:33 +01:00
38ae42b11a
tape: changer - add export-slot config
2021-01-06 11:06:50 +01:00
c4b2b9ab41
tape: only query volume stats if we can read MAM
2021-01-06 09:20:36 +01:00
ef942e04c2
tape: add function to classify tape-alert-flags
2021-01-05 17:23:30 +01:00
b40ab10d38
tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus
2021-01-05 13:43:17 +01:00
f8ccbfdedd
tape: implement read_volume_statistics
2021-01-05 12:58:18 +01:00
470f1c798a
tape: status - show thape alert flags
2021-01-04 13:15:30 +01:00
5c012b392a
tape: use LP 12h TapeAlert Response to query tape alert flags
2021-01-04 13:14:02 +01:00
165b641c1d
tape: changer status - show full slots (for cartridge without barcode)
2021-01-04 12:06:05 +01:00
66e42bec05
tape: further PoolWriter cleanups
2021-01-03 12:08:40 +01:00
c503ea7045
tape: cleanup - rename 'info' to 'media_id'
...
Second try.
2021-01-03 11:38:00 +01:00
745ec187ce
Revert "tape: cleanup - rename 'info' to 'media_id'"
...
This reverts commit f046313c0e
2021-01-03 11:14:58 +01:00
f046313c0e
tape: cleanup - rename 'info' to 'media_id'
2021-01-03 10:37:42 +01:00
74595b8821
tape: sg-tape-cmd tape-alert-flags
2021-01-03 10:09:43 +01:00
c9fdd142a4
tape: commit missing file
2021-01-02 13:39:34 +01:00
abaa6d0ac9
tape: decode TapeAlertFlags in cartridge-memory command
2021-01-02 10:55:30 +01:00
cfae8f0656
tape: merge MediaStateDatabase into Inventory
2021-01-01 16:15:13 +01:00
54f4ecd46a
tape: implement MediaPool flag to consider offline media
...
For standalone tape drives.
2021-01-01 10:03:59 +01:00
b9b4b31284
tape: add basic restore api/command
2020-12-31 10:26:48 +01:00
b4772d1c43
tape: new inventory helper - lookup_media_set_pool
2020-12-31 10:03:17 +01:00
08ac90f920
api: allow tokens to list users
...
their owner, or all if they have the appropriate privileges.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:49 +01:00
13f5863561
api: improve error messages for restricted endpoints
...
the old variant attempted to parse a tokenid as userid and returned the
cryptic parsing error to the client, which is rather confusing.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:09 +01:00
81764111fe
tape: media_change - log all errors
2020-12-30 19:17:18 +01:00
cb022525ff
tape: only log to stdout in CLI environment
2020-12-30 19:01:39 +01:00
75656a78c6
tape: improve inline docu
2020-12-30 17:28:33 +01:00
284eb5daff
tape: cleanup/simplify media_change code
2020-12-30 17:16:57 +01:00
ff58c51919
tape: improve media request/load
2020-12-30 13:09:28 +01:00
12299b333b
tape: set minimal media label length to 2
2020-12-30 10:15:02 +01:00
b017bbc441
tape: add restore code, implement catalog api/command
2020-12-30 09:48:18 +01:00
9e8c0d2e33
tape: cleanup - remove debug messages
2020-12-30 08:41:30 +01:00
250c29edd2
tape: correctly sort media api entries
2020-12-29 12:09:51 +01:00
c431659d05
cleanup: remove debug output
2020-12-29 11:59:57 +01:00
a33389c391
tape: implement media content list api
2020-12-29 11:58:26 +01:00
3460565414
tape: create the MediaCatalog when we label a tape
2020-12-29 10:55:20 +01:00
26b62138ee
cleanup: disable debug message when we detect a stopped worker task
2020-12-29 10:53:16 +01:00
afb0220642
tape: cleanup LinuxDriveStatus - make density optional
2020-12-29 09:10:30 +01:00
0993923ed5
tape: factor out get_drive_and_media_status
2020-12-29 08:39:06 +01:00
e0362b0d0f
tape: correctly parse mtx import/export slots
2020-12-28 13:32:56 +01:00
b27c32821c
tape: install new sg-tape-cmd setuid binary
2020-12-28 11:10:25 +01:00
76b15a035f
tape: MediaCatalog: write magic number before content
2020-12-26 11:05:25 +01:00
eb8feb1281
tape: add LTO1 to TapeDensity
2020-12-26 10:48:32 +01:00
fc6ce9835b
tape: fix non-rewinding tape device check
2020-12-25 15:38:29 +01:00
8ae9f4efc2
tape: minor cleanups
2020-12-25 13:45:26 +01:00
c9d13b0fc4
tape: expose check_tape_is_linux_tape_device
2020-12-24 15:51:49 +01:00
bfacc1d8c3
tape: cleanup - factor out open_linux_tape_device
2020-12-24 11:24:45 +01:00
5ae86dfaa1
tape: return media usage info with status command
2020-12-23 11:24:34 +01:00
dbe7e556b0
tape: implement binding for libsgutils2
...
So that we can read cartridge memory without calling "sg_raw". In future,
we may need further low level command to control the tape..
2020-12-23 09:44:53 +01:00
4799280ccd
http_client: add timeouts for critical connects
...
Use timeout futures for sections that might hang in certain error
conditions. This is mostly intended to be used as a safeguard, not a
first line of defense - i.e. best-effort avoidance of total hangs.
Not every future used for the HttpClient/H2Client is changed, only those
where a quick response is to be expected. For example, the response
reading futures are left alone, so data transfer is never capped with
timeout, only the initial server connect.
It is also used for upgrading to H2 connections, as that can take a long
time on overloaded servers.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-12-22 13:31:10 +01:00
cb80d900b3
tape: add drive status api
2020-12-22 10:42:22 +01:00
ee01737e87
tape: rename 'mam' api to 'cartridge-memory'
2020-12-22 09:27:34 +01:00
2012825913
depend on proxmox 0.9.0
2020-12-22 08:52:24 +01:00
b2362a1207
adaptions for proxmox 0.9 and proxmox-api-macro 0.3
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-22 07:31:05 +01:00
1e20f819d5
tape: add command to read cartridge memory (MAM)
...
Thsi add an additional dependency to sg3-utils (small).
2020-12-21 12:12:33 +01:00
9aa58f0143
cleanup: rename mtfsf into forward_space_count_files
2020-12-18 16:57:49 +01:00
8835664653
tape: add tape backup api
2020-12-18 15:32:12 +01:00
d37da6b7fc
tape: add PoolWriter
2020-12-18 15:27:44 +01:00
b9ee86efe1
tape: use SnapshotReader to create snapshot archive
2020-12-18 12:11:29 +01:00
d108b610fd
tape: fix write_media_set_label - move to correct position
2020-12-18 12:11:29 +01:00
0ec79339f7
tools/daemon: improve reload behaviour
...
it seems that sometimes, the child process signal gets handled
before the parent process signal. Systemd then ignores the
childs signal (finished reloading) and only after going into
reloading state because of the parent. this will never finish.
Instead, wait for the state to change to 'reloading' after sending
that signal in the parent, an only fork afterwards. This way
we ensure that systemd knows about the reloading before actually trying
to do it.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-12-18 10:30:37 +01:00
2afdc7f27d
tape: MediaPool::with_config() - remove name parameter
...
Not required, because config already contains the pool name.
2020-12-18 08:14:24 +01:00
26aa9aca40
tape: return current_file_number as u64
2020-12-18 07:44:50 +01:00
3e2984bcb9
tools/process_locker: Decrement writer count in drop handler
...
of ProcessLockSharedGuard.
We use a counter to determine if we can unlock the file again, but
we never actually decremented the writer count, so we held the
lock forever.
This fixes the issue that we could not start a garbage collect after
a reload, as long as the old process is still running, even when that
process has no active backup anymore but another long running task
(e.g. file download, terminal, etc.).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-18 07:15:08 +01:00
a7a5406c32
acl: rustfmt module
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:07:01 +01:00
4f727a783e
acl: reformat privileges
...
for better readability, and tell rustfmt to leave those definitions
alone.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:45 +01:00
23dc68fdea
acl: add docs and adapt visibility
...
document all public things, add some doc links and make some
previously-public things only available for test cases or within the
crate:
previously public, now private:
- AclTreeNode::extract_user_roles (we have extract_roles())
- AclTreeNode::extract_group_roles (same)
- AclTreeNode::delete_group_role (exists on AclTree)
- AclTreeNode::delete_user_role (same)
- AclTreeNode::insert_group_role (same)
- AclTreeNode::insert_user_role (same)
- AclTree::write_config (we have save_config())
- AclTree::load (we have config()/cached_config())
previously public, now crate-internal:
- AclTree::from_raw (only used by tests)
- split_acl_path (used by some test binaries)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:11 +01:00
b532dd00c4
tape: add helper to read snapshot contents
...
- lock the snapshot for reading
- use openat to open files
- provides an iterator over all chunks
2020-12-17 13:07:52 +01:00
c01742855a
KeyConfig: bail on wrong fingerprint
...
instead of just logging the error. this should never happen in practice
unless someone is messing with the keyfile, in which case, it's better
to abort.
update tests accordingly (wrong fingerprint should fail, no fingerprint
should get the expected one).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 11:27:06 +01:00
9c953dd260
tape: add code to write backup snapshot files (without chunks) to tape
2020-12-17 08:28:47 +01:00
3fbf2d2fcd
tape: cleanup MediaCatalog
2020-12-17 08:05:53 +01:00
e0af222ec3
KeyConfig: always calculate fingerprint
...
and warn if stored and calculated fingerprint don't match.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:52:55 +01:00
73b5011786
KeyConfig: add encrypt/decrypt test
...
the RSA key and the encryption key itself are hard-coded to avoid
stalling the test runs because of lack of entropy, they have no special
significance otherwise.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:47:45 +01:00
7137630d43
client: add 'import-with-master-key' command
...
to import an encrypted encryption key using a master key.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:46:24 +01:00
8acfd15d6e
key: move RSA-encryption to KeyConfig
...
since that is what gets encrypted, and not a CryptConfig.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:43:34 +01:00
48fbbfeb7e
fix #3197 : skip fingerprint check when restoring key
...
when restoring an encrypted key, the original one is obviously not
available to check the fingerprint with.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:37:54 +01:00
9990af3042
master key: store blob name in constant
...
since we will use it in more than one place.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:36:06 +01:00
fe6c19383b
tape: remove MediaLabelInfo, use MediaId instead
...
The additional content_uuid was quite useless...
2020-12-16 13:31:32 +01:00
9839d3f778
tape: improve docu
2020-12-16 12:43:51 +01:00
dd59e3c2a1
tape: improve docu
2020-12-16 12:23:52 +01:00
0b7432ae09
tape: add chunk archive reader/writer
2020-12-16 12:08:34 +01:00
c1c2c8f635
tape: cleanup MediaLocation type for direct use with API
2020-12-16 10:49:01 +01:00
42298d5896
tape: add magic number to identify media catalog files
2020-12-16 09:00:14 +01:00
205e187613
tape: add MediaCatalog implementation
2020-12-15 13:40:49 +01:00
a78348acbb
tape: rename DriveLabel to MediaLabel
2020-12-14 17:37:16 +01:00
410611b4f2
tape: improve file format docu
2020-12-14 17:29:57 +01:00
af07ec8f29
tape: minor code cleanup
2020-12-14 16:56:26 +01:00
3f803af00b
tape: scan - print more debug info
2020-12-14 13:16:18 +01:00
ac461bd651
tape: implement scan command (useful for debug)
2020-12-14 12:55:49 +01:00
ce955e1635
tape: implement eod cli command (debug tool)
2020-12-14 09:56:59 +01:00
e20d008c6a
tape: rename cli 'media media-destroy' toö 'media destroy'
2020-12-14 09:30:32 +01:00
fb657d8ee5
tape: implement destroy_media
2020-12-14 08:58:40 +01:00
fba0b77469
tape: add media api
2020-12-14 07:55:57 +01:00
b5c1296eaa
tape: make changer get_status async
2020-12-14 07:14:24 +01:00
065df12872
tape: split api type definitions for changers into extra file
2020-12-13 09:31:02 +01:00
7e1d4712b8
tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA
2020-12-13 09:22:08 +01:00
49c965a497
tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA
2020-12-13 09:18:16 +01:00
6fe9aedd0b
tape: correctly call Async handler in proxmox-tape
2020-12-12 09:58:47 +01:00
42cb9bd6a5
tape: avoid executor blocking in changer api
2020-12-12 09:45:08 +01:00
66dbe5639e
tape: avoid executor blocking in drive API
...
By using tokio::task::spawn_blocking().
2020-12-12 09:20:04 +01:00
663ef85992
tape: use WorkerTask for erase and rewind
2020-12-11 11:19:33 +01:00
e92c75815b
tape: split inventory api
...
inventory: sync, list labels with uuids,
update_inventory: WorkerTask, updates database
2020-12-11 10:42:29 +01:00
6dbad5b4b5
tape: run label commands as WorkerTask (threads)
2020-12-11 09:10:22 +01:00
bff7e3f3e4
tape: implement barcode-label-mdedia
2020-12-11 07:50:19 +01:00
83abc7497d
tape: implement inventory command
2020-12-11 07:39:28 +01:00
be1a8c94ae
fix build: add missing file
2020-12-10 13:40:20 +01:00
4606f34353
tape: implement read-label command
2020-12-10 13:20:39 +01:00
7bb720cb4d
tape: implement label command
2020-12-10 12:30:27 +01:00
c4d8542ec1
tape: add media pool handling
2020-12-10 11:41:35 +01:00
9700d5374a
tape: add media pool cli
2020-12-10 11:13:12 +01:00
05e90d6463
tape: add media pool config api
2020-12-10 10:52:27 +01:00
55118ca18e
tape: correctly sort drive api subdir
2020-12-10 10:09:12 +01:00
f70d8091d3
tape: implement option changer-drive-id
2020-12-10 09:09:06 +01:00
a3c709ef21
tape: cli cleanup - avoid api redefinition
2020-12-10 08:35:11 +01:00
4917f1e2d4
tape: implement delete property for drive update command
2020-12-10 08:25:46 +01:00
93829fc680
tape: cleanup load-slot api
2020-12-10 08:04:55 +01:00
5605ca5619
tape: cli cleanup - rename scana-for-* into scan
2020-12-10 07:58:45 +01:00
e49f0c03d9
tape: implement load-media command
2020-12-10 07:52:56 +01:00
0098b712a5
tape: implement eject
2020-12-09 17:50:48 +01:00
5fb694e8c0
tape: implement rewind
2020-12-09 17:43:38 +01:00
583a68a446
tape: implement erase media
2020-12-09 17:35:31 +01:00
e6604cf391
tape: add command line interface proxmox-tape
2020-12-09 13:00:20 +01:00
43cfb3c35a
tape: do not remove changer while still used
2020-12-09 12:55:54 +01:00
8a16c571d2
tape: add changer property to drive create api
2020-12-09 12:55:10 +01:00
314652a499
tape: set protected flag for configuration change api methods
2020-12-09 12:02:55 +01:00
6b68e5d597
client: move connect_to_localhost into client module
2020-12-09 11:59:50 +01:00
cafd51bf42
tape: add media state database
2020-12-09 11:21:56 +01:00
9b93c62044
remove unused descriptions from api macros
...
these are now a hard error in the api macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-09 10:55:18 +01:00
5d90860688
tape: expose basic tape/changer functionality at api2/tape/
2020-12-08 15:42:50 +01:00
5ba83ed099
tape: check digest on config update
2020-12-08 11:24:38 +01:00
50bf10ad56
tape: add changer configuration API
2020-12-08 09:04:56 +01:00
16d444c979
tape: add tape drive configuration API
2020-12-07 13:04:32 +01:00
fa9c9be737
tape: add tape device driver
2020-12-07 08:29:22 +01:00
2e7014e31d
tape: add BlockeReader/BlockedWriter streams
...
This is the basic format used to write data to tapes.
2020-12-06 12:09:55 +01:00
a84050c1f0
tape: add BlockHeader impl
2020-12-06 10:26:24 +01:00
7c9835465e
tape: add helpers to emulate tape read/write behavior
2020-12-06 09:41:16 +01:00
ec00200411
fix bug #3189 : fix change_password permission checks, run protected
2020-12-05 16:20:29 +01:00
b107fdb99a
tape: add tape changer support using 'mtx' command
2020-12-05 14:54:12 +01:00
7320e9ff4b
tape: add media invenotry
2020-12-05 12:54:15 +01:00
c4d2d54a6d
tape: define useful constants
2020-12-05 12:20:46 +01:00
1142350e8d
tape: add media pool config
2020-12-05 11:59:38 +01:00
d735b31345
tape: add tape read trait
2020-12-05 10:54:38 +01:00
e211fee562
tape: add tape write trait
2020-12-05 10:51:34 +01:00
8c15560b68
tape: add file format definitions
2020-12-05 10:45:08 +01:00
327e93711f
commit missing file: tape api type definitions
2020-12-04 16:00:52 +01:00
a076571470
tape support: add drive configuration
2020-12-04 15:42:32 +01:00
179145dc24
backup/datastore: move manifest locking to /run
...
this fixes the issue that on some filesystems, you cannot recursively
remove a directory when you hold a lock on a file inside (e.g. nfs/cifs)
it is not really backwards compatible (so during an upgrade, there
could be two daemons have the lock), but since the locking was
broken before (see previous patch) it should not really matter
(also it seems very unlikely that someone will trigger this)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 09:56:42 +01:00
6bd0a00c46
backup/datastore: really lock manifest on delete
...
'lock_manifest' returns a Result<File, Error> so we always got the result,
even when we did not get the lock, but we acted like we had.
bubble the locking error up
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 14:37:05 +01:00
f6e28f4e62
client/pull: log how many groups to pull were found
...
if no groups were found, the task log was very confusing as it
contained no real information why nothing was synced, e.g.:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
this patch simply logs how many groups were found and are about to be synced:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
found 0 groups to sync
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 07:22:50 +01:00
2260f065d4
cleanup: use extra file for StoreProgress
2020-12-01 06:34:33 +01:00
6eff8dec4f
cleanup: remove unnecessary StoreProgress clone()
2020-12-01 06:29:11 +01:00
7e25b9aaaa
verify: use same progress as pull
...
percentage of verified groups, interpolating based on snapshot count
within the group. in most cases, this will also be closer to 'real'
progress since added snapshots (those which will be verified) in active
backup groups will be roughly evenly distributed, while number of total
snapshots per group will be heavily skewed towards those groups which
have existed the longest, even though most of those old snapshots will
only be re-verified very infrequently.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:55 +01:00
f867ef9c4a
progress: add format variants
...
for iterating over a single group, or iterating just on the group level
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:12 +01:00
fc8920e35d
pull: factor out interpolated progress
...
and add group/snapshot count info.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:13:11 +01:00
7f3b0f67e7
remove BackupGroup::list_groups
...
BackupInfo::list_backup_groups is identical code-wise, and makes more
sense as entry point for listing groups.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:09:44 +01:00
844660036b
gc: don't limit index listing to same filesystem
...
WalkDir does not follow symlinks by default anyway, and this behaviour
is not documented anywhere. e.g., if a sysadmin mounts 'extra storage'
for some backup group or type (not knowing that only metadata is stored
in those directories), GC will ignore all the indices contained within
and happily garbage collect their chunks..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:07:09 +01:00
efcac39d34
gc: remove duplicate variable
...
list_images already returns absolute paths, we don't need to prepend
anything.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:51 +01:00
cb4b721cb0
gc: log index files found outside of expected scheme
...
for safety reason, GC finds and marks all index files below the
datastore base path. as a result of regular operations, only index files
within the expected scheme of <TYPE>/<ID>/<TIMESTAMP> should exist.
add a small check + warning if the index list contains index files out
side of this expected scheme, so that an admin with shell access can
investigate.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:17 +01:00
7956877f14
gc: shorten progress messages
...
we have messages starting the phases anyway, and limit the number of
progress updates so that context remains available at all times.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:04:13 +01:00
Dietmar Maurer
Dominik Csapak
Thomas Lamprecht
Wolfgang Bumiller
Stefan Reiter
Fabian Grünbichler
Oguz Bektas