074503f288
tape: implement fast catalog restore
2021-03-24 08:40:34 +01:00
30316192b3
tape: improve locking (lock media-sets)
...
- new helper: lock_media_set()
- MediaPool: lock media set
- Expose Inventory::new() to avoid double loading
- do not lock pool on restore (only lock media-set)
- change pool lock name to ".pool-{name}"
2021-03-24 06:32:59 +01:00
54fcb7f5d8
api2/tape/backup: wait indefinitely for lock in scheduled backup jobs
...
so that a user can schedule multiple backup jobs onto a single
media pool without having to consider timing them apart
this makes sense since we can backup multiple datastores onto
the same media-set but can only specify one datastore per backup job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 09:04:32 +01:00
4abd4dbe38
api2/tape/backup: include a summary on notification e-mails
...
for now only contains the list of included snapshots (if any),
as well as the backup duration
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-19 09:03:52 +01:00
32b75d36a8
tape: backup media catalogs
2021-03-19 06:58:46 +01:00
237314ad0d
tape: improve catalog consistency checks
...
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.
Note: this changes the catalog format again.
2021-03-18 08:43:55 +01:00
54722acada
tape: store datastore name in tape archives and media catalog
...
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
2021-03-17 11:08:51 +01:00
e36135031d
ui: tape/Restore: let the user choose an owner
...
so that the tape backup can be restored as any user, given
the current logged in user has the correct permission.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 08:55:42 +01:00
5c4755ad08
tape: speedup backup by doing read/write in parallel
2021-03-16 08:51:49 +01:00
2c10410b0d
tape: improve backup task log
2021-03-11 08:43:13 +01:00
d1d74c4367
typo fixes all over the place
...
found and semi-manually replaced by using:
codespell -L mut -L crate -i 3 -w
Mostly in comments, but also email notification and two occurrences
of misspelled 'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 16:39:57 +01:00
6396bace3d
tape: improve backup task log (show percentage)
2021-03-10 10:59:13 +01:00
2a06e08618
api2/tape/backup: continue on vanishing snapshots
...
when we do a prune during a tape backup, do not cancel the tape backup,
but continue with a warning
the task still fails and prompts the user to check the log
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-09 10:20:54 +01:00
af76234112
tape: improve MediaPool allocation by sorting tapes by ctime and label_text
2021-03-09 08:33:21 +01:00
8703a68a31
tape/backup: add success/error notifications
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:06 +01:00
d95c74c6e7
tape: test tape alert flags after drive clean
2021-03-08 09:08:55 +01:00
c9793d47f9
api2/tape: add notify_user to backup(-jobs) and restore api calls
...
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-05 17:20:37 +01:00
9152a0077f
config/tape_job: add notify-user config to tape jobs
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-03-05 12:51:25 +01:00
0b90c67fb4
tape: avoid Updatable for now
2021-03-05 12:51:25 +01:00
b4975d3102
tape: finish api permission checks
2021-03-05 11:49:07 +01:00
ee33795b72
acl: use /tape/device instead of /tape/drive and /tape/changer
2021-03-05 10:06:19 +01:00
90e16be3ae
tape: improve PoolWriter logging
...
Log reason why we allocate a new media set.
2021-03-05 09:59:38 +01:00
cf90a369e2
cleanup: rename token_user into auth_id_filter
2021-03-05 08:36:18 +01:00
ab77d660cc
tape: improve media status in list_media
2021-03-04 10:17:10 +01:00
8a76e71129
tape: impl access permission for media content list
2021-03-03 13:16:15 +01:00
396fd747a6
tape: impl access permissions for tape jobs
2021-03-03 12:48:06 +01:00
ccdf327ac8
tape: impl access permissions for encryption key config
2021-03-03 12:21:55 +01:00
8cd63df0dc
tape: impl api permissions for drive/changer/pool configuration
2021-03-03 12:10:00 +01:00
347cde827b
dashboard: fix datastore full estimate for not changing usage
...
If the usage of a datastore did not change, we did not
return an estimate. The ui interpreted this as 'not enough data', but
it should actually be 'never'.
Fixing this by always setting the estimate first to 0 and overwriting
if we successfully calculated one, and checking for 'undefined' in the ui.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 17:51:13 +01:00
8bf5769382
api2/tape/drive: add state to DriveListEntry
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:51:19 +01:00
db04d10d14
tape: add comment to media pool config
2021-03-02 11:42:58 +01:00
1d14c31658
Revert "tape: also abort backup/restore on server shutdown"
...
This reverts commit 9bd81bb384
2021-03-02 08:00:10 +01:00
9bd81bb384
tape: also abort backup/restore on server shutdown
2021-03-02 07:19:31 +01:00
1552d9699c
tape: add 'Restore Key' button
2021-03-01 11:11:38 +01:00
16f9ea6708
tape: restore - do not check if pool is configured
2021-03-01 09:43:23 +01:00
955f4aefcd
tape: show media inventory/content, even if there is no pool configuration
2021-03-01 09:21:11 +01:00
08ec39be0c
tape: add api to set media status
2021-02-26 10:44:07 +01:00
5830e5620d
tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig
2021-02-25 09:44:31 +01:00
0dadf66dc7
tape: implement backup using latest-only flag
2021-02-24 13:51:53 +01:00
21e3ed3449
tape: add 'latest-only' property to backup job config
2021-02-24 11:19:12 +01:00
52d8db7925
api2/config/tape_backup_job: fix duplicate id parameter
...
since the PUT api call is using the 'Updater', the 'id' parameter is
already encoded in there, tripping up the api verify tests with
'Duplicate keys found in AllOf schema: id'
"fixing" it by removing the explicit id from the api call and
taking it from the Updater (and failing if it does not exists there;
even though that should never happen)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 17:23:47 +01:00
875d375d7a
api2/types: ArchiveEntry: fix inverse 'leaf' value
...
a 'leaf' node is every file *except* directories, so we have
to reverse the logtic here
this fixes the pxar.didx browser in the web ui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 17:23:47 +01:00
4961404c7c
api2/config/tape_backup_job: enable update api call
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 13:17:49 +01:00
043018cfbe
doc: fix wrong api method description
2021-02-22 12:10:34 +01:00
7f07991035
docs: improve MEDIA_SET_NAMING_TEMPLATE_SCHEMA description
2021-02-22 09:50:16 +01:00
18ce01caff
tape: use correct schema for changer path (SCSI_CHANGER_PATH_SCHEMA)
2021-02-22 09:43:13 +01:00
5bc8e80a99
cleanup: cleanup use clause (avoid *)
2021-02-22 09:33:29 +01:00
6252df4c18
docs: fix EXPORT_SLOT_LIST_SCHEMA description
2021-02-22 09:05:07 +01:00
86d9f4e733
tape: extend MediaChange trait to return MtxStatus
2021-02-20 10:23:16 +01:00
3f16f1b006
tape: update changer status inside ScsiMediaChange implementation
2021-02-20 09:56:27 +01:00
cbd9899389
tape: update changer status cache after load/unload
2021-02-20 09:06:17 +01:00
4188fd59a0
tape: cache changer state
2021-02-19 16:48:19 +01:00
926d05ef0b
api2/tape/{backup, restore}, proxmox-tape: set device state
...
set the drive device state everywhere we lock it, so that we
know what it currently does
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:16:30 +01:00
8be48ddfc7
api2/tape/changer: add drive state to changer status output
...
if we can find the drive in the config and it has a state
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:15:15 +01:00
41e66bfaf6
api2/tape/drive: wrap some api calls in run_drive_blocking_task
...
those calls could also block, so we have to run them in a blocking
tokio task, as to not block the current thread
nice side effect is that we now also update the state for that
drive in those instances
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:14:08 +01:00
47a7241410
api2/tape/drive: use run_drive_blocking_task where possible
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:13:36 +01:00
54c77b3d62
api2/tape/drive: add wrapper for tokio::task::spawn_blocking
...
similar to the worker wrapper, lock, write status, run code, unset status
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:13:21 +01:00
a1c5575308
api2/tape/drive: use 'run_drive_worker' where possible
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:11:04 +01:00
a44c934b5d
api2/tape/drive: add run_drive_worker wrapper
...
a wrapper for locking, starting the worker and writing the state
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:08:27 +01:00
1917ea3ce1
cleanup: use serde(flatten) for VerificationJobStatus, improve code reuse
2021-02-19 09:50:25 +01:00
70842b9ef2
cleanup: use serde(flatten) for SyncJobStatus, improve code reuse
2021-02-19 09:36:39 +01:00
7690a8e7bd
api: list tape backup jobs with scheduling status
2021-02-19 09:02:13 +01:00
1689296d46
api: add new type JobScheduleStatus
2021-02-19 08:23:41 +01:00
9bbd83b1f2
tape: media_list API - allow to update online status for a single changer
2021-02-18 10:59:33 +01:00
d0647e5a02
tape: use worker tasks for media load/unload
2021-02-18 09:04:51 +01:00
2e21948156
api2/admin/datastore: refactor create_zip into pxar/extract
...
we will reuse that code in the client, so we need to move it to
where we can access it from the client
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[clippy fixes]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
5279ee745f
api2/admin/datastore: accept "/" as path for root
...
makes more sense than sending "root'"
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
227501c063
api2/admin/datastore: refactor list_dir_content in catalog_reader
...
we will reuse that later in the client, so we need it somewhere
we can use from there
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
[add strongly typed ArchiveEntry and put api code into helpers.rs]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
3cdd1a3424
tape: erase_media - automatically load media (with tape libraries).
2021-02-17 06:55:53 +01:00
be61c56c21
api2/tape/drive: add optional label-text to erase
...
if given, erases the tape only iff the inserted tape contains that label
used to safeguard tape erasing from ui for standalone drives
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
159100b944
tape: ui - avoid changer status query in backup content panel
2021-02-15 12:17:36 +01:00
41a8db3576
tape: implement API to run tape backup jobs
2021-02-15 11:09:30 +01:00
fe291ab794
cleanup: tape_backup_job.rs - do not use #[allow(non_camel_case_types)]
2021-02-15 10:41:04 +01:00
8513626b9f
proxmox-backup-proxy: schedule tape backup jobs
2021-02-15 08:30:42 +01:00
42200c405a
fix api sort order
2021-02-15 06:35:45 +01:00
be327dbccd
tape: add tape backup job configuration
2021-02-14 10:35:51 +01:00
7b1bf4c098
tape: erase_media - automatically erase catalog and inventory
2021-02-12 11:30:34 +01:00
7c9fb570cc
api2/types/tape/media_location: fix deserialization of MediaLocation::Vault
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:23:54 +01:00
56d22c66c0
api2/tape/media: add 'move' api call
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:23:27 +01:00
8f02db04f9
docs: use new type_text option for calendar events
2021-02-11 11:51:22 +01:00
7f9d8438ab
tfa: derive WebauthnConfigUpdater via api macro
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-10 10:47:47 +01:00
afcf8b3ed6
api2/tape/drive: add missing locks for some drive actions
...
else a backup/restore can easily be disrupted
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-09 08:46:03 +01:00
42c0f784e2
key: add show-master-pubkey command
...
and print public key when generating/importing..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:55:21 +01:00
25aa55b5f5
tape: correctly lock the tape device
2021-02-05 10:50:21 +01:00
cdf39e62b3
tape: MediaPool - replace use_offline_media with changer_name
...
This way, we can improve location_is_available, because we only
consider media from that changer as available.
2021-02-04 10:15:18 +01:00
f47e035721
tape: cleanup - move tape file readers/writers into src/tape/file_formats folder
2021-02-04 07:59:37 +01:00
a80d72f999
tape: allow to abort restore tasks
2021-02-04 07:05:43 +01:00
8de9a9917f
cleanup: use task_log macro
2021-02-04 06:55:18 +01:00
50a4797fb1
api2/types/tape/media: add media_set_ctime to MediaContentEntry
...
to be able to better sort in the ui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:45:54 +01:00
d0f11b66f7
thape: add read_tapedev_options, display driver options with status command
2021-02-02 06:40:40 +01:00
6248e51797
change half-ticket time range from -120..240 to -60..600
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-01 15:13:11 +01:00
9883b54cba
tape: remove drive from pool config
2021-02-01 09:14:28 +01:00
b5b99a52cd
tape: API type cleanup, use serde flatten to derive types
2021-01-30 09:36:54 +01:00
9586ce2f46
tape: move scan_drives API code to correct file
2021-01-30 08:03:17 +01:00
c297835b01
tape: proxmox-tape - use API instead of direct functions calls
2021-01-29 11:49:11 +01:00
5243df4712
tape: proxmox-tape - use API instead of direct functions calls
2021-01-29 10:50:11 +01:00
1f2c4713ef
tape: improve backup task abort behaviour
2021-01-29 09:23:39 +01:00
271764deb9
tape: make it possible to abort tape backup tasks (check_abort)
...
Also use task_log makro instead of worker.log.
2021-01-29 09:07:55 +01:00
bdb6e6b83f
api2/reader: asyncify the reader worker task
...
this way, the code is much more readable
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 06:59:25 +01:00
41dacd5d3d
tape: use worker task for eject-media api
2021-01-28 16:49:08 +01:00
Dietmar Maurer
Dominik Csapak
Thomas Lamprecht
Wolfgang Bumiller
Fabian Grünbichler