tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,989 Commits 2 Branches 54 Tags 14 MiB
d4877712f8
Commit Graph

228 Commits

Author SHA1 Message Date
Dietmar Maurer
28570d19a6 tape restore: avoid multiple stat calls for same chunk 2021-04-16 13:17:17 +02:00
Dietmar Maurer
1369bcdbba tape restore: verify if all chunks exist 2021-04-16 12:20:44 +02:00
Dietmar Maurer
5e4d81e957 tape restore: simplify log (list datastores on single line) 2021-04-16 11:35:05 +02:00
Dietmar Maurer
0f4721f305 tape restore: fix datastore locking 2021-04-16 09:09:05 +02:00
Dietmar Maurer
80acdd71fa tape: do not try to backup unfinished backups 2021-04-15 10:24:14 +02:00
Dominik Csapak
1dd1c9eb5c api2/tape/restore: restore_chunk_archive: only ignore tape related errors 
when we get an error from the tape, we possibly want to ignore it,
i.e. when the file was incomplete, but we still want to error
out if the error came from e.g, the datastore, so we have to move
the error checking code to the 'next_chunk' call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-04-14 10:38:26 +02:00
Dietmar Maurer
90461b76fb TapeRead: add skip_data() 2021-04-13 13:32:45 +02:00
Dietmar Maurer
318b310638 tape: improve EOT error handling 2021-04-12 13:27:34 +02:00
Dietmar Maurer
109ccd300f cleanup: move tape SCSI code to src/tape/drive/lto/sg_tape/ 2021-04-09 11:34:45 +02:00
Dietmar Maurer
e29f456efc tape: implement format/erase 2021-04-08 07:28:30 +02:00
Dietmar Maurer
a79082a0dd tape: implement LTO userspace driver 2021-04-08 07:28:30 +02:00
Dietmar Maurer
0db5712493 tape: introduce trait BlockRead 2021-04-08 07:28:30 +02:00
Dietmar Maurer
c553407e98 tape: add --scan option for catalog restore 2021-03-25 13:08:34 +01:00
Dietmar Maurer
4830de408b tape: avoid writing catalogs for empty backup tasks 2021-03-25 12:50:40 +01:00
Dominik Csapak
4c4e5c2b1e api2/tape/restore: enable restore mapping of datastores 
by changing the 'store' parameter of the restore api call to a
list of mappings (or a single default datastore)

for example giving:
a=b,c=d,e

would restore
datastore 'a' from tape to local datastore 'b'
datastore 'c' from tape to local datastore 'e'
all other datastores to 'e'

this way, only a single datastore can also be restored, by only
giving a single mapping, e.g. 'a=b'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-25 07:46:12 +01:00
Dietmar Maurer
870681013a tape: fix catalog restore 
We need to rewind the tape if fast_catalog_restore() fail ...
 2021-03-24 10:09:23 +01:00
Dietmar Maurer
8b1289f3e4 tape: skip catalog archives in restore 2021-03-24 09:33:39 +01:00
Dietmar Maurer
074503f288 tape: implement fast catalog restore 2021-03-24 08:40:34 +01:00
Dietmar Maurer
30316192b3 tape: improve locking (lock media-sets) 
- new helper: lock_media_set()

- MediaPool: lock media set

- Expose Inventory::new() to avoid double loading

- do not lock pool on restore (only lock media-set)

- change pool lock name to ".pool-{name}"
 2021-03-24 06:32:59 +01:00
Dominik Csapak
54fcb7f5d8 api2/tape/backup: wait indefinitely for lock in scheduled backup jobs 
so that a user can schedule multiple backup jobs onto a single
media pool without having to consider timing them apart

this makes sense since we can backup multiple datastores onto
the same media-set but can only specify one datastore per backup job

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-19 09:04:32 +01:00
Dominik Csapak
4abd4dbe38 api2/tape/backup: include a summary on notification e-mails 
for now only contains the list of included snapshots (if any),
as well as the backup duration

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-19 09:03:52 +01:00
Dietmar Maurer
32b75d36a8 tape: backup media catalogs 2021-03-19 06:58:46 +01:00
Dietmar Maurer
237314ad0d tape: improve catalog consistency checks 
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.

Note: this changes the catalog format again.
 2021-03-18 08:43:55 +01:00
Dietmar Maurer
54722acada tape: store datastore name in tape archives and media catalog 
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
 2021-03-17 11:08:51 +01:00
Dominik Csapak
e36135031d ui: tape/Restore: let the user choose an owner 
so that the tape backup can be restored as any user, given
the current logged in user has the correct permission.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-16 08:55:42 +01:00
Dietmar Maurer
5c4755ad08 tape: speedup backup by doing read/write in parallel 2021-03-16 08:51:49 +01:00
Dietmar Maurer
2c10410b0d tape: improve backup task log 2021-03-11 08:43:13 +01:00
Thomas Lamprecht
d1d74c4367 typo fixes all over the place 
found and semi-manually replaced by using:
 codespell -L mut -L crate -i 3 -w

Mostly in comments, but also email notification and two occurrences
of misspelled  'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-03-10 16:39:57 +01:00
Dietmar Maurer
6396bace3d tape: improve backup task log (show percentage) 2021-03-10 10:59:13 +01:00
Dominik Csapak
2a06e08618 api2/tape/backup: continue on vanishing snapshots 
when we do a prune during a tape backup, do not cancel the tape backup,
but continue with a warning

the task still fails and prompts the user to check the log

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-09 10:20:54 +01:00
Dietmar Maurer
af76234112 tape: improve MediaPool allocation by sorting tapes by ctime and label_text 2021-03-09 08:33:21 +01:00
Dominik Csapak
8703a68a31 tape/backup: add success/error notifications 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-08 14:29:06 +01:00
Dietmar Maurer
d95c74c6e7 tape: test tape alert flags after drive clean 2021-03-08 09:08:55 +01:00
Dominik Csapak
c9793d47f9 api2/tape: add notify_user to backup(-jobs) and restore api calls 
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-05 17:20:37 +01:00
Dietmar Maurer
b4975d3102 tape: finish api permission checks 2021-03-05 11:49:07 +01:00
Dietmar Maurer
90e16be3ae tape: improve PoolWriter logging 
Log reason why we allocate a new media set.
 2021-03-05 09:59:38 +01:00
Dietmar Maurer
ab77d660cc tape: improve media status in list_media 2021-03-04 10:17:10 +01:00
Dietmar Maurer
8a76e71129 tape: impl access permission for media content list 2021-03-03 13:16:15 +01:00
Dietmar Maurer
396fd747a6 tape: impl access permissions for tape jobs 2021-03-03 12:48:06 +01:00
Dietmar Maurer
8cd63df0dc tape: impl api permissions for drive/changer/pool configuration 2021-03-03 12:10:00 +01:00
Dominik Csapak
8bf5769382 api2/tape/drive: add state to DriveListEntry 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-02 12:51:19 +01:00
Dietmar Maurer
1d14c31658 Revert "tape: also abort backup/restore on server shutdown" 
This reverts commit 9bd81bb384.

Turns out this is not really a good idea.
 2021-03-02 08:00:10 +01:00
Dietmar Maurer
9bd81bb384 tape: also abort backup/restore on server shutdown 2021-03-02 07:19:31 +01:00
Dietmar Maurer
1552d9699c tape: add 'Restore Key' button 2021-03-01 11:11:38 +01:00
Dietmar Maurer
16f9ea6708 tape: restore - do not check if pool is configured 2021-03-01 09:43:23 +01:00
Dietmar Maurer
955f4aefcd tape: show media inventory/content, even if there is no pool configuration 2021-03-01 09:21:11 +01:00
Dietmar Maurer
08ec39be0c tape: add api to set media status 2021-02-26 10:44:07 +01:00
Dietmar Maurer
5830e5620d tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig 2021-02-25 09:44:31 +01:00
Dietmar Maurer
0dadf66dc7 tape: implement backup using latest-only flag 2021-02-24 13:51:53 +01:00
Dietmar Maurer
86d9f4e733 tape: extend MediaChange trait to return MtxStatus 2021-02-20 10:23:16 +01:00
Dietmar Maurer
3f16f1b006 tape: update changer status inside ScsiMediaChange implementation 2021-02-20 09:56:27 +01:00
Dietmar Maurer
cbd9899389 tape: update changer status cache after load/unload 2021-02-20 09:06:17 +01:00
Dietmar Maurer
4188fd59a0 tape: cache changer state 2021-02-19 16:48:19 +01:00
Dominik Csapak
926d05ef0b api2/tape/{backup, restore}, proxmox-tape: set device state 
set the drive device state everywhere we lock it, so that we
know what it currently does

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:16:30 +01:00
Dominik Csapak
8be48ddfc7 api2/tape/changer: add drive state to changer status output 
if we can find the drive in the config and it has a state

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:15:15 +01:00
Dominik Csapak
41e66bfaf6 api2/tape/drive: wrap some api calls in run_drive_blocking_task 
those calls could also block, so we have to run them in a blocking
tokio task, as to not block the current thread

nice side effect is that we now also update the state for that
drive in those instances

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:14:08 +01:00
Dominik Csapak
47a7241410 api2/tape/drive: use run_drive_blocking_task where possible 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:13:36 +01:00
Dominik Csapak
54c77b3d62 api2/tape/drive: add wrapper for tokio::task::spawn_blocking 
similar to the worker wrapper, lock, write status, run code, unset status

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:13:21 +01:00
Dominik Csapak
a1c5575308 api2/tape/drive: use 'run_drive_worker' where possible 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:11:04 +01:00
Dominik Csapak
a44c934b5d api2/tape/drive: add run_drive_worker wrapper 
a wrapper for locking, starting the worker and writing the state

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:08:27 +01:00
Dietmar Maurer
7690a8e7bd api: list tape backup jobs with scheduling status 2021-02-19 09:02:13 +01:00
Dietmar Maurer
9bbd83b1f2 tape: media_list API - allow to update online status for a single changer 2021-02-18 10:59:33 +01:00
Dietmar Maurer
d0647e5a02 tape: use worker tasks for media load/unload 2021-02-18 09:04:51 +01:00
Dietmar Maurer
3cdd1a3424 tape: erase_media - automatically load media (with tape libraries). 2021-02-17 06:55:53 +01:00
Dominik Csapak
be61c56c21 api2/tape/drive: add optional label-text to erase 
if given, erases the tape only iff the inserted tape contains that label
used to safeguard tape erasing from ui for standalone drives

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-17 06:55:53 +01:00
Dietmar Maurer
159100b944 tape: ui - avoid changer status query in backup content panel 2021-02-15 12:17:36 +01:00
Dietmar Maurer
41a8db3576 tape: implement API to run tape backup jobs 2021-02-15 11:09:30 +01:00
Dietmar Maurer
8513626b9f proxmox-backup-proxy: schedule tape backup jobs 2021-02-15 08:30:42 +01:00
Dietmar Maurer
7b1bf4c098 tape: erase_media - automatically erase catalog and inventory 2021-02-12 11:30:34 +01:00
Dominik Csapak
56d22c66c0 api2/tape/media: add 'move' api call 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-11 14:23:27 +01:00
Dominik Csapak
afcf8b3ed6 api2/tape/drive: add missing locks for some drive actions 
else a backup/restore can easily be disrupted

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-09 08:46:03 +01:00
Dietmar Maurer
25aa55b5f5 tape: correctly lock the tape device 2021-02-05 10:50:21 +01:00
Dietmar Maurer
cdf39e62b3 tape: MediaPool - replace use_offline_media with changer_name 
This way, we can improve location_is_available, because we only
consider media from that changer as available.
 2021-02-04 10:15:18 +01:00
Dietmar Maurer
f47e035721 tape: cleanup - move tape file readers/writers into src/tape/file_formats folder 2021-02-04 07:59:37 +01:00
Dietmar Maurer
a80d72f999 tape: allow to abort restore tasks 2021-02-04 07:05:43 +01:00
Dietmar Maurer
8de9a9917f cleanup: use task_log macro 2021-02-04 06:55:18 +01:00
Dominik Csapak
50a4797fb1 api2/types/tape/media: add media_set_ctime to MediaContentEntry 
to be able to better sort in the ui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-02 14:45:54 +01:00
Dietmar Maurer
9883b54cba tape: remove drive from pool config 2021-02-01 09:14:28 +01:00
Dietmar Maurer
b5b99a52cd tape: API type cleanup, use serde flatten to derive types 2021-01-30 09:36:54 +01:00
Dietmar Maurer
9586ce2f46 tape: move scan_drives API code to correct file 2021-01-30 08:03:17 +01:00
Dietmar Maurer
c297835b01 tape: proxmox-tape - use API instead of direct functions calls 2021-01-29 11:49:11 +01:00
Dietmar Maurer
5243df4712 tape: proxmox-tape - use API instead of direct functions calls 2021-01-29 10:50:11 +01:00
Dietmar Maurer
1f2c4713ef tape: improve backup task abort behaviour 2021-01-29 09:23:39 +01:00
Dietmar Maurer
271764deb9 tape: make it possible to abort tape backup tasks (check_abort) 
Also use task_log makro instead of worker.log.
 2021-01-29 09:07:55 +01:00
Dietmar Maurer
41dacd5d3d tape: use worker task for eject-media api 2021-01-28 16:49:08 +01:00
Dietmar Maurer
eb1dfb02b5 tape: proxmox-tape - use api for erase-media and rewind 2021-01-28 16:36:10 +01:00
Dominik Csapak
cef4654ff4 api2/tape/drive: change methods of some api calls from put to get 
makes more sense to have retrieving api calls as get instead of put

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:02:52 +01:00
Dominik Csapak
f45dceeb73 api2/tape/drive: add load_media as api call 
code was already there, just add it as api call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:02:13 +01:00
Dominik Csapak
18262a88c9 api2/tape/changer: add changer filter to list_drives api call 
so that an api user can get the drives belonging to a changer
without having to parse the config listing themselves

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:01:41 +01:00
Dietmar Maurer
d737adc6be tape: rename changer_drive_id to changer_drivenum 2021-01-28 11:29:59 +01:00
Dominik Csapak
5fdaecf6f4 api2/tape/drive: reorganize drive api 
similar to the changers, create a listing at /tape/drive and put
the specific api calls below that

move the scan api call up one level

remove the status info from the config listing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 11:15:17 +01:00
Dominik Csapak
d8792b88ef api2/types/tape/drive: add changer_drivenum 
so that an api user can see which drive belongs to which drivenum of a changer
for ones with multiple drives

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 11:14:28 +01:00
Dietmar Maurer
887f1cb90c cleanup: move scan changers API implementation 2021-01-27 09:58:16 +01:00
Dominik Csapak
740dc9d1d4 api2/tape/changer: reorganize api 
add a changer listing here (copied from api2/config/changer)
and put the status and transfer api calls below that

puts the changer scan into the top level tape api
and removes the (now redundant) info from the config api path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-26 12:47:34 +01:00
Dietmar Maurer
697c41c584 tape: add/use rust scsi changer implementation using libsgutil2 2021-01-25 13:14:07 +01:00
Dietmar Maurer
bf78f70885 improve code docs in api2 
Note: API methos should be declared pub, so that they show up in the generated docu.
 2021-01-22 15:57:42 +01:00
Dietmar Maurer
5f34d69bcc tape: add volume-statistics api/command 2021-01-22 08:45:35 +01:00
Dietmar Maurer
8e6459a818 tape: set encryption key on restore 2021-01-22 07:26:42 +01:00
Dietmar Maurer
645a044bf6 tape: further hierarchy improvements 2021-01-21 17:25:32 +01:00
Dietmar Maurer
37796ff73f tape: change code hierarchy to improve docs 2021-01-21 17:12:01 +01:00