tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,161 Commits 2 Branches 54 Tags 14 MiB
2d5d264f99
Commit Graph

269 Commits

Author SHA1 Message Date
Dietmar Maurer
166a48f903 tape: cleanup - split PoolWriter into several files 2021-03-19 08:19:13 +01:00
Dietmar Maurer
82775c4764 tape: make sure we only commit/write valid catalogs 2021-03-19 07:50:32 +01:00
Dietmar Maurer
88bc9635aa tape: store media_uuid in PoolWriterState 
This is mainly a cleanup, avoiding to access the catalog_set to get the uuid.
 2021-03-19 07:33:59 +01:00
Dietmar Maurer
1037f2bc2d tape: cleanup - rename CatalogBuilder to CatalogSet 2021-03-19 07:22:54 +01:00
Dietmar Maurer
32b75d36a8 tape: backup media catalogs 2021-03-19 06:58:46 +01:00
Dietmar Maurer
237314ad0d tape: improve catalog consistency checks 
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.

Note: this changes the catalog format again.
 2021-03-18 08:43:55 +01:00
Dietmar Maurer
6ee3035523 tape: define magic number for catalog archives 2021-03-17 13:35:23 +01:00
Dietmar Maurer
b627ebbf40 tape: improve catalog parser 2021-03-17 11:29:23 +01:00
Dietmar Maurer
54722acada tape: store datastore name in tape archives and media catalog 
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
 2021-03-17 11:08:51 +01:00
Dietmar Maurer
0e2bf3aa1d SnapshotReader: add self.datastore_name() helper 2021-03-17 10:16:34 +01:00
Dietmar Maurer
365126efa9 tape: PoolWriter - remove unnecessary move_to_eom 2021-03-17 10:16:34 +01:00
Dietmar Maurer
776dabfb2e tape: use MB/s for backup speed (to match drive speed specification) 2021-03-16 08:51:49 +01:00
Dietmar Maurer
5c4755ad08 tape: speedup backup by doing read/write in parallel 2021-03-16 08:51:49 +01:00
Dietmar Maurer
e8913fea12 tape: write_chunk_archive - do not consume partially written chunk at EOT 
So that it is re-written to the next tape.
 2021-03-12 07:14:50 +01:00
Dominik Csapak
2892624783 tape/send_load_media_email: move to server/email_notifications 
and reuse 'send_job_status_mail' there so that we get consistent
formatted mails from pbs (e.g. html part and author)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-11 09:56:12 +01:00
Dietmar Maurer
2c10410b0d tape: improve backup task log 2021-03-11 08:43:13 +01:00
Thomas Lamprecht
d1d74c4367 typo fixes all over the place 
found and semi-manually replaced by using:
 codespell -L mut -L crate -i 3 -w

Mostly in comments, but also email notification and two occurrences
of misspelled  'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-03-10 16:39:57 +01:00
Dietmar Maurer
5ef4c7bcd3 tape: fix scsi volume_statistics and cartridge_memory for quantum drives 2021-03-10 14:13:48 +01:00
Dietmar Maurer
713a128adf tape: improve backup task log format 2021-03-10 09:54:51 +01:00
Dietmar Maurer
affc224aca tape: read_tape_mam - pass correct allocation len 2021-03-10 09:24:38 +01:00
Dietmar Maurer
6f82d32977 tape: cleanup - remove wrong inline comment 2021-03-10 08:11:51 +01:00
Dietmar Maurer
1057b1f5a5 tape: lock artificial "__UNASSIGNED__" pool to avoid races 2021-03-09 10:00:26 +01:00
Dietmar Maurer
af76234112 tape: improve MediaPool allocation by sorting tapes by ctime and label_text 2021-03-09 08:33:21 +01:00
Dominik Csapak
1d7fcbece8 tape/linux_tape: remove debug prints 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-08 14:29:19 +01:00
Dietmar Maurer
9ce2481a69 tape: clean - fix if cleaning tape is already loaded 2021-03-08 09:08:55 +01:00
Dominik Csapak
c9793d47f9 api2/tape: add notify_user to backup(-jobs) and restore api calls 
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-05 17:20:37 +01:00
Dietmar Maurer
90e16be3ae tape: improve PoolWriter logging 
Log reason why we allocate a new media set.
 2021-03-05 09:59:38 +01:00
Dietmar Maurer
cd5d6103ea tape: fix media_expire_time 2021-03-04 16:58:24 +01:00
Dietmar Maurer
1bed3aedc8 tape: fix wrong media set expire time 2021-03-04 12:38:28 +01:00
Dietmar Maurer
ab77d660cc tape: improve media status in list_media 2021-03-04 10:17:10 +01:00
Dominik Csapak
78593b5b5c tape/drive: improve error and email handling for requesting a tape load 
Try once first to load the correct tape before sending an email to
insert the correct one. This way, the admin does not get a mail
if the correct tape is already inserted.

Also include the error we got that prompted the email to insert the
tape. This means that if the admin gets prompted to insert e.g.
"FOO" but inserts "BAR", he'll get an email that the wrong
tape is inserted.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-03 16:09:39 +01:00
Dietmar Maurer
1d14c31658 Revert "tape: also abort backup/restore on server shutdown" 
This reverts commit 9bd81bb384.

Turns out this is not really a good idea.
 2021-03-02 08:00:10 +01:00
Dietmar Maurer
9bd81bb384 tape: also abort backup/restore on server shutdown 2021-03-02 07:19:31 +01:00
Dietmar Maurer
d984a9acf0 tape: add Inventory.media_list() helper 2021-03-01 09:30:45 +01:00
Dietmar Maurer
08ec39be0c tape: add api to set media status 2021-02-26 10:44:07 +01:00
Dietmar Maurer
25350f3370 tape: set media status if we detect damaged medium at start of backup 2021-02-26 09:00:50 +01:00
Dietmar Maurer
3fbf2311e4 tape: improve backup task logging 2021-02-23 12:58:44 +01:00
Dietmar Maurer
86d9f4e733 tape: extend MediaChange trait to return MtxStatus 2021-02-20 10:23:16 +01:00
Dietmar Maurer
3f16f1b006 tape: update changer status inside ScsiMediaChange implementation 2021-02-20 09:56:27 +01:00
Dietmar Maurer
cd44fb8d84 tape: create tmp dirs early at server startup 2021-02-19 18:10:02 +01:00
Dietmar Maurer
4188fd59a0 tape: cache changer state 2021-02-19 16:48:19 +01:00
Dominik Csapak
546d2653ee tape/drive: add get/set status functions 
simply writes into/reads from a file in /run, we will use this
for writing the upid (or potential other states) per drive

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:05:09 +01:00
Dominik Csapak
33c06b3388 tape/drive: add test_device_path_lock 
this makes it possible to detect if the drive was locked in a
non-blocking manner

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:03:30 +01:00
Dietmar Maurer
9bbd83b1f2 tape: media_list API - allow to update online status for a single changer 2021-02-18 10:59:33 +01:00
Dietmar Maurer
8b910bb6bc tape: add regression test for compute_media_state 2021-02-17 06:57:29 +01:00
Dietmar Maurer
dbaef7d04d tape: add regression test for alloc_writable_media 2021-02-17 06:55:49 +01:00
Dietmar Maurer
ced7838de4 tape: fix path for test files 2021-02-06 09:32:50 +01:00
Dominik Csapak
f197c286d5 tape/inventory: fix missing parameter in error message 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-05 16:12:18 +01:00
Dietmar Maurer
25aa55b5f5 tape: correctly lock the tape device 2021-02-05 10:50:21 +01:00
Dietmar Maurer
cdf39e62b3 tape: MediaPool - replace use_offline_media with changer_name 
This way, we can improve location_is_available, because we only
consider media from that changer as available.
 2021-02-04 10:15:18 +01:00
Dietmar Maurer
b81e37f6ab tape: improve code reuse 2021-02-04 09:39:16 +01:00
Dietmar Maurer
ddebbb52fd tape: fix tests for BlockedReader 2021-02-04 08:54:54 +01:00
Dietmar Maurer
983e929e25 tape: add multi volume reader/writer implementations 
We currently do not use it. Added anaways, to show the possibility.
 2021-02-04 08:36:35 +01:00
Dietmar Maurer
f47e035721 tape: cleanup - move tape file readers/writers into src/tape/file_formats folder 2021-02-04 07:59:37 +01:00
Dietmar Maurer
7d2c156eb1 tape: BlockedReader - always consume EOF 2021-02-03 13:25:59 +01:00
Dietmar Maurer
b22e8c3632 tape: add media pool regression tests 2021-02-03 10:23:04 +01:00
Dietmar Maurer
7929292618 tape: add regresion test for media state 2021-02-03 09:34:31 +01:00
Dietmar Maurer
cc2a0b12f8 test: define tape tests as submodule 2021-02-02 14:38:15 +01:00
Dietmar Maurer
988e8de122 tape: set correct ownership on lock file 2021-02-02 14:18:57 +01:00
Dietmar Maurer
2f8809c6bc test: src/tape/inventory.rs - avoid chown when running tests 2021-02-02 13:43:16 +01:00
Dietmar Maurer
8937c65951 tape: add pmt stoptions/stsethoptions/stclearoptions 2021-02-02 08:58:02 +01:00
Dietmar Maurer
d0f11b66f7 thape: add read_tapedev_options, display driver options with status command 2021-02-02 06:40:40 +01:00
Dietmar Maurer
ca953d831f cleanup: remove MT_ST_ prefix from SetDrvBufferOptions 2021-02-01 17:54:53 +01:00
Dietmar Maurer
b8cbe5d65b tape: fix tape alert flag decoding 2021-02-01 12:18:55 +01:00
Dietmar Maurer
83b8949a98 tape: add pmt weof 2021-01-31 17:33:07 +01:00
Dietmar Maurer
1f31d06f48 tape: add pmt bsf 2021-01-31 17:00:15 +01:00
Dietmar Maurer
2f2e83c890 tape: add pmt fsf 2021-01-31 16:54:16 +01:00
Dietmar Maurer
1e041082bb tape: add pmt command line tool 
Experimental, not installed by now.
 2021-01-31 16:19:53 +01:00
Dietmar Maurer
b5b99a52cd tape: API type cleanup, use serde flatten to derive types 2021-01-30 09:36:54 +01:00
Dietmar Maurer
271764deb9 tape: make it possible to abort tape backup tasks (check_abort) 
Also use task_log makro instead of worker.log.
 2021-01-29 09:07:55 +01:00
Dietmar Maurer
f2ca03d7d0 cleanup: avoid compiler warning 2021-01-28 15:32:21 +01:00
Dominik Csapak
00ac86c31b tape/drive/linux_tape: fix and refactor usage of sg-tape-cmd 
when executing this code as non-root, we use sg-tape-cmd (a setuid binary)
to execute various ioctls on the tape device

we give the command the open tape device fd as stdin, but did not
dup it, so the std::process:Stdio handle closed it on drop,
which let subsequent operation on that file fail (since it was closed)

fix it by dup'ing it before giving it to the command, and also refactor
the calling code, so that we do not forget to do this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:24:32 +01:00
Dominik Csapak
4be4736603 tape/changer: refactor marking of import/export slots from config 
we did this for 'mtx', but missed it for the sg_pt_changer code
refactor it into the MtxStatus strut, and call it from both
code paths

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:10:55 +01:00
Dominik Csapak
2da7aca8e8 tape/changer: add vendor/model to DriveStatus 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:10:31 +01:00
Dietmar Maurer
d737adc6be tape: rename changer_drive_id to changer_drivenum 2021-01-28 11:29:59 +01:00
Dietmar Maurer
16b4d78400 tape: rename retry_command to execute_scsi_command, make retry a flag 2021-01-27 09:34:24 +01:00
Dietmar Maurer
ec8d9c6b80 tape: repeat changer scsi command until successful 2021-01-27 08:59:10 +01:00
Dominik Csapak
e1740f3f01 tape/changer/mtx: add mtx parser test 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-26 12:51:26 +01:00
Dominik Csapak
bbf01b644c tape: fix typos 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-26 12:39:54 +01:00
Dietmar Maurer
c3747b93c8 tape: add new command line tool "pmtx" 
Also improve sgutil2 error reporting
 2021-01-26 11:57:15 +01:00
Dietmar Maurer
2b96a43879 tape: cleanup - use ScsiMediaChange trait instead of mtx_status() 2021-01-25 13:25:22 +01:00
Dietmar Maurer
697c41c584 tape: add/use rust scsi changer implementation using libsgutil2 2021-01-25 13:14:07 +01:00
Dietmar Maurer
0107fd323c cleanup: avoid compiler warnings 2021-01-23 17:34:26 +01:00
Dietmar Maurer
979dccc7ec tape: avoid error when clearing encryption key 
Simply ignore clear request when sg_spin_data_encryption_caps fails.
Assume those are tapes without hardware encryption support.
 2021-01-23 10:20:43 +01:00
Dietmar Maurer
45212a8c78 fix mtx parser bug: s/strip_suffix/strip_prefix/ 2021-01-22 11:00:56 +01:00
Dietmar Maurer
5f34d69bcc tape: add volume-statistics api/command 2021-01-22 08:45:35 +01:00
Dietmar Maurer
337ff5a3cc tape: add estimated medium wearout to status 2021-01-22 08:06:25 +01:00
Dietmar Maurer
9372c0787d renamed src/tape/sgutils2.rs -> src/tools/sgutils2.rs 2021-01-21 17:57:17 +01:00
Dietmar Maurer
83fb2da53e tape: move MediaCatalog magic number into struct (doc cleanup) 2021-01-21 17:48:07 +01:00
Dietmar Maurer
645a044bf6 tape: further hierarchy improvements 2021-01-21 17:25:32 +01:00
Dietmar Maurer
37796ff73f tape: change code hierarchy to improve docs 2021-01-21 17:12:01 +01:00
Dietmar Maurer
e1fdcb1678 tape: do not export/doc low level libsgutils2 bindings 2021-01-21 16:38:24 +01:00
Dietmar Maurer
edda5039d4 tape: improve code docs 2021-01-21 13:19:07 +01:00
Dietmar Maurer
d543587d34 Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup 2021-01-21 10:56:52 +01:00
Dietmar Maurer
2b191385ea tape: use specialized encryption key per media-set 2021-01-21 10:31:49 +01:00
Fabian Grünbichler
1d928b25fe clippy: remove some unnecessary reference taking 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
f2f81791d1 clippy: fix for_kv_map 
and allow it in the one case where the entry loop is intended, but the
code is not yet implemented fully.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
382f10a0cc clippy: fix/allow needless_range_loop 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
09faa9ee95 clippy: pass &str/&[..] instead of &String/&Vec 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
ccec086e25 clippy: remove unnecessary &mut 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00