tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,731 Commits 2 Branches 54 Tags 14 MiB
608806e884
Commit Graph

326 Commits

Author SHA1 Message Date
Dominik Csapak
e5950360ca tape/drive: improve tape device locking behaviour 
by implementing a custom error type that is either 'TimeOut' or
'Other'.

In the api, check in the worker loop for exactly 'TimeOut' errors and continue only
then. All other errors lead to a aborted task.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-06-02 17:08:00 +02:00
Dietmar Maurer
0bca966ec5 fix typo: s/dies/does/ 2021-05-31 11:01:15 +02:00
Dominik Csapak
84737fb33f lto/sg_tape/encryption: remove non lto-4 supported byte 
from the SspDataEncryptionCapabilityPage

it seems we do not need it, since the EXTDECC flag is only used for
determining if the drive is capable to be configured via
ADI (Automation/Drive Interface) which we do not use at all.

this makes the call work with LTO-4 again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-31 10:58:38 +02:00
Dominik Csapak
d19010481d tape/test: repair tests after changing 'start_write_session' 
i added a parameter and forgot to adapt the tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-14 10:01:54 +02:00
Dominik Csapak
e953029e8f api2/tape/backup: add 'force-media-set' parameter to manual backup 
so that a user can force a new media set, e.g. if he uses the
allocation policy 'continue', but wants to manually start a new
media-set.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-14 08:58:33 +02:00
Dominik Csapak
13f435caab tape/inventory: add completion helper for tape snapshots 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-11 12:56:55 +02:00
Dietmar Maurer
1bff50afea tape locate_file: fix off by one error 2021-05-11 12:37:04 +02:00
Dominik Csapak
2d5d264f99 tape/pool_writer: do not unwrap on channel send 
if the reader thread is already gone here, we panic here, resulting in
a nondescript error message, so simply ignore/warn in that case and
return gracefully

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-11 09:07:45 +02:00
Dietmar Maurer
3e234af16e tape: improve inline docs for READ POSITION LONG 2021-05-06 11:45:40 +02:00
Dietmar Maurer
bbbf662d20 tape: use LOCATE(16) SCSI command 
Turns out this works on LTO4 and newer.
 2021-05-06 10:51:59 +02:00
Dominik Csapak
2e41dbe828 tape/media_catalog: add helpers to look for snapshot/chunk files 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-06 07:58:03 +02:00
Dominik Csapak
56d36ca439 tape/drive: add 'move_to_file' to TapeDriver trait 
so that we can directly move to a specified file on the tape

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-05-06 07:55:08 +02:00
Dietmar Maurer
5e169f387c tape: add read_medium_configuration_page() to detect WORM media 
And use it inside format_media().
 2021-04-27 11:37:03 +02:00
Dietmar Maurer
73cef112eb tape: remove MediumType struct, which is only valid on IBM drives 
HP drives do not return this information.

Note: This breaks format on WORM media, because we have not way
to detect WOREM media (how?).
 2021-04-27 09:58:27 +02:00
Dietmar Maurer
8e898895cc tape: do not query density_code in SgTape::new() 
Because this can fail with NoSense/MediumChanged and other informational
Sense codes.
 2021-04-23 09:56:44 +02:00
Dietmar Maurer
4be6beab6f tape: format_media - implement special case for WORM media 2021-04-23 08:33:13 +02:00
Dietmar Maurer
a3b4b5b50e tape: define and use MediumType enum 2021-04-23 07:54:42 +02:00
Dietmar Maurer
33b8d7e5e8 tape: use loaded media_type in format_media (instead of drive_density) 
Required to format LTO4 media loaded in LTO5 drive).

Also contains some SCSI code cleanups.
 2021-04-23 07:27:30 +02:00
Dietmar Maurer
b9e0fcbdcd tape: implement report_desnity 2021-04-22 13:54:31 +02:00
Dietmar Maurer
a7188b3a75 tape: fix FORMAT for LTO-4 drives 
FORMAT requires LTO-5 or newer, so we do a rewind/erase if FORMAT fails.
 2021-04-22 11:44:49 +02:00
Dominik Csapak
71e83e1b1f tape/changer/sg_pt_changer: read whole descriptor size for each entry 
Some changer seem to append more data than we expect, but correctly
annotates that size in the subheader.

For each descriptor entry, read as much as the size given in the
subheader (or until the end of the reader), else our position in
the reader is wrong for the next entry, and we will parse
incorrect data.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-04-21 14:07:41 +02:00
Dietmar Maurer
4bf2ab1109 cleanup: remove debug println 2021-04-14 10:39:29 +02:00
Dietmar Maurer
2631e57d20 fix regression tests 2021-04-13 14:02:37 +02:00
Dietmar Maurer
90461b76fb TapeRead: add skip_data() 2021-04-13 13:32:45 +02:00
Dietmar Maurer
6a15cce540 tape: SgTapeReader::read_block - disable reading beyond EOF 2021-04-13 11:46:30 +02:00
Dietmar Maurer
f281b8d3a9 tape: cleanup MediaCatalog on tape reuse 2021-04-13 11:46:30 +02:00
Dietmar Maurer
ce5327badc tape: fix regression tests 2021-04-12 14:08:05 +02:00
Dietmar Maurer
318b310638 tape: improve EOT error handling 2021-04-12 13:27:34 +02:00
Dietmar Maurer
109ccd300f cleanup: move tape SCSI code to src/tape/drive/lto/sg_tape/ 2021-04-09 11:34:45 +02:00
Dietmar Maurer
c560cfddca tape: read_drive_status - ignore media changed sense info 2021-04-09 09:46:19 +02:00
Dietmar Maurer
8204d9b095 tape: avoid unneccessary SCSI request in Drop 2021-04-08 11:26:08 +02:00
Dietmar Maurer
fad95a334a tape: clear encryption key after backup (for security reasons) 2021-04-08 10:37:49 +02:00
Dietmar Maurer
973e985d73 cleanup: remove unused linux tape driver code 2021-04-08 10:15:52 +02:00
Dietmar Maurer
15d1435789 tape: add vendor, product and revision to LtoDriveAndMediaStatus 2021-04-08 08:34:46 +02:00
Dietmar Maurer
80ea23e1b9 tape: pmt - implement options command 2021-04-08 08:34:45 +02:00
Dietmar Maurer
5d6379f8db tape: implement locate_file without LOCATE(10) 2021-04-08 08:34:45 +02:00
Dietmar Maurer
566b946f9b tape: pmt - re-implement lock/unlock command 2021-04-08 07:28:30 +02:00
Dietmar Maurer
7f7459677d tape: pmt - re-implement fsr/bsr 2021-04-08 07:28:30 +02:00
Dietmar Maurer
0892a512bc tape: correctly set/display drive option 2021-04-08 07:28:30 +02:00
Dietmar Maurer
7b11a8098d tape: make sure there is a filemark at the end of the tape 2021-04-08 07:28:30 +02:00
Dietmar Maurer
8b2c6f5dbc tape: make fsf/bsf driver specific 
Because the virtual tape driver behaves different than LTO drives.
 2021-04-08 07:28:30 +02:00
Dietmar Maurer
d26985a600 tape: fix LEOM handling 2021-04-08 07:28:30 +02:00
Dietmar Maurer
e29f456efc tape: implement format/erase 2021-04-08 07:28:30 +02:00
Dietmar Maurer
a79082a0dd tape: implement LTO userspace driver 2021-04-08 07:28:30 +02:00
Dietmar Maurer
1336ae8249 tape: introduce trait BlockWrite 2021-04-08 07:28:30 +02:00
Dietmar Maurer
0db5712493 tape: introduce trait BlockRead 2021-04-08 07:28:30 +02:00
Dietmar Maurer
cf1e117fc7 sgutils2: use enum for ScsiError 
This avoids string allocation when we return SenseInfo.
 2021-03-27 15:57:48 +01:00
Dietmar Maurer
11f5d59396 tape: page-align BlockHeader so that we can use it with SG_IO 2021-03-27 15:36:35 +01:00
Dietmar Maurer
2843ba9017 avoid compiler warning 2021-03-25 12:25:23 +01:00
Dietmar Maurer
589c4dad9e tape: add fsf/bsf to TapeDriver trait 2021-03-25 10:10:16 +01:00
Dietmar Maurer
c046739461 tape: fix MediaPool regression tests 2021-03-24 09:44:30 +01:00
Dietmar Maurer
c6f55139f8 tape: impl. MediaCatalog::parse_catalog_header 
This is just an optimization, avoiding to read the catalog into memory.

We also expose create_temporary_database_file() now (will be
used for catalog restore).
 2021-03-24 06:32:59 +01:00
Dietmar Maurer
20cc25d749 tape: add TapeDriver::move_to_last_file 2021-03-24 06:32:59 +01:00
Dietmar Maurer
30316192b3 tape: improve locking (lock media-sets) 
- new helper: lock_media_set()

- MediaPool: lock media set

- Expose Inventory::new() to avoid double loading

- do not lock pool on restore (only lock media-set)

- change pool lock name to ".pool-{name}"
 2021-03-24 06:32:59 +01:00
Dietmar Maurer
e93263be1e taoe: implement MediaCatalog::destroy_unrelated_catalog() helper 2021-03-22 12:03:11 +01:00
Dietmar Maurer
2ab2ca9c24 tape: add MediaPool::lock_unassigned_media_pool() helper 2021-03-19 10:13:38 +01:00
Dietmar Maurer
eac1beef3c tape: cleanup PoolWriter - factor out common code 2021-03-19 08:56:14 +01:00
Dietmar Maurer
166a48f903 tape: cleanup - split PoolWriter into several files 2021-03-19 08:19:13 +01:00
Dietmar Maurer
82775c4764 tape: make sure we only commit/write valid catalogs 2021-03-19 07:50:32 +01:00
Dietmar Maurer
88bc9635aa tape: store media_uuid in PoolWriterState 
This is mainly a cleanup, avoiding to access the catalog_set to get the uuid.
 2021-03-19 07:33:59 +01:00
Dietmar Maurer
1037f2bc2d tape: cleanup - rename CatalogBuilder to CatalogSet 2021-03-19 07:22:54 +01:00
Dietmar Maurer
32b75d36a8 tape: backup media catalogs 2021-03-19 06:58:46 +01:00
Dietmar Maurer
237314ad0d tape: improve catalog consistency checks 
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.

Note: this changes the catalog format again.
 2021-03-18 08:43:55 +01:00
Dietmar Maurer
6ee3035523 tape: define magic number for catalog archives 2021-03-17 13:35:23 +01:00
Dietmar Maurer
b627ebbf40 tape: improve catalog parser 2021-03-17 11:29:23 +01:00
Dietmar Maurer
54722acada tape: store datastore name in tape archives and media catalog 
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
 2021-03-17 11:08:51 +01:00
Dietmar Maurer
0e2bf3aa1d SnapshotReader: add self.datastore_name() helper 2021-03-17 10:16:34 +01:00
Dietmar Maurer
365126efa9 tape: PoolWriter - remove unnecessary move_to_eom 2021-03-17 10:16:34 +01:00
Dietmar Maurer
776dabfb2e tape: use MB/s for backup speed (to match drive speed specification) 2021-03-16 08:51:49 +01:00
Dietmar Maurer
5c4755ad08 tape: speedup backup by doing read/write in parallel 2021-03-16 08:51:49 +01:00
Dietmar Maurer
e8913fea12 tape: write_chunk_archive - do not consume partially written chunk at EOT 
So that it is re-written to the next tape.
 2021-03-12 07:14:50 +01:00
Dominik Csapak
2892624783 tape/send_load_media_email: move to server/email_notifications 
and reuse 'send_job_status_mail' there so that we get consistent
formatted mails from pbs (e.g. html part and author)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-11 09:56:12 +01:00
Dietmar Maurer
2c10410b0d tape: improve backup task log 2021-03-11 08:43:13 +01:00
Thomas Lamprecht
d1d74c4367 typo fixes all over the place 
found and semi-manually replaced by using:
 codespell -L mut -L crate -i 3 -w

Mostly in comments, but also email notification and two occurrences
of misspelled  'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-03-10 16:39:57 +01:00
Dietmar Maurer
5ef4c7bcd3 tape: fix scsi volume_statistics and cartridge_memory for quantum drives 2021-03-10 14:13:48 +01:00
Dietmar Maurer
713a128adf tape: improve backup task log format 2021-03-10 09:54:51 +01:00
Dietmar Maurer
affc224aca tape: read_tape_mam - pass correct allocation len 2021-03-10 09:24:38 +01:00
Dietmar Maurer
6f82d32977 tape: cleanup - remove wrong inline comment 2021-03-10 08:11:51 +01:00
Dietmar Maurer
1057b1f5a5 tape: lock artificial "__UNASSIGNED__" pool to avoid races 2021-03-09 10:00:26 +01:00
Dietmar Maurer
af76234112 tape: improve MediaPool allocation by sorting tapes by ctime and label_text 2021-03-09 08:33:21 +01:00
Dominik Csapak
1d7fcbece8 tape/linux_tape: remove debug prints 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-08 14:29:19 +01:00
Dietmar Maurer
9ce2481a69 tape: clean - fix if cleaning tape is already loaded 2021-03-08 09:08:55 +01:00
Dominik Csapak
c9793d47f9 api2/tape: add notify_user to backup(-jobs) and restore api calls 
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-05 17:20:37 +01:00
Dietmar Maurer
90e16be3ae tape: improve PoolWriter logging 
Log reason why we allocate a new media set.
 2021-03-05 09:59:38 +01:00
Dietmar Maurer
cd5d6103ea tape: fix media_expire_time 2021-03-04 16:58:24 +01:00
Dietmar Maurer
1bed3aedc8 tape: fix wrong media set expire time 2021-03-04 12:38:28 +01:00
Dietmar Maurer
ab77d660cc tape: improve media status in list_media 2021-03-04 10:17:10 +01:00
Dominik Csapak
78593b5b5c tape/drive: improve error and email handling for requesting a tape load 
Try once first to load the correct tape before sending an email to
insert the correct one. This way, the admin does not get a mail
if the correct tape is already inserted.

Also include the error we got that prompted the email to insert the
tape. This means that if the admin gets prompted to insert e.g.
"FOO" but inserts "BAR", he'll get an email that the wrong
tape is inserted.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-03 16:09:39 +01:00
Dietmar Maurer
1d14c31658 Revert "tape: also abort backup/restore on server shutdown" 
This reverts commit 9bd81bb384.

Turns out this is not really a good idea.
 2021-03-02 08:00:10 +01:00
Dietmar Maurer
9bd81bb384 tape: also abort backup/restore on server shutdown 2021-03-02 07:19:31 +01:00
Dietmar Maurer
d984a9acf0 tape: add Inventory.media_list() helper 2021-03-01 09:30:45 +01:00
Dietmar Maurer
08ec39be0c tape: add api to set media status 2021-02-26 10:44:07 +01:00
Dietmar Maurer
25350f3370 tape: set media status if we detect damaged medium at start of backup 2021-02-26 09:00:50 +01:00
Dietmar Maurer
3fbf2311e4 tape: improve backup task logging 2021-02-23 12:58:44 +01:00
Dietmar Maurer
86d9f4e733 tape: extend MediaChange trait to return MtxStatus 2021-02-20 10:23:16 +01:00
Dietmar Maurer
3f16f1b006 tape: update changer status inside ScsiMediaChange implementation 2021-02-20 09:56:27 +01:00
Dietmar Maurer
cd44fb8d84 tape: create tmp dirs early at server startup 2021-02-19 18:10:02 +01:00
Dietmar Maurer
4188fd59a0 tape: cache changer state 2021-02-19 16:48:19 +01:00
Dominik Csapak
546d2653ee tape/drive: add get/set status functions 
simply writes into/reads from a file in /run, we will use this
for writing the upid (or potential other states) per drive

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:05:09 +01:00
Dominik Csapak
33c06b3388 tape/drive: add test_device_path_lock 
this makes it possible to detect if the drive was locked in a
non-blocking manner

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:03:30 +01:00