2631e57d20
fix regression tests
2021-04-13 14:02:37 +02:00
90461b76fb
TapeRead: add skip_data()
2021-04-13 13:32:45 +02:00
6a15cce540
tape: SgTapeReader::read_block - disable reading beyond EOF
2021-04-13 11:46:30 +02:00
f281b8d3a9
tape: cleanup MediaCatalog on tape reuse
2021-04-13 11:46:30 +02:00
ce5327badc
tape: fix regression tests
2021-04-12 14:08:05 +02:00
318b310638
tape: improve EOT error handling
2021-04-12 13:27:34 +02:00
109ccd300f
cleanup: move tape SCSI code to src/tape/drive/lto/sg_tape/
2021-04-09 11:34:45 +02:00
c560cfddca
tape: read_drive_status - ignore media changed sense info
2021-04-09 09:46:19 +02:00
8204d9b095
tape: avoid unneccessary SCSI request in Drop
2021-04-08 11:26:08 +02:00
fad95a334a
tape: clear encryption key after backup (for security reasons)
2021-04-08 10:37:49 +02:00
973e985d73
cleanup: remove unused linux tape driver code
2021-04-08 10:15:52 +02:00
15d1435789
tape: add vendor, product and revision to LtoDriveAndMediaStatus
2021-04-08 08:34:46 +02:00
80ea23e1b9
tape: pmt - implement options command
2021-04-08 08:34:45 +02:00
5d6379f8db
tape: implement locate_file without LOCATE(10)
2021-04-08 08:34:45 +02:00
566b946f9b
tape: pmt - re-implement lock/unlock command
2021-04-08 07:28:30 +02:00
7f7459677d
tape: pmt - re-implement fsr/bsr
2021-04-08 07:28:30 +02:00
0892a512bc
tape: correctly set/display drive option
2021-04-08 07:28:30 +02:00
7b11a8098d
tape: make sure there is a filemark at the end of the tape
2021-04-08 07:28:30 +02:00
8b2c6f5dbc
tape: make fsf/bsf driver specific
...
Because the virtual tape driver behaves different than LTO drives.
2021-04-08 07:28:30 +02:00
d26985a600
tape: fix LEOM handling
2021-04-08 07:28:30 +02:00
e29f456efc
tape: implement format/erase
2021-04-08 07:28:30 +02:00
a79082a0dd
tape: implement LTO userspace driver
2021-04-08 07:28:30 +02:00
1336ae8249
tape: introduce trait BlockWrite
2021-04-08 07:28:30 +02:00
0db5712493
tape: introduce trait BlockRead
2021-04-08 07:28:30 +02:00
cf1e117fc7
sgutils2: use enum for ScsiError
...
This avoids string allocation when we return SenseInfo.
2021-03-27 15:57:48 +01:00
11f5d59396
tape: page-align BlockHeader so that we can use it with SG_IO
2021-03-27 15:36:35 +01:00
2843ba9017
avoid compiler warning
2021-03-25 12:25:23 +01:00
589c4dad9e
tape: add fsf/bsf to TapeDriver trait
2021-03-25 10:10:16 +01:00
c046739461
tape: fix MediaPool regression tests
2021-03-24 09:44:30 +01:00
c6f55139f8
tape: impl. MediaCatalog::parse_catalog_header
...
This is just an optimization, avoiding to read the catalog into memory.
We also expose create_temporary_database_file() now (will be
used for catalog restore).
2021-03-24 06:32:59 +01:00
20cc25d749
tape: add TapeDriver::move_to_last_file
2021-03-24 06:32:59 +01:00
30316192b3
tape: improve locking (lock media-sets)
...
- new helper: lock_media_set()
- MediaPool: lock media set
- Expose Inventory::new() to avoid double loading
- do not lock pool on restore (only lock media-set)
- change pool lock name to ".pool-{name}"
2021-03-24 06:32:59 +01:00
e93263be1e
taoe: implement MediaCatalog::destroy_unrelated_catalog() helper
2021-03-22 12:03:11 +01:00
2ab2ca9c24
tape: add MediaPool::lock_unassigned_media_pool() helper
2021-03-19 10:13:38 +01:00
eac1beef3c
tape: cleanup PoolWriter - factor out common code
2021-03-19 08:56:14 +01:00
166a48f903
tape: cleanup - split PoolWriter into several files
2021-03-19 08:19:13 +01:00
82775c4764
tape: make sure we only commit/write valid catalogs
2021-03-19 07:50:32 +01:00
88bc9635aa
tape: store media_uuid in PoolWriterState
...
This is mainly a cleanup, avoiding to access the catalog_set to get the uuid.
2021-03-19 07:33:59 +01:00
1037f2bc2d
tape: cleanup - rename CatalogBuilder to CatalogSet
2021-03-19 07:22:54 +01:00
32b75d36a8
tape: backup media catalogs
2021-03-19 06:58:46 +01:00
237314ad0d
tape: improve catalog consistency checks
...
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.
Note: this changes the catalog format again.
2021-03-18 08:43:55 +01:00
6ee3035523
tape: define magic number for catalog archives
2021-03-17 13:35:23 +01:00
b627ebbf40
tape: improve catalog parser
2021-03-17 11:29:23 +01:00
54722acada
tape: store datastore name in tape archives and media catalog
...
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
2021-03-17 11:08:51 +01:00
0e2bf3aa1d
SnapshotReader: add self.datastore_name() helper
2021-03-17 10:16:34 +01:00
365126efa9
tape: PoolWriter - remove unnecessary move_to_eom
2021-03-17 10:16:34 +01:00
776dabfb2e
tape: use MB/s for backup speed (to match drive speed specification)
2021-03-16 08:51:49 +01:00
5c4755ad08
tape: speedup backup by doing read/write in parallel
2021-03-16 08:51:49 +01:00
e8913fea12
tape: write_chunk_archive - do not consume partially written chunk at EOT
...
So that it is re-written to the next tape.
2021-03-12 07:14:50 +01:00
2892624783
tape/send_load_media_email: move to server/email_notifications
...
and reuse 'send_job_status_mail' there so that we get consistent
formatted mails from pbs (e.g. html part and author)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-11 09:56:12 +01:00
2c10410b0d
tape: improve backup task log
2021-03-11 08:43:13 +01:00
d1d74c4367
typo fixes all over the place
...
found and semi-manually replaced by using:
codespell -L mut -L crate -i 3 -w
Mostly in comments, but also email notification and two occurrences
of misspelled 'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 16:39:57 +01:00
5ef4c7bcd3
tape: fix scsi volume_statistics and cartridge_memory for quantum drives
2021-03-10 14:13:48 +01:00
713a128adf
tape: improve backup task log format
2021-03-10 09:54:51 +01:00
affc224aca
tape: read_tape_mam - pass correct allocation len
2021-03-10 09:24:38 +01:00
6f82d32977
tape: cleanup - remove wrong inline comment
2021-03-10 08:11:51 +01:00
1057b1f5a5
tape: lock artificial "__UNASSIGNED__" pool to avoid races
2021-03-09 10:00:26 +01:00
af76234112
tape: improve MediaPool allocation by sorting tapes by ctime and label_text
2021-03-09 08:33:21 +01:00
1d7fcbece8
tape/linux_tape: remove debug prints
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:19 +01:00
9ce2481a69
tape: clean - fix if cleaning tape is already loaded
2021-03-08 09:08:55 +01:00
c9793d47f9
api2/tape: add notify_user to backup(-jobs) and restore api calls
...
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-05 17:20:37 +01:00
90e16be3ae
tape: improve PoolWriter logging
...
Log reason why we allocate a new media set.
2021-03-05 09:59:38 +01:00
cd5d6103ea
tape: fix media_expire_time
2021-03-04 16:58:24 +01:00
1bed3aedc8
tape: fix wrong media set expire time
2021-03-04 12:38:28 +01:00
ab77d660cc
tape: improve media status in list_media
2021-03-04 10:17:10 +01:00
78593b5b5c
tape/drive: improve error and email handling for requesting a tape load
...
Try once first to load the correct tape before sending an email to
insert the correct one. This way, the admin does not get a mail
if the correct tape is already inserted.
Also include the error we got that prompted the email to insert the
tape. This means that if the admin gets prompted to insert e.g.
"FOO" but inserts "BAR", he'll get an email that the wrong
tape is inserted.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
1d14c31658
Revert "tape: also abort backup/restore on server shutdown"
...
This reverts commit 9bd81bb384
2021-03-02 08:00:10 +01:00
9bd81bb384
tape: also abort backup/restore on server shutdown
2021-03-02 07:19:31 +01:00
d984a9acf0
tape: add Inventory.media_list() helper
2021-03-01 09:30:45 +01:00
08ec39be0c
tape: add api to set media status
2021-02-26 10:44:07 +01:00
25350f3370
tape: set media status if we detect damaged medium at start of backup
2021-02-26 09:00:50 +01:00
3fbf2311e4
tape: improve backup task logging
2021-02-23 12:58:44 +01:00
86d9f4e733
tape: extend MediaChange trait to return MtxStatus
2021-02-20 10:23:16 +01:00
3f16f1b006
tape: update changer status inside ScsiMediaChange implementation
2021-02-20 09:56:27 +01:00
cd44fb8d84
tape: create tmp dirs early at server startup
2021-02-19 18:10:02 +01:00
4188fd59a0
tape: cache changer state
2021-02-19 16:48:19 +01:00
546d2653ee
tape/drive: add get/set status functions
...
simply writes into/reads from a file in /run, we will use this
for writing the upid (or potential other states) per drive
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:05:09 +01:00
33c06b3388
tape/drive: add test_device_path_lock
...
this makes it possible to detect if the drive was locked in a
non-blocking manner
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:03:30 +01:00
9bbd83b1f2
tape: media_list API - allow to update online status for a single changer
2021-02-18 10:59:33 +01:00
8b910bb6bc
tape: add regression test for compute_media_state
2021-02-17 06:57:29 +01:00
dbaef7d04d
tape: add regression test for alloc_writable_media
2021-02-17 06:55:49 +01:00
ced7838de4
tape: fix path for test files
2021-02-06 09:32:50 +01:00
f197c286d5
tape/inventory: fix missing parameter in error message
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:12:18 +01:00
25aa55b5f5
tape: correctly lock the tape device
2021-02-05 10:50:21 +01:00
cdf39e62b3
tape: MediaPool - replace use_offline_media with changer_name
...
This way, we can improve location_is_available, because we only
consider media from that changer as available.
2021-02-04 10:15:18 +01:00
b81e37f6ab
tape: improve code reuse
2021-02-04 09:39:16 +01:00
ddebbb52fd
tape: fix tests for BlockedReader
2021-02-04 08:54:54 +01:00
983e929e25
tape: add multi volume reader/writer implementations
...
We currently do not use it. Added anaways, to show the possibility.
2021-02-04 08:36:35 +01:00
f47e035721
tape: cleanup - move tape file readers/writers into src/tape/file_formats folder
2021-02-04 07:59:37 +01:00
7d2c156eb1
tape: BlockedReader - always consume EOF
2021-02-03 13:25:59 +01:00
b22e8c3632
tape: add media pool regression tests
2021-02-03 10:23:04 +01:00
7929292618
tape: add regresion test for media state
2021-02-03 09:34:31 +01:00
cc2a0b12f8
test: define tape tests as submodule
2021-02-02 14:38:15 +01:00
988e8de122
tape: set correct ownership on lock file
2021-02-02 14:18:57 +01:00
2f8809c6bc
test: src/tape/inventory.rs - avoid chown when running tests
2021-02-02 13:43:16 +01:00
8937c65951
tape: add pmt stoptions/stsethoptions/stclearoptions
2021-02-02 08:58:02 +01:00
d0f11b66f7
thape: add read_tapedev_options, display driver options with status command
2021-02-02 06:40:40 +01:00
ca953d831f
cleanup: remove MT_ST_ prefix from SetDrvBufferOptions
2021-02-01 17:54:53 +01:00
b8cbe5d65b
tape: fix tape alert flag decoding
2021-02-01 12:18:55 +01:00
83b8949a98
tape: add pmt weof
2021-01-31 17:33:07 +01:00
1f31d06f48
tape: add pmt bsf
2021-01-31 17:00:15 +01:00
2f2e83c890
tape: add pmt fsf
2021-01-31 16:54:16 +01:00
1e041082bb
tape: add pmt command line tool
...
Experimental, not installed by now.
2021-01-31 16:19:53 +01:00
b5b99a52cd
tape: API type cleanup, use serde flatten to derive types
2021-01-30 09:36:54 +01:00
271764deb9
tape: make it possible to abort tape backup tasks (check_abort)
...
Also use task_log makro instead of worker.log.
2021-01-29 09:07:55 +01:00
f2ca03d7d0
cleanup: avoid compiler warning
2021-01-28 15:32:21 +01:00
00ac86c31b
tape/drive/linux_tape: fix and refactor usage of sg-tape-cmd
...
when executing this code as non-root, we use sg-tape-cmd (a setuid binary)
to execute various ioctls on the tape device
we give the command the open tape device fd as stdin, but did not
dup it, so the std::process:Stdio handle closed it on drop,
which let subsequent operation on that file fail (since it was closed)
fix it by dup'ing it before giving it to the command, and also refactor
the calling code, so that we do not forget to do this
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:24:32 +01:00
4be4736603
tape/changer: refactor marking of import/export slots from config
...
we did this for 'mtx', but missed it for the sg_pt_changer code
refactor it into the MtxStatus strut, and call it from both
code paths
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:10:55 +01:00
2da7aca8e8
tape/changer: add vendor/model to DriveStatus
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:10:31 +01:00
d737adc6be
tape: rename changer_drive_id to changer_drivenum
2021-01-28 11:29:59 +01:00
16b4d78400
tape: rename retry_command to execute_scsi_command, make retry a flag
2021-01-27 09:34:24 +01:00
ec8d9c6b80
tape: repeat changer scsi command until successful
2021-01-27 08:59:10 +01:00
e1740f3f01
tape/changer/mtx: add mtx parser test
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:51:26 +01:00
bbf01b644c
tape: fix typos
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:39:54 +01:00
c3747b93c8
tape: add new command line tool "pmtx"
...
Also improve sgutil2 error reporting
2021-01-26 11:57:15 +01:00
2b96a43879
tape: cleanup - use ScsiMediaChange trait instead of mtx_status()
2021-01-25 13:25:22 +01:00
697c41c584
tape: add/use rust scsi changer implementation using libsgutil2
2021-01-25 13:14:07 +01:00
0107fd323c
cleanup: avoid compiler warnings
2021-01-23 17:34:26 +01:00
979dccc7ec
tape: avoid error when clearing encryption key
...
Simply ignore clear request when sg_spin_data_encryption_caps fails.
Assume those are tapes without hardware encryption support.
2021-01-23 10:20:43 +01:00
45212a8c78
fix mtx parser bug: s/strip_suffix/strip_prefix/
2021-01-22 11:00:56 +01:00
5f34d69bcc
tape: add volume-statistics api/command
2021-01-22 08:45:35 +01:00
337ff5a3cc
tape: add estimated medium wearout to status
2021-01-22 08:06:25 +01:00
9372c0787d
renamed src/tape/sgutils2.rs -> src/tools/sgutils2.rs
2021-01-21 17:57:17 +01:00
83fb2da53e
tape: move MediaCatalog magic number into struct (doc cleanup)
2021-01-21 17:48:07 +01:00
645a044bf6
tape: further hierarchy improvements
2021-01-21 17:25:32 +01:00
37796ff73f
tape: change code hierarchy to improve docs
2021-01-21 17:12:01 +01:00
e1fdcb1678
tape: do not export/doc low level libsgutils2 bindings
2021-01-21 16:38:24 +01:00
edda5039d4
tape: improve code docs
2021-01-21 13:19:07 +01:00
d543587d34
Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup
2021-01-21 10:56:52 +01:00
2b191385ea
tape: use specialized encryption key per media-set
2021-01-21 10:31:49 +01:00
1d928b25fe
clippy: remove some unnecessary reference taking
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
f2f81791d1
clippy: fix for_kv_map
...
and allow it in the one case where the entry loop is intended, but the
code is not yet implemented fully.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
382f10a0cc
clippy: fix/allow needless_range_loop
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
09faa9ee95
clippy: pass &str/&[..] instead of &String/&Vec
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
ccec086e25
clippy: remove unnecessary &mut
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
05725ac9a4
clippy: remove unnecessary `let` binding
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
e062ebbc29
clippy: us *_or_else with function calls
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
ea368a06cd
clippy: misc. fixes
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a375df6f4c
clippy: use copied/cloned instead of map
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a3775bb4e8
clippy: shorten assignments
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
6334bdc1c5
clippy: collapse nested ifs
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3b82f3eea5
clippy: avoid useless format!
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
38556bf60d
clippy: remove explicit returns
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
397356096a
clippy: remove needless bool literals
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:52 +01:00
365915da9a
clippy: use strip_prefix instead of manual stripping
...
it's less error-prone (off-by-one!)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
87152fbac6
clippy: drop redundant 'static lifetime
...
those declarations are already const/static..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
22a9189ee0
clippy: remove unnecessary closures
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
4428818412
clippy: remove unnecessary clones
...
and from::<T>(T)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
6dd0513546
tape: allocate new media set when pool encryption key changes
2021-01-20 15:43:39 +01:00
82a103c8f9
add "password hint" to KeyConfig
2021-01-20 15:43:19 +01:00
Dietmar Maurer
Dominik Csapak
Thomas Lamprecht
Fabian Grünbichler