tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,161 Commits 2 Branches 54 Tags 14 MiB
2d5d264f99
Commit Graph

269 Commits

Author SHA1 Message Date
Dietmar Maurer
b81e37f6ab tape: improve code reuse 2021-02-04 09:39:16 +01:00
Dietmar Maurer
ddebbb52fd tape: fix tests for BlockedReader 2021-02-04 08:54:54 +01:00
Dietmar Maurer
983e929e25 tape: add multi volume reader/writer implementations 
We currently do not use it. Added anaways, to show the possibility.
 2021-02-04 08:36:35 +01:00
Dietmar Maurer
f47e035721 tape: cleanup - move tape file readers/writers into src/tape/file_formats folder 2021-02-04 07:59:37 +01:00
Dietmar Maurer
7d2c156eb1 tape: BlockedReader - always consume EOF 2021-02-03 13:25:59 +01:00
Dietmar Maurer
b22e8c3632 tape: add media pool regression tests 2021-02-03 10:23:04 +01:00
Dietmar Maurer
7929292618 tape: add regresion test for media state 2021-02-03 09:34:31 +01:00
Dietmar Maurer
cc2a0b12f8 test: define tape tests as submodule 2021-02-02 14:38:15 +01:00
Dietmar Maurer
988e8de122 tape: set correct ownership on lock file 2021-02-02 14:18:57 +01:00
Dietmar Maurer
2f8809c6bc test: src/tape/inventory.rs - avoid chown when running tests 2021-02-02 13:43:16 +01:00
Dietmar Maurer
8937c65951 tape: add pmt stoptions/stsethoptions/stclearoptions 2021-02-02 08:58:02 +01:00
Dietmar Maurer
d0f11b66f7 thape: add read_tapedev_options, display driver options with status command 2021-02-02 06:40:40 +01:00
Dietmar Maurer
ca953d831f cleanup: remove MT_ST_ prefix from SetDrvBufferOptions 2021-02-01 17:54:53 +01:00
Dietmar Maurer
b8cbe5d65b tape: fix tape alert flag decoding 2021-02-01 12:18:55 +01:00
Dietmar Maurer
83b8949a98 tape: add pmt weof 2021-01-31 17:33:07 +01:00
Dietmar Maurer
1f31d06f48 tape: add pmt bsf 2021-01-31 17:00:15 +01:00
Dietmar Maurer
2f2e83c890 tape: add pmt fsf 2021-01-31 16:54:16 +01:00
Dietmar Maurer
1e041082bb tape: add pmt command line tool 
Experimental, not installed by now.
 2021-01-31 16:19:53 +01:00
Dietmar Maurer
b5b99a52cd tape: API type cleanup, use serde flatten to derive types 2021-01-30 09:36:54 +01:00
Dietmar Maurer
271764deb9 tape: make it possible to abort tape backup tasks (check_abort) 
Also use task_log makro instead of worker.log.
 2021-01-29 09:07:55 +01:00
Dietmar Maurer
f2ca03d7d0 cleanup: avoid compiler warning 2021-01-28 15:32:21 +01:00
Dominik Csapak
00ac86c31b tape/drive/linux_tape: fix and refactor usage of sg-tape-cmd 
when executing this code as non-root, we use sg-tape-cmd (a setuid binary)
to execute various ioctls on the tape device

we give the command the open tape device fd as stdin, but did not
dup it, so the std::process:Stdio handle closed it on drop,
which let subsequent operation on that file fail (since it was closed)

fix it by dup'ing it before giving it to the command, and also refactor
the calling code, so that we do not forget to do this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:24:32 +01:00
Dominik Csapak
4be4736603 tape/changer: refactor marking of import/export slots from config 
we did this for 'mtx', but missed it for the sg_pt_changer code
refactor it into the MtxStatus strut, and call it from both
code paths

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:10:55 +01:00
Dominik Csapak
2da7aca8e8 tape/changer: add vendor/model to DriveStatus 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:10:31 +01:00
Dietmar Maurer
d737adc6be tape: rename changer_drive_id to changer_drivenum 2021-01-28 11:29:59 +01:00
Dietmar Maurer
16b4d78400 tape: rename retry_command to execute_scsi_command, make retry a flag 2021-01-27 09:34:24 +01:00
Dietmar Maurer
ec8d9c6b80 tape: repeat changer scsi command until successful 2021-01-27 08:59:10 +01:00
Dominik Csapak
e1740f3f01 tape/changer/mtx: add mtx parser test 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-26 12:51:26 +01:00
Dominik Csapak
bbf01b644c tape: fix typos 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-26 12:39:54 +01:00
Dietmar Maurer
c3747b93c8 tape: add new command line tool "pmtx" 
Also improve sgutil2 error reporting
 2021-01-26 11:57:15 +01:00
Dietmar Maurer
2b96a43879 tape: cleanup - use ScsiMediaChange trait instead of mtx_status() 2021-01-25 13:25:22 +01:00
Dietmar Maurer
697c41c584 tape: add/use rust scsi changer implementation using libsgutil2 2021-01-25 13:14:07 +01:00
Dietmar Maurer
0107fd323c cleanup: avoid compiler warnings 2021-01-23 17:34:26 +01:00
Dietmar Maurer
979dccc7ec tape: avoid error when clearing encryption key 
Simply ignore clear request when sg_spin_data_encryption_caps fails.
Assume those are tapes without hardware encryption support.
 2021-01-23 10:20:43 +01:00
Dietmar Maurer
45212a8c78 fix mtx parser bug: s/strip_suffix/strip_prefix/ 2021-01-22 11:00:56 +01:00
Dietmar Maurer
5f34d69bcc tape: add volume-statistics api/command 2021-01-22 08:45:35 +01:00
Dietmar Maurer
337ff5a3cc tape: add estimated medium wearout to status 2021-01-22 08:06:25 +01:00
Dietmar Maurer
9372c0787d renamed src/tape/sgutils2.rs -> src/tools/sgutils2.rs 2021-01-21 17:57:17 +01:00
Dietmar Maurer
83fb2da53e tape: move MediaCatalog magic number into struct (doc cleanup) 2021-01-21 17:48:07 +01:00
Dietmar Maurer
645a044bf6 tape: further hierarchy improvements 2021-01-21 17:25:32 +01:00
Dietmar Maurer
37796ff73f tape: change code hierarchy to improve docs 2021-01-21 17:12:01 +01:00
Dietmar Maurer
e1fdcb1678 tape: do not export/doc low level libsgutils2 bindings 2021-01-21 16:38:24 +01:00
Dietmar Maurer
edda5039d4 tape: improve code docs 2021-01-21 13:19:07 +01:00
Dietmar Maurer
d543587d34 Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup 2021-01-21 10:56:52 +01:00
Dietmar Maurer
2b191385ea tape: use specialized encryption key per media-set 2021-01-21 10:31:49 +01:00
Fabian Grünbichler
1d928b25fe clippy: remove some unnecessary reference taking 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
f2f81791d1 clippy: fix for_kv_map 
and allow it in the one case where the entry loop is intended, but the
code is not yet implemented fully.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
382f10a0cc clippy: fix/allow needless_range_loop 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
09faa9ee95 clippy: pass &str/&[..] instead of &String/&Vec 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
ccec086e25 clippy: remove unnecessary &mut 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
05725ac9a4 clippy: remove unnecessary let binding 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
e062ebbc29 clippy: us *_or_else with function calls 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
ea368a06cd clippy: misc. fixes 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
a375df6f4c clippy: use copied/cloned instead of map 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
a3775bb4e8 clippy: shorten assignments 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
6334bdc1c5 clippy: collapse nested ifs 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
3b82f3eea5 clippy: avoid useless format! 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
38556bf60d clippy: remove explicit returns 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
397356096a clippy: remove needless bool literals 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:52 +01:00
Fabian Grünbichler
365915da9a clippy: use strip_prefix instead of manual stripping 
it's less error-prone (off-by-one!)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:22:59 +01:00
Fabian Grünbichler
87152fbac6 clippy: drop redundant 'static lifetime 
those declarations are already const/static..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:22:59 +01:00
Fabian Grünbichler
22a9189ee0 clippy: remove unnecessary closures 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:22:59 +01:00
Fabian Grünbichler
4428818412 clippy: remove unnecessary clones 
and from::<T>(T)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:22:59 +01:00
Dietmar Maurer
6dd0513546 tape: allocate new media set when pool encryption key changes 2021-01-20 15:43:39 +01:00
Dietmar Maurer
82a103c8f9 add "password hint" to KeyConfig 2021-01-20 15:43:19 +01:00
Dietmar Maurer
feb1645f37 tape: generate random encryptions keys and store key_config on media 2021-01-19 11:20:07 +01:00
Dietmar Maurer
8a0046f519 tape: implement encrypted backup - simple version 
This is just a proof of concept, only storing the encryption key fingerprint
inside the media-set label.
 2021-01-18 13:38:22 +01:00
Dietmar Maurer
619554af2b tape: clear encryption key before writing labels 
We always write labels unencrypted.
 2021-01-18 13:38:22 +01:00
Dietmar Maurer
d5a48b5ce4 tape: add hardware encryption key managenent api 2021-01-18 13:38:22 +01:00
Dietmar Maurer
b22b6c2299 tape: encryption scsi command cleanup 2021-01-16 18:24:04 +01:00
Dietmar Maurer
90950c9c20 tape: add scsi commands to control drive hardware encryption 2021-01-16 15:59:05 +01:00
Dietmar Maurer
0c5b9e7820 tape: sgutils2.rs - add do_out_command() 
Make it possible to run commands that writes data.
 2021-01-16 15:59:05 +01:00
Dietmar Maurer
a08a198577 tape: do not abort backup if tape drive does not support tape-alert-flags 2021-01-15 11:43:17 +01:00
Dietmar Maurer
2d50a6192f tape: sg-tape-cmd - add more ways to specify devices 2021-01-14 13:05:26 +01:00
Dietmar Maurer
25e464c5ce tape: MediaPool - allow to allocate free tapes 2021-01-13 14:25:51 +01:00
Dietmar Maurer
8446fbca85 tape: rename changer_id to label_text 2021-01-13 13:26:59 +01:00
Dietmar Maurer
6543214dde tape: MediaListEntry - add ctime 2021-01-12 12:01:21 +01:00
Dietmar Maurer
a484c9cf96 tape: automatically reload tapes inside autoloader 
We always automatically unload tapes to free library slots,
so it should not happen that an ejected tape resides inside the drive.

This is just a safe guard to handle the situation in case it happens ...

You can manually produce the situation by ejecting a tape without unloading:

 mt -f /dev/nst0 eject

Note: Our "proxmox-tape eject" does automatic unload
 2021-01-12 09:49:05 +01:00
Dietmar Maurer
5654d8ceba tape: make eject/export more reliable, improve logging 2021-01-12 09:16:16 +01:00
Dietmar Maurer
31cf625af5 tape: improve backup logs 2021-01-11 13:23:12 +01:00
Dietmar Maurer
93be18ffd2 tape: fix tape alert flag values 2021-01-11 13:23:12 +01:00
Dietmar Maurer
6638c034d2 tape: remove unused eject_on_unload method 2021-01-10 16:20:18 +01:00
Dietmar Maurer
04df41cec1 tape: more MediaChange cleanups 
Try to provide generic implementation for complex operations:

- unload_to_free_slot
- load_media
- export media
- clean drive
- online_media_changer_ids
 2021-01-10 15:32:52 +01:00
Dietmar Maurer
483da89d03 tape: improve export media to directly export from drive, add CLI 2021-01-10 13:44:44 +01:00
Dietmar Maurer
c92e3832bf tape: cleanup: s/transfer/transfer_media/, avoid compiler warnings 2021-01-10 12:18:30 +01:00
Dietmar Maurer
edb90f6afa tape: backup - implement export-media-set option 2021-01-10 11:59:55 +01:00
Dietmar Maurer
0057f0e580 tape: MediaChange - add transfer, implement export 2021-01-10 11:51:09 +01:00
Dietmar Maurer
e6217b8b36 tape: renamed src/tape/changer/linux_tape.rs -> src/tape/changer/mtx.rs 2021-01-10 10:07:40 +01:00
Dietmar Maurer
6fe16039b9 tape: simplify media changer implementation - new struct MtxMediaChanger 2021-01-10 10:02:01 +01:00
Dietmar Maurer
42967bf185 tape: backup - implement --eject-media option 2021-01-09 15:17:03 +01:00
Dietmar Maurer
5843268c47 tape: abort backup when we detect critical tape alert flags 2021-01-09 12:34:00 +01:00
Dietmar Maurer
7273ba3de2 tape: change default media set naming template to "%c" 2021-01-09 10:51:51 +01:00
Dietmar Maurer
0bf1c314da tape: show catalog status in media list 2021-01-09 10:24:48 +01:00
Dietmar Maurer
c7926d8e8c tape: split MediaSet into extra file 2021-01-09 08:54:58 +01:00
Dietmar Maurer
df69a4fc59 tape: implement drive clean 2021-01-08 11:32:56 +01:00
Dietmar Maurer
25d3965769 tape: correctly skip cleaning tapes (not regular tapes) 2021-01-08 09:16:42 +01:00
Dietmar Maurer
46a1863f88 tape: improve MediaChange trait 
We expose the whole MtxStatus, and we can load/store from/to
specified slot numbers.
 2021-01-07 14:26:43 +01:00
Dietmar Maurer
d5035c5600 tape: mtx_status - consider new export-slots property 2021-01-06 11:53:33 +01:00
Dietmar Maurer
c4b2b9ab41 tape: only query volume stats if we can read MAM 2021-01-06 09:20:36 +01:00
Dietmar Maurer
ef942e04c2 tape: add function to classify tape-alert-flags 2021-01-05 17:23:30 +01:00
Dietmar Maurer
b40ab10d38 tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus 2021-01-05 13:43:17 +01:00
Dietmar Maurer
f8ccbfdedd tape: implement read_volume_statistics 2021-01-05 12:58:18 +01:00
Dietmar Maurer
470f1c798a tape: status - show thape alert flags 2021-01-04 13:15:30 +01:00
Dietmar Maurer
5c012b392a tape: use LP 12h TapeAlert Response to query tape alert flags 2021-01-04 13:14:02 +01:00
Dietmar Maurer
66e42bec05 tape: further PoolWriter cleanups 2021-01-03 12:08:40 +01:00
Dietmar Maurer
c503ea7045 tape: cleanup - rename 'info' to 'media_id' 
Second try.
 2021-01-03 11:38:00 +01:00
Dietmar Maurer
745ec187ce Revert "tape: cleanup - rename 'info' to 'media_id'" 
This reverts commit f046313c0e.

media_id is already use as parameter, so this commit is totally buggy.
 2021-01-03 11:14:58 +01:00
Dietmar Maurer
f046313c0e tape: cleanup - rename 'info' to 'media_id' 2021-01-03 10:37:42 +01:00
Dietmar Maurer
74595b8821 tape: sg-tape-cmd tape-alert-flags 2021-01-03 10:09:43 +01:00
Dietmar Maurer
c9fdd142a4 tape: commit missing file 2021-01-02 13:39:34 +01:00
Dietmar Maurer
abaa6d0ac9 tape: decode TapeAlertFlags in cartridge-memory command 2021-01-02 10:55:30 +01:00
Dietmar Maurer
cfae8f0656 tape: merge MediaStateDatabase into Inventory 2021-01-01 16:15:13 +01:00
Dietmar Maurer
54f4ecd46a tape: implement MediaPool flag to consider offline media 
For standalone tape drives.
 2021-01-01 10:03:59 +01:00
Dietmar Maurer
b4772d1c43 tape: new inventory helper - lookup_media_set_pool 2020-12-31 10:03:17 +01:00
Dietmar Maurer
81764111fe tape: media_change - log all errors 2020-12-30 19:17:18 +01:00
Dietmar Maurer
75656a78c6 tape: improve inline docu 2020-12-30 17:28:33 +01:00
Dietmar Maurer
284eb5daff tape: cleanup/simplify media_change code 2020-12-30 17:16:57 +01:00
Dietmar Maurer
ff58c51919 tape: improve media request/load 2020-12-30 13:09:28 +01:00
Dietmar Maurer
9e8c0d2e33 tape: cleanup - remove debug messages 2020-12-30 08:41:30 +01:00
Dietmar Maurer
c431659d05 cleanup: remove debug output 2020-12-29 11:59:57 +01:00
Dietmar Maurer
afb0220642 tape: cleanup LinuxDriveStatus - make density optional 2020-12-29 09:10:30 +01:00
Dietmar Maurer
0993923ed5 tape: factor out get_drive_and_media_status 2020-12-29 08:39:06 +01:00
Dietmar Maurer
e0362b0d0f tape: correctly parse mtx import/export slots 2020-12-28 13:32:56 +01:00
Dietmar Maurer
b27c32821c tape: install new sg-tape-cmd setuid binary 2020-12-28 11:10:25 +01:00
Dietmar Maurer
76b15a035f tape: MediaCatalog: write magic number before content 2020-12-26 11:05:25 +01:00
Dietmar Maurer
fc6ce9835b tape: fix non-rewinding tape device check 2020-12-25 15:38:29 +01:00
Dietmar Maurer
8ae9f4efc2 tape: minor cleanups 2020-12-25 13:45:26 +01:00
Dietmar Maurer
c9d13b0fc4 tape: expose check_tape_is_linux_tape_device 2020-12-24 15:51:49 +01:00
Dietmar Maurer
bfacc1d8c3 tape: cleanup - factor out open_linux_tape_device 2020-12-24 11:24:45 +01:00
Dietmar Maurer
5ae86dfaa1 tape: return media usage info with status command 2020-12-23 11:24:34 +01:00
Dietmar Maurer
dbe7e556b0 tape: implement binding for libsgutils2 
So that we can read cartridge memory without calling "sg_raw". In future,
we may need further low level command to control the tape..
 2020-12-23 09:44:53 +01:00
Dietmar Maurer
cb80d900b3 tape: add drive status api 2020-12-22 10:42:22 +01:00
Dietmar Maurer
1e20f819d5 tape: add command to read cartridge memory (MAM) 
Thsi add an additional dependency to sg3-utils (small).
 2020-12-21 12:12:33 +01:00
Dietmar Maurer
9aa58f0143 cleanup: rename mtfsf into forward_space_count_files 2020-12-18 16:57:49 +01:00
Dietmar Maurer
d37da6b7fc tape: add PoolWriter 2020-12-18 15:27:44 +01:00
Dietmar Maurer
b9ee86efe1 tape: use SnapshotReader to create snapshot archive 2020-12-18 12:11:29 +01:00
Dietmar Maurer
d108b610fd tape: fix write_media_set_label - move to correct position 2020-12-18 12:11:29 +01:00
Dietmar Maurer
2afdc7f27d tape: MediaPool::with_config() - remove name parameter 
Not required, because config already contains the pool name.
 2020-12-18 08:14:24 +01:00
Dietmar Maurer
26aa9aca40 tape: return current_file_number as u64 2020-12-18 07:44:50 +01:00
Dietmar Maurer
b532dd00c4 tape: add helper to read snapshot contents 
- lock the snapshot for reading
- use openat to open files
- provides an iterator over all chunks
 2020-12-17 13:07:52 +01:00
Dietmar Maurer
9c953dd260 tape: add code to write backup snapshot files (without chunks) to tape 2020-12-17 08:28:47 +01:00
Dietmar Maurer
3fbf2d2fcd tape: cleanup MediaCatalog 2020-12-17 08:05:53 +01:00
Dietmar Maurer
fe6c19383b tape: remove MediaLabelInfo, use MediaId instead 
The additional content_uuid was quite useless...
 2020-12-16 13:31:32 +01:00
Dietmar Maurer
9839d3f778 tape: improve docu 2020-12-16 12:43:51 +01:00
Dietmar Maurer
dd59e3c2a1 tape: improve docu 2020-12-16 12:23:52 +01:00
Dietmar Maurer
0b7432ae09 tape: add chunk archive reader/writer 2020-12-16 12:08:34 +01:00
Dietmar Maurer
c1c2c8f635 tape: cleanup MediaLocation type for direct use with API 2020-12-16 10:49:01 +01:00
Dietmar Maurer
42298d5896 tape: add magic number to identify media catalog files 2020-12-16 09:00:14 +01:00
Dietmar Maurer
205e187613 tape: add MediaCatalog implementation 2020-12-15 13:40:49 +01:00
Dietmar Maurer
a78348acbb tape: rename DriveLabel to MediaLabel 2020-12-14 17:37:16 +01:00