Commit Graph

3350 Commits

Author SHA1 Message Date
Dietmar Maurer 25e464c5ce tape: MediaPool - allow to allocate free tapes 2021-01-13 14:25:51 +01:00
Dietmar Maurer 8446fbca85 tape: rename changer_id to label_text 2021-01-13 13:26:59 +01:00
Dietmar Maurer 0bce2118e7 tape: improve docu 2021-01-12 16:37:23 +01:00
Dietmar Maurer 6543214dde tape: MediaListEntry - add ctime 2021-01-12 12:01:21 +01:00
Dietmar Maurer a484c9cf96 tape: automatically reload tapes inside autoloader
We always automatically unload tapes to free library slots,
so it should not happen that an ejected tape resides inside the drive.

This is just a safe guard to handle the situation in case it happens ...

You can manually produce the situation by ejecting a tape without unloading:

 mt -f /dev/nst0 eject

Note: Our "proxmox-tape eject" does automatic unload
2021-01-12 09:49:05 +01:00
Dietmar Maurer 5654d8ceba tape: make eject/export more reliable, improve logging 2021-01-12 09:16:16 +01:00
Dietmar Maurer 31cf625af5 tape: improve backup logs 2021-01-11 13:23:12 +01:00
Dietmar Maurer 93be18ffd2 tape: fix tape alert flag values 2021-01-11 13:23:12 +01:00
Wolfgang Bumiller ad0ed40a59 api: return "invalid" as CSRF token for partial tickets
So that old clients don't `unwrap` a `None` value.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
Wolfgang Bumiller 7ad33e8052 tfa: use UNAUTHORIZED http status in password check
to trigger our 3s delay in the rest handler

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
Wolfgang Bumiller abfe0c0e70 tfa: fixup for challenge file split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
Wolfgang Bumiller f22dfb5ece tfa: remove tfa user when a user is deleted
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:10 +01:00
Wolfgang Bumiller 4bda51688b tfa: improve user existence check
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller eab25e2f33 tfa: allow deletion of entries of non-existent users
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller 94bd11bae2 typo fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller 759af9f00c tfa api: return types and 'pub' structs/methods
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller f58e5132aa tfa: entry access/iteration cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller d831846706 tfa: r#type parameter name
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller 1fc9ac0433 tfa: _entry api method name suffix consistency
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
Wolfgang Bumiller 7f066a9b21 proxy: expose qrcodejs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Wolfgang Bumiller 027ef213aa api: tfa management and login
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Wolfgang Bumiller dc1fdd6267 config: add tfa configuration
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Wolfgang Bumiller 96918252e5 buildcfg: add rundir helper macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Wolfgang Bumiller 014dc5f9d7 tools: add create_run_dir helper
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Wolfgang Bumiller 59e94227af add tools::serde_filter submodule
can be used to perform filtering at parse time

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Dietmar Maurer 6638c034d2 tape: remove unused eject_on_unload method 2021-01-10 16:20:18 +01:00
Dietmar Maurer 04df41cec1 tape: more MediaChange cleanups
Try to provide generic implementation for complex operations:

- unload_to_free_slot
- load_media
- export media
- clean drive
- online_media_changer_ids
2021-01-10 15:32:52 +01:00
Dietmar Maurer 483da89d03 tape: improve export media to directly export from drive, add CLI 2021-01-10 13:44:44 +01:00
Dietmar Maurer c92e3832bf tape: cleanup: s/transfer/transfer_media/, avoid compiler warnings 2021-01-10 12:18:30 +01:00
Dietmar Maurer edb90f6afa tape: backup - implement export-media-set option 2021-01-10 11:59:55 +01:00
Dietmar Maurer 0057f0e580 tape: MediaChange - add transfer, implement export 2021-01-10 11:51:09 +01:00
Dietmar Maurer e6217b8b36 tape: renamed src/tape/changer/linux_tape.rs -> src/tape/changer/mtx.rs 2021-01-10 10:07:40 +01:00
Dietmar Maurer 6fe16039b9 tape: simplify media changer implementation - new struct MtxMediaChanger 2021-01-10 10:02:01 +01:00
Dietmar Maurer 42967bf185 tape: backup - implement --eject-media option 2021-01-09 15:17:03 +01:00
Dietmar Maurer 5843268c47 tape: abort backup when we detect critical tape alert flags 2021-01-09 12:34:00 +01:00
Dietmar Maurer 7273ba3de2 tape: change default media set naming template to "%c" 2021-01-09 10:51:51 +01:00
Dietmar Maurer 0bf1c314da tape: show catalog status in media list 2021-01-09 10:24:48 +01:00
Dietmar Maurer c7926d8e8c tape: split MediaSet into extra file 2021-01-09 08:54:58 +01:00
Dietmar Maurer df69a4fc59 tape: implement drive clean 2021-01-08 11:32:56 +01:00
Dietmar Maurer 25d3965769 tape: correctly skip cleaning tapes (not regular tapes) 2021-01-08 09:16:42 +01:00
Dietmar Maurer 879569d73f tape: changer transfer - make name parameter optional 2021-01-07 17:09:47 +01:00
Dietmar Maurer b63f833d36 tape: fix paramater name - s/slot/source-slot/ 2021-01-07 15:39:25 +01:00
Dietmar Maurer 482c6e33dd tape: changer status command: make changer name optional 2021-01-07 15:12:19 +01:00
Dietmar Maurer 46a1863f88 tape: improve MediaChange trait
We expose the whole MtxStatus, and we can load/store from/to
specified slot numbers.
2021-01-07 14:26:43 +01:00
Dietmar Maurer d5035c5600 tape: mtx_status - consider new export-slots property 2021-01-06 11:53:33 +01:00
Dietmar Maurer 38ae42b11a tape: changer - add export-slot config 2021-01-06 11:06:50 +01:00
Dietmar Maurer c4b2b9ab41 tape: only query volume stats if we can read MAM 2021-01-06 09:20:36 +01:00
Dietmar Maurer ef942e04c2 tape: add function to classify tape-alert-flags 2021-01-05 17:23:30 +01:00
Dietmar Maurer b40ab10d38 tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus 2021-01-05 13:43:17 +01:00
Dietmar Maurer f8ccbfdedd tape: implement read_volume_statistics 2021-01-05 12:58:18 +01:00
Dietmar Maurer 470f1c798a tape: status - show thape alert flags 2021-01-04 13:15:30 +01:00
Dietmar Maurer 5c012b392a tape: use LP 12h TapeAlert Response to query tape alert flags 2021-01-04 13:14:02 +01:00
Dietmar Maurer 165b641c1d tape: changer status - show full slots (for cartridge without barcode) 2021-01-04 12:06:05 +01:00
Dietmar Maurer 66e42bec05 tape: further PoolWriter cleanups 2021-01-03 12:08:40 +01:00
Dietmar Maurer c503ea7045 tape: cleanup - rename 'info' to 'media_id'
Second try.
2021-01-03 11:38:00 +01:00
Dietmar Maurer 745ec187ce Revert "tape: cleanup - rename 'info' to 'media_id'"
This reverts commit f046313c0e.

media_id is already use as parameter, so this commit is totally buggy.
2021-01-03 11:14:58 +01:00
Dietmar Maurer f046313c0e tape: cleanup - rename 'info' to 'media_id' 2021-01-03 10:37:42 +01:00
Dietmar Maurer 74595b8821 tape: sg-tape-cmd tape-alert-flags 2021-01-03 10:09:43 +01:00
Dietmar Maurer c9fdd142a4 tape: commit missing file 2021-01-02 13:39:34 +01:00
Dietmar Maurer abaa6d0ac9 tape: decode TapeAlertFlags in cartridge-memory command 2021-01-02 10:55:30 +01:00
Dietmar Maurer cfae8f0656 tape: merge MediaStateDatabase into Inventory 2021-01-01 16:15:13 +01:00
Dietmar Maurer 54f4ecd46a tape: implement MediaPool flag to consider offline media
For standalone tape drives.
2021-01-01 10:03:59 +01:00
Dietmar Maurer b9b4b31284 tape: add basic restore api/command 2020-12-31 10:26:48 +01:00
Dietmar Maurer b4772d1c43 tape: new inventory helper - lookup_media_set_pool 2020-12-31 10:03:17 +01:00
Fabian Grünbichler 08ac90f920 api: allow tokens to list users
their owner, or all if they have the appropriate privileges.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:49 +01:00
Fabian Grünbichler 13f5863561 api: improve error messages for restricted endpoints
the old variant attempted to parse a tokenid as userid and returned the
cryptic parsing error to the client, which is rather confusing.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:09 +01:00
Dietmar Maurer 81764111fe tape: media_change - log all errors 2020-12-30 19:17:18 +01:00
Dietmar Maurer cb022525ff tape: only log to stdout in CLI environment 2020-12-30 19:01:39 +01:00
Dietmar Maurer 75656a78c6 tape: improve inline docu 2020-12-30 17:28:33 +01:00
Dietmar Maurer 284eb5daff tape: cleanup/simplify media_change code 2020-12-30 17:16:57 +01:00
Dietmar Maurer ff58c51919 tape: improve media request/load 2020-12-30 13:09:28 +01:00
Dietmar Maurer 12299b333b tape: set minimal media label length to 2 2020-12-30 10:15:02 +01:00
Dietmar Maurer b017bbc441 tape: add restore code, implement catalog api/command 2020-12-30 09:48:18 +01:00
Dietmar Maurer 9e8c0d2e33 tape: cleanup - remove debug messages 2020-12-30 08:41:30 +01:00
Dietmar Maurer 250c29edd2 tape: correctly sort media api entries 2020-12-29 12:09:51 +01:00
Dietmar Maurer c431659d05 cleanup: remove debug output 2020-12-29 11:59:57 +01:00
Dietmar Maurer a33389c391 tape: implement media content list api 2020-12-29 11:58:26 +01:00
Dietmar Maurer 3460565414 tape: create the MediaCatalog when we label a tape 2020-12-29 10:55:20 +01:00
Dietmar Maurer 26b62138ee cleanup: disable debug message when we detect a stopped worker task 2020-12-29 10:53:16 +01:00
Dietmar Maurer afb0220642 tape: cleanup LinuxDriveStatus - make density optional 2020-12-29 09:10:30 +01:00
Dietmar Maurer 0993923ed5 tape: factor out get_drive_and_media_status 2020-12-29 08:39:06 +01:00
Dietmar Maurer e0362b0d0f tape: correctly parse mtx import/export slots 2020-12-28 13:32:56 +01:00
Dietmar Maurer b27c32821c tape: install new sg-tape-cmd setuid binary 2020-12-28 11:10:25 +01:00
Dietmar Maurer 76b15a035f tape: MediaCatalog: write magic number before content 2020-12-26 11:05:25 +01:00
Dietmar Maurer eb8feb1281 tape: add LTO1 to TapeDensity 2020-12-26 10:48:32 +01:00
Dietmar Maurer fc6ce9835b tape: fix non-rewinding tape device check 2020-12-25 15:38:29 +01:00
Dietmar Maurer 8ae9f4efc2 tape: minor cleanups 2020-12-25 13:45:26 +01:00
Dietmar Maurer c9d13b0fc4 tape: expose check_tape_is_linux_tape_device 2020-12-24 15:51:49 +01:00
Dietmar Maurer bfacc1d8c3 tape: cleanup - factor out open_linux_tape_device 2020-12-24 11:24:45 +01:00
Dietmar Maurer 5ae86dfaa1 tape: return media usage info with status command 2020-12-23 11:24:34 +01:00
Dietmar Maurer dbe7e556b0 tape: implement binding for libsgutils2
So that we can read cartridge memory without calling "sg_raw". In future,
we may need further low level command to control the tape..
2020-12-23 09:44:53 +01:00
Stefan Reiter 4799280ccd http_client: add timeouts for critical connects
Use timeout futures for sections that might hang in certain error
conditions. This is mostly intended to be used as a safeguard, not a
first line of defense - i.e. best-effort avoidance of total hangs.

Not every future used for the HttpClient/H2Client is changed, only those
where a quick response is to be expected. For example, the response
reading futures are left alone, so data transfer is never capped with
timeout, only the initial server connect.

It is also used for upgrading to H2 connections, as that can take a long
time on overloaded servers.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-12-22 13:31:10 +01:00
Dietmar Maurer cb80d900b3 tape: add drive status api 2020-12-22 10:42:22 +01:00
Dietmar Maurer ee01737e87 tape: rename 'mam' api to 'cartridge-memory' 2020-12-22 09:27:34 +01:00
Dietmar Maurer 2012825913 depend on proxmox 0.9.0 2020-12-22 08:52:24 +01:00
Wolfgang Bumiller b2362a1207 adaptions for proxmox 0.9 and proxmox-api-macro 0.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-22 07:31:05 +01:00
Dietmar Maurer 1e20f819d5 tape: add command to read cartridge memory (MAM)
Thsi add an additional dependency to sg3-utils (small).
2020-12-21 12:12:33 +01:00
Dietmar Maurer 9aa58f0143 cleanup: rename mtfsf into forward_space_count_files 2020-12-18 16:57:49 +01:00
Dietmar Maurer 8835664653 tape: add tape backup api 2020-12-18 15:32:12 +01:00
Dietmar Maurer d37da6b7fc tape: add PoolWriter 2020-12-18 15:27:44 +01:00
Dietmar Maurer b9ee86efe1 tape: use SnapshotReader to create snapshot archive 2020-12-18 12:11:29 +01:00
Dietmar Maurer d108b610fd tape: fix write_media_set_label - move to correct position 2020-12-18 12:11:29 +01:00
Dominik Csapak 0ec79339f7 tools/daemon: improve reload behaviour
it seems that sometimes, the child process signal gets handled
before the parent process signal. Systemd then ignores the
childs signal (finished reloading) and only after going into
reloading state because of the parent. this will never finish.

Instead, wait for the state to change to 'reloading' after sending
that signal in the parent, an only fork afterwards. This way
we ensure that systemd knows about the reloading before actually trying
to do it.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-12-18 10:30:37 +01:00
Dietmar Maurer 2afdc7f27d tape: MediaPool::with_config() - remove name parameter
Not required, because config already contains the pool name.
2020-12-18 08:14:24 +01:00
Dietmar Maurer 26aa9aca40 tape: return current_file_number as u64 2020-12-18 07:44:50 +01:00
Dominik Csapak 3e2984bcb9 tools/process_locker: Decrement writer count in drop handler
of ProcessLockSharedGuard.

We use a counter to determine if we can unlock the file again, but
we never actually decremented the writer count, so we held the
lock forever.

This fixes the issue that we could not start a garbage collect after
a reload, as long as the old process is still running, even when that
process has no active backup anymore but another long running task
(e.g. file download, terminal, etc.).

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-18 07:15:08 +01:00
Fabian Grünbichler a7a5406c32 acl: rustfmt module
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:07:01 +01:00
Fabian Grünbichler 4f727a783e acl: reformat privileges
for better readability, and tell rustfmt to leave those definitions
alone.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:45 +01:00
Fabian Grünbichler 23dc68fdea acl: add docs and adapt visibility
document all public things, add some doc links and make some
previously-public things only available for test cases or within the
crate:

previously public, now private:
- AclTreeNode::extract_user_roles (we have extract_roles())
- AclTreeNode::extract_group_roles (same)
- AclTreeNode::delete_group_role (exists on AclTree)
- AclTreeNode::delete_user_role (same)
- AclTreeNode::insert_group_role (same)
- AclTreeNode::insert_user_role (same)
- AclTree::write_config (we have save_config())
- AclTree::load (we have config()/cached_config())

previously public, now crate-internal:
- AclTree::from_raw (only used by tests)
- split_acl_path (used by some test binaries)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:11 +01:00
Dietmar Maurer b532dd00c4 tape: add helper to read snapshot contents
- lock the snapshot for reading
- use openat to open files
- provides an iterator over all chunks
2020-12-17 13:07:52 +01:00
Fabian Grünbichler c01742855a KeyConfig: bail on wrong fingerprint
instead of just logging the error. this should never happen in practice
unless someone is messing with the keyfile, in which case, it's better
to abort.

update tests accordingly (wrong fingerprint should fail, no fingerprint
should get the expected one).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 11:27:06 +01:00
Dietmar Maurer 9c953dd260 tape: add code to write backup snapshot files (without chunks) to tape 2020-12-17 08:28:47 +01:00
Dietmar Maurer 3fbf2d2fcd tape: cleanup MediaCatalog 2020-12-17 08:05:53 +01:00
Fabian Grünbichler e0af222ec3 KeyConfig: always calculate fingerprint
and warn if stored and calculated fingerprint don't match.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:52:55 +01:00
Fabian Grünbichler 73b5011786 KeyConfig: add encrypt/decrypt test
the RSA key and the encryption key itself are hard-coded to avoid
stalling the test runs because of lack of entropy, they have no special
significance otherwise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:47:45 +01:00
Fabian Grünbichler 7137630d43 client: add 'import-with-master-key' command
to import an encrypted encryption key using a master key.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:46:24 +01:00
Fabian Grünbichler 8acfd15d6e key: move RSA-encryption to KeyConfig
since that is what gets encrypted, and not a CryptConfig.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:43:34 +01:00
Fabian Grünbichler 48fbbfeb7e fix #3197: skip fingerprint check when restoring key
when restoring an encrypted key, the original one is obviously not
available to check the fingerprint with.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:37:54 +01:00
Fabian Grünbichler 9990af3042 master key: store blob name in constant
since we will use it in more than one place.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:36:06 +01:00
Dietmar Maurer fe6c19383b tape: remove MediaLabelInfo, use MediaId instead
The additional content_uuid was quite useless...
2020-12-16 13:31:32 +01:00
Dietmar Maurer 9839d3f778 tape: improve docu 2020-12-16 12:43:51 +01:00
Dietmar Maurer dd59e3c2a1 tape: improve docu 2020-12-16 12:23:52 +01:00
Dietmar Maurer 0b7432ae09 tape: add chunk archive reader/writer 2020-12-16 12:08:34 +01:00
Dietmar Maurer c1c2c8f635 tape: cleanup MediaLocation type for direct use with API 2020-12-16 10:49:01 +01:00
Dietmar Maurer 42298d5896 tape: add magic number to identify media catalog files 2020-12-16 09:00:14 +01:00
Dietmar Maurer 205e187613 tape: add MediaCatalog implementation 2020-12-15 13:40:49 +01:00
Dietmar Maurer a78348acbb tape: rename DriveLabel to MediaLabel 2020-12-14 17:37:16 +01:00
Dietmar Maurer 410611b4f2 tape: improve file format docu 2020-12-14 17:29:57 +01:00
Dietmar Maurer af07ec8f29 tape: minor code cleanup 2020-12-14 16:56:26 +01:00
Dietmar Maurer 3f803af00b tape: scan - print more debug info 2020-12-14 13:16:18 +01:00
Dietmar Maurer ac461bd651 tape: implement scan command (useful for debug) 2020-12-14 12:55:49 +01:00
Dietmar Maurer ce955e1635 tape: implement eod cli command (debug tool) 2020-12-14 09:56:59 +01:00
Dietmar Maurer e20d008c6a tape: rename cli 'media media-destroy' toö 'media destroy' 2020-12-14 09:30:32 +01:00
Dietmar Maurer fb657d8ee5 tape: implement destroy_media 2020-12-14 08:58:40 +01:00
Dietmar Maurer fba0b77469 tape: add media api 2020-12-14 07:55:57 +01:00
Dietmar Maurer b5c1296eaa tape: make changer get_status async 2020-12-14 07:14:24 +01:00
Dietmar Maurer 065df12872 tape: split api type definitions for changers into extra file 2020-12-13 09:31:02 +01:00
Dietmar Maurer 7e1d4712b8 tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA 2020-12-13 09:22:08 +01:00
Dietmar Maurer 49c965a497 tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA 2020-12-13 09:18:16 +01:00
Dietmar Maurer 6fe9aedd0b tape: correctly call Async handler in proxmox-tape 2020-12-12 09:58:47 +01:00
Dietmar Maurer 42cb9bd6a5 tape: avoid executor blocking in changer api 2020-12-12 09:45:08 +01:00
Dietmar Maurer 66dbe5639e tape: avoid executor blocking in drive API
By using tokio::task::spawn_blocking().
2020-12-12 09:20:04 +01:00
Dietmar Maurer 663ef85992 tape: use WorkerTask for erase and rewind 2020-12-11 11:19:33 +01:00
Dietmar Maurer e92c75815b tape: split inventory api
inventory: sync, list labels with uuids,
update_inventory: WorkerTask, updates database
2020-12-11 10:42:29 +01:00
Dietmar Maurer 6dbad5b4b5 tape: run label commands as WorkerTask (threads) 2020-12-11 09:10:22 +01:00
Dietmar Maurer bff7e3f3e4 tape: implement barcode-label-mdedia 2020-12-11 07:50:19 +01:00
Dietmar Maurer 83abc7497d tape: implement inventory command 2020-12-11 07:39:28 +01:00
Dietmar Maurer be1a8c94ae fix build: add missing file 2020-12-10 13:40:20 +01:00
Dietmar Maurer 4606f34353 tape: implement read-label command 2020-12-10 13:20:39 +01:00
Dietmar Maurer 7bb720cb4d tape: implement label command 2020-12-10 12:30:27 +01:00
Dietmar Maurer c4d8542ec1 tape: add media pool handling 2020-12-10 11:41:35 +01:00
Dietmar Maurer 9700d5374a tape: add media pool cli 2020-12-10 11:13:12 +01:00
Dietmar Maurer 05e90d6463 tape: add media pool config api 2020-12-10 10:52:27 +01:00
Dietmar Maurer 55118ca18e tape: correctly sort drive api subdir 2020-12-10 10:09:12 +01:00
Dietmar Maurer f70d8091d3 tape: implement option changer-drive-id 2020-12-10 09:09:06 +01:00
Dietmar Maurer a3c709ef21 tape: cli cleanup - avoid api redefinition 2020-12-10 08:35:11 +01:00
Dietmar Maurer 4917f1e2d4 tape: implement delete property for drive update command 2020-12-10 08:25:46 +01:00
Dietmar Maurer 93829fc680 tape: cleanup load-slot api 2020-12-10 08:04:55 +01:00
Dietmar Maurer 5605ca5619 tape: cli cleanup - rename scana-for-* into scan 2020-12-10 07:58:45 +01:00
Dietmar Maurer e49f0c03d9 tape: implement load-media command 2020-12-10 07:52:56 +01:00
Dietmar Maurer 0098b712a5 tape: implement eject 2020-12-09 17:50:48 +01:00
Dietmar Maurer 5fb694e8c0 tape: implement rewind 2020-12-09 17:43:38 +01:00
Dietmar Maurer 583a68a446 tape: implement erase media 2020-12-09 17:35:31 +01:00
Dietmar Maurer e6604cf391 tape: add command line interface proxmox-tape 2020-12-09 13:00:20 +01:00
Dietmar Maurer 43cfb3c35a tape: do not remove changer while still used 2020-12-09 12:55:54 +01:00
Dietmar Maurer 8a16c571d2 tape: add changer property to drive create api 2020-12-09 12:55:10 +01:00
Dietmar Maurer 314652a499 tape: set protected flag for configuration change api methods 2020-12-09 12:02:55 +01:00
Dietmar Maurer 6b68e5d597 client: move connect_to_localhost into client module 2020-12-09 11:59:50 +01:00
Dietmar Maurer cafd51bf42 tape: add media state database 2020-12-09 11:21:56 +01:00
Wolfgang Bumiller 9b93c62044 remove unused descriptions from api macros
these are now a hard error in the api macro

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-09 10:55:18 +01:00
Dietmar Maurer 5d90860688 tape: expose basic tape/changer functionality at api2/tape/ 2020-12-08 15:42:50 +01:00
Dietmar Maurer 5ba83ed099 tape: check digest on config update 2020-12-08 11:24:38 +01:00
Dietmar Maurer 50bf10ad56 tape: add changer configuration API 2020-12-08 09:04:56 +01:00
Dietmar Maurer 16d444c979 tape: add tape drive configuration API 2020-12-07 13:04:32 +01:00
Dietmar Maurer fa9c9be737 tape: add tape device driver 2020-12-07 08:29:22 +01:00
Dietmar Maurer 2e7014e31d tape: add BlockeReader/BlockedWriter streams
This is the basic format used to write data to tapes.
2020-12-06 12:09:55 +01:00
Dietmar Maurer a84050c1f0 tape: add BlockHeader impl 2020-12-06 10:26:24 +01:00
Dietmar Maurer 7c9835465e tape: add helpers to emulate tape read/write behavior 2020-12-06 09:41:16 +01:00
Dietmar Maurer ec00200411 fix bug #3189: fix change_password permission checks, run protected 2020-12-05 16:20:29 +01:00
Dietmar Maurer b107fdb99a tape: add tape changer support using 'mtx' command 2020-12-05 14:54:12 +01:00
Dietmar Maurer 7320e9ff4b tape: add media invenotry 2020-12-05 12:54:15 +01:00
Dietmar Maurer c4d2d54a6d tape: define useful constants 2020-12-05 12:20:46 +01:00
Dietmar Maurer 1142350e8d tape: add media pool config 2020-12-05 11:59:38 +01:00
Dietmar Maurer d735b31345 tape: add tape read trait 2020-12-05 10:54:38 +01:00
Dietmar Maurer e211fee562 tape: add tape write trait 2020-12-05 10:51:34 +01:00
Dietmar Maurer 8c15560b68 tape: add file format definitions 2020-12-05 10:45:08 +01:00
Dietmar Maurer 327e93711f commit missing file: tape api type definitions 2020-12-04 16:00:52 +01:00
Dietmar Maurer a076571470 tape support: add drive configuration 2020-12-04 15:42:32 +01:00
Dominik Csapak 179145dc24 backup/datastore: move manifest locking to /run
this fixes the issue that on some filesystems, you cannot recursively
remove a directory when you hold a lock on a file inside (e.g. nfs/cifs)

it is not really backwards compatible (so during an upgrade, there
could be two daemons have the lock), but since the locking was
broken before (see previous patch) it should not really matter
(also it seems very unlikely that someone will trigger this)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 09:56:42 +01:00
Dominik Csapak 6bd0a00c46 backup/datastore: really lock manifest on delete
'lock_manifest' returns a Result<File, Error> so we always got the result,
even when we did not get the lock, but we acted like we had.

bubble the locking error up

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 14:37:05 +01:00
Dominik Csapak f6e28f4e62 client/pull: log how many groups to pull were found
if no groups were found, the task log was very confusing as it
contained no real information why nothing was synced, e.g.:

 Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
 Sync datastore 'local-datastore' from 'remote/datastore'
 sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
 TASK OK

this patch simply logs how many groups were found and are about to be synced:

 Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
 Sync datastore 'local-datastore' from 'remote/datastore'
 found 0 groups to sync
 sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
 TASK OK

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 07:22:50 +01:00
Dietmar Maurer 2260f065d4 cleanup: use extra file for StoreProgress 2020-12-01 06:34:33 +01:00
Dietmar Maurer 6eff8dec4f cleanup: remove unnecessary StoreProgress clone() 2020-12-01 06:29:11 +01:00
Fabian Grünbichler 7e25b9aaaa verify: use same progress as pull
percentage of verified groups, interpolating based on snapshot count
within the group. in most cases, this will also be closer to 'real'
progress since added snapshots (those which will be verified) in active
backup groups will be roughly evenly distributed, while number of total
snapshots per group will be heavily skewed towards those groups which
have existed the longest, even though most of those old snapshots will
only be re-verified very infrequently.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:55 +01:00
Fabian Grünbichler f867ef9c4a progress: add format variants
for iterating over a single group, or iterating just on the group level

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:12 +01:00
Fabian Grünbichler fc8920e35d pull: factor out interpolated progress
and add group/snapshot count info.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:13:11 +01:00
Fabian Grünbichler 7f3b0f67e7 remove BackupGroup::list_groups
BackupInfo::list_backup_groups is identical code-wise, and makes more
sense as entry point for listing groups.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:09:44 +01:00
Fabian Grünbichler 844660036b gc: don't limit index listing to same filesystem
WalkDir does not follow symlinks by default anyway, and this behaviour
is not documented anywhere. e.g., if a sysadmin mounts 'extra storage'
for some backup group or type (not knowing that only metadata is stored
in those directories), GC will ignore all the indices contained within
and happily garbage collect their chunks..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:07:09 +01:00
Fabian Grünbichler efcac39d34 gc: remove duplicate variable
list_images already returns absolute paths, we don't need to prepend
anything.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:51 +01:00
Fabian Grünbichler cb4b721cb0 gc: log index files found outside of expected scheme
for safety reason, GC finds and marks all index files below the
datastore base path. as a result of regular operations, only index files
within the expected scheme of <TYPE>/<ID>/<TIMESTAMP> should exist.

add a small check + warning if the index list contains index files out
side of this expected scheme, so that an admin with shell access can
investigate.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:17 +01:00