90950c9c20
tape: add scsi commands to control drive hardware encryption
2021-01-16 15:59:05 +01:00
0c5b9e7820
tape: sgutils2.rs - add do_out_command()
...
Make it possible to run commands that writes data.
2021-01-16 15:59:05 +01:00
a670b99db1
tfa: add webauthn configuration API entry points
...
Currently there's not yet a node config and the WA config is
somewhat "tightly coupled" to the user entries in that
changing it can lock them all out, so for now I opted for
fewer reorganization and just use a digest of the
canonicalized config here, and keep it all in the tfa.json
file.
Experimentally using the flatten feature on the methods with
an`Updater` struct similar to what the api macro is supposed
to be able to derive on its own in the future.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
aefd74197a
bakckup::manifest: use tools::json for canonical representation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
9ff747ef50
add tools::json for canonical json generation
...
moving this from backup::manifest, no functional changes
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
a08a198577
tape: do not abort backup if tape drive does not support tape-alert-flags
2021-01-15 11:43:17 +01:00
6bbe49aa14
access: restrict password changes on @pam realm to superuser
...
for behavior consistency with `update_user`
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-15 08:49:22 +01:00
5aa1019010
access: limit editing pam credentials to superuser
...
modifying @pam users credentials should be only possible for root@pam,
otherwise it can have unintended consequences.
also enforce the same limit on user creation (except self_service check,
since it makes no sense during user creation)
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-15 08:49:22 +01:00
29a59b380c
proxmox 0.10: adapt to moved ParameterSchema
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0bfcea6a11
cleanup: remove unnecessary 'mut' and '.clone()'
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
89e9134a3f
hyper: use new hyper::upgrade
...
the old Body::on_upgrade method is no more
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
b5a202acb6
tokio 1.0: update to new Signal interface
...
Signal does not yet re-implement Stream (and is not yet wrapped in
tokio-stream either).
see https://github.com/tokio-rs/tokio/pull/3383
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0f860f712f
tokio 1.0: update to new tokio-openssl interface
...
connect/accept are now happening on pinned SslStreams
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
7c66701366
tokio 1.0: use ReceiverStream from tokio-stream
...
to wrap a Receiver in a Stream. this will likely move back into tokio
proper once we have a std Stream..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
585e90c0de
tokio: adapt to 1.0 process:Child changes
...
Child itself is no longer a Future, but it has a new wait() async fn
that does the same thing
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
5c852d5b82
tokio: adapt to 1.0 runtime changes
...
enter() now returns a guard, and the builder got revamped to make the
choice between MT and current thread explicit.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
484172b5f8
tokio 1.0: AsyncRead/Seek with ReadBuf
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
d148958b67
proxmox 0.10: use tokio::time::timeout directly
...
TimeoutFutureExt is no more
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0a8d773ad0
tokio 1.0: delay -> sleep
...
almost the same thing, new name(s), no longer Unpin
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
427d90e6c1
update to tokio 1.0
...
and various related crates
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
1a0b410554
manager: user/token list: fix rendering 0 (never) expire date
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-14 13:59:08 +01:00
2d50a6192f
tape: sg-tape-cmd - add more ways to specify devices
2021-01-14 13:05:26 +01:00
781da7f6f0
tape: add --inventorize flag to read-label API/CLI
2021-01-14 11:51:23 +01:00
25e464c5ce
tape: MediaPool - allow to allocate free tapes
2021-01-13 14:25:51 +01:00
8446fbca85
tape: rename changer_id to label_text
2021-01-13 13:26:59 +01:00
0bce2118e7
tape: improve docu
2021-01-12 16:37:23 +01:00
6543214dde
tape: MediaListEntry - add ctime
2021-01-12 12:01:21 +01:00
a484c9cf96
tape: automatically reload tapes inside autoloader
...
We always automatically unload tapes to free library slots,
so it should not happen that an ejected tape resides inside the drive.
This is just a safe guard to handle the situation in case it happens ...
You can manually produce the situation by ejecting a tape without unloading:
mt -f /dev/nst0 eject
Note: Our "proxmox-tape eject" does automatic unload
2021-01-12 09:49:05 +01:00
5654d8ceba
tape: make eject/export more reliable, improve logging
2021-01-12 09:16:16 +01:00
31cf625af5
tape: improve backup logs
2021-01-11 13:23:12 +01:00
93be18ffd2
tape: fix tape alert flag values
2021-01-11 13:23:12 +01:00
ad0ed40a59
api: return "invalid" as CSRF token for partial tickets
...
So that old clients don't `unwrap` a `None` value.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
7ad33e8052
tfa: use UNAUTHORIZED http status in password check
...
to trigger our 3s delay in the rest handler
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
abfe0c0e70
tfa: fixup for challenge file split
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
f22dfb5ece
tfa: remove tfa user when a user is deleted
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:10 +01:00
4bda51688b
tfa: improve user existence check
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
eab25e2f33
tfa: allow deletion of entries of non-existent users
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
94bd11bae2
typo fixups
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
759af9f00c
tfa api: return types and 'pub' structs/methods
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
f58e5132aa
tfa: entry access/iteration cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
d831846706
tfa: r#type parameter name
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
1fc9ac0433
tfa: _entry api method name suffix consistency
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
7f066a9b21
proxy: expose qrcodejs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
027ef213aa
api: tfa management and login
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
dc1fdd6267
config: add tfa configuration
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
96918252e5
buildcfg: add rundir helper macro
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
014dc5f9d7
tools: add create_run_dir helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
59e94227af
add tools::serde_filter submodule
...
can be used to perform filtering at parse time
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
6638c034d2
tape: remove unused eject_on_unload method
2021-01-10 16:20:18 +01:00
04df41cec1
tape: more MediaChange cleanups
...
Try to provide generic implementation for complex operations:
- unload_to_free_slot
- load_media
- export media
- clean drive
- online_media_changer_ids
2021-01-10 15:32:52 +01:00
483da89d03
tape: improve export media to directly export from drive, add CLI
2021-01-10 13:44:44 +01:00
c92e3832bf
tape: cleanup: s/transfer/transfer_media/, avoid compiler warnings
2021-01-10 12:18:30 +01:00
edb90f6afa
tape: backup - implement export-media-set option
2021-01-10 11:59:55 +01:00
0057f0e580
tape: MediaChange - add transfer, implement export
2021-01-10 11:51:09 +01:00
e6217b8b36
tape: renamed src/tape/changer/linux_tape.rs -> src/tape/changer/mtx.rs
2021-01-10 10:07:40 +01:00
6fe16039b9
tape: simplify media changer implementation - new struct MtxMediaChanger
2021-01-10 10:02:01 +01:00
42967bf185
tape: backup - implement --eject-media option
2021-01-09 15:17:03 +01:00
5843268c47
tape: abort backup when we detect critical tape alert flags
2021-01-09 12:34:00 +01:00
7273ba3de2
tape: change default media set naming template to "%c"
2021-01-09 10:51:51 +01:00
0bf1c314da
tape: show catalog status in media list
2021-01-09 10:24:48 +01:00
c7926d8e8c
tape: split MediaSet into extra file
2021-01-09 08:54:58 +01:00
df69a4fc59
tape: implement drive clean
2021-01-08 11:32:56 +01:00
25d3965769
tape: correctly skip cleaning tapes (not regular tapes)
2021-01-08 09:16:42 +01:00
879569d73f
tape: changer transfer - make name parameter optional
2021-01-07 17:09:47 +01:00
b63f833d36
tape: fix paramater name - s/slot/source-slot/
2021-01-07 15:39:25 +01:00
482c6e33dd
tape: changer status command: make changer name optional
2021-01-07 15:12:19 +01:00
46a1863f88
tape: improve MediaChange trait
...
We expose the whole MtxStatus, and we can load/store from/to
specified slot numbers.
2021-01-07 14:26:43 +01:00
d5035c5600
tape: mtx_status - consider new export-slots property
2021-01-06 11:53:33 +01:00
38ae42b11a
tape: changer - add export-slot config
2021-01-06 11:06:50 +01:00
c4b2b9ab41
tape: only query volume stats if we can read MAM
2021-01-06 09:20:36 +01:00
ef942e04c2
tape: add function to classify tape-alert-flags
2021-01-05 17:23:30 +01:00
b40ab10d38
tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus
2021-01-05 13:43:17 +01:00
f8ccbfdedd
tape: implement read_volume_statistics
2021-01-05 12:58:18 +01:00
470f1c798a
tape: status - show thape alert flags
2021-01-04 13:15:30 +01:00
5c012b392a
tape: use LP 12h TapeAlert Response to query tape alert flags
2021-01-04 13:14:02 +01:00
165b641c1d
tape: changer status - show full slots (for cartridge without barcode)
2021-01-04 12:06:05 +01:00
66e42bec05
tape: further PoolWriter cleanups
2021-01-03 12:08:40 +01:00
c503ea7045
tape: cleanup - rename 'info' to 'media_id'
...
Second try.
2021-01-03 11:38:00 +01:00
745ec187ce
Revert "tape: cleanup - rename 'info' to 'media_id'"
...
This reverts commit f046313c0e
2021-01-03 11:14:58 +01:00
f046313c0e
tape: cleanup - rename 'info' to 'media_id'
2021-01-03 10:37:42 +01:00
74595b8821
tape: sg-tape-cmd tape-alert-flags
2021-01-03 10:09:43 +01:00
c9fdd142a4
tape: commit missing file
2021-01-02 13:39:34 +01:00
abaa6d0ac9
tape: decode TapeAlertFlags in cartridge-memory command
2021-01-02 10:55:30 +01:00
cfae8f0656
tape: merge MediaStateDatabase into Inventory
2021-01-01 16:15:13 +01:00
54f4ecd46a
tape: implement MediaPool flag to consider offline media
...
For standalone tape drives.
2021-01-01 10:03:59 +01:00
b9b4b31284
tape: add basic restore api/command
2020-12-31 10:26:48 +01:00
b4772d1c43
tape: new inventory helper - lookup_media_set_pool
2020-12-31 10:03:17 +01:00
08ac90f920
api: allow tokens to list users
...
their owner, or all if they have the appropriate privileges.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:49 +01:00
13f5863561
api: improve error messages for restricted endpoints
...
the old variant attempted to parse a tokenid as userid and returned the
cryptic parsing error to the client, which is rather confusing.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:09 +01:00
81764111fe
tape: media_change - log all errors
2020-12-30 19:17:18 +01:00
cb022525ff
tape: only log to stdout in CLI environment
2020-12-30 19:01:39 +01:00
75656a78c6
tape: improve inline docu
2020-12-30 17:28:33 +01:00
284eb5daff
tape: cleanup/simplify media_change code
2020-12-30 17:16:57 +01:00
ff58c51919
tape: improve media request/load
2020-12-30 13:09:28 +01:00
12299b333b
tape: set minimal media label length to 2
2020-12-30 10:15:02 +01:00
b017bbc441
tape: add restore code, implement catalog api/command
2020-12-30 09:48:18 +01:00
9e8c0d2e33
tape: cleanup - remove debug messages
2020-12-30 08:41:30 +01:00
250c29edd2
tape: correctly sort media api entries
2020-12-29 12:09:51 +01:00
c431659d05
cleanup: remove debug output
2020-12-29 11:59:57 +01:00
a33389c391
tape: implement media content list api
2020-12-29 11:58:26 +01:00
3460565414
tape: create the MediaCatalog when we label a tape
2020-12-29 10:55:20 +01:00
26b62138ee
cleanup: disable debug message when we detect a stopped worker task
2020-12-29 10:53:16 +01:00
afb0220642
tape: cleanup LinuxDriveStatus - make density optional
2020-12-29 09:10:30 +01:00
0993923ed5
tape: factor out get_drive_and_media_status
2020-12-29 08:39:06 +01:00
e0362b0d0f
tape: correctly parse mtx import/export slots
2020-12-28 13:32:56 +01:00
b27c32821c
tape: install new sg-tape-cmd setuid binary
2020-12-28 11:10:25 +01:00
76b15a035f
tape: MediaCatalog: write magic number before content
2020-12-26 11:05:25 +01:00
eb8feb1281
tape: add LTO1 to TapeDensity
2020-12-26 10:48:32 +01:00
fc6ce9835b
tape: fix non-rewinding tape device check
2020-12-25 15:38:29 +01:00
8ae9f4efc2
tape: minor cleanups
2020-12-25 13:45:26 +01:00
c9d13b0fc4
tape: expose check_tape_is_linux_tape_device
2020-12-24 15:51:49 +01:00
bfacc1d8c3
tape: cleanup - factor out open_linux_tape_device
2020-12-24 11:24:45 +01:00
5ae86dfaa1
tape: return media usage info with status command
2020-12-23 11:24:34 +01:00
dbe7e556b0
tape: implement binding for libsgutils2
...
So that we can read cartridge memory without calling "sg_raw". In future,
we may need further low level command to control the tape..
2020-12-23 09:44:53 +01:00
4799280ccd
http_client: add timeouts for critical connects
...
Use timeout futures for sections that might hang in certain error
conditions. This is mostly intended to be used as a safeguard, not a
first line of defense - i.e. best-effort avoidance of total hangs.
Not every future used for the HttpClient/H2Client is changed, only those
where a quick response is to be expected. For example, the response
reading futures are left alone, so data transfer is never capped with
timeout, only the initial server connect.
It is also used for upgrading to H2 connections, as that can take a long
time on overloaded servers.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-12-22 13:31:10 +01:00
cb80d900b3
tape: add drive status api
2020-12-22 10:42:22 +01:00
ee01737e87
tape: rename 'mam' api to 'cartridge-memory'
2020-12-22 09:27:34 +01:00
2012825913
depend on proxmox 0.9.0
2020-12-22 08:52:24 +01:00
b2362a1207
adaptions for proxmox 0.9 and proxmox-api-macro 0.3
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-22 07:31:05 +01:00
1e20f819d5
tape: add command to read cartridge memory (MAM)
...
Thsi add an additional dependency to sg3-utils (small).
2020-12-21 12:12:33 +01:00
9aa58f0143
cleanup: rename mtfsf into forward_space_count_files
2020-12-18 16:57:49 +01:00
8835664653
tape: add tape backup api
2020-12-18 15:32:12 +01:00
d37da6b7fc
tape: add PoolWriter
2020-12-18 15:27:44 +01:00
b9ee86efe1
tape: use SnapshotReader to create snapshot archive
2020-12-18 12:11:29 +01:00
d108b610fd
tape: fix write_media_set_label - move to correct position
2020-12-18 12:11:29 +01:00
0ec79339f7
tools/daemon: improve reload behaviour
...
it seems that sometimes, the child process signal gets handled
before the parent process signal. Systemd then ignores the
childs signal (finished reloading) and only after going into
reloading state because of the parent. this will never finish.
Instead, wait for the state to change to 'reloading' after sending
that signal in the parent, an only fork afterwards. This way
we ensure that systemd knows about the reloading before actually trying
to do it.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-12-18 10:30:37 +01:00
2afdc7f27d
tape: MediaPool::with_config() - remove name parameter
...
Not required, because config already contains the pool name.
2020-12-18 08:14:24 +01:00
26aa9aca40
tape: return current_file_number as u64
2020-12-18 07:44:50 +01:00
3e2984bcb9
tools/process_locker: Decrement writer count in drop handler
...
of ProcessLockSharedGuard.
We use a counter to determine if we can unlock the file again, but
we never actually decremented the writer count, so we held the
lock forever.
This fixes the issue that we could not start a garbage collect after
a reload, as long as the old process is still running, even when that
process has no active backup anymore but another long running task
(e.g. file download, terminal, etc.).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-18 07:15:08 +01:00
a7a5406c32
acl: rustfmt module
...
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:07:01 +01:00
4f727a783e
acl: reformat privileges
...
for better readability, and tell rustfmt to leave those definitions
alone.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:45 +01:00
23dc68fdea
acl: add docs and adapt visibility
...
document all public things, add some doc links and make some
previously-public things only available for test cases or within the
crate:
previously public, now private:
- AclTreeNode::extract_user_roles (we have extract_roles())
- AclTreeNode::extract_group_roles (same)
- AclTreeNode::delete_group_role (exists on AclTree)
- AclTreeNode::delete_user_role (same)
- AclTreeNode::insert_group_role (same)
- AclTreeNode::insert_user_role (same)
- AclTree::write_config (we have save_config())
- AclTree::load (we have config()/cached_config())
previously public, now crate-internal:
- AclTree::from_raw (only used by tests)
- split_acl_path (used by some test binaries)
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:11 +01:00
b532dd00c4
tape: add helper to read snapshot contents
...
- lock the snapshot for reading
- use openat to open files
- provides an iterator over all chunks
2020-12-17 13:07:52 +01:00
c01742855a
KeyConfig: bail on wrong fingerprint
...
instead of just logging the error. this should never happen in practice
unless someone is messing with the keyfile, in which case, it's better
to abort.
update tests accordingly (wrong fingerprint should fail, no fingerprint
should get the expected one).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 11:27:06 +01:00
9c953dd260
tape: add code to write backup snapshot files (without chunks) to tape
2020-12-17 08:28:47 +01:00
3fbf2d2fcd
tape: cleanup MediaCatalog
2020-12-17 08:05:53 +01:00
e0af222ec3
KeyConfig: always calculate fingerprint
...
and warn if stored and calculated fingerprint don't match.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:52:55 +01:00
73b5011786
KeyConfig: add encrypt/decrypt test
...
the RSA key and the encryption key itself are hard-coded to avoid
stalling the test runs because of lack of entropy, they have no special
significance otherwise.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:47:45 +01:00
7137630d43
client: add 'import-with-master-key' command
...
to import an encrypted encryption key using a master key.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:46:24 +01:00
8acfd15d6e
key: move RSA-encryption to KeyConfig
...
since that is what gets encrypted, and not a CryptConfig.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:43:34 +01:00
48fbbfeb7e
fix #3197 : skip fingerprint check when restoring key
...
when restoring an encrypted key, the original one is obviously not
available to check the fingerprint with.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:37:54 +01:00
9990af3042
master key: store blob name in constant
...
since we will use it in more than one place.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:36:06 +01:00
fe6c19383b
tape: remove MediaLabelInfo, use MediaId instead
...
The additional content_uuid was quite useless...
2020-12-16 13:31:32 +01:00
9839d3f778
tape: improve docu
2020-12-16 12:43:51 +01:00
dd59e3c2a1
tape: improve docu
2020-12-16 12:23:52 +01:00
0b7432ae09
tape: add chunk archive reader/writer
2020-12-16 12:08:34 +01:00
c1c2c8f635
tape: cleanup MediaLocation type for direct use with API
2020-12-16 10:49:01 +01:00
42298d5896
tape: add magic number to identify media catalog files
2020-12-16 09:00:14 +01:00
205e187613
tape: add MediaCatalog implementation
2020-12-15 13:40:49 +01:00
a78348acbb
tape: rename DriveLabel to MediaLabel
2020-12-14 17:37:16 +01:00
410611b4f2
tape: improve file format docu
2020-12-14 17:29:57 +01:00
af07ec8f29
tape: minor code cleanup
2020-12-14 16:56:26 +01:00
3f803af00b
tape: scan - print more debug info
2020-12-14 13:16:18 +01:00
ac461bd651
tape: implement scan command (useful for debug)
2020-12-14 12:55:49 +01:00
ce955e1635
tape: implement eod cli command (debug tool)
2020-12-14 09:56:59 +01:00
e20d008c6a
tape: rename cli 'media media-destroy' toö 'media destroy'
2020-12-14 09:30:32 +01:00
fb657d8ee5
tape: implement destroy_media
2020-12-14 08:58:40 +01:00
fba0b77469
tape: add media api
2020-12-14 07:55:57 +01:00
b5c1296eaa
tape: make changer get_status async
2020-12-14 07:14:24 +01:00
065df12872
tape: split api type definitions for changers into extra file
2020-12-13 09:31:02 +01:00
7e1d4712b8
tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA
2020-12-13 09:22:08 +01:00
49c965a497
tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA
2020-12-13 09:18:16 +01:00
6fe9aedd0b
tape: correctly call Async handler in proxmox-tape
2020-12-12 09:58:47 +01:00
42cb9bd6a5
tape: avoid executor blocking in changer api
2020-12-12 09:45:08 +01:00
66dbe5639e
tape: avoid executor blocking in drive API
...
By using tokio::task::spawn_blocking().
2020-12-12 09:20:04 +01:00
663ef85992
tape: use WorkerTask for erase and rewind
2020-12-11 11:19:33 +01:00
e92c75815b
tape: split inventory api
...
inventory: sync, list labels with uuids,
update_inventory: WorkerTask, updates database
2020-12-11 10:42:29 +01:00
6dbad5b4b5
tape: run label commands as WorkerTask (threads)
2020-12-11 09:10:22 +01:00
bff7e3f3e4
tape: implement barcode-label-mdedia
2020-12-11 07:50:19 +01:00
83abc7497d
tape: implement inventory command
2020-12-11 07:39:28 +01:00
be1a8c94ae
fix build: add missing file
2020-12-10 13:40:20 +01:00
4606f34353
tape: implement read-label command
2020-12-10 13:20:39 +01:00
7bb720cb4d
tape: implement label command
2020-12-10 12:30:27 +01:00
c4d8542ec1
tape: add media pool handling
2020-12-10 11:41:35 +01:00
9700d5374a
tape: add media pool cli
2020-12-10 11:13:12 +01:00
05e90d6463
tape: add media pool config api
2020-12-10 10:52:27 +01:00
55118ca18e
tape: correctly sort drive api subdir
2020-12-10 10:09:12 +01:00
f70d8091d3
tape: implement option changer-drive-id
2020-12-10 09:09:06 +01:00
a3c709ef21
tape: cli cleanup - avoid api redefinition
2020-12-10 08:35:11 +01:00
4917f1e2d4
tape: implement delete property for drive update command
2020-12-10 08:25:46 +01:00
93829fc680
tape: cleanup load-slot api
2020-12-10 08:04:55 +01:00
5605ca5619
tape: cli cleanup - rename scana-for-* into scan
2020-12-10 07:58:45 +01:00
e49f0c03d9
tape: implement load-media command
2020-12-10 07:52:56 +01:00
0098b712a5
tape: implement eject
2020-12-09 17:50:48 +01:00
5fb694e8c0
tape: implement rewind
2020-12-09 17:43:38 +01:00
583a68a446
tape: implement erase media
2020-12-09 17:35:31 +01:00
e6604cf391
tape: add command line interface proxmox-tape
2020-12-09 13:00:20 +01:00
43cfb3c35a
tape: do not remove changer while still used
2020-12-09 12:55:54 +01:00
8a16c571d2
tape: add changer property to drive create api
2020-12-09 12:55:10 +01:00
314652a499
tape: set protected flag for configuration change api methods
2020-12-09 12:02:55 +01:00
6b68e5d597
client: move connect_to_localhost into client module
2020-12-09 11:59:50 +01:00
cafd51bf42
tape: add media state database
2020-12-09 11:21:56 +01:00
9b93c62044
remove unused descriptions from api macros
...
these are now a hard error in the api macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-09 10:55:18 +01:00
5d90860688
tape: expose basic tape/changer functionality at api2/tape/
2020-12-08 15:42:50 +01:00
5ba83ed099
tape: check digest on config update
2020-12-08 11:24:38 +01:00
50bf10ad56
tape: add changer configuration API
2020-12-08 09:04:56 +01:00
16d444c979
tape: add tape drive configuration API
2020-12-07 13:04:32 +01:00
fa9c9be737
tape: add tape device driver
2020-12-07 08:29:22 +01:00
2e7014e31d
tape: add BlockeReader/BlockedWriter streams
...
This is the basic format used to write data to tapes.
2020-12-06 12:09:55 +01:00
a84050c1f0
tape: add BlockHeader impl
2020-12-06 10:26:24 +01:00
7c9835465e
tape: add helpers to emulate tape read/write behavior
2020-12-06 09:41:16 +01:00
ec00200411
fix bug #3189 : fix change_password permission checks, run protected
2020-12-05 16:20:29 +01:00
b107fdb99a
tape: add tape changer support using 'mtx' command
2020-12-05 14:54:12 +01:00
7320e9ff4b
tape: add media invenotry
2020-12-05 12:54:15 +01:00
c4d2d54a6d
tape: define useful constants
2020-12-05 12:20:46 +01:00
1142350e8d
tape: add media pool config
2020-12-05 11:59:38 +01:00
d735b31345
tape: add tape read trait
2020-12-05 10:54:38 +01:00
e211fee562
tape: add tape write trait
2020-12-05 10:51:34 +01:00
8c15560b68
tape: add file format definitions
2020-12-05 10:45:08 +01:00
327e93711f
commit missing file: tape api type definitions
2020-12-04 16:00:52 +01:00
a076571470
tape support: add drive configuration
2020-12-04 15:42:32 +01:00
179145dc24
backup/datastore: move manifest locking to /run
...
this fixes the issue that on some filesystems, you cannot recursively
remove a directory when you hold a lock on a file inside (e.g. nfs/cifs)
it is not really backwards compatible (so during an upgrade, there
could be two daemons have the lock), but since the locking was
broken before (see previous patch) it should not really matter
(also it seems very unlikely that someone will trigger this)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 09:56:42 +01:00
6bd0a00c46
backup/datastore: really lock manifest on delete
...
'lock_manifest' returns a Result<File, Error> so we always got the result,
even when we did not get the lock, but we acted like we had.
bubble the locking error up
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 14:37:05 +01:00
f6e28f4e62
client/pull: log how many groups to pull were found
...
if no groups were found, the task log was very confusing as it
contained no real information why nothing was synced, e.g.:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
this patch simply logs how many groups were found and are about to be synced:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
found 0 groups to sync
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 07:22:50 +01:00
2260f065d4
cleanup: use extra file for StoreProgress
2020-12-01 06:34:33 +01:00
6eff8dec4f
cleanup: remove unnecessary StoreProgress clone()
2020-12-01 06:29:11 +01:00
7e25b9aaaa
verify: use same progress as pull
...
percentage of verified groups, interpolating based on snapshot count
within the group. in most cases, this will also be closer to 'real'
progress since added snapshots (those which will be verified) in active
backup groups will be roughly evenly distributed, while number of total
snapshots per group will be heavily skewed towards those groups which
have existed the longest, even though most of those old snapshots will
only be re-verified very infrequently.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:55 +01:00
f867ef9c4a
progress: add format variants
...
for iterating over a single group, or iterating just on the group level
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:12 +01:00
fc8920e35d
pull: factor out interpolated progress
...
and add group/snapshot count info.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:13:11 +01:00
7f3b0f67e7
remove BackupGroup::list_groups
...
BackupInfo::list_backup_groups is identical code-wise, and makes more
sense as entry point for listing groups.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:09:44 +01:00
844660036b
gc: don't limit index listing to same filesystem
...
WalkDir does not follow symlinks by default anyway, and this behaviour
is not documented anywhere. e.g., if a sysadmin mounts 'extra storage'
for some backup group or type (not knowing that only metadata is stored
in those directories), GC will ignore all the indices contained within
and happily garbage collect their chunks..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:07:09 +01:00
efcac39d34
gc: remove duplicate variable
...
list_images already returns absolute paths, we don't need to prepend
anything.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:51 +01:00
cb4b721cb0
gc: log index files found outside of expected scheme
...
for safety reason, GC finds and marks all index files below the
datastore base path. as a result of regular operations, only index files
within the expected scheme of <TYPE>/<ID>/<TIMESTAMP> should exist.
add a small check + warning if the index list contains index files out
side of this expected scheme, so that an admin with shell access can
investigate.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:17 +01:00
7956877f14
gc: shorten progress messages
...
we have messages starting the phases anyway, and limit the number of
progress updates so that context remains available at all times.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:04:13 +01:00
43e60ceb41
file logger: remove test.log after test as well
...
and a doc formatting fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:13:21 +01:00
b760d8a23f
derive PartialEq for Userid
...
the manual implementation is equivalent
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:10:17 +01:00
2c1592263d
tiny clippy hint
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:03:43 +01:00
616533823c
don't enforce Vec and String in tools::join
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:56:59 +01:00
913dddea85
minor cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:56:21 +01:00
3530430365
tools avoid unnecessary copying of parameters/properties
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:53:49 +01:00
a4ba60be8f
minor cleanups
...
whitespace, formatting and superfluous lifetime annotations
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:47:31 +01:00
99e98f605c
network helpers: fix fd leak in get_network_interfaces
...
This one always leaked.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
935ee97b17
use fd_change_cloexec helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
6b9bfd7fe9
minor cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
dd519bbad1
pxar: stricter file descriptor guards
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
35fe981c7d
client: use tools::pipe instead of nix
...
nix::unistd::pipe returns unguarded RawFds which should be
avoided
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
b6570abe79
changes for proxmox 0.8
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
490560e0c6
restore: print to STDERR
...
else restoring to STDOUT is broken..
Reported-by: Dominic Jäger <d.jaeger@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 14:38:02 +01:00
0c9209b04c
cli: rename command "upload-log" to "snapshot upload-log"
2020-11-25 07:57:39 +01:00
edebd52374
cli: rename command "forget" to "snapshot forget"
2020-11-25 07:57:39 +01:00
61205f00fb
cli: rename command "files" to "snapshot files"
2020-11-25 07:57:39 +01:00
a303e00289
fingerprint: add new() method
2020-11-25 07:57:39 +01:00
af9f72e9d8
fingerprint: add bytes() accessor
...
needed for libproxmox-backup-qemu0
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 06:34:34 +01:00
731eeef25b
cli: use new alias feature for "snapshots"
...
Now maps to "snapshot list".
2020-11-24 13:26:43 +01:00
a65e3e4bc0
client: add 'snapshot notes show/update' command
...
to show and update snapshot notes from the cli
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 11:44:19 +01:00
035c40e638
list_snapshots: return manifest fingerprint
...
for display in clients.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
79c535955d
refactor BackupInfo -> SnapshotListItem helper
...
before adding more fields to the tuple, let's just create the struct
inside the match arms to improve readability.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
8b7f8d3f3d
expose previous backup time in backup env
...
and use this information to add more information to client backup log
and guide the download manifest decision.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
23e4e90540
verification: fix message in notification mail
...
the errors Vec can contain failed groups as well (e.g., if a group has
no or an invalid owner).
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:33:20 +01:00
a4fa3fc241
verification job: log failed dirs
...
else users have to manually search through a potentially very long task
log to find the entries that are different.. this is the same summary
printed at the end of a manual verify task.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:33:20 +01:00
Dietmar Maurer
Wolfgang Bumiller
Oguz Bektas
Fabian Grünbichler
Thomas Lamprecht
Stefan Reiter
Dominik Csapak