Commit Graph

6606 Commits

Author SHA1 Message Date
Thomas Lamprecht 2ad96e1635 api: tape restore: split/rework datastore/namespace map implementation
The split out helpers will (partially) be used in later patches for
call sites where we only need parts of the info assembled here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 16:47:27 +02:00
Thomas Lamprecht 7bc2e240b1 api: tape restore: use HumanByte for friendlier total/throughput reporting
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:45:13 +02:00
Thomas Lamprecht 20a04cf07c api: tape restore: refactor some code parts shorter
not wanting to play code golf here, but bloat in code makes it often
also harder to read, so try to reduce some of that without making it
to terse.

No semantic change intended.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:42:47 +02:00
Thomas Lamprecht a40ffb92ac code formatting fixups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:38:33 +02:00
Thomas Lamprecht e2aeff40eb tape: use inline variable in formats for code reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:38:33 +02:00
Thomas Lamprecht d20137e5a9 tree wide: typo fixes through codespell
Most, not all, found and fixes using `codespell -wci3 -L crate`

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:34:10 +02:00
Thomas Lamprecht 6a35698796 bump version to 2.2.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 16:30:20 +02:00
Thomas Lamprecht 2981cdd4c0 api: datastore status: use cheaper any_privs_below over can_access_any_namespace
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:34:42 +02:00
Thomas Lamprecht 8c9c6c0755 api: list datastore: avoid iterating over NS for priv check, use AclTree
Make the assumption that if a user has any privilege that would make
an NS and (parts) of its content visible they also should be able to
know about the datastore and very basic errors on lookup (path
existence and maintenance mode) even if that NS doesn't even exists
(yet), as they could, e.g., make or view a backup and find out
anyway.

This avoids iterating over parts of the whole datastore folder tree
on disk, doing a priv check on each, swapping IO to virtual in memory
checks on info we got available already anyway, is always a good idea
after all

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:34:42 +02:00
Thomas Lamprecht 2c69b69108 config: cached user info: expose new any_privs_below
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:29:45 +02:00
Thomas Lamprecht 0bed1f2956 config: any_priv_below: plural name & switch to slice of &str for path
s/any_priv_below/any_privs_below/ for consistency and switch from a
single &str for the path param to the slice-ref string variant, as
that allows to use it more often without allocation.

Also allow passing the whole path as single &str element in the slice
by splitting each component on '/' like we do in other parts
nowadays. Note though that we need to omit the leading slash then.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:29:45 +02:00
Thomas Lamprecht 4ef6b7d1f0 config: s/propagating/only_propagated/ and style nits
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:29:45 +02:00
Stefan Sterz 87d8aa4278 pbs-config: acl-tree: add any_priv_below
`any_priv_below()` checks if a given AuthId has any given privileges
on a sub-tree of the AclTree. to do so, it first takes into account
propagating privileges on the path itself and then uses a depth-first
search to check if any of the provided privileges are set on any
node of the sub-tree pointed to by the path.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:29:45 +02:00
Thomas Lamprecht 51d900d187 datastore: swap ConfigVersionCache with digest for change detection
We got the digest available anyway, and it's only 16 bytes more to
save (compared to last_generation and the recently removed last_time,
both being 64 bit = 8 bytes each)

Side benefit, we detect config changes made manually (e.g., `vim
datacenter.cfg`) immediately.

Note that we could restructure the maintenance mode checking to only
be done after checking if there's a cached datastore, in which case
using the generation could make sense to decide if we need to re-load
it again before blindly loading the config anyway. As that's not only
some (not exactly hard but not really trivial like a typo fix either)
restructuring work but also means we'd lose the "detect manual
changes" again I'd rather keep using the digest.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-04 15:26:50 +02:00
Thomas Lamprecht 519ca9d010 datastore: make unsafe fn public again, useful for example/test
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 17:10:17 +02:00
Thomas Lamprecht 615a50c108 datastore: make unsafe functions only visible in their own crate
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 13:42:42 +02:00
Thomas Lamprecht f418f4e48b api: list datastores: avoid unsafe datastore open
to avoid the problematic open fresh datastore with fresh chunkstore
with, and that's the actual problematic part, fresh process locker.
As the latter uses posix record locks which are pretty dangreous as
they operate on a path level (not FD level) and thus closing any file
opened (even if it wasn't opened for locking at all) drops all active
locks on the same file on completely unrelated file descriptors -.-

Also, no operation wasn't exactly correct for this thing in the first
place, but we cannot use Operation::Lookup either, as we're currently
indeed using a rather stupid-simple way and *are* reading.

So until we optimize this to allow querying the AclTree if there's
any priv XYZ below a path, use the Operation::Read.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 13:31:29 +02:00
Thomas Lamprecht c66fa32c08 datastore: add safety doc comment for unsafe opens
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 10:58:33 +02:00
Thomas Lamprecht 2515ff35c2 datastore: reduce chunk store open visibility and comment pitfalls
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 10:15:41 +02:00
Thomas Lamprecht 33a1ef7aae datastore: rename non-telling `map` to `datastore_cache`
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 10:11:09 +02:00
Thomas Lamprecht 9c12e82006 datastore: drop bogus last_update stale-cache mechanism
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 10:04:16 +02:00
Thomas Lamprecht 9f19057036 config: version cache: fix ordering of datastore generation increase
Fixes: 118deb4d (pbs-datastore: use ConfigVersionCache for datastore)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-03 09:18:09 +02:00
Thomas Lamprecht c7f7236b88 datastore: more concise comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 17:48:08 +02:00
Thomas Lamprecht fdefe192ac bump version to 2.2.2-3
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 17:38:52 +02:00
Thomas Lamprecht 1ed8698b7e docs: faq: more specific eol date
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 17:38:52 +02:00
Dominik Csapak 0bd9c87010 datastore: lookup: reuse ChunkStore on stale datastore re-open
When re-opening a datastore due to the cached entry being stale
(config change) but also if the last re-open was >60s ago). On
datastore open the chunk store was also re-opened, which in turn
creates a new ProcessLocker, loosing any existing shared lock which
can cause conflicts between long running (24h+) backups  and GC.

To fix this, reuse the existing ChunkStore, and thus  its
ProcessLocker, when creating a up-to-date datastore instance on
lookup, since only the datastore config should be reloaded. This is
fine as the ChunkStore path is not updatable over our API.

This was always a potential issue but got exposed in practice by
commit 118deb4db8 which introduced the
unconditional "re-open after 60s" mechanism.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: reword commit message a bit and reference commit that made the
   issue much more likely ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 17:00:49 +02:00
Thomas Lamprecht fbfb64a6b2 tree wide: clippy lint fixes
most (not all) where done automatically

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:59:55 +02:00
Thomas Lamprecht c39852abdc client: clippy lints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:57:33 +02:00
Thomas Lamprecht 1ec167ee8c api types: clippy lints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:57:07 +02:00
Fabian Grünbichler 11ca834317 update to nix 0.24 / rustyline 9 / proxmox-sys 0.3
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-02 14:33:33 +02:00
Fabian Grünbichler 68a6e970d4 bump tokio-util to 0.7
along with the rest of tokio/futures/hyper/openssl being updated - this
is the only one we explicitly depend on that had a non-compatible
version number.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-02 09:41:38 +02:00
Thomas Lamprecht 4e851c26a2 bump version to 2.2.2-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 17:00:02 +02:00
Thomas Lamprecht ceb815d295 server: remove jobstate: ignore removal error due to file not found
we want to remove lock and state file anyway, so not found is all
right

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 16:40:09 +02:00
Thomas Lamprecht 14433718fb bump version to 2.2.2-1
same story as last time

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 15:09:43 +02:00
Thomas Lamprecht 3dc8783af7 manager cli: output more info when transforming prune jobs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 15:09:20 +02:00
Thomas Lamprecht 6d89534929 bump version to 2.2.2-1
re-bump for small fixes discovered before any upload

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:34:03 +02:00
Thomas Lamprecht aa19d5b917 manager cli: output more info when skipping prune tranforms
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:31:53 +02:00
Thomas Lamprecht a8d3f1943b api types: prune keep options: also check weekly in keeps_something
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:30:24 +02:00
Thomas Lamprecht 3cf12ffac9 bump version to 2.2.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 13:04:37 +02:00
Fabian Grünbichler 2017a47eec Cargo.toml: add missing patch sections
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-01 11:01:23 +02:00
Thomas Lamprecht 21185350fb ui: add prune job worker task description and renderer
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-31 13:11:23 +02:00
Thomas Lamprecht 17b079918e ui: prune & gc: relay activate/deactivate events to sub panels
which allows us also to drop the initial manual load in the init,
which would also trigger if the tab isn't visible.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-31 10:02:12 +02:00
Thomas Lamprecht fbfc439372 ui: system config: improve bottom margins and scroll behavior
setting scrollable on the parent tab panel makes not much sense and
will always add a scroll bar that can scroll a few pixels, even if
there's enough space.
Rather set it to true (= auto) in the actual panels that hold the
content.

Also set a bottom margin so that users can see the "end" of the panel
at the bottom, otherwise it looked like it had a start and sides, but
no bottom.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-31 07:03:31 +02:00
Thomas Lamprecht 27d3a232d0 ui: prune jobs: avoid duplicate params through nested input panels
input panel collect all form fields below them, so nesting two
input panels needs a bit of special care to avoid that each of the
panels adds the data of the deeper nested ones, resulting in
duplicate parameters that the backend then chokes one.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 15:15:42 +02:00
Thomas Lamprecht 1fa6083bc8 ui: prune & gc: disallow collapse and add bottom margin
the intra-panel margin is still the same (10 + 0 == 7 + 3) but one
can now see the bottom border.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 15:02:08 +02:00
Wolfgang Bumiller aa32a46171 api: disable setting prune options in datastore.cfg
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:48:15 +02:00
Wolfgang Bumiller 6283d7d13a stop executing datastore prune job
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:47:57 +02:00
Wolfgang Bumiller d4dd7ac842 api: don't use PRUNE perms for prune jobs
just stick to MODIFY so we don't need to give the prune jobs
an owner for now

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:33:06 +02:00
Wolfgang Bumiller 451da4923b drop unused import
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:01:22 +02:00
Thomas Lamprecht f15e094408 d/postinst: transform prune tasks from datastore cfg to new prune job
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 13:58:45 +02:00