Commit Graph

6581 Commits

Author SHA1 Message Date
Dominik Csapak 0bd9c87010 datastore: lookup: reuse ChunkStore on stale datastore re-open
When re-opening a datastore due to the cached entry being stale
(config change) but also if the last re-open was >60s ago). On
datastore open the chunk store was also re-opened, which in turn
creates a new ProcessLocker, loosing any existing shared lock which
can cause conflicts between long running (24h+) backups  and GC.

To fix this, reuse the existing ChunkStore, and thus  its
ProcessLocker, when creating a up-to-date datastore instance on
lookup, since only the datastore config should be reloaded. This is
fine as the ChunkStore path is not updatable over our API.

This was always a potential issue but got exposed in practice by
commit 118deb4db8 which introduced the
unconditional "re-open after 60s" mechanism.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ T: reword commit message a bit and reference commit that made the
   issue much more likely ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 17:00:49 +02:00
Thomas Lamprecht fbfb64a6b2 tree wide: clippy lint fixes
most (not all) where done automatically

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:59:55 +02:00
Thomas Lamprecht c39852abdc client: clippy lints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:57:33 +02:00
Thomas Lamprecht 1ec167ee8c api types: clippy lints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:57:07 +02:00
Fabian Grünbichler 11ca834317 update to nix 0.24 / rustyline 9 / proxmox-sys 0.3
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-02 14:33:33 +02:00
Fabian Grünbichler 68a6e970d4 bump tokio-util to 0.7
along with the rest of tokio/futures/hyper/openssl being updated - this
is the only one we explicitly depend on that had a non-compatible
version number.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-02 09:41:38 +02:00
Thomas Lamprecht 4e851c26a2 bump version to 2.2.2-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 17:00:02 +02:00
Thomas Lamprecht ceb815d295 server: remove jobstate: ignore removal error due to file not found
we want to remove lock and state file anyway, so not found is all
right

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 16:40:09 +02:00
Thomas Lamprecht 14433718fb bump version to 2.2.2-1
same story as last time

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 15:09:43 +02:00
Thomas Lamprecht 3dc8783af7 manager cli: output more info when transforming prune jobs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 15:09:20 +02:00
Thomas Lamprecht 6d89534929 bump version to 2.2.2-1
re-bump for small fixes discovered before any upload

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:34:03 +02:00
Thomas Lamprecht aa19d5b917 manager cli: output more info when skipping prune tranforms
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:31:53 +02:00
Thomas Lamprecht a8d3f1943b api types: prune keep options: also check weekly in keeps_something
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:30:24 +02:00
Thomas Lamprecht 3cf12ffac9 bump version to 2.2.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 13:04:37 +02:00
Fabian Grünbichler 2017a47eec Cargo.toml: add missing patch sections
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-06-01 11:01:23 +02:00
Thomas Lamprecht 21185350fb ui: add prune job worker task description and renderer
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-31 13:11:23 +02:00
Thomas Lamprecht 17b079918e ui: prune & gc: relay activate/deactivate events to sub panels
which allows us also to drop the initial manual load in the init,
which would also trigger if the tab isn't visible.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-31 10:02:12 +02:00
Thomas Lamprecht fbfc439372 ui: system config: improve bottom margins and scroll behavior
setting scrollable on the parent tab panel makes not much sense and
will always add a scroll bar that can scroll a few pixels, even if
there's enough space.
Rather set it to true (= auto) in the actual panels that hold the
content.

Also set a bottom margin so that users can see the "end" of the panel
at the bottom, otherwise it looked like it had a start and sides, but
no bottom.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-31 07:03:31 +02:00
Thomas Lamprecht 27d3a232d0 ui: prune jobs: avoid duplicate params through nested input panels
input panel collect all form fields below them, so nesting two
input panels needs a bit of special care to avoid that each of the
panels adds the data of the deeper nested ones, resulting in
duplicate parameters that the backend then chokes one.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 15:15:42 +02:00
Thomas Lamprecht 1fa6083bc8 ui: prune & gc: disallow collapse and add bottom margin
the intra-panel margin is still the same (10 + 0 == 7 + 3) but one
can now see the bottom border.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 15:02:08 +02:00
Wolfgang Bumiller aa32a46171 api: disable setting prune options in datastore.cfg
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:48:15 +02:00
Wolfgang Bumiller 6283d7d13a stop executing datastore prune job
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:47:57 +02:00
Wolfgang Bumiller d4dd7ac842 api: don't use PRUNE perms for prune jobs
just stick to MODIFY so we don't need to give the prune jobs
an owner for now

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:33:06 +02:00
Wolfgang Bumiller 451da4923b drop unused import
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 14:01:22 +02:00
Thomas Lamprecht f15e094408 d/postinst: transform prune tasks from datastore cfg to new prune job
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 13:58:45 +02:00
Wolfgang Bumiller 134779664e manager: hidden command to move datastore prune opts into jobs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Thomas Lamprecht 9ce2f903fb ui: rework prune job view/edit
Fix missing load on initial view, re-use the prune input panel for
editing and avoid using a tab panel for a single tab, rework also
some columns widths and various other small parts-

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 13:58:43 +02:00
Thomas Lamprecht 6802a68356 ui: re-integrate prune into prune & GC panel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller c69884a459 ui: add ui for prune jobs
similar to verification/sync jobs, the prune settings on the
datastore are deprecated

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller 93205cbe92 tests: switch to PruneJobOptions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller 434dd3cc84 client: switch to PruneJobsOptions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller dba37e212b add prune jobs api
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller db4b8683cf add prune job config
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller 5557af0efb api-types: add PruneJobConfig
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller 8721b42e2f api: add some missing sorted macro calls
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Thomas Lamprecht 5408e30ab1 d/postinst: fix upper version for applying sync.cfg remove-vanished default
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-30 13:40:24 +02:00
Thomas Lamprecht 70493f1823 ui: datastore content: better cope with restricted privs on parent namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 16:09:48 +02:00
Thomas Lamprecht 069720f510 ui: datastore content: only mask the treeview, not the top bar
so that an user can try to reload again easily for non-persistent
errors

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 16:06:21 +02:00
Thomas Lamprecht a93c96823c ui: datastore content: avoid duplicate masking on load error
we already handle that manually in the onLoad and want to further
extend that, so drop the more generic monStoreError

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 16:02:44 +02:00
Thomas Lamprecht 2393943fbb api: namespace list: fix restrictive priv checking
This endpoint only lists all accessible namespace, and one doesn't
necessarily needs to have permissions on the parent itself just to
have OK ACLs on deeper down NS.

So, drop the upfront check on parent but explicitly avoid leaking if
a NS exists or not, i.e., only do so if they got access on the parent
NS.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 11:14:01 +02:00
Thomas Lamprecht 49d604aec1 ui: datastore options: avoid breakage if rrd store cannot be queried
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 10:59:42 +02:00
Thomas Lamprecht 246275e203 ui: datastore options: avoid breakage if active-ops cannot be queried
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 10:59:25 +02:00
Thomas Lamprecht c9fb0f3887 ui: datastore summary: cope with optional gc-stats
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 10:58:38 +02:00
Thomas Lamprecht 84de101272 api: status: include empty entry for stores with ns-only privs
I.e., for those that only got permissions on a sub namespace and
those that onlöy got BACKUP_READ, as both they could just list and
count themselves too after all, so not exactly secret info.

The UI needs some adaptions to cope with gc-stats and usage being
optional, will be done in a next commit.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-27 10:55:48 +02:00
Thomas Lamprecht de77a20d3d api: move can_access_any_namespace helper to hierarchy
to prepare for reuse

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 13:35:33 +02:00
Thomas Lamprecht 997c96d6a3 datastore status: impl empty-status constructor for item type
we can now use it for the error case and will further use it for the
can access namespace but not datastore case in a future patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 13:34:00 +02:00
Thomas Lamprecht 513da8ed10 docs: fix yet another typo
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 13:26:56 +02:00
Thomas Lamprecht e87e4499fd docs: fix some typos
The s/Namesapce/Namespace/ one was reported in the forum [0] and so I
figured I do a quick scan for others too using codespell.

[0]: https://forum.proxmox.com/threads/109724/post-472744

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 13:08:52 +02:00
Thomas Lamprecht a19b8c2e24 pbs-config: clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 11:42:13 +02:00
Thomas Lamprecht b8858d5186 datastore: avoid unsafe transmute, use to_ne_bytes
which is stable since rustc 1.32 but wasn't available in out
toolchain when this was originally written in commit 7bc1d7277

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 11:42:13 +02:00