Commit Graph

6493 Commits

Author SHA1 Message Date
Thomas Lamprecht e22ad28302 GC scheduling: avoid triggering operation tracking error for upfront checks
without that one gets a "failed to lookup datastore X" in the log for
every datastore that is in read-only or offline maintenance mode,
even if they aren't scheduled for GC anyway.

Avoid that by first opening the datastore through a Lookup operation,
and only re-open it as Write op once we know that GC needs to get
scheduled for it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Thomas Lamprecht 0408f60b58 datastore: add new Lookup for operations tracking
We sometimes need to do some in-memory only stuff, e.g., to check if
GC is already running for a datastore, which is a try_lock on a mutex
that is in-memory.

Actually the whole thing would be nicer if we could guarantee to hold
the correct contract statically, e.g., like
https://docs.rust-embedded.org/book/static-guarantees/design-contracts.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Hannes Laimer d4d730e589 proxy: rrd: skip update disk stats for offline datastores
RDD update did not use lookup_datastore() and therefore bypassed
the maintenance mode checks. This adds the needed check directly.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-05-12 11:36:56 +02:00
Wolfgang Bumiller 5b460ef525 client: add --ns parameters to snapshot commands
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 11:02:06 +02:00
Wolfgang Bumiller 03d4f43d5a client: rename --backup-ns to --ns in backup command
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 10:43:56 +02:00
Stoiko Ivanov 5225817de6 docs: zfs: update documentation about ZED
This closely follows commit aa425868069818167ff0a3cca5c64a2acc88173e
in pve-docs.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 0ae5f76277 docs: local-zfs: minor cleanup and adaptation
fixes a few small glitches in the markup.

rephrases a few PVEisms (PBS will not swap when starting a backup to
an external storage)

add zstd to available compression algorithms

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 09d903034f docs: system-booting: (re)add screenshots
add the grub+systemdboot screen from a PBS system (taken via
spice-viewer).

The alingment of left/right looked better to me than keeping both on the
right).

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 9eb804006c docs: add system-booting from pve-docs
and transform to reST.

semantic changes to the content are:
* s/{pve}/`Proxmox Backup`_/g
* changing footnotes to parenthesized notes (did not see footnote use in
  the current docs)
* removed the comment about systems setup before the introduction of
  p-b-t (which was introduced before pbs)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 6e3391c85b docs: sysadmin: adapt kernel-specifics for PBS
while all statements here are technically true - adding all
virtualization improvements is not relevant for proxmox backup in most
cases.
The intel nic driver seems like a left-over from a time (pre PVE 5.1)
where the pve-kernel included the out-of-tree drivers.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Thomas Lamprecht 71139be203 bump version to 2.1.9-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:48:51 +02:00
Thomas Lamprecht fbca018229 ui: content: code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 7e8b24bd8c ui: content: show namespaces also inline and rework node type detection
this not only makes the action disable/hide checks simpler, but also
prepares the view a bit for the idea of adding a new API endpoint
that returns the whole datastore content tree as structured JSON so
that it can be directly loaded into a tree store.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht fe79687c59 pull group: add error context for cleanup_unreferenced_files
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 9ccf933be5 datastore: move update_manifest into BackupDir impl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 87cdc327b9 sync: pull snapshot: use template variables for bloat reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 5566099849 datastore: move cleanup_unreferenced_files to BackupDir impl and fix NS awareness
sync failed on cleanup due to always trying to do so in the root NS

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 92b9cc1554 ui: remote target selectors: code cleanups
just a small start...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 0e3de42aa7 ui: sync job: use namespace selector for localNS
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 8c29bca57c ui: move remote target datastore/ns selectors to own file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Fabian Grünbichler d895b26bb9 ui: add namespace fields to sync
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Fabian Grünbichler c06c1b4bd7 sync/pull: make namespace aware
Allow pulling all groups from a certain source namespace, and
possibly sub namespaces until max-depth, into a target namespace.

If any sub-namespaces get pulled, they will be mapped relatively from
the source parent namespace to the target parent namespace.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 31aa38b684 ui: verify job: fix add-job on datastore-agnostic level
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 9d8090626c ui: namespace selector: allow to set datastore dynamically
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6b4d057370 api-types: rework BackupNamespace::map_prefix
to use slice::strip_prefix() from std

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 53d073ec1a datastore: minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 30ccc3003e datastore: relative path fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht bc4af01559 ui: datastore content: make verify-all more flexible
allow to specify the namespace, max_depth and also the re-verify/skip
behavior.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht d83ce0d0c7 ui: fix group backup comment NS awareness
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht ad7741a294 ui: verify job: make namespace and max-depth aware
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht a327f918af ui: add verifyOutdatedAfter component
mainly as separate component for the trigger

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 0b1edf297b verify job: support max-depth config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 59229bd7f1 api: verify: support namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 8e82cc807c add ns-recursive and acl/authid aware backup group iter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler d4037525a8 remote scan/completion: add namespace support
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 40d495de6d api: add DatastoreWithNamespace helper struct
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler d3a570eb79 ui: fix wrong call to htmlEncode
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 9f8fb928f1 ui: add namespace renderer
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 226a4e68da client: add basic namespace commands
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 473063e9ec api: ns management: fix permission checks
we do not have normal GET variables available in the checks provided
by the rest server from the api macro, so do it manually.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 93b0659ff2 ui: datastore: more NS awareness
verify is actually not yet ready in the backend

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht e8112eb37b ui: datastore content: show root node for better UX with NS
that way it's easier to see on which NS one currently operates and
allows better distinguishing of root NS and some sub ns named "Root"

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6f5753cfa3 api-types: allow empty namespace
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 3c09413a0a client: don't pass empty backup-ns
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 028346e42c ui: content view: improve empty text
reference NS so that users get a hint where they are currently
hierarchy-wise, and clarify that we found no *accessible* snapshots,
on this level, i.e., there can be some that we just cannot see due to
only having access on lover level NS or being different owners.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler bc06c7b4e9 api: namespace: return popped component
helpful for places where namespaces need to be (re)created

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 7a404dc53d api: datastore: further unify check helpers
this is the most common sequence of checks we have in this file, so
let's have a single place where we implement it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler c939698414 api: datastore: load datastore & check owner helper
these happen together very often.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 1909ece229 api: datastore: lookup after checking privs
else this could leak existence of datastore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 2bc2435a96 api: datastore: refactor priv checks
the helper now takes both high-privilege and lesser-privilege privs, so
the resulting bool can be used to quickly check whether additional
checks like group ownership are needed or not.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00