Commit Graph

6365 Commits

Author SHA1 Message Date
Thomas Lamprecht f07e660153 ui: move max NS prefix length logic to reduced max-depth selector
for better re-usability in the future and it felt a bit odd to have
such specific logic in the sync job edit directly

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 13:15:02 +02:00
Thomas Lamprecht addcb7803e datastore: inline some format variables
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:42:41 +02:00
Thomas Lamprecht 1ddfae5499 api types: set NS_MAX_DEPTH schema default to MAX_NAMESPACE_DEPTH
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:32:25 +02:00
Thomas Lamprecht 54d315c951 ui: group filter: make also local filter NS aware
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:32:25 +02:00
Fabian Grünbichler 9dde8cd625 ui: sync: add reduced max-depth selector
that allows setting the limit based on sync namespace prefix lengths.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:20:29 +02:00
Fabian Grünbichler 87be232d1c pull/sync: clamp (local) max-depth if unset
to handle the unlikely case of `ns` being deeper than `remote-ns`,
`max-depth` being set to `None` and a too-deep sub-ns of `ns` existing.
such a sub-ns cannot have been created by a previous run of this sync
job, so avoid unexpectedly removing it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler e40c7fb906 api: split max-depth schema/types
into the regular one (with default == MAX) and the one used for
pull/sync, where the default is 'None' which actually means the remote
end reduces the scope of sync automatically (or, if needed,
backwards-compat mode without any remote namespaces at all).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 66abc4cb7d namespaces: move max-depth check to api type
and use it when creating a sync job, and simplify the check on updating
(only check the final, resulting config instead of each intermediate
version).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 11567dfbad pull/sync: correctly query with remote-ns as parent
else (grand)-parents and siblings/cousins of remote-ns are also
included, and mapping the remote-ns prefix fails.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 7a3e777ded pull/sync: detect remote lack of namespace support
and fall back to only syncing the root namespace, if possible. the sync
job will still be marked as failed to prompt the admin to resolve the
situation:
- explicitly mark the job as syncing *only* the root namespace
- or upgrade remote end to support namespaces

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 17:00:38 +02:00
Fabian Grünbichler b9310489cf pull/sync: treat unset max-depth as full recursion
to be consistent with tape backup and verification jobs.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 17:00:38 +02:00
Fabian Grünbichler d9aad37f2f pull: pass params as non-ref in pull_store
so that it's possible to modify them in-place without cloning.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 16:26:26 +02:00
Thomas Lamprecht 2a088b9975 datastore: drop bogus chunk size check, can cause trouble
other sizes can happen in legitimate and illegitimate ways:
 - illegitimate: encryped chunks and bad actor client
 - legitimate: same chunk but newer zstd version (or compression
   level) can compress it better (or worse) so the

Ideally we could take the actual smaller chunk so that improved zstd
tech gets leveraged, but we could only allow to do that for
un-encrypted ones.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 15:41:20 +02:00
Thomas Lamprecht 78e1ee5230 bump version to 2.1.9-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 14:28:16 +02:00
Dominik Csapak c7d42dac97 ui: navigation tree: fix losing datastore selection on store load
instead of using 'replaceChild', simply set the appropriate
properties. When using the 'nodeUpdate' (protected function of extjs,
intended to be overwritten) instead of the private 'updateNode', it
will be called when the properties change

This way, the treenode stays the same and it can keep the selection

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-12 14:28:16 +02:00
Wolfgang Bumiller 8ca7cccf5f file-restore: add namespace support to qemu part
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 13:35:34 +02:00
Thomas Lamprecht e30a2e9058 ui: content: fix various tree-checks from action handlers
they all still used some odd side effects of the tree structure to
decided what record type they operated on, just move them over to the
new `ty` record.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 13:28:20 +02:00
Thomas Lamprecht 08982a3746 rest: example: fix comment width
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:57:51 +02:00
Thomas Lamprecht 42fb291c7c cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:54:21 +02:00
Fabian Ebner e9b9f33aee rest server: daemon: update PID file before sending MAINPID notification
There is a race upon reload, where it can happen that:
1. systemd forks off /bin/kill -HUP $MAINPID
2. Current instance forks off new one and notifies systemd with the
   new MAINPID.
3. systemd sets new MAINPID.
4. systemd receives SIGCHLD for the kill process (which is the current
   control process for the service) and reads the PID of the old
   instance from the PID file, resetting MAINPID to the PID of the old
   instance.
5. Old instance exits.
6. systemd receives SIGCHLD for the old instance, reads the PID of the
   old instance from the PID file once more. systemd sees that the
   MAINPID matches the child PID and considers the service exited.
7. systemd receivese notification from the new PID and is confused.
   The service won't get active, because the notification wasn't
   handled.

To fix it, update the PID file before sending the MAINPID
notification, similar to what a comment in systemd's
src/core/service.c suggests:
> /* Forking services may occasionally move to a new PID.
>  * As long as they update the PID file before exiting the old
>  * PID, they're fine. */
but for our Type=notify "before sending the notification" rather than
"before exiting", because otherwise, the mix-up in 4. could still
happen (although it might not actually be problematic without the
mix-up in 6., it still seems better to avoid).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-05-12 11:53:54 +02:00
Thomas Lamprecht f4d246072d ui: avoid ascending to upper NS on double click of current
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:48:27 +02:00
Thomas Lamprecht 15808a9023 ui: add namespace: preselect current NS as parent for new one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:47:14 +02:00
Thomas Lamprecht e22ad28302 GC scheduling: avoid triggering operation tracking error for upfront checks
without that one gets a "failed to lookup datastore X" in the log for
every datastore that is in read-only or offline maintenance mode,
even if they aren't scheduled for GC anyway.

Avoid that by first opening the datastore through a Lookup operation,
and only re-open it as Write op once we know that GC needs to get
scheduled for it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Thomas Lamprecht 0408f60b58 datastore: add new Lookup for operations tracking
We sometimes need to do some in-memory only stuff, e.g., to check if
GC is already running for a datastore, which is a try_lock on a mutex
that is in-memory.

Actually the whole thing would be nicer if we could guarantee to hold
the correct contract statically, e.g., like
https://docs.rust-embedded.org/book/static-guarantees/design-contracts.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Hannes Laimer d4d730e589 proxy: rrd: skip update disk stats for offline datastores
RDD update did not use lookup_datastore() and therefore bypassed
the maintenance mode checks. This adds the needed check directly.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-05-12 11:36:56 +02:00
Wolfgang Bumiller 5b460ef525 client: add --ns parameters to snapshot commands
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 11:02:06 +02:00
Wolfgang Bumiller 03d4f43d5a client: rename --backup-ns to --ns in backup command
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 10:43:56 +02:00
Stoiko Ivanov 5225817de6 docs: zfs: update documentation about ZED
This closely follows commit aa425868069818167ff0a3cca5c64a2acc88173e
in pve-docs.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 0ae5f76277 docs: local-zfs: minor cleanup and adaptation
fixes a few small glitches in the markup.

rephrases a few PVEisms (PBS will not swap when starting a backup to
an external storage)

add zstd to available compression algorithms

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 09d903034f docs: system-booting: (re)add screenshots
add the grub+systemdboot screen from a PBS system (taken via
spice-viewer).

The alingment of left/right looked better to me than keeping both on the
right).

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 9eb804006c docs: add system-booting from pve-docs
and transform to reST.

semantic changes to the content are:
* s/{pve}/`Proxmox Backup`_/g
* changing footnotes to parenthesized notes (did not see footnote use in
  the current docs)
* removed the comment about systems setup before the introduction of
  p-b-t (which was introduced before pbs)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 6e3391c85b docs: sysadmin: adapt kernel-specifics for PBS
while all statements here are technically true - adding all
virtualization improvements is not relevant for proxmox backup in most
cases.
The intel nic driver seems like a left-over from a time (pre PVE 5.1)
where the pve-kernel included the out-of-tree drivers.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Thomas Lamprecht 71139be203 bump version to 2.1.9-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:48:51 +02:00
Thomas Lamprecht fbca018229 ui: content: code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 7e8b24bd8c ui: content: show namespaces also inline and rework node type detection
this not only makes the action disable/hide checks simpler, but also
prepares the view a bit for the idea of adding a new API endpoint
that returns the whole datastore content tree as structured JSON so
that it can be directly loaded into a tree store.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht fe79687c59 pull group: add error context for cleanup_unreferenced_files
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 9ccf933be5 datastore: move update_manifest into BackupDir impl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 87cdc327b9 sync: pull snapshot: use template variables for bloat reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 5566099849 datastore: move cleanup_unreferenced_files to BackupDir impl and fix NS awareness
sync failed on cleanup due to always trying to do so in the root NS

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 92b9cc1554 ui: remote target selectors: code cleanups
just a small start...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 0e3de42aa7 ui: sync job: use namespace selector for localNS
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 8c29bca57c ui: move remote target datastore/ns selectors to own file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Fabian Grünbichler d895b26bb9 ui: add namespace fields to sync
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Fabian Grünbichler c06c1b4bd7 sync/pull: make namespace aware
Allow pulling all groups from a certain source namespace, and
possibly sub namespaces until max-depth, into a target namespace.

If any sub-namespaces get pulled, they will be mapped relatively from
the source parent namespace to the target parent namespace.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 31aa38b684 ui: verify job: fix add-job on datastore-agnostic level
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 9d8090626c ui: namespace selector: allow to set datastore dynamically
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6b4d057370 api-types: rework BackupNamespace::map_prefix
to use slice::strip_prefix() from std

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 53d073ec1a datastore: minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 30ccc3003e datastore: relative path fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht bc4af01559 ui: datastore content: make verify-all more flexible
allow to specify the namespace, max_depth and also the re-verify/skip
behavior.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00