Commit Graph

6500 Commits

Author SHA1 Message Date
Wolfgang Bumiller 8ca7cccf5f file-restore: add namespace support to qemu part
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 13:35:34 +02:00
Thomas Lamprecht e30a2e9058 ui: content: fix various tree-checks from action handlers
they all still used some odd side effects of the tree structure to
decided what record type they operated on, just move them over to the
new `ty` record.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 13:28:20 +02:00
Thomas Lamprecht 08982a3746 rest: example: fix comment width
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:57:51 +02:00
Thomas Lamprecht 42fb291c7c cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:54:21 +02:00
Fabian Ebner e9b9f33aee rest server: daemon: update PID file before sending MAINPID notification
There is a race upon reload, where it can happen that:
1. systemd forks off /bin/kill -HUP $MAINPID
2. Current instance forks off new one and notifies systemd with the
   new MAINPID.
3. systemd sets new MAINPID.
4. systemd receives SIGCHLD for the kill process (which is the current
   control process for the service) and reads the PID of the old
   instance from the PID file, resetting MAINPID to the PID of the old
   instance.
5. Old instance exits.
6. systemd receives SIGCHLD for the old instance, reads the PID of the
   old instance from the PID file once more. systemd sees that the
   MAINPID matches the child PID and considers the service exited.
7. systemd receivese notification from the new PID and is confused.
   The service won't get active, because the notification wasn't
   handled.

To fix it, update the PID file before sending the MAINPID
notification, similar to what a comment in systemd's
src/core/service.c suggests:
> /* Forking services may occasionally move to a new PID.
>  * As long as they update the PID file before exiting the old
>  * PID, they're fine. */
but for our Type=notify "before sending the notification" rather than
"before exiting", because otherwise, the mix-up in 4. could still
happen (although it might not actually be problematic without the
mix-up in 6., it still seems better to avoid).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-05-12 11:53:54 +02:00
Thomas Lamprecht f4d246072d ui: avoid ascending to upper NS on double click of current
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:48:27 +02:00
Thomas Lamprecht 15808a9023 ui: add namespace: preselect current NS as parent for new one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:47:14 +02:00
Thomas Lamprecht e22ad28302 GC scheduling: avoid triggering operation tracking error for upfront checks
without that one gets a "failed to lookup datastore X" in the log for
every datastore that is in read-only or offline maintenance mode,
even if they aren't scheduled for GC anyway.

Avoid that by first opening the datastore through a Lookup operation,
and only re-open it as Write op once we know that GC needs to get
scheduled for it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Thomas Lamprecht 0408f60b58 datastore: add new Lookup for operations tracking
We sometimes need to do some in-memory only stuff, e.g., to check if
GC is already running for a datastore, which is a try_lock on a mutex
that is in-memory.

Actually the whole thing would be nicer if we could guarantee to hold
the correct contract statically, e.g., like
https://docs.rust-embedded.org/book/static-guarantees/design-contracts.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Hannes Laimer d4d730e589 proxy: rrd: skip update disk stats for offline datastores
RDD update did not use lookup_datastore() and therefore bypassed
the maintenance mode checks. This adds the needed check directly.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-05-12 11:36:56 +02:00
Wolfgang Bumiller 5b460ef525 client: add --ns parameters to snapshot commands
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 11:02:06 +02:00
Wolfgang Bumiller 03d4f43d5a client: rename --backup-ns to --ns in backup command
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 10:43:56 +02:00
Stoiko Ivanov 5225817de6 docs: zfs: update documentation about ZED
This closely follows commit aa425868069818167ff0a3cca5c64a2acc88173e
in pve-docs.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 0ae5f76277 docs: local-zfs: minor cleanup and adaptation
fixes a few small glitches in the markup.

rephrases a few PVEisms (PBS will not swap when starting a backup to
an external storage)

add zstd to available compression algorithms

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 09d903034f docs: system-booting: (re)add screenshots
add the grub+systemdboot screen from a PBS system (taken via
spice-viewer).

The alingment of left/right looked better to me than keeping both on the
right).

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 9eb804006c docs: add system-booting from pve-docs
and transform to reST.

semantic changes to the content are:
* s/{pve}/`Proxmox Backup`_/g
* changing footnotes to parenthesized notes (did not see footnote use in
  the current docs)
* removed the comment about systems setup before the introduction of
  p-b-t (which was introduced before pbs)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Stoiko Ivanov 6e3391c85b docs: sysadmin: adapt kernel-specifics for PBS
while all statements here are technically true - adding all
virtualization improvements is not relevant for proxmox backup in most
cases.
The intel nic driver seems like a left-over from a time (pre PVE 5.1)
where the pve-kernel included the out-of-tree drivers.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2022-05-12 10:08:30 +02:00
Thomas Lamprecht 71139be203 bump version to 2.1.9-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:48:51 +02:00
Thomas Lamprecht fbca018229 ui: content: code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 7e8b24bd8c ui: content: show namespaces also inline and rework node type detection
this not only makes the action disable/hide checks simpler, but also
prepares the view a bit for the idea of adding a new API endpoint
that returns the whole datastore content tree as structured JSON so
that it can be directly loaded into a tree store.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht fe79687c59 pull group: add error context for cleanup_unreferenced_files
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 9ccf933be5 datastore: move update_manifest into BackupDir impl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 87cdc327b9 sync: pull snapshot: use template variables for bloat reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 5566099849 datastore: move cleanup_unreferenced_files to BackupDir impl and fix NS awareness
sync failed on cleanup due to always trying to do so in the root NS

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 92b9cc1554 ui: remote target selectors: code cleanups
just a small start...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 0e3de42aa7 ui: sync job: use namespace selector for localNS
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 8c29bca57c ui: move remote target datastore/ns selectors to own file
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Fabian Grünbichler d895b26bb9 ui: add namespace fields to sync
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Fabian Grünbichler c06c1b4bd7 sync/pull: make namespace aware
Allow pulling all groups from a certain source namespace, and
possibly sub namespaces until max-depth, into a target namespace.

If any sub-namespaces get pulled, they will be mapped relatively from
the source parent namespace to the target parent namespace.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 31aa38b684 ui: verify job: fix add-job on datastore-agnostic level
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 9d8090626c ui: namespace selector: allow to set datastore dynamically
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6b4d057370 api-types: rework BackupNamespace::map_prefix
to use slice::strip_prefix() from std

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 53d073ec1a datastore: minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 30ccc3003e datastore: relative path fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht bc4af01559 ui: datastore content: make verify-all more flexible
allow to specify the namespace, max_depth and also the re-verify/skip
behavior.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht d83ce0d0c7 ui: fix group backup comment NS awareness
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht ad7741a294 ui: verify job: make namespace and max-depth aware
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht a327f918af ui: add verifyOutdatedAfter component
mainly as separate component for the trigger

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 0b1edf297b verify job: support max-depth config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 59229bd7f1 api: verify: support namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 8e82cc807c add ns-recursive and acl/authid aware backup group iter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler d4037525a8 remote scan/completion: add namespace support
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 40d495de6d api: add DatastoreWithNamespace helper struct
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler d3a570eb79 ui: fix wrong call to htmlEncode
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 9f8fb928f1 ui: add namespace renderer
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 226a4e68da client: add basic namespace commands
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 473063e9ec api: ns management: fix permission checks
we do not have normal GET variables available in the checks provided
by the rest server from the api macro, so do it manually.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 93b0659ff2 ui: datastore: more NS awareness
verify is actually not yet ready in the backend

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht e8112eb37b ui: datastore content: show root node for better UX with NS
that way it's easier to see on which NS one currently operates and
allows better distinguishing of root NS and some sub ns named "Root"

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6f5753cfa3 api-types: allow empty namespace
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00