Commit Graph

6445 Commits

Author SHA1 Message Date
Thomas Lamprecht cf1b029b3f ui: ACL edit: set default focus on a non-combobox element
to avoid making it "jump" in the users face by immediately opening
the picker on window open.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-14 14:50:40 +02:00
Thomas Lamprecht 4f897c8cf9 ui: namespace selector: set queryMode to local
to avoid that the comobox triggers automatic API request with the
queryParam default `query` GET param on manual typing (e.g., for
filtering) from the user, we have all data already loaded and locally
available.

https://docs.sencha.com/extjs/7.0.0/classic/Ext.form.field.ComboBox.html#cfg-queryMode

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-14 14:48:36 +02:00
Thomas Lamprecht 2e63a46414 ui: trigger datastore update after maintenance mode edit
This provides immediate feedback for adding the respective icon in
the navigation tree entry most of the time, and we can then increase
the query period of the datastore list store to the original 15
again, as it was lowered to 5 seconds for just this reason in commit
fbd6f54f39

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-14 14:39:02 +02:00
Thomas Lamprecht 9f4d9abbf6 ui: fix storeId casing to register store correctly
we query that store to add the datastore specific ACL paths to
improve UX there, this failed a while due the StoreManager lookup
always failing as the store wasn't registered in the StoreManager due
to using storeid vs. correct storeId

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-14 12:28:03 +02:00
Thomas Lamprecht 8122eaadaa cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 16:59:32 +02:00
Thomas Lamprecht 22cfad134f api: datastore status: make counts recurse over all accesible namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 16:49:42 +02:00
Thomas Lamprecht de27ebc6b6 hierachy: add lifetime to ListAccessibleBackupGroups so that owner can be ref
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 16:48:56 +02:00
Dominik Csapak 74391d1c32 docs: tape: add information about namespaces
which are backed up, how to use the new parameters and how to map
them during restore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 15:45:17 +02:00
Dominik Csapak fca84a4b94 docs: tape/restore: mention single snapshot restore
what it is, how to use it and the caveats

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 15:45:17 +02:00
Dominik Csapak 602319f9fc docs: tape: remove note about global content namespace
that is actually not true, we save the datastore in the chunk archives
as well as the snapshot archives, otherwise we could not backup
multiple datastores to a single media-set.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 15:45:17 +02:00
Dominik Csapak 8d2a9b2904 cli: proxmox-tape: fix ns/depth parameter
was forgotten after recent rebase

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 15:45:17 +02:00
Thomas Lamprecht c8e93b31ff bump version to 2.1.10-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 14:26:38 +02:00
Thomas Lamprecht cf99333b83 ui: adapt to s/backup-ns/ns/ api param change
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 14:21:52 +02:00
Dominik Csapak b70a12e723 ui: tape/Restore: allow simple namespace mapping
add a default namespace selector (of the current default store)
and a namespace selector per target datastore (for media-sets with
multiple datastores).

to achieve that we have to change the way we handle the mapping field a bit:
* don't use it as field directly (otherwise the value gets stringified),
  but use the 'getValue' method in 'onGetValues'.
* set the defaultStore there, not only that we have one
  (with this we can now easily show it as emptytext for each store)
* add a reference to the widgets to the record so that we can access
  them in the respective change handler (also clean those references up,
  else we have a cyclic reference between record <-> widget)

in onGetValues, if we have multiple datastores, the mapping grid does
all the work for us, otherwise, we have to create the ns mapping
ourselves there.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:09:53 +02:00
Dominik Csapak f6b09e83cb ui: tape/BackupJobEdit: add onlineHelp
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:09:53 +02:00
Dominik Csapak 6f836d3ffa ui: tape/Backup: add namespace + max-depth to backup job edit window
like we do for sync

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:09:53 +02:00
Dominik Csapak 80df7caded ui: tape/Backup: add namespace and recursion field for manual backup
and change the layout to two columns, because the window was getting
too tall.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:09:53 +02:00
Dominik Csapak 12d334615b api: tape/backup: fix namespace/max-depth parameters
by adding the 'default' serde hint and renaming 'recursion_depth' to
'max_depth' (to be in line with sync job config)

also add the logic to actually add/update the tape backup job config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:09:53 +02:00
Dominik Csapak 1e37156a6b ui: tape/BackupOverview: show namespaces as their own level above groups
since the namespaces are in the snapshot path we get here, we must parse
them out, else we confuse the first namespace with the group.

for now, show all namespaces on the same level (so not nested), and
do not allow for preselecting a namespace for restoring

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Fabian Grünbichler e49bd1e98f tape: media catalog: use match for magic check
like in other parts of the code

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 14:08:32 +02:00
Fabian Grünbichler 707c48ad46 tape: bump catalog/snapshot archive magic
the snapshot string format is not backwards compatible since it now has
an in-line namespace prefix. it's possible to select which magic to use
at the start of the backup, since a tape backup job knows whether it
operates on non-root namespaces up-front.

the MediaCatalog itself also has a similar incompatible change, but
there
- updating existing catalogs in-place
- not knowing what the catalog will contain in the future when initially
  creating/opening it
makes bumping the magic there harder. since the tape contents are
sufficiently guarded by the other two bumps, ignoring the
backwards-incomaptible change of the on-disk catalogs seems like an okay
tradeoff.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak 07ffb86451 api: tape/restore: add namespace mapping
by adding a new parameter 'namespaces', which contains a mapping
for a namespace like this:

store=datastore,source=foo,target=bar,max-depth=2

if source or target are omitted the root namespace is used for its value

this mapping can be given several times (on the cli) or as an array (via
api) to have mappings for multiple datastores

if a specific snapshot list is given simultaneously, the given snapshots
will be restored according to this mapping, or to the source namespace
if no mapping was found.

to do this, we reutilize the restore_list_worker, but change it so that
it does not hold a lock for the duration of the restore, but fails
if the snapshot does exist at the end. also the snapshot will now
be temporarily restored into the target datastore into the
'.tmp/<media-set-uuid>' folder.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak fc99c2791b api: tape/restore: check and create target namespace
checks the privilegs for the target namespace. If that does not exist,
try to recursively create them while checking the privileges.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak 6b61d319c5 api: tape/restore: add optional namespace map to DataStoreMap
and change the interface from 'get_datastore' to 'get_targets'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak be97e0a55b tape: add namespaces mapping type
and the relevant parser for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak 999293bbca tape: add namespaces/recursion depth to tape backup jobs
and manual api via TapeBackupJobSetup

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:31 +02:00
Dominik Csapak 9c65e6ab4a tape: fix snapshot path in catalog and snapshot_archive
both used the 'Display' trait of pbs_datastore::BackupDir, which is not
intended to be serialized anywhere. Instead, manually format the path
using the print_ns_and_snapshot helper, and conversely, parse with
'parse_ns_and_snapshot'. to be a bit safer, change the register_snapshot
signature to take a BackupNamespace and BackupDir instead of a string.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 13:52:50 +02:00
Dominik Csapak 1e4e1514d3 pbs-api-types: add parse and print ns_and_snapshot
these are helpers for the few cases where we want to print and parse
from a format that has the namespace and snapshot combined, like for
the on-tape catalog and snapshot archive.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 13:52:50 +02:00
Dominik Csapak 05b7175a56 tape: notify when arriving at end of media
when continuing a media set, we first move to the end of the tape and
start with the next (chunk) archive. If that takes long, the task logs
last line is 'moving to end of media' even if we already startet
writing. To make this less confusing, log that we arrived at the
end of the media.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 13:52:50 +02:00
Wolfgang Bumiller bc21ade293 tree-wide: rename 'backup-ns' API parameters to 'ns'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-13 13:46:13 +02:00
Thomas Lamprecht f07e660153 ui: move max NS prefix length logic to reduced max-depth selector
for better re-usability in the future and it felt a bit odd to have
such specific logic in the sync job edit directly

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 13:15:02 +02:00
Thomas Lamprecht addcb7803e datastore: inline some format variables
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:42:41 +02:00
Thomas Lamprecht 1ddfae5499 api types: set NS_MAX_DEPTH schema default to MAX_NAMESPACE_DEPTH
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:32:25 +02:00
Thomas Lamprecht 54d315c951 ui: group filter: make also local filter NS aware
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:32:25 +02:00
Fabian Grünbichler 9dde8cd625 ui: sync: add reduced max-depth selector
that allows setting the limit based on sync namespace prefix lengths.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:20:29 +02:00
Fabian Grünbichler 87be232d1c pull/sync: clamp (local) max-depth if unset
to handle the unlikely case of `ns` being deeper than `remote-ns`,
`max-depth` being set to `None` and a too-deep sub-ns of `ns` existing.
such a sub-ns cannot have been created by a previous run of this sync
job, so avoid unexpectedly removing it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler e40c7fb906 api: split max-depth schema/types
into the regular one (with default == MAX) and the one used for
pull/sync, where the default is 'None' which actually means the remote
end reduces the scope of sync automatically (or, if needed,
backwards-compat mode without any remote namespaces at all).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 66abc4cb7d namespaces: move max-depth check to api type
and use it when creating a sync job, and simplify the check on updating
(only check the final, resulting config instead of each intermediate
version).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 11567dfbad pull/sync: correctly query with remote-ns as parent
else (grand)-parents and siblings/cousins of remote-ns are also
included, and mapping the remote-ns prefix fails.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 7a3e777ded pull/sync: detect remote lack of namespace support
and fall back to only syncing the root namespace, if possible. the sync
job will still be marked as failed to prompt the admin to resolve the
situation:
- explicitly mark the job as syncing *only* the root namespace
- or upgrade remote end to support namespaces

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 17:00:38 +02:00
Fabian Grünbichler b9310489cf pull/sync: treat unset max-depth as full recursion
to be consistent with tape backup and verification jobs.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 17:00:38 +02:00
Fabian Grünbichler d9aad37f2f pull: pass params as non-ref in pull_store
so that it's possible to modify them in-place without cloning.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 16:26:26 +02:00
Thomas Lamprecht 2a088b9975 datastore: drop bogus chunk size check, can cause trouble
other sizes can happen in legitimate and illegitimate ways:
 - illegitimate: encryped chunks and bad actor client
 - legitimate: same chunk but newer zstd version (or compression
   level) can compress it better (or worse) so the

Ideally we could take the actual smaller chunk so that improved zstd
tech gets leveraged, but we could only allow to do that for
un-encrypted ones.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 15:41:20 +02:00
Thomas Lamprecht 78e1ee5230 bump version to 2.1.9-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 14:28:16 +02:00
Dominik Csapak c7d42dac97 ui: navigation tree: fix losing datastore selection on store load
instead of using 'replaceChild', simply set the appropriate
properties. When using the 'nodeUpdate' (protected function of extjs,
intended to be overwritten) instead of the private 'updateNode', it
will be called when the properties change

This way, the treenode stays the same and it can keep the selection

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-12 14:28:16 +02:00
Wolfgang Bumiller 8ca7cccf5f file-restore: add namespace support to qemu part
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-12 13:35:34 +02:00
Thomas Lamprecht e30a2e9058 ui: content: fix various tree-checks from action handlers
they all still used some odd side effects of the tree structure to
decided what record type they operated on, just move them over to the
new `ty` record.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 13:28:20 +02:00
Thomas Lamprecht 08982a3746 rest: example: fix comment width
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:57:51 +02:00
Thomas Lamprecht 42fb291c7c cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:54:21 +02:00
Fabian Ebner e9b9f33aee rest server: daemon: update PID file before sending MAINPID notification
There is a race upon reload, where it can happen that:
1. systemd forks off /bin/kill -HUP $MAINPID
2. Current instance forks off new one and notifies systemd with the
   new MAINPID.
3. systemd sets new MAINPID.
4. systemd receives SIGCHLD for the kill process (which is the current
   control process for the service) and reads the PID of the old
   instance from the PID file, resetting MAINPID to the PID of the old
   instance.
5. Old instance exits.
6. systemd receives SIGCHLD for the old instance, reads the PID of the
   old instance from the PID file once more. systemd sees that the
   MAINPID matches the child PID and considers the service exited.
7. systemd receivese notification from the new PID and is confused.
   The service won't get active, because the notification wasn't
   handled.

To fix it, update the PID file before sending the MAINPID
notification, similar to what a comment in systemd's
src/core/service.c suggests:
> /* Forking services may occasionally move to a new PID.
>  * As long as they update the PID file before exiting the old
>  * PID, they're fine. */
but for our Type=notify "before sending the notification" rather than
"before exiting", because otherwise, the mix-up in 4. could still
happen (although it might not actually be problematic without the
mix-up in 6., it still seems better to avoid).

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-05-12 11:53:54 +02:00