Commit Graph

163 Commits

Author SHA1 Message Date
Thomas Lamprecht 1ec167ee8c api types: clippy lints
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:57:07 +02:00
Thomas Lamprecht a8d3f1943b api types: prune keep options: also check weekly in keeps_something
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-01 14:30:24 +02:00
Wolfgang Bumiller 134779664e manager: hidden command to move datastore prune opts into jobs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller dba37e212b add prune jobs api
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Wolfgang Bumiller 5557af0efb api-types: add PruneJobConfig
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-05-30 13:58:43 +02:00
Thomas Lamprecht 997c96d6a3 datastore status: impl empty-status constructor for item type
we can now use it for the error case and will further use it for the
can access namespace but not datastore case in a future patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-26 13:34:00 +02:00
Fabian Grünbichler abd8248520 tree-wide: remove DatastoreWithNamespace
instead move the acl_path helper to BackupNamespace, and introduce a new
helper for printing a store+ns when logging/generating error messages.

Suggested-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-26 11:42:10 +02:00
Fabian Grünbichler efa62d44d4 api: add new priv to priv name helper
for usage in permission check error messages, to allow easily indicating
which privs are missing.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-25 17:18:56 +02:00
Fabian Grünbichler 0aa5815fb6 verify_job: fix priv check
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-25 17:18:56 +02:00
Fabian Grünbichler 83e3000349 sync job: don't require privs on datastore
syncing to a namespace only requires privileges on the namespace (and
potentially its children during execution).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-25 17:18:56 +02:00
Fabian Grünbichler b9b2d635fe sync job: fix worker ID parsing
the namespace is optional, but should be captured to allow ACL checks
for unprivileged non-job-owners.

also add FIXME for other job types and workers that (might) need
updating.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-25 17:18:56 +02:00
Thomas Lamprecht 8772ca727c api types: verify job: fix doc comment typo
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-18 15:45:55 +02:00
Thomas Lamprecht 7f3b4a94e6 api types: verify job: allow outdated-afer == 0 for backward compat
We can have those in existing verify jobs configs, and that'd break
stuff. So, even while the "bad" commit got released only recently
with `2.1.6-1` (14 April 2022), we still need to cope with those that
used it, and using some serde parser magic to transform on read only
is hard here due to section config (json-value and verify currently
happen before we can do anything about it)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-18 15:39:59 +02:00
Thomas Lamprecht 327d14b3d1 Revert "verify: allow '0' days for reverification"
This reverts commit 7a1a5d206d.

We could already cause the behavior by simply setting ignore-verified
to false, aas that flag is basically an on/off switch for even
considering outdated-after or not.

So avoid the extra logic and just make the gui use the previously
existing way.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-18 12:53:08 +02:00
Thomas Lamprecht 63e98028cc api types: namespace: fix typo in error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-16 09:50:17 +02:00
Thomas Lamprecht 6bfc94ea19 api types: BackupNamespace: fix depth check on pushing subdir to ns
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-16 09:37:38 +02:00
Thomas Lamprecht 1f2126fd7c api types: BackupNamespace: remove unused, commented out code
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-16 09:37:38 +02:00
Dominik Csapak 12d334615b api: tape/backup: fix namespace/max-depth parameters
by adding the 'default' serde hint and renaming 'recursion_depth' to
'max_depth' (to be in line with sync job config)

also add the logic to actually add/update the tape backup job config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:09:53 +02:00
Dominik Csapak 07ffb86451 api: tape/restore: add namespace mapping
by adding a new parameter 'namespaces', which contains a mapping
for a namespace like this:

store=datastore,source=foo,target=bar,max-depth=2

if source or target are omitted the root namespace is used for its value

this mapping can be given several times (on the cli) or as an array (via
api) to have mappings for multiple datastores

if a specific snapshot list is given simultaneously, the given snapshots
will be restored according to this mapping, or to the source namespace
if no mapping was found.

to do this, we reutilize the restore_list_worker, but change it so that
it does not hold a lock for the duration of the restore, but fails
if the snapshot does exist at the end. also the snapshot will now
be temporarily restored into the target datastore into the
'.tmp/<media-set-uuid>' folder.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak be97e0a55b tape: add namespaces mapping type
and the relevant parser for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak 999293bbca tape: add namespaces/recursion depth to tape backup jobs
and manual api via TapeBackupJobSetup

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:31 +02:00
Dominik Csapak 1e4e1514d3 pbs-api-types: add parse and print ns_and_snapshot
these are helpers for the few cases where we want to print and parse
from a format that has the namespace and snapshot combined, like for
the on-tape catalog and snapshot archive.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 13:52:50 +02:00
Thomas Lamprecht addcb7803e datastore: inline some format variables
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:42:41 +02:00
Thomas Lamprecht 1ddfae5499 api types: set NS_MAX_DEPTH schema default to MAX_NAMESPACE_DEPTH
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-13 12:32:25 +02:00
Fabian Grünbichler e40c7fb906 api: split max-depth schema/types
into the regular one (with default == MAX) and the one used for
pull/sync, where the default is 'None' which actually means the remote
end reduces the scope of sync automatically (or, if needed,
backwards-compat mode without any remote namespaces at all).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler 66abc4cb7d namespaces: move max-depth check to api type
and use it when creating a sync job, and simplify the check on updating
(only check the final, resulting config instead of each intermediate
version).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 12:07:22 +02:00
Fabian Grünbichler b9310489cf pull/sync: treat unset max-depth as full recursion
to be consistent with tape backup and verification jobs.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-12 17:00:38 +02:00
Thomas Lamprecht 42fb291c7c cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:54:21 +02:00
Thomas Lamprecht 0408f60b58 datastore: add new Lookup for operations tracking
We sometimes need to do some in-memory only stuff, e.g., to check if
GC is already running for a datastore, which is a try_lock on a mutex
that is in-memory.

Actually the whole thing would be nicer if we could guarantee to hold
the correct contract statically, e.g., like
https://docs.rust-embedded.org/book/static-guarantees/design-contracts.html

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 11:36:56 +02:00
Fabian Grünbichler c06c1b4bd7 sync/pull: make namespace aware
Allow pulling all groups from a certain source namespace, and
possibly sub namespaces until max-depth, into a target namespace.

If any sub-namespaces get pulled, they will be mapped relatively from
the source parent namespace to the target parent namespace.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6b4d057370 api-types: rework BackupNamespace::map_prefix
to use slice::strip_prefix() from std

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 0b1edf297b verify job: support max-depth config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 59229bd7f1 api: verify: support namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 40d495de6d api: add DatastoreWithNamespace helper struct
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6f5753cfa3 api-types: allow empty namespace
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler bc06c7b4e9 api: namespace: return popped component
helpful for places where namespaces need to be (re)created

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 133d718fe4 split the namespace out of BackupGroup/Dir api types
We decided to go this route because it'll most likely be
safer in the API as we need to explicitly add namespaces
support to the various API endpoints this way.

For example, 'pull' should have 2 namespaces: local and
remote, and the GroupFilter (which would otherwise contain
exactly *one* namespace parameter) needs to be applied for
both sides (to decide what to pull from the remote, and what
to *remove* locally as cleanup).

The *datastore* types still contain the namespace and have a
`.backup_ns()` getter.

Note that the datastore's `Display` implementations are no
longer safe to use as a deserializable string.

Additionally, some datastore based methods now have been
exposed via the BackupGroup/BackupDir types to avoid a
"round trip" in code.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 18934ae56b api: namespace management endpoints
allow to list any namespace with privileges on it and allow to create
and delete namespaces if the user has modify permissions on the parent
namespace.

Creation is only allowed if the parent NS already exists.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 08aa5fe7aa api: add NS_MAX_DEPTH_SCHEMA
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 [ T: renamed from NAMESPACE_RECURSION_DEPTH_SCHEMA & moved to from
   jobs to datastore ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler e687d1b8ee api: add prefix-mapping helper to BackupNamespace
given a namespace, a source prefix and a target prefix this helper
strips the source prefix and replaces it with the target one (erroring
out if the prefix doesn't match).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler c12a075b83 api: derive UpdaterType for BackupNamespace
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 6dd8a2ced0 BackupNamespace: fix deserialize of root NS
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller be5b3ebfdd api-types: fixup backup-ns being optional
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 352e13db9d api types: BackupNamespace add pop & parent helpers
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 220b66077c api-types: more regex fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 2772159692 api-types: add missing slash in optional ns path regex
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 02ec2ae9b8 api types: namespace: add from_parent_ns helper
will be used in the (recursive) namespace iterator

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht c2425132c4 api types: namespace: include problematic component in error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 8c74349b08 api-types: add namespace to BackupGroup
Make it easier by adding an helper accepting either group or
directory

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 42103c467d ns: max depth: set constant to upper inclusive boundary
makes usage a bit simpler, e.g., the api maximum can use that 1:1
then.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00