Commit Graph

6407 Commits

Author SHA1 Message Date
Fabian Grünbichler d3a570eb79 ui: fix wrong call to htmlEncode
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 9f8fb928f1 ui: add namespace renderer
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 226a4e68da client: add basic namespace commands
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 473063e9ec api: ns management: fix permission checks
we do not have normal GET variables available in the checks provided
by the rest server from the api macro, so do it manually.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 93b0659ff2 ui: datastore: more NS awareness
verify is actually not yet ready in the backend

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht e8112eb37b ui: datastore content: show root node for better UX with NS
that way it's easier to see on which NS one currently operates and
allows better distinguishing of root NS and some sub ns named "Root"

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 6f5753cfa3 api-types: allow empty namespace
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 3c09413a0a client: don't pass empty backup-ns
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 028346e42c ui: content view: improve empty text
reference NS so that users get a hint where they are currently
hierarchy-wise, and clarify that we found no *accessible* snapshots,
on this level, i.e., there can be some that we just cannot see due to
only having access on lover level NS or being different owners.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler bc06c7b4e9 api: namespace: return popped component
helpful for places where namespaces need to be (re)created

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 7a404dc53d api: datastore: further unify check helpers
this is the most common sequence of checks we have in this file, so
let's have a single place where we implement it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler c939698414 api: datastore: load datastore & check owner helper
these happen together very often.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 1909ece229 api: datastore: lookup after checking privs
else this could leak existence of datastore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 2bc2435a96 api: datastore: refactor priv checks
the helper now takes both high-privilege and lesser-privilege privs, so
the resulting bool can be used to quickly check whether additional
checks like group ownership are needed or not.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler a724f5fd47 api: datastore: unify access checks
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 133d718fe4 split the namespace out of BackupGroup/Dir api types
We decided to go this route because it'll most likely be
safer in the API as we need to explicitly add namespaces
support to the various API endpoints this way.

For example, 'pull' should have 2 namespaces: local and
remote, and the GroupFilter (which would otherwise contain
exactly *one* namespace parameter) needs to be applied for
both sides (to decide what to pull from the remote, and what
to *remove* locally as cleanup).

The *datastore* types still contain the namespace and have a
`.backup_ns()` getter.

Note that the datastore's `Display` implementations are no
longer safe to use as a deserializable string.

Additionally, some datastore based methods now have been
exposed via the BackupGroup/BackupDir types to avoid a
"round trip" in code.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 1baf9030ad ui: datastore prune: support passing namespace
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 2f5417f845 prune: allow passing namespace
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht a7f5e64154 ui: datastore content: allow to create new namespace
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 55ffd4a946 ui: utils: also provided me.SAFE_ID_RE
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 94135ccca2 ui: datastore content: allow to select namespace to show
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 968270ae3d ui: add namespace selector combobox
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht d45506d4a4 api: backup create: enforce that namespace exists
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht cabda57f0a api: backup create: make permission check namespace aware
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 7d6fc15b20 api: datastore: make permission checks namespace aware
We probably can combine the base permission + owner check, but for
now add explicit ones to upfront so that the change is simpler as
only one thing is done.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 18934ae56b api: namespace management endpoints
allow to list any namespace with privileges on it and allow to create
and delete namespaces if the user has modify permissions on the parent
namespace.

Creation is only allowed if the parent NS already exists.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 15a9272495 datastore: add max-depth to recursive namespace iter
on depth == 0 we only yield the anchor ns, this simplifies usage in
the API as for, e.g. list-snapthos the depth == 0 means only the
snapshots from the passed namespace, nothing below.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 08aa5fe7aa api: add NS_MAX_DEPTH_SCHEMA
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 [ T: renamed from NAMESPACE_RECURSION_DEPTH_SCHEMA & moved to from
   jobs to datastore ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler e687d1b8ee api: add prefix-mapping helper to BackupNamespace
given a namespace, a source prefix and a target prefix this helper
strips the source prefix and replaces it with the target one (erroring
out if the prefix doesn't match).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler c12a075b83 api: derive UpdaterType for BackupNamespace
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht c5648f1920 config: acl tree: allow path components to be paths too
will be used for namespaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht dc3d716bdb datastore: add create_namespace
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Fabian Grünbichler 6dd8a2ced0 BackupNamespace: fix deserialize of root NS
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller be5b3ebfdd api-types: fixup backup-ns being optional
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller c18d481fd7 pbs-client: don't include empty backup-ns in requests
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 68857aecb3 client: add --ns parameter to snapshot list
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 352e13db9d api types: BackupNamespace add pop & parent helpers
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 220b66077c api-types: more regex fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 2772159692 api-types: add missing slash in optional ns path regex
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 89ae3c3255 client: more backup namespace support
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 13d6de3787 datastore: include namespace in full_path
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 33f2c2a1bf api: add remaining missing backup-ns parameters
these are the ones for non-#[api] methods, also fill in the
namespace in prune operations

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 4c7cc5b39e datastore: add helpers to destroy whole namespaces
The behavior on "any snapshot was protected" isn't yet ideal, as we
then do not cleanup any (sub) namespace, even if some of them where
cleaned from groups & snapshots completely. But that isn't easy to do
with our current depth-first pre-order iterator behavior, and it's
also not completely wrong either, the user can re-do the removal on
the sub-namespaces, so leave that for later.

Should get moved to a datastore::BackupNamespace type once/if we get
one

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 90e3869690 datastore: add single-level and recursive namespace iterators
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 02ec2ae9b8 api types: namespace: add from_parent_ns helper
will be used in the (recursive) namespace iterator

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht c2425132c4 api types: namespace: include problematic component in error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 11ffd737e3 datastore: add backup_ns accessor
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 8c74349b08 api-types: add namespace to BackupGroup
Make it easier by adding an helper accepting either group or
directory

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 42103c467d ns: max depth: set constant to upper inclusive boundary
makes usage a bit simpler, e.g., the api maximum can use that 1:1
then.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht b68bd900c1 api-types: add BackupNamespace type
The idea is to have namespaces in a datastore to allow grouping and
namespacing backups from different (but similar trusted) sources,
e.g., different PVE clusters, geo sites, use-cases or company
service-branches, without separating the underlying
deduplication domain and thus blowing up data and (GC/verify)
resource usage.

To avoid namespace ID clashes with anything existing or future
usecases use a intermediate `ns` level on *each* depth.

The current implementation treats that as internal and thus hides
that fact from the API, iow., the namespace path the users passes
along or gets returned won't include the `ns` level, they do not
matter there at all.

The max-depth of 8 is chosen with the following in mind:
- assume that end-users already are in a deeper level of a hierarchy,
  most often they'll start at level one or two, as the higher ones
  are used by the seller/admin to namespace different users/groups,
  so lower than four would be very limiting for a lot of target use
  cases

- all the more, a PBS could be used as huge second level archive in a
  big company, so one could imagine a namespace structure like:
  /<state>/<intra-state-location>/<datacenter>/<company-branch>/<workload-type>/<service-type>/
  e.g.: /us/east-coast/dc12345/financial/report-storage/cassandra/
  that's six levels that one can imagine for a reasonable use-case,
  leave some room for the ones harder to imagine ;-)

- on the other hand, we do not want to allow unlimited levels as we
  have request parameter limits and deep nesting can create other
  issues as well (e.g., stack exhaustion), so doubling the minimum
  level of 4 (1st point) we got room to breath even for the
  more odd (or huge) use cases (2nd point)

- a per-level length of 32 (-1 due to separator) is enough to use
  telling names, making lives of users and admin simpler, but not
  blowing up parameter total length with the max depth of 8

- 8 * 32 = 256 which is nice buffer size

Much thanks for Wolfgang for all the great work on the type
implementation and assisting greatly with the design.

Co-authored-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00