that way it's easier to see on which NS one currently operates and
allows better distinguishing of root NS and some sub ns named "Root"
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
reference NS so that users get a hint where they are currently
hierarchy-wise, and clarify that we found no *accessible* snapshots,
on this level, i.e., there can be some that we just cannot see due to
only having access on lover level NS or being different owners.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
We probably can combine the base permission + owner check, but for
now add explicit ones to upfront so that the change is simpler as
only one thing is done.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
allow to list any namespace with privileges on it and allow to create
and delete namespaces if the user has modify permissions on the parent
namespace.
Creation is only allowed if the parent NS already exists.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
on depth == 0 we only yield the anchor ns, this simplifies usage in
the API as for, e.g. list-snapthos the depth == 0 means only the
snapshots from the passed namespace, nothing below.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The behavior on "any snapshot was protected" isn't yet ideal, as we
then do not cleanup any (sub) namespace, even if some of them where
cleaned from groups & snapshots completely. But that isn't easy to do
with our current depth-first pre-order iterator behavior, and it's
also not completely wrong either, the user can re-do the removal on
the sub-namespaces, so leave that for later.
Should get moved to a datastore::BackupNamespace type once/if we get
one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The idea is to have namespaces in a datastore to allow grouping and
namespacing backups from different (but similar trusted) sources,
e.g., different PVE clusters, geo sites, use-cases or company
service-branches, without separating the underlying
deduplication domain and thus blowing up data and (GC/verify)
resource usage.
To avoid namespace ID clashes with anything existing or future
usecases use a intermediate `ns` level on *each* depth.
The current implementation treats that as internal and thus hides
that fact from the API, iow., the namespace path the users passes
along or gets returned won't include the `ns` level, they do not
matter there at all.
The max-depth of 8 is chosen with the following in mind:
- assume that end-users already are in a deeper level of a hierarchy,
most often they'll start at level one or two, as the higher ones
are used by the seller/admin to namespace different users/groups,
so lower than four would be very limiting for a lot of target use
cases
- all the more, a PBS could be used as huge second level archive in a
big company, so one could imagine a namespace structure like:
/<state>/<intra-state-location>/<datacenter>/<company-branch>/<workload-type>/<service-type>/
e.g.: /us/east-coast/dc12345/financial/report-storage/cassandra/
that's six levels that one can imagine for a reasonable use-case,
leave some room for the ones harder to imagine ;-)
- on the other hand, we do not want to allow unlimited levels as we
have request parameter limits and deep nesting can create other
issues as well (e.g., stack exhaustion), so doubling the minimum
level of 4 (1st point) we got room to breath even for the
more odd (or huge) use cases (2nd point)
- a per-level length of 32 (-1 due to separator) is enough to use
telling names, making lives of users and admin simpler, but not
blowing up parameter total length with the max depth of 8
- 8 * 32 = 256 which is nice buffer size
Much thanks for Wolfgang for all the great work on the type
implementation and assisting greatly with the design.
Co-authored-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Co-authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
thanks to commit 70142e607dda43fc778f39d52dc7bb3bba088cd3 from
proxmox repos's proxmox-http crate
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
data blobs can only appear in a BackupDir (snapshot) in the backup
hierachy, so makes more sense that it lives in there.
As it wasn't widely used anyway it's easy to move the single
non-package call site over to the new one directly and drop the
implementation from Datastore completely.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
the manifest is owned by the backup dir (snapshot) so it should also
handle locking, makes no sense to have the implementation somewhere
higher up.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
move the check for directory before doing the OSString -> String
conversion, which should be a bit more efficient.
Also let the match return the entry in the non-skip/return case to
reduce indentation level for the inner "yield element" part, making
it slightly easier to follow.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
> The hash set will be able to hold at least capacity elements
> without reallocating. If capacity is 0, the hash set will not
> allocate.
-- rustdoc, HashSet::with_capacity
So, the number we pass is the amount of chunk "IDs" we safe, which is
then 64Ki, not 16Ki and thus the size we can reference too is also
256 GiB, not 64 GiB.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
will be more used in the future, when the upend-datastore master plan
comes in effect.
also a preparatory work for namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Avoid collecting the whole group list in memory only to iterate and
filter over it again.
Note that the change could result in a indentation change, so best
viewed with `-w` flag.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
While currently it's still only used in a collected() way, most call
sites can be switched over to use the iterator directly, as often
they already convert the not-so-cheap, in-memory vector back in
.into_iter() anyway.
somewhat also preparatory (yak shaving) work for namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Having that as static method in BackupInfo makes zero sense and just
complicates call sites, which need to extract the base_path from the
store manually upfront.
Mark old fn as deprecated so that we can do the move in a separate
step.
It's also planned to add an Iterator impl for this to allow more
efficient usage in the future.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
without this the store stayed active in the background and kept
updating every 3s for every datastore the ui was opened.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
The entries in a file go from oldest end-time in the first time to
newest end-time in the last line. So, just because the first line is
older than the cut-off time, the remaining one doesn't necessarily
have to be old enough too. What we can know for sure that older than
the current checked rotations of the task archive are definitively up
for deletion.
Another possibility would be to check the last line, but as scanning
backwards is more expensive/complex to do while only being an actual
improvement in a very specific edge case (it's more likely to have a
mixed time-cutoff vs. task-log-file boundary than that those are
aligned)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
commenting that version_cache.increase_datastore_generation increases
the, well, version is rather superfluous. Also avoid the use of "we",
which is always ambiguous in code comments.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
avoid that the acme renewal is skipped due to bailing out earlier
from a subscription or apt update error.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Thomas Lamprecht
