Commit Graph

29 Commits

Author SHA1 Message Date
Thomas Lamprecht
b298e9f16e datastore: s/fail_if_not_exist/assert_exists/
avoid putting whole sentences in parameter names

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:10:34 +02:00
Thomas Lamprecht
cc295e2c7a datastore: improve backup group/snapshot iters
move the check for directory before doing the OSString -> String
conversion, which should be a bit more efficient.

Also let the match return the entry in the non-skip/return case to
reduce indentation level for the inner "yield element" part, making
it slightly easier to follow.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:02:58 +02:00
Thomas Lamprecht
4b77d300a2 datastore: replace manual path assembly by group/dir full_path
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 19:57:20 +02:00
Thomas Lamprecht
df5c6a11cd datastore: list snapshots iter: report group dir in error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 19:57:10 +02:00
Wolfgang Bumiller
6da20161f0 reference the datastore in BackupGroup/Dir
And drop the base_path parameter on a first bunch of
functions (more reordering will follow).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 15:31:04 +02:00
Wolfgang Bumiller
6b0c6492f7 datastore: cleanup and document backup group/dir openers
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 13:24:57 +02:00
Wolfgang Bumiller
db87d93efc make datastore BackupGroup/Dir ctors private
And use the api-types for their contents.

These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:56:23 +02:00
Wolfgang Bumiller
1f6a45c938 rename BackupDir's group_path to relative_group_path
datastore's group_path will be moved to BackupDir soon and
this is required to be able to properly distinguish them

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 10:04:02 +02:00
Wolfgang Bumiller
988d575dbb api-types: introduce BackupType enum and Group/Dir api types
The type is a real enum.

All are API types and implement Display and FromStr. The
ordering is the same as it is in pbs-datastore.

Also, they are now flattened into a few structs instead of
being copied manually.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 13:12:46 +02:00
Thomas Lamprecht
33eb23d57e datastore: add snapshot iterator and provide example
will be more used in the future, when the upend-datastore master plan
comes in effect.

also a preparatory work for namespaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
7b125de3e1 datastore: add helper to get a iterator for backup groups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
de015ce7e1 datastore: implement Iterator for backup group listing
While currently it's still only used in a collected() way, most call
sites can be switched over to use the iterator directly, as often
they already convert the not-so-cheap, in-memory vector back in
.into_iter() anyway.

somewhat also preparatory (yak shaving) work for namespaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
c90dbb5c7b datastore: move list_backup_groups into Datastore impl
Having that as static method in BackupInfo makes zero sense and just
complicates call sites, which need to extract the base_path from the
store manually upfront.

Mark old fn as deprecated so that we can do the move in a separate
step.

It's also planned to add an Iterator impl for this to allow more
efficient usage in the future.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
42c2b5bec9 datastore: rustfmt whole package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 13:27:53 +02:00
Hannes Laimer
4bc84a6549 pbs-datastore: add active operations tracking
Saves the currently active read/write operation counts in a file. The
file is updated whenever a reference returned by lookup_datastore is
dropped and whenever a reference is returned by lookup_datastore. The
files are locked before every access, there is one file per datastore.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Hannes Laimer
e9d2fc9362 datastore: add check for maintenance in lookup
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Dominik Csapak
118deb4db8 pbs-datastore: use ConfigVersionCache for datastore
instead of relying on the content of some configs

previously, we always read and parsed the config file, and only
generated a new config object when the path or the 'verify-new' option
changed.

now, we increase the datastore generation on config save, and if that
changed (or the last load is 1 minute in the past), we always
generate a new object

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-01 08:16:27 +01:00
Dominik Csapak
fef61684b4 datastore: add tuning option for chunk order
currently, we sort chunks by inode when verifying or backing up to tape.
we get the inode# by stat'ing each chunk, which may be more expensive
than the gains of reading the chunks in order

Since that is highly dependent on the underlying storage of the datastore,
introduce a tuning option  so that the admin can tune that behaviour
for each datastore.

The default stays the same (sorting by inode)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-02-23 09:06:03 +01:00
Fabian Grünbichler
dcf5a0f62d misc clippy fixes
the trivial ones ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 14:57:16 +01:00
Fabian Grünbichler
540fca5c9e tree-wide: cleanup manual map/flatten
found with clippy, best viewed with `-w` ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Dietmar Maurer
25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Thomas Lamprecht
a58a5cf795 move HumanByte to pbs-abi-types crate
Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-20 19:35:24 +01:00
Dietmar Maurer
d5790a9f27 use new proxmox-sys crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 11:06:35 +01:00
Dominik Csapak
8292d3d20e api2/admin/datastore: add get/set_protection
for gettin/setting the protected flag for snapshots (akin to notes)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 11:31:11 +02:00
Dominik Csapak
de91418b79 backup/datastore: prevent protected snapshots to be removed
by throwing an error for remove_backup_dir, and skipping for
remove_backup_group

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 11:31:00 +02:00
Dietmar Maurer
e0a19d3313 use new fsync parameter to replace_file and atomic_open_or_create
Depend on proxmox 0.15.0 and proxmox-openid 0.8.1

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-10-21 07:28:32 +02:00
Wolfgang Bumiller
6ef1b649d9 update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Dietmar Maurer
6d5d305d9d move src/backup/datastore.rs into pbs_datastore crate 2021-09-27 09:11:38 +02:00