Commit Graph

758 Commits

Author SHA1 Message Date
Thomas Lamprecht 9ccf933be5 datastore: move update_manifest into BackupDir impl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:40:43 +02:00
Thomas Lamprecht 59229bd7f1 api: verify: support namespaces
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 8e82cc807c add ns-recursive and acl/authid aware backup group iter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 133d718fe4 split the namespace out of BackupGroup/Dir api types
We decided to go this route because it'll most likely be
safer in the API as we need to explicitly add namespaces
support to the various API endpoints this way.

For example, 'pull' should have 2 namespaces: local and
remote, and the GroupFilter (which would otherwise contain
exactly *one* namespace parameter) needs to be applied for
both sides (to decide what to pull from the remote, and what
to *remove* locally as cleanup).

The *datastore* types still contain the namespace and have a
`.backup_ns()` getter.

Note that the datastore's `Display` implementations are no
longer safe to use as a deserializable string.

Additionally, some datastore based methods now have been
exposed via the BackupGroup/BackupDir types to avoid a
"round trip" in code.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Wolfgang Bumiller 8c74349b08 api-types: add namespace to BackupGroup
Make it easier by adding an helper accepting either group or
directory

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Thomas Lamprecht 0f198b82f5 cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 10:26:00 +02:00
Thomas Lamprecht 1eef52c206 datastore: move blob loading into BackupDir impl and adapt call sites
data blobs can only appear in a BackupDir (snapshot) in the backup
hierachy, so makes more sense that it lives in there.

As it wasn't widely used anyway it's easy to move the single
non-package call site over to the new one directly and drop the
implementation from Datastore completely.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:16:58 +02:00
Wolfgang Bumiller 6da20161f0 reference the datastore in BackupGroup/Dir
And drop the base_path parameter on a first bunch of
functions (more reordering will follow).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 15:31:04 +02:00
Wolfgang Bumiller 5116453b6d datastore: drop Ord from BackupGroup
This one is supposed to be linked to a datastore instance,
so it won't be Ord for now.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 12:20:30 +02:00
Wolfgang Bumiller db87d93efc make datastore BackupGroup/Dir ctors private
And use the api-types for their contents.

These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:56:23 +02:00
Wolfgang Bumiller 988d575dbb api-types: introduce BackupType enum and Group/Dir api types
The type is a real enum.

All are API types and implement Display and FromStr. The
ordering is the same as it is in pbs-datastore.

Also, they are now flattened into a few structs instead of
being copied manually.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 13:12:46 +02:00
Thomas Lamprecht 249dde8b63 backup: switch over to streaming Iterator improving memory usage
Avoid collecting the whole group list in memory only to iterate and
filter over it again.

Note that the change could result in a indentation change, so best
viewed with `-w` flag.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht 7d9cb8c458 replace deprecated list_backup_group from BackupInfo with Datastore one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht 9531d2c570 rust fmt for pbs src
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:03:46 +02:00
Dietmar Maurer 260147bd73 ParallelHandler: avoid re-export (cleanup)
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-02-14 14:12:39 +01:00
Dominik Csapak 7a1a5d206d verify: allow '0' days for reverification
and let it mean that we will always reverify

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-27 15:31:55 +01:00
Fabian Grünbichler 9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Dietmar Maurer 25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Dietmar Maurer d5790a9f27 use new proxmox-sys crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 11:06:35 +01:00
Wolfgang Bumiller 6ef1b649d9 update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Dietmar Maurer 8a23ea4656 move src/backup/read_chunk.rs to pbs-datastore/src/local_chunk_reader.rs 2021-09-27 11:10:14 +02:00
Dietmar Maurer c95c1c83b0 move src/backup/snapshot_reader.rs to pbs_datastore crate 2021-09-27 09:58:20 +02:00
Dietmar Maurer b446fa14c5 WorkerTaskContext: make it Send + Sync 2021-09-27 09:11:38 +02:00
Dietmar Maurer 6d5d305d9d move src/backup/datastore.rs into pbs_datastore crate 2021-09-27 09:11:38 +02:00
Dietmar Maurer 0fd55b08d9 WorkerTaskContext: add shutdown_requested() and fail_on_shutdown() 2021-09-24 12:04:31 +02:00
Dietmar Maurer c8449217dc rename TaskState to WorkerTaskContext 2021-09-24 10:33:49 +02:00
Dietmar Maurer b9700a9fe5 move worker_task.rs into proxmox-rest-server crate
Also moved pbs-datastore/src/task.rs to pbs-tools, which now depends on 'log'.
2021-09-24 10:28:17 +02:00
Dietmar Maurer fd6d243843 move ApiConfig, FileLogger and CommandoSocket to proxmox-rest-server workspace
ApiConfig: avoid using  pbs_config::backup_user()
CommandoSocket: avoid using  pbs_config::backup_user()
FileLogger: avoid using  pbs_config::backup_user()
- use atomic_open_or_create_file()

Auth Trait: moved definitions to proxmox-rest-server/src/lib.rs
- removed CachedUserInfo patrameter
- return user as String (not Authid)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-21 08:46:41 +02:00
Dietmar Maurer ccc3896ff3 avoid type re-exports 2021-09-14 08:35:43 +02:00
Dietmar Maurer cef5c72682 move src/tape/helpers/snapshot_reader.rs to src/backup/snapshot_reader.rs 2021-09-14 07:42:06 +02:00
Dietmar Maurer 6227654ad8 more api type cleanups: avoid re-exports 2021-09-10 12:25:32 +02:00
Dietmar Maurer e7d4be9d85 move datastore config to pbs_config workspace 2021-09-10 08:40:58 +02:00
Dietmar Maurer bbdda58b35 moved key_derivation.rs from pbs_datastore to pbs-config/src/key_config.rs
Also moved pbs-datastore/src/crypt_config.rs to pbs-tools/src/crypt_config.rs.
We do not want to depend on pbs-api-types there, so I use [u8;32] instead of
Fingerprint.
2021-09-07 10:12:17 +02:00
Dietmar Maurer 2121174827 start new pbs-config workspace
moved src/config/domains.rs
2021-09-02 12:58:20 +02:00
Wolfgang Bumiller 6c76aa434d split proxmox-file-restore into its own crate
This also moves a couple of required utilities such as
logrotate and some file descriptor methods to pbs-tools.

Note that the logrotate usage and run-dir handling should be
improved to work as a regular user as this *should* (IMHO)
be a regular unprivileged command (including running
qemu given the kvm privileges...)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-01 12:23:29 +02:00
Wolfgang Bumiller b2065dc7d2 cleanup proxmox_backup::backup module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-08-30 14:14:04 +02:00
Wolfgang Bumiller ef4df211ab move CachedChunkReader to pbs-datastore
this was actually still missing from the previous commit

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-21 14:20:03 +02:00
Wolfgang Bumiller eb5e0ae65a move remaining client tools to pbs-tools/datastore
pbs-datastore now ended up depending on tokio after all, but
that's fine for now

for the fuse code I added pbs-fuse-loop (has the old
fuse_loop and its 'loopdev' module)
ultimately only binaries should depend on this to avoid the
library link

the only thins remaining to move out the client binary are
the api method return types, those will need to be moved to
pbs-api-types...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-21 14:12:24 +02:00
Dietmar Maurer ac81ed17b9 fix regression test file permission problems
By simply using the current user/group instead of backup:backup
2021-07-21 09:30:22 +02:00
Dietmar Maurer a301c362e3 add helpers to write configuration files 2021-07-20 18:54:23 +02:00
Dietmar Maurer 7526d86419 use new atomic_open_or_create_file
Factor out open_backup_lockfile() method to acquire locks owned by
user backup with permission 0660.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 18:54:23 +02:00
Wolfgang Bumiller 6c221244df move lru cachers to pbs-tools
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-20 10:57:22 +02:00
Wolfgang Bumiller 38629c3961 move ChunkStream to pbs-client
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-20 10:52:21 +02:00
Wolfgang Bumiller 2b7f8dd5ea move client to pbs-client subcrate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 12:58:43 +02:00
Dominik Csapak 8e0b852f24 server/prune_job: add proper permission checks to 'prune_datastore'
checks for PRIV_DATASTORE_MODIFY, or else if the auth_id is the backup
owner, and skips the group if not.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:39:01 +02:00
Dominik Csapak 9751ef4b36 backup/datastore: refactor check_backup_owner there
and add a 'owns_backup' convenience function

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:36:02 +02:00
Wolfgang Bumiller ea584a7510 move more api types for the client
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-09 15:17:53 +02:00
Wolfgang Bumiller 2f02e431b0 moving more code to pbs-datastore
prune and fixed/dynamic index

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-09 10:40:14 +02:00
Wolfgang Bumiller e64f38cb6b move chunk_stat, read_chunk to pbs-datastore
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-09 10:40:14 +02:00
Wolfgang Bumiller c23192d34e move chunk_store to pbs-datastore
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 14:37:47 +02:00