Commit Graph

6240 Commits

Author SHA1 Message Date
Thomas Lamprecht
0f198b82f5 cargo fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-28 10:26:00 +02:00
Thomas Lamprecht
a0781d7b9e bump version to 2.1.7-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:54:28 +02:00
Hannes Laimer
f732942089 ui: add tooltip to datastore in maintenance mode
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-27 19:21:19 +02:00
Hannes Laimer
1b7479c968 ui: utils: add function for parsing maintenance mode
...since the same code is used is more than one place

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-27 19:21:19 +02:00
Hannes Laimer
fbd6f54f39 ui: update datastore list more often
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-27 19:21:19 +02:00
Hannes Laimer
adf5dcba8d ui: update icon in datastore list when in maintenance mode
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-27 19:21:19 +02:00
Hannes Laimer
e022d13cf3 api2: DataStoreListItem add maintenance info
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-27 19:21:19 +02:00
Hannes Laimer
dd09432a90 ui: add summary mask when in maintenance mode
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-27 19:21:19 +02:00
Dominik Csapak
6ddd69c5ce file-restore: add 'timeout' and 'json-error' parameter
timeout limits the code with the given timeout in seconds, and
'json-error' return json to stdout when the call returns an error like
this:

{
    "msg": "error message",
    "error": true,
    "code": <HTTP_STATUS_CODE>, // if it was an http error
}

with both options set, a client can more easily determine if the call
ran into a timeout (since it will return a 503 error), and can poll
it again

both is done behind new parameters, so that we can stay backwards-compatible

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:19:57 +02:00
Dominik Csapak
25be1fa0d7 file-restore: factor out 'list_files'
we'll want to reuse that in a later patch

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:19:57 +02:00
Dominik Csapak
8eaa46ffea restore-daemon: avoid auto-mounting zpools
the duration of mounting zpools not only correspond to the number of disks,
but also to the content (many subvols for example) which we cannot know
beforehand. so avoid mounting them at the start, and mount it only when
the user requests a listing/extraction with the zpool in path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:19:57 +02:00
Dominik Csapak
4d76ab91e4 restore-daemon: put blocking code into 'block_in_place'
DISK_STATE.lock() and '.resolve()' can both block since they access
the disks. Putting them into a 'block_in_place' makes tokio move it
out in its own thread to avoid that the executor isn't able to
progress any other futures in the mean time.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:18:44 +02:00
Dominik Csapak
436a48d611 restore-daemon: start disk initialization in parallel to the api
this way, the vm can start up faster, and the actual disk init happens
in parallel. this avoids unnecessary timeouts when starting the vm

if the call panics, we still abort the vm with an error

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-27 19:18:44 +02:00
Thomas Lamprecht
274ac755a1 api types: datastore status: reword doc comment of estimated_full_date
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-25 11:48:25 +02:00
Thomas Lamprecht
579362f743 ui: update generated OnlineHelpInfo map
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-25 10:17:21 +02:00
Stefan Sterz
f3b02a9b86 fix #3067: ui: add a separate notes view for longer markdown notes
since markdown notes might be rather long, this commit adds a tab
similar to pve's datacenter or node notes. requires a bump of the
widget toolkit in order to use the `pmxNotesView`.

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-04-25 08:39:39 +02:00
Stefan Sterz
684a402931 fix #3067: docs: add markdown primer from pve to pbs
this copies the markdown primer from the pve docs to allow access to
it via the help buttons in the gui

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-04-25 08:39:39 +02:00
Thomas Lamprecht
1eef52c206 datastore: move blob loading into BackupDir impl and adapt call sites
data blobs can only appear in a BackupDir (snapshot) in the backup
hierachy, so makes more sense that it lives in there.

As it wasn't widely used anyway it's easy to move the single
non-package call site over to the new one directly and drop the
implementation from Datastore completely.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:16:58 +02:00
Thomas Lamprecht
f03649b8f3 datastore: move destroying group or dir into respective impl
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:14:39 +02:00
Thomas Lamprecht
5c9c23b6b2 datastore: move manifest locking into BackupDir impl
the manifest is owned by the backup dir (snapshot) so it should also
handle locking, makes no sense to have the implementation somewhere
higher up.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:10:43 +02:00
Thomas Lamprecht
b298e9f16e datastore: s/fail_if_not_exist/assert_exists/
avoid putting whole sentences in parameter names

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:10:34 +02:00
Thomas Lamprecht
cc295e2c7a datastore: improve backup group/snapshot iters
move the check for directory before doing the OSString -> String
conversion, which should be a bit more efficient.

Also let the match return the entry in the non-skip/return case to
reduce indentation level for the inner "yield element" part, making
it slightly easier to follow.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 20:02:58 +02:00
Thomas Lamprecht
4b77d300a2 datastore: replace manual path assembly by group/dir full_path
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 19:57:20 +02:00
Thomas Lamprecht
df5c6a11cd datastore: list snapshots iter: report group dir in error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-24 19:57:10 +02:00
Dominik Csapak
07a683d266 pbs-client: extract: add top-level dir in tar.zst
when download a folder, include that folder as first entry (except '/')

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-22 11:35:55 +02:00
Dominik Csapak
7098f5d885 pbs-client: extract: rewrite create_zip with sequential decoder
instead of an async recursive function. Not only is it less code,
recursive futures are not really nice and it should be faster too.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-22 11:35:53 +02:00
Thomas Lamprecht
f37d8540e1 server pull: fix comment w.r.t. initial downloaded chunk capacity
> The hash set will be able to hold at least capacity elements
> without reallocating. If capacity is 0, the hash set will not
> allocate.
-- rustdoc, HashSet::with_capacity

So, the number we pass is the amount of chunk "IDs" we safe, which is
then 64Ki, not 16Ki and thus the size we can reference too is also
256 GiB, not 64 GiB.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-21 15:55:03 +02:00
Dietmar Maurer
eb1cd24e21 pbs-tape: sgutils2: check sense data when status is 'CHECK_CONDITION'
Some raid controllers return a 'transport error' when we expected a
'sense error'. it seems the correct way to check the sense data is when
either the result category is 'SENSE' or when the status is 'CHECK_CONDITION',
so do that. (similar to how 'sg_raw' returns the errors)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-04-21 09:35:52 +02:00
Wolfgang Bumiller
6da20161f0 reference the datastore in BackupGroup/Dir
And drop the base_path parameter on a first bunch of
functions (more reordering will follow).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 15:31:04 +02:00
Wolfgang Bumiller
bb628c295a api-types: DataStoreConfig::new for testing
so our examples can more easily access a datastore without
going over a configuration & cache

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 15:31:04 +02:00
Fabian Grünbichler
2c88dc97fd api2: read_remote: also return RemoteWithoutPassword
like for the index, instead of manually stripping it.

this (and the previous change) is backwards-compatible since `Remote`
already skipped serializing empty strings, so the returned JSON is
identical.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-04-20 13:58:41 +02:00
Wolfgang Bumiller
6b0c6492f7 datastore: cleanup and document backup group/dir openers
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 13:24:57 +02:00
Wolfgang Bumiller
10a0059602 datastore: drop Hash from BackupGroup
same as for Eq/Ord/...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 13:08:44 +02:00
Wolfgang Bumiller
5203cfcff9 datastore: drop PartialEq and PartialOrd from BackupGroup
Same as previous commits: this will be linked to a
particular DataStore and Eq/Ord is now only part of the
api types, for now.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 12:23:14 +02:00
Wolfgang Bumiller
cf320b6ba1 datastore: drop Eq and PartialEq from BackupDir
Same as previous commit: this is supposed to be connected to
a datastore and Eq/PartialEq only make sense for the
api-type part.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 12:22:57 +02:00
Wolfgang Bumiller
5116453b6d datastore: drop Ord from BackupGroup
This one is supposed to be linked to a datastore instance,
so it won't be Ord for now.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 12:20:30 +02:00
Wolfgang Bumiller
db87d93efc make datastore BackupGroup/Dir ctors private
And use the api-types for their contents.

These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:56:23 +02:00
Wolfgang Bumiller
38aa71fcc8 api-types: use BackupType for GroupFilter::BackupType
instead of a string

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:49:01 +02:00
Wolfgang Bumiller
1f6a45c938 rename BackupDir's group_path to relative_group_path
datastore's group_path will be moved to BackupDir soon and
this is required to be able to properly distinguish them

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 10:04:02 +02:00
Wolfgang Bumiller
b444eb68af api-types: datastore type improvements
let BackupGroup implement Hash

let BackupGroup and BackupDir be AsRef<BackupGroup>
let BackupDir be AsRef<BackupDir>

the pbs-datastore types will implement these AsRefs as well

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 10:03:39 +02:00
Wolfgang Bumiller
2d5c20c8f5 datastore: remove unused list_files function
it also doesn't belong into this type

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 10:00:33 +02:00
Wolfgang Bumiller
c4b2d26cdb datastore: move last_backup from BackupInfo to BackupGroup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 10:00:25 +02:00
Dietmar Maurer
fe94c9962e AuthId: derive Ord and PartialOrd
So the we can sort...

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-04-20 09:58:52 +02:00
Dietmar Maurer
24cb5c7a81 RemoteWithoutPassword: new API type
To make it explicit that we do not return the password.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-04-20 09:42:46 +02:00
Wolfgang Bumiller
988d575dbb api-types: introduce BackupType enum and Group/Dir api types
The type is a real enum.

All are API types and implement Display and FromStr. The
ordering is the same as it is in pbs-datastore.

Also, they are now flattened into a few structs instead of
being copied manually.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 13:12:46 +02:00
Thomas Lamprecht
33eb23d57e datastore: add snapshot iterator and provide example
will be more used in the future, when the upend-datastore master plan
comes in effect.

also a preparatory work for namespaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
249dde8b63 backup: switch over to streaming Iterator improving memory usage
Avoid collecting the whole group list in memory only to iterate and
filter over it again.

Note that the change could result in a indentation change, so best
viewed with `-w` flag.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
7b125de3e1 datastore: add helper to get a iterator for backup groups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
de015ce7e1 datastore: implement Iterator for backup group listing
While currently it's still only used in a collected() way, most call
sites can be switched over to use the iterator directly, as often
they already convert the not-so-cheap, in-memory vector back in
.into_iter() anyway.

somewhat also preparatory (yak shaving) work for namespaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht
72f8154571 api datastore: some code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00