Commit Graph

6493 Commits

Author SHA1 Message Date
Stefan Reiter 5446bfbba8 tools: add AsyncLruCache as a wrapper around sync LruCache
Supports concurrent 'access' calls to the same key via a
BroadcastFuture. These are stored in a seperate HashMap, the LruCache
underneath is only modified once a valid value has been retrieved.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:34 +02:00
Stefan Reiter 400885e620 tools/BroadcastFuture: add testcase for better understanding
Explicitly test that data will stay available and can be retrieved
immediately via listen(), even if the future producing the data and
notifying the consumers was already run in the past.

Wasn't broken or anything, but helps with understanding IMO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:29 +02:00
Dominik Csapak f960fc3b6f fix #3433: use PVE's wearout logic in PBS
in PVE, the logic how wearout gets read from the smartctl output was
changed from a vendor -> id map to a sorted list of specific
attribute field names.

copy that list to pbs (in the same order), and use that to get the
wearout

in the future we might want to split the disk logic into its own crate
and reuse it in pve

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-08 08:31:37 +02:00
Thomas Lamprecht ddfa4d679a ui: tape: DriveSelector: make wider and fine-tune column flex
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:57:45 +02:00
Thomas Lamprecht 10e8026786 ui: tape: DriveSelector: code cleanup, group config together
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:30:57 +02:00
Dominik Csapak 2527c039df ui: tape: TapeBackupJob: use correct default value for pbsUserSelector
if we want the empty value as a valid default value in a combogrid,
we have to explicitely select 'null' else the field will be marked as
dirty

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:30:29 +02:00
Dominik Csapak 93d8a2044e ui: tape: DriveSelector: do not autoselect the drive
in case an invalid drive was configured, now it marks the field
invalid instead of autoselecting the first valid one

this could have lead to users configuring the wrong drive in a
tape-backup-job when they edited one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:30:13 +02:00
Dominik Csapak d2354a16cd client/pull: log snapshots that are skipped because of time
we skip snapshots that are older than the newest snapshot of the group in
the target datastore, log it so the user can know why it is not synced

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-07 10:51:25 +02:00
Dominik Csapak 34ee1f1c76 ui: DataStoreList: add remove button
so that a user can remove a datastore from the gui,
though no data is deleted, this has to be done elsewhere (for now)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-04 09:41:52 +02:00
Dominik Csapak 2de4dc3a81 backup/chunk_store: optionally log progress on creation
and enable it for the worker variants

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-04 09:32:09 +02:00
Dietmar Maurer b90036dadd cleanup: factor out config::datastore::lock_config() 2021-06-04 09:04:14 +02:00
Dominik Csapak 4708f4fc21 api2/config/datastore: change create datastore api call to a worker
so that longer running creates (e.g. a slow storage), does not
run in a timeout and we can follow its creation

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 09:02:05 +02:00
Dominik Csapak 062cf75cdf proxmox-backup-proxy: fix leftover references on datastore removal
when we remove a datastore via api/cli, the proxy
has sometimes leftover references to that datastore in its
DATASTORE_MAP which includes an open filehandle on the
'.lock' file

this prevents unmounting/exporting the datastore even after removal,
only a reload/restart of the proxy did help

add a command to our command socket, which removes all non
configured datastores from the map, dropping the open filehandle

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 08:22:53 +02:00
Dominik Csapak e5950360ca tape/drive: improve tape device locking behaviour
by implementing a custom error type that is either 'TimeOut' or
'Other'.

In the api, check in the worker loop for exactly 'TimeOut' errors and continue only
then. All other errors lead to a aborted task.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-02 17:08:00 +02:00
Dominik Csapak 5b358ff0b1 server/prune_job: fix locking during prune jobs
removing the backup dir must acquire the snapshot lock, else it can
happen that we remove a snapshot while it is being restored
or backed up to tape

the original commit that adds the force flag
(c9756b40d1)
mentions that the prune checks itself if the snapshot is in use,
but i could not find such code, so simply set force to false

to avoid failing and aborting the prune job, warn if it could not
and continue

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-02 17:04:49 +02:00
Dominik Csapak 4c00391d78 ui: dashboard/TaskSummary: add type 'close' to the close tool
otherwise the button is not visible

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-01 16:38:07 +02:00
Dominik Csapak 9594362e35 ui: datastore/DataStoreListSummary: catch and show errors per datastore
so that the update does not get canceled because of a bad datastore
hide the irrelevant fields in that case

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-01 16:38:06 +02:00
Fabian Grünbichler 3420029b5e Revert "file-restore-daemon: work around tokio DuplexStream bug"
This reverts commit 75f9f40922, which is
no longer needed now that we use tokio >= 1.6 which contains the proper
fix.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-01 10:31:19 +02:00
Fabian Grünbichler f432a1c927 bump tokio dependency to 1.6
it contains a bug fix that allows dropping the workaround in

75f9f40922 file-restore-daemon: work around tokio DuplexStream bug

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-01 10:30:57 +02:00
Dietmar Maurer e8b32f2d87 bump version to 1.1.9-1 2021-06-01 08:27:18 +02:00
Fabian Grünbichler 3e3b505cc8 reorder serde usage/derive
this is deprecated with rustc 1.52+, and will become a hard error at
some point:

https://github.com/rust-lang/rust/issues/79202

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-31 14:53:08 +02:00
Dietmar Maurer 0bca966ec5 fix typo: s/dies/does/ 2021-05-31 11:01:15 +02:00
Dominik Csapak 84737fb33f lto/sg_tape/encryption: remove non lto-4 supported byte
from the SspDataEncryptionCapabilityPage

it seems we do not need it, since the EXTDECC flag is only used for
determining if the drive is capable to be configured via
ADI (Automation/Drive Interface) which we do not use at all.

this makes the call work with LTO-4 again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-31 10:58:38 +02:00
Thomas Lamprecht e21a15ab17 ui: tape: s/Restore Wizard/Restore/
Mostly to avoid an extra translation text for basically no gain.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-27 11:26:46 +02:00
Dominik Csapak 90066d22a0 ui: MainView: use new beforeChangePath signature
subpath can be optional in extjs 7.0, so handle that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak dbf5dad1c4 ui: css: fix text-align pmx-button-badge
this was previously set on the button class, but has since been removed
add it here to have the badge number centered again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak c793da1edc ui: MainView: navigation: use different ui class
by default the treelist gets the 'nav' ui, which in newer extjs
versions has a custom styling (unlike before)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak f8735e5988 ui: datastore/Summary: change destroy listener
by using beforedestroy instead of destroy (like we do everywhere else)
to avoid race condition when the controller has
already removed some handlers on destruction

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak e9805b2486 ui: panel/UsageChart: change downloadServerUrl
to not have the sencha url by default

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak eb90405a78 ui: form/CalendarEvent: do not set displayField
we use displayTpl here, setting displayField will override the template

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak ecf5f468c3 ui: MainView: do not use unnecessary panels
using container here is fine, we do not need panel behaviour which
is more bloated. Removes two ARIA warnings.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Thomas Lamprecht 51aee8cac8 ui: tape restore wizard: set emptyText to media set selector
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:10:16 +02:00
Thomas Lamprecht 7d5049c350 ui: tape restore wizard: always show snapshot grid
looks (almost confusingly) empty else and no real disadvantage in
showing the disabled one until a media-set is selected and loaded

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:09:36 +02:00
Thomas Lamprecht 01a99f5651 ui: tape overview: rename to "Restore Wizard" and use icons
To create a correlation with the restore action column

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:05:31 +02:00
Thomas Lamprecht 2914e99ff3 ui: tape overview: use correct icon for Media-Pools
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:04:07 +02:00
Thomas Lamprecht f9b824ac30 ui: tape overview: include more context in restore tooltips
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:03:35 +02:00
Thomas Lamprecht 9a535ec77b ui: tape restore: small code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:00:12 +02:00
Dominik Csapak ffba023c91 ui: tape/TapeRestore: fix some properties
remove leftover from when it was an Proxmox.window.Edit, and
add the missing 'modal'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak e01689978e ui: tape/TapeRestore: allow preselecting a datastore
for that we need to split the prefilter additions, else
we always filter the snaphots too and giving 'undefined' filters
all snapshots...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak 68ac8976eb ui: tape/TapeRestore: don't send snapshotlist when restoring whole datastores
for the case that the user selects only whole datastores, we do not
want to send and (exhaustive) list of snapshots that get restored,
but we only want to honor the mapping the user gives

this avoids using the backup restore codepath that iterates twice
over the tapes and would generally be slower for a lot of snapshots

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak afb790db73 ui: tape/BackupOverview: add generic 'Restore' button
this will open the restore window without anything preselected

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak 0732de361a ui: tape/TapeRestore: add MediaSetSelector
when no uuid/mediaset is given.
we change a bit how we use the uuid by moving it into the viewmodel
(instead of a simple property on the view) so that we can always
use the selected one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak d455270fa1 ui: tape: add MediaSetSelector
so that we can let the user select a media-set

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak 1336be16c9 ui: tape/BackupOverview: rename action column to restore
to make it clear that this button is for restore and for
now we do not have any plans to add buttons here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak 03380db560 api2/tape: add api call to list media sets
we want a 'media-set' selector in the gui, this makes it
very easy to do and is not as costly as reusing the media list,
since we do not need to iterate over all media (e.g. unassigned)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak 927ebc702c ui: tape/BackupOverview: expand pools by default
normally, users will not have many tape media pools,
and are more interested in the actual media-sets, so
expand those nodes by default

if the list gets very long, the user can collapse some pools anyway

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak c24cb13382 api: node/journal: fix parameter extraction of /nodes/node/journal
by extracting them via the api macro into the function signature

this fixes an issue, where giving 'since' and 'until' where not
used since we tried to extract them as 'str' while they were numbers.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-25 13:26:51 +02:00
Stefan Reiter 3a804a8a20 file-restore-daemon: limit concurrent download calls
While the issue with vsock packets starving kernel memory is mostly
worked around by the '64k -> 4k buffer' patch in
'proxmox-backup-restore-image', let's be safe and also limit the number
of concurrent transfers. 8 downloads per VM seems like a fair value.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Stefan Reiter 1fde4167ea file-restore-daemon: watchdog: add inhibit for long downloads
The extract API call may be active for more than the watchdog timeout,
so a simple ping is not enough.

This adds an "inhibit" API, which will stop the watchdog from completing
as long as at least one WatchdogInhibitor instance is alive. Keep one in
the download task, so it will be dropped once it completes (or errors).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Stefan Reiter 75f9f40922 file-restore-daemon: work around tokio DuplexStream bug
See this PR for more info: https://github.com/tokio-rs/tokio/pull/3756

As a workaround use a pair of connected unix sockets - this obviously
incurs some overhead, albeit not measureable on my machine. Once tokio
includes the fix we can go back to a DuplexStream for performance and
simplicity.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00