Commit Graph

6050 Commits

Author SHA1 Message Date
Dietmar Maurer
2165f0d450 api: define and use REALM_ID_SCHEMA 2021-06-10 11:10:00 +02:00
Wolfgang Bumiller
1e7639bfc4 fixup minimum lru capacity
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-08 10:13:46 +02:00
Stefan Reiter
4121628d99 tools/lru_cache: make minimum capacity 1
Setting this to 0 is not just useless, but breaks the logic horribly
enough to cause random segfaults - better forbid this, to avoid someone
else having to debug it again ;)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:55 +02:00
Stefan Reiter
da78b90f9c backup: remove AsyncIndexReader
superseded by CachedChunkReader, with less code and more speed

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:46 +02:00
Stefan Reiter
1ef6e8b6a7 replace AsyncIndexReader with SeekableCachedChunkReader
admin/datastore reads linearly only, so no need for cache (capacity of 1
basically means no cache except for the currently active chunk).
mount can do random access too, so cache last 8 chunks for possibly a
mild performance improvement.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:44 +02:00
Stefan Reiter
10351f7075 backup: add AsyncRead/Seek to CachedChunkReader
Implemented as a seperate struct SeekableCachedChunkReader that contains
the original as an Arc, since the read_at future captures the
CachedChunkReader, which would otherwise not work with the lifetimes
required by AsyncRead. This is also the reason we cannot use a shared
read buffer and have to allocate a new one for every read. It also means
that the struct items required for AsyncRead/Seek do not need to be
included in a regular CachedChunkReader.

This is intended as a replacement for AsyncIndexReader, so we have less
code duplication and can utilize the LRU cache there too (even though
actual request concurrency is not supported in these traits).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:40 +02:00
Stefan Reiter
70a152deb7 backup: add CachedChunkReader utilizing AsyncLruCache
Provides a fast arbitrary read implementation with full async and
concurrency support.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:37 +02:00
Stefan Reiter
5446bfbba8 tools: add AsyncLruCache as a wrapper around sync LruCache
Supports concurrent 'access' calls to the same key via a
BroadcastFuture. These are stored in a seperate HashMap, the LruCache
underneath is only modified once a valid value has been retrieved.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:34 +02:00
Stefan Reiter
400885e620 tools/BroadcastFuture: add testcase for better understanding
Explicitly test that data will stay available and can be retrieved
immediately via listen(), even if the future producing the data and
notifying the consumers was already run in the past.

Wasn't broken or anything, but helps with understanding IMO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:29 +02:00
Dominik Csapak
f960fc3b6f fix #3433: use PVE's wearout logic in PBS
in PVE, the logic how wearout gets read from the smartctl output was
changed from a vendor -> id map to a sorted list of specific
attribute field names.

copy that list to pbs (in the same order), and use that to get the
wearout

in the future we might want to split the disk logic into its own crate
and reuse it in pve

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-08 08:31:37 +02:00
Thomas Lamprecht
ddfa4d679a ui: tape: DriveSelector: make wider and fine-tune column flex
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:57:45 +02:00
Thomas Lamprecht
10e8026786 ui: tape: DriveSelector: code cleanup, group config together
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:30:57 +02:00
Dominik Csapak
2527c039df ui: tape: TapeBackupJob: use correct default value for pbsUserSelector
if we want the empty value as a valid default value in a combogrid,
we have to explicitely select 'null' else the field will be marked as
dirty

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:30:29 +02:00
Dominik Csapak
93d8a2044e ui: tape: DriveSelector: do not autoselect the drive
in case an invalid drive was configured, now it marks the field
invalid instead of autoselecting the first valid one

this could have lead to users configuring the wrong drive in a
tape-backup-job when they edited one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-07 16:30:13 +02:00
Dominik Csapak
d2354a16cd client/pull: log snapshots that are skipped because of time
we skip snapshots that are older than the newest snapshot of the group in
the target datastore, log it so the user can know why it is not synced

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-07 10:51:25 +02:00
Dominik Csapak
34ee1f1c76 ui: DataStoreList: add remove button
so that a user can remove a datastore from the gui,
though no data is deleted, this has to be done elsewhere (for now)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-04 09:41:52 +02:00
Dominik Csapak
2de4dc3a81 backup/chunk_store: optionally log progress on creation
and enable it for the worker variants

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-04 09:32:09 +02:00
Dietmar Maurer
b90036dadd cleanup: factor out config::datastore::lock_config() 2021-06-04 09:04:14 +02:00
Dominik Csapak
4708f4fc21 api2/config/datastore: change create datastore api call to a worker
so that longer running creates (e.g. a slow storage), does not
run in a timeout and we can follow its creation

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 09:02:05 +02:00
Dominik Csapak
062cf75cdf proxmox-backup-proxy: fix leftover references on datastore removal
when we remove a datastore via api/cli, the proxy
has sometimes leftover references to that datastore in its
DATASTORE_MAP which includes an open filehandle on the
'.lock' file

this prevents unmounting/exporting the datastore even after removal,
only a reload/restart of the proxy did help

add a command to our command socket, which removes all non
configured datastores from the map, dropping the open filehandle

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 08:22:53 +02:00
Dominik Csapak
e5950360ca tape/drive: improve tape device locking behaviour
by implementing a custom error type that is either 'TimeOut' or
'Other'.

In the api, check in the worker loop for exactly 'TimeOut' errors and continue only
then. All other errors lead to a aborted task.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-02 17:08:00 +02:00
Dominik Csapak
5b358ff0b1 server/prune_job: fix locking during prune jobs
removing the backup dir must acquire the snapshot lock, else it can
happen that we remove a snapshot while it is being restored
or backed up to tape

the original commit that adds the force flag
(c9756b40d1)
mentions that the prune checks itself if the snapshot is in use,
but i could not find such code, so simply set force to false

to avoid failing and aborting the prune job, warn if it could not
and continue

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-02 17:04:49 +02:00
Dominik Csapak
4c00391d78 ui: dashboard/TaskSummary: add type 'close' to the close tool
otherwise the button is not visible

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-01 16:38:07 +02:00
Dominik Csapak
9594362e35 ui: datastore/DataStoreListSummary: catch and show errors per datastore
so that the update does not get canceled because of a bad datastore
hide the irrelevant fields in that case

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-01 16:38:06 +02:00
Fabian Grünbichler
3420029b5e Revert "file-restore-daemon: work around tokio DuplexStream bug"
This reverts commit 75f9f40922, which is
no longer needed now that we use tokio >= 1.6 which contains the proper
fix.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-01 10:31:19 +02:00
Fabian Grünbichler
f432a1c927 bump tokio dependency to 1.6
it contains a bug fix that allows dropping the workaround in

75f9f40922 file-restore-daemon: work around tokio DuplexStream bug

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-01 10:30:57 +02:00
Dietmar Maurer
e8b32f2d87 bump version to 1.1.9-1 2021-06-01 08:27:18 +02:00
Fabian Grünbichler
3e3b505cc8 reorder serde usage/derive
this is deprecated with rustc 1.52+, and will become a hard error at
some point:

https://github.com/rust-lang/rust/issues/79202

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-31 14:53:08 +02:00
Dietmar Maurer
0bca966ec5 fix typo: s/dies/does/ 2021-05-31 11:01:15 +02:00
Dominik Csapak
84737fb33f lto/sg_tape/encryption: remove non lto-4 supported byte
from the SspDataEncryptionCapabilityPage

it seems we do not need it, since the EXTDECC flag is only used for
determining if the drive is capable to be configured via
ADI (Automation/Drive Interface) which we do not use at all.

this makes the call work with LTO-4 again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-31 10:58:38 +02:00
Thomas Lamprecht
e21a15ab17 ui: tape: s/Restore Wizard/Restore/
Mostly to avoid an extra translation text for basically no gain.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-27 11:26:46 +02:00
Dominik Csapak
90066d22a0 ui: MainView: use new beforeChangePath signature
subpath can be optional in extjs 7.0, so handle that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak
dbf5dad1c4 ui: css: fix text-align pmx-button-badge
this was previously set on the button class, but has since been removed
add it here to have the badge number centered again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak
c793da1edc ui: MainView: navigation: use different ui class
by default the treelist gets the 'nav' ui, which in newer extjs
versions has a custom styling (unlike before)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak
f8735e5988 ui: datastore/Summary: change destroy listener
by using beforedestroy instead of destroy (like we do everywhere else)
to avoid race condition when the controller has
already removed some handlers on destruction

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak
e9805b2486 ui: panel/UsageChart: change downloadServerUrl
to not have the sencha url by default

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak
eb90405a78 ui: form/CalendarEvent: do not set displayField
we use displayTpl here, setting displayField will override the template

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Dominik Csapak
ecf5f468c3 ui: MainView: do not use unnecessary panels
using container here is fine, we do not need panel behaviour which
is more bloated. Removes two ARIA warnings.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 19:18:40 +02:00
Thomas Lamprecht
51aee8cac8 ui: tape restore wizard: set emptyText to media set selector
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:10:16 +02:00
Thomas Lamprecht
7d5049c350 ui: tape restore wizard: always show snapshot grid
looks (almost confusingly) empty else and no real disadvantage in
showing the disabled one until a media-set is selected and loaded

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:09:36 +02:00
Thomas Lamprecht
01a99f5651 ui: tape overview: rename to "Restore Wizard" and use icons
To create a correlation with the restore action column

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:05:31 +02:00
Thomas Lamprecht
2914e99ff3 ui: tape overview: use correct icon for Media-Pools
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:04:07 +02:00
Thomas Lamprecht
f9b824ac30 ui: tape overview: include more context in restore tooltips
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:03:35 +02:00
Thomas Lamprecht
9a535ec77b ui: tape restore: small code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-26 19:00:12 +02:00
Dominik Csapak
ffba023c91 ui: tape/TapeRestore: fix some properties
remove leftover from when it was an Proxmox.window.Edit, and
add the missing 'modal'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak
e01689978e ui: tape/TapeRestore: allow preselecting a datastore
for that we need to split the prefilter additions, else
we always filter the snaphots too and giving 'undefined' filters
all snapshots...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak
68ac8976eb ui: tape/TapeRestore: don't send snapshotlist when restoring whole datastores
for the case that the user selects only whole datastores, we do not
want to send and (exhaustive) list of snapshots that get restored,
but we only want to honor the mapping the user gives

this avoids using the backup restore codepath that iterates twice
over the tapes and would generally be slower for a lot of snapshots

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak
afb790db73 ui: tape/BackupOverview: add generic 'Restore' button
this will open the restore window without anything preselected

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak
0732de361a ui: tape/TapeRestore: add MediaSetSelector
when no uuid/mediaset is given.
we change a bit how we use the uuid by moving it into the viewmodel
(instead of a simple property on the view) so that we can always
use the selected one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak
d455270fa1 ui: tape: add MediaSetSelector
so that we can let the user select a media-set

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00