Commit Graph

3962 Commits

Author SHA1 Message Date
Dietmar Maurer
414be8b675 tape: fix LTO locate_file for HP drives
Add test code to the first locate_file command, compute locate_offset.
Subsequent locate_file commands use that offset.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-30 09:08:58 +02:00
Dietmar Maurer
fda19dcc6f fix CachedUserInfo by using a shared memory version counter 2021-06-30 08:54:30 +02:00
Dietmar Maurer
3b7b1dfb8e api: add openid redirect/login API 2021-06-30 08:54:30 +02:00
Dietmar Maurer
d8a47ec649 cleanup user/token is_active() check 2021-06-30 08:54:30 +02:00
Dietmar Maurer
252cd3b781 implement new helper is_active_user_id() 2021-06-30 08:54:30 +02:00
Dietmar Maurer
0decd11efb cli: add CLI to manage openid realms. 2021-06-30 08:54:30 +02:00
Dietmar Maurer
b84d2592fb add API to manage openid realms 2021-06-30 08:54:30 +02:00
Dietmar Maurer
0219ba2cc5 check_acl_path: add /access/domains and /access/openid 2021-06-30 08:54:30 +02:00
Dietmar Maurer
bbff6c4968 config: new domains.cfg to configure openid realm
Or other realmy types...
2021-06-30 08:54:30 +02:00
Dominik Csapak
85417b2a88 docs: build api-viewer from widget-toolkit-dev
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:26:40 +02:00
Dominik Csapak
5667b76381 fix #3393 (again): pxar/create: try to read xattrs/fcaps/acls by default
we have a static list of filesystems and their capabilities regarding
file attributes and fs features (e.g. sockets/fifos/etc) which also
includes xattrs,acls and fcaps

if we did not know a filesystem by its magic number (for example cephfs),
we did not even attempt to read xattrs, etc.

this patch adds those flags by default to unknown filesystems, and
removes them when we encounter EOPNOTSUPP (to remove the number
of syscalls)

with this, we should be able to catch xattrs/acls/fcaps on all
(unknown) fs types that support them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-28 14:04:22 +02:00
Stefan Reiter
d9b318a444 file-restore/disk: support ZFS subvols with mountpoint=legacy
These require mounting using the regular 'mount' syscall.
Auto-generates an appropriate mount path.

Note that subvols with mountpoint=none cannot be mounted this way, and
would require setting the mountpoint property, which is not possible as
the zpools have to be imported with readonly=on.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-28 13:58:41 +02:00
Stefan Reiter
86ce56f193 file-restore/disk: support ZFS pools
Uses the ZFS utils to detect, import and mount zpools. These are
available as a new Bucket type 'zpool'.

Requires some minor changes to the existing disk and partiton detection
code, so the ZFS-specific part can use the information gathered in the
previous pass to associate drive names with their 'drive-xxxN.img.fidx'
node.

For detecting size, the zpool has to be imported. This is only done with
pools containing 5 or less disks, as anything else might take too long
(and should be seldomly found within VMs).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-28 13:58:41 +02:00
Stefan Reiter
8d72c2c32e file-restore: increase RAM for ZFS and disable ARC
Even through best efforts at keeping it small, including the ZFS tools
in the initramfs seems to have exhausted the small overhead we had left
- give it a bit more RAM to compensate.

Also disable the ZFS ARC, as it's no use in such a memory constrained
environment, and we cache on the QEMU/rust layer anyway.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-28 13:58:41 +02:00
Stefan Reiter
c48c38ab8c async_lru_cache: fix handling of errors in fetch
The future needs to be removed from the pending map in any case, even if
it returned an error, else all upcoming calls to access this key will
always return the same error.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-28 13:48:26 +02:00
Dominik Csapak
3d3769830b tape/helpers/snapshot_reader: sort chunks by inode (per index)
sort the chunks we want to backup to tape by inode, to gain some
speed on spinning disks. this is done per index, not globally.

costs a bit memory, but not too much, about 16 bytes per chunk which
would mean ~4MiB for a 1TiB index with 4MiB chunks.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 12:16:14 +02:00
Dominik Csapak
4921a411ad backup/datastore: refactor chunk inode sorting to the datastore
so that we can reuse that information

the removal of the adding to the corrupted list is ok, since
'get_chunks_in_order' returns them at the end of the list
and we do the same if the loading fails later in 'verify_index_chunks'
so we still mark them corrupt
(assuming that the load will fail if the stat does)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 12:14:52 +02:00
Dominik Csapak
81c767efce proxmox-backup-manager: show task log on datastore create
since the output:
Result: "<UPID>"
is not really interesting, show instead the task log while
the datastore is creating, since it is now run in a worker

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 12:09:55 +02:00
Hannes Laimer
60abf03f05 close #3459: manager: add --ignore-verified and --outdated-after parameters
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 11:03:51 +02:00
Hannes Laimer
dcbf29e71b api: add ignore-verified and outdated-after to datastore verify endpoint
preparatory change for fixing #3459

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 11:03:51 +02:00
Hannes Laimer
037e6c0ca8 verify-job: move snapshot filter into function
preparatory steps for fixing #3459

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Reviewed-By: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 11:03:44 +02:00
Fabian Grünbichler
90ff75f85c update to zstd 0.6
compatible with libzstd from bullseye.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-14 13:01:43 +02:00
Dietmar Maurer
2165f0d450 api: define and use REALM_ID_SCHEMA 2021-06-10 11:10:00 +02:00
Wolfgang Bumiller
1e7639bfc4 fixup minimum lru capacity
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-08 10:13:46 +02:00
Stefan Reiter
4121628d99 tools/lru_cache: make minimum capacity 1
Setting this to 0 is not just useless, but breaks the logic horribly
enough to cause random segfaults - better forbid this, to avoid someone
else having to debug it again ;)

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:55 +02:00
Stefan Reiter
da78b90f9c backup: remove AsyncIndexReader
superseded by CachedChunkReader, with less code and more speed

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:46 +02:00
Stefan Reiter
1ef6e8b6a7 replace AsyncIndexReader with SeekableCachedChunkReader
admin/datastore reads linearly only, so no need for cache (capacity of 1
basically means no cache except for the currently active chunk).
mount can do random access too, so cache last 8 chunks for possibly a
mild performance improvement.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:44 +02:00
Stefan Reiter
10351f7075 backup: add AsyncRead/Seek to CachedChunkReader
Implemented as a seperate struct SeekableCachedChunkReader that contains
the original as an Arc, since the read_at future captures the
CachedChunkReader, which would otherwise not work with the lifetimes
required by AsyncRead. This is also the reason we cannot use a shared
read buffer and have to allocate a new one for every read. It also means
that the struct items required for AsyncRead/Seek do not need to be
included in a regular CachedChunkReader.

This is intended as a replacement for AsyncIndexReader, so we have less
code duplication and can utilize the LRU cache there too (even though
actual request concurrency is not supported in these traits).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:40 +02:00
Stefan Reiter
70a152deb7 backup: add CachedChunkReader utilizing AsyncLruCache
Provides a fast arbitrary read implementation with full async and
concurrency support.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:37 +02:00
Stefan Reiter
5446bfbba8 tools: add AsyncLruCache as a wrapper around sync LruCache
Supports concurrent 'access' calls to the same key via a
BroadcastFuture. These are stored in a seperate HashMap, the LruCache
underneath is only modified once a valid value has been retrieved.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:34 +02:00
Stefan Reiter
400885e620 tools/BroadcastFuture: add testcase for better understanding
Explicitly test that data will stay available and can be retrieved
immediately via listen(), even if the future producing the data and
notifying the consumers was already run in the past.

Wasn't broken or anything, but helps with understanding IMO.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:29 +02:00
Dominik Csapak
f960fc3b6f fix #3433: use PVE's wearout logic in PBS
in PVE, the logic how wearout gets read from the smartctl output was
changed from a vendor -> id map to a sorted list of specific
attribute field names.

copy that list to pbs (in the same order), and use that to get the
wearout

in the future we might want to split the disk logic into its own crate
and reuse it in pve

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-08 08:31:37 +02:00
Dominik Csapak
d2354a16cd client/pull: log snapshots that are skipped because of time
we skip snapshots that are older than the newest snapshot of the group in
the target datastore, log it so the user can know why it is not synced

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-07 10:51:25 +02:00
Dominik Csapak
2de4dc3a81 backup/chunk_store: optionally log progress on creation
and enable it for the worker variants

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-04 09:32:09 +02:00
Dietmar Maurer
b90036dadd cleanup: factor out config::datastore::lock_config() 2021-06-04 09:04:14 +02:00
Dominik Csapak
4708f4fc21 api2/config/datastore: change create datastore api call to a worker
so that longer running creates (e.g. a slow storage), does not
run in a timeout and we can follow its creation

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 09:02:05 +02:00
Dominik Csapak
062cf75cdf proxmox-backup-proxy: fix leftover references on datastore removal
when we remove a datastore via api/cli, the proxy
has sometimes leftover references to that datastore in its
DATASTORE_MAP which includes an open filehandle on the
'.lock' file

this prevents unmounting/exporting the datastore even after removal,
only a reload/restart of the proxy did help

add a command to our command socket, which removes all non
configured datastores from the map, dropping the open filehandle

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 08:22:53 +02:00
Dominik Csapak
e5950360ca tape/drive: improve tape device locking behaviour
by implementing a custom error type that is either 'TimeOut' or
'Other'.

In the api, check in the worker loop for exactly 'TimeOut' errors and continue only
then. All other errors lead to a aborted task.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-02 17:08:00 +02:00
Dominik Csapak
5b358ff0b1 server/prune_job: fix locking during prune jobs
removing the backup dir must acquire the snapshot lock, else it can
happen that we remove a snapshot while it is being restored
or backed up to tape

the original commit that adds the force flag
(c9756b40d1)
mentions that the prune checks itself if the snapshot is in use,
but i could not find such code, so simply set force to false

to avoid failing and aborting the prune job, warn if it could not
and continue

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-02 17:04:49 +02:00
Fabian Grünbichler
3420029b5e Revert "file-restore-daemon: work around tokio DuplexStream bug"
This reverts commit 75f9f40922, which is
no longer needed now that we use tokio >= 1.6 which contains the proper
fix.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-06-01 10:31:19 +02:00
Fabian Grünbichler
3e3b505cc8 reorder serde usage/derive
this is deprecated with rustc 1.52+, and will become a hard error at
some point:

https://github.com/rust-lang/rust/issues/79202

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-31 14:53:08 +02:00
Dietmar Maurer
0bca966ec5 fix typo: s/dies/does/ 2021-05-31 11:01:15 +02:00
Dominik Csapak
84737fb33f lto/sg_tape/encryption: remove non lto-4 supported byte
from the SspDataEncryptionCapabilityPage

it seems we do not need it, since the EXTDECC flag is only used for
determining if the drive is capable to be configured via
ADI (Automation/Drive Interface) which we do not use at all.

this makes the call work with LTO-4 again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-31 10:58:38 +02:00
Dominik Csapak
03380db560 api2/tape: add api call to list media sets
we want a 'media-set' selector in the gui, this makes it
very easy to do and is not as costly as reusing the media list,
since we do not need to iterate over all media (e.g. unassigned)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak
c24cb13382 api: node/journal: fix parameter extraction of /nodes/node/journal
by extracting them via the api macro into the function signature

this fixes an issue, where giving 'since' and 'until' where not
used since we tried to extract them as 'str' while they were numbers.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-25 13:26:51 +02:00
Stefan Reiter
3a804a8a20 file-restore-daemon: limit concurrent download calls
While the issue with vsock packets starving kernel memory is mostly
worked around by the '64k -> 4k buffer' patch in
'proxmox-backup-restore-image', let's be safe and also limit the number
of concurrent transfers. 8 downloads per VM seems like a fair value.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Stefan Reiter
1fde4167ea file-restore-daemon: watchdog: add inhibit for long downloads
The extract API call may be active for more than the watchdog timeout,
so a simple ping is not enough.

This adds an "inhibit" API, which will stop the watchdog from completing
as long as at least one WatchdogInhibitor instance is alive. Keep one in
the download task, so it will be dropped once it completes (or errors).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Stefan Reiter
75f9f40922 file-restore-daemon: work around tokio DuplexStream bug
See this PR for more info: https://github.com/tokio-rs/tokio/pull/3756

As a workaround use a pair of connected unix sockets - this obviously
incurs some overhead, albeit not measureable on my machine. Once tokio
includes the fix we can go back to a DuplexStream for performance and
simplicity.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Thomas Lamprecht
e9c2638f90 apt: fix removal of non-existant http-proxy config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-25 11:54:46 +02:00
Oguz Bektas
338c545f85 tasks: fix typos in API description
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-05-25 07:54:57 +02:00
Stefan Reiter
e379b4a31c file-restore-daemon: disk: add RawFs bucket type
Used to specify a filesystem placed directly on a disk, without a
partition table inbetween. Detected by simply attempting to mount the
disk itself.

A helper "make_dev_node" is extracted to avoid code duplication.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 07:53:22 +02:00
Stefan Reiter
3d7ca2bdb9 file-restore-daemon: disk: allow arbitrary component count per bucket
A bucket might contain multiple (or 0) layers of components in its path
specification, so allow a mapping between bucket type strings and
expected component depth. For partitions, this is 1, as there is only
the partition number layer below the "part" node.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 07:53:22 +02:00
Stefan Reiter
d34019e246 file-restore-daemon: disk: ignore "invalid fs" error
Mainly just causes log spam, we print a more useful error in the end if
all mounts fail anyway.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 07:53:22 +02:00
Thomas Lamprecht
64591e731e api: status: graceful-degrade when a datastore lookup fails
This can happen if the underlying storage failed, in which case we do
not want to fail the whole API call, as it should report the status
of all datastores. So rather add the error inline to the related
store entry and continue.

Allows to nicely visualize those stores in the gui.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-21 13:20:55 +02:00
Thomas Lamprecht
64e0786aa9 api: datastore status: refactor reused rrd get-data code into closure
Nicer and shorter than just using a variable for the common parameters

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-21 13:20:55 +02:00
Thomas Lamprecht
90761f0f62 api: datastore status: code cleanup, reduce indentation level
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-21 13:20:55 +02:00
Wolfgang Bumiller
1d781c5b20 update proxmox-http dependency
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:29:24 +02:00
Fabian Grünbichler
7d2be91bc9 move SimpleHttp to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:32:33 +02:00
Fabian Grünbichler
578895336a SimpleHttp: factor out product-specific bits
in preparation of moving the abstraction to proxmox_http

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:32:22 +02:00
Fabian Grünbichler
8c090937f5 move tools::http to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:31:54 +02:00
Fabian Grünbichler
4229633d98 move ProxyConfig to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:31:27 +02:00
Fabian Grünbichler
3ed7e87538 HttpsConnector: make keepalive configurable
it's the only PBS-specific part in there, so let's make it
product-agnostic before moving it off to proxmox-http.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:31:15 +02:00
Fabian Grünbichler
5b43cc4487 move MaybeTlsStream wrapper to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:30:05 +02:00
Fabian Grünbichler
3241392117 refactor: move socket helper to proxmox crate
and constant to tools module.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:29:42 +02:00
Fabian Grünbichler
c474a66b41 move websocket to new 'proxmox_http' crate
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:26:41 +02:00
Dominik Csapak
f32791b4b2 api2/admin/datastore: add delete for groups
so that a user can delete a whole group at once, until now, the fastest
way for this was to prune to one snapshot, and delete that

code is basically a copy/paste from the snapshot delete, sans
the 'backup-time' parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-17 08:45:10 +02:00
Dominik Csapak
d19010481d tape/test: repair tests after changing 'start_write_session'
i added a parameter and forgot to adapt the tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-14 10:01:54 +02:00
Dominik Csapak
e953029e8f api2/tape/backup: add 'force-media-set' parameter to manual backup
so that a user can force a new media set, e.g. if he uses the
allocation policy 'continue', but wants to manually start a new
media-set.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-14 08:58:33 +02:00
Wolfgang Bumiller
126ccbcfa6 acme: improve errors when account loading fails
if the account does not exist, error with its name
if file loading fails, the error includes the full path
if the content fails to parse, show file & parse error
and in each case mention that it's about loading the acme account file

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-12 12:22:21 +02:00
Dietmar Maurer
440472cb32 correctly set apt proxy configuration 2021-05-12 12:19:24 +02:00
Dietmar Maurer
4ce7da516d reload cert inside command socket handler 2021-05-12 12:03:27 +02:00
Thomas Lamprecht
9fe4c79005 api: acme accounts: use name as worker ID
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 18:07:03 +02:00
Thomas Lamprecht
f09f4d5fd5 config: acme: rust format
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 17:35:54 +02:00
Thomas Lamprecht
38b4f9b534 config: acme: fall-back to the "default" account
syncs behavior with both, the displayed state in the PBS
web-interface, and the behavior of PVE/PMG.

Without this a standard setup would result in a Error like:
> TASK ERROR: no acme client configured

which was pretty confusing, as the actual error was something else
(no account configured), and the web-interface showed "default" as
selected account, so a user had no idea what actually was wrong and
how to fix it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 17:33:07 +02:00
Wolfgang Bumiller
fca1cef29f hot-reload proxy certificate when updating via the API
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller
45b8a0327f refactor send_command
- refactor the combinators,
- make it take a `&T: Serialize` instead of a Value, and
  allow sending the raw string via `send_raw_command`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller
a723c08715 proxy: implement 'reload-certificate' command
to be used via the command socket

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller
c381a162fb proxy: factor out tls acceptor creation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller
b4931192c3 proxy: Arc usage cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller
cc269b9ff9 proxy: "continue on error" for the accept call, too
as this gets rid of 2 levels of indentation

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller
a5e3be4992 proxy: factor out accept_connection
no functional changes, moved code and named the channel's
type for more readability

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Fabian Grünbichler
85f4e834d8 client: use stderr for all fingerprint confirm msgs
an interactive client might still want machine-readable output on
stdout.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Fabian Grünbichler
065013ccec client: refactor verification callback
return a result with optional fingerprint instead of tuple, allowing
easy extraction of a meaningful error message.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Fabian Grünbichler
56d98ba966 client: improve fingerprint variable names
and pass as reference instead of cloning.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Fabian Grünbichler
dda1b4fa44 fix #3391: improve mismatched fingerprint handling
if the expected fingerprint and the one returned by the server don't
match, print a warning and allow confirmation and proceeding if running
interactive.

previous:

$ proxmox-backup-client ...
Error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1915:

new:

$ proxmox-backup-client ...
WARNING: certificate fingerprint does not match expected fingerprint!
expected:    ac:cb:6a:bc:d6:b7:b4:77:3e:17:05:d6:b6:29:dd:1f:05:9c:2b:3a:df:84:3b:4d:f9:06:2c:be:da:06:52:12
fingerprint: ab:cb:6a:bc:d6:b7:b4:77:3e:17:05:d6:b6:29:dd:1f:05:9c:2b:3a:df:84:3b:4d:f9:06:2c:be:da:06:52:12
Are you sure you want to continue connecting? (y/n): n
Error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1915:

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Dominik Csapak
0ecdaa0dc0 bin/proxmox-tape: add optional snapshots to restore command
and add the appropriate completion helper

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:57:14 +02:00
Dominik Csapak
13f435caab tape/inventory: add completion helper for tape snapshots
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:56:55 +02:00
Dominik Csapak
ff99780303 api2/tape/restore: add optional snapshots to 'restore'
this makes it possible to only restore some snapshots from a tape media-set
instead of the whole. If the user selects only a small part, this will
probably be faster (and definitely uses less space on the target
datastores).

the user has to provide a list of snapshots to restore in the form of
'store:type/group/id'
e.g. 'mystore:ct/100/2021-01-01T00:00:00Z'

we achieve this by first restoring the index to a temp dir, retrieving
a list of chunks, and using the catalog, we generate a list of
media/files that we need to (partially) restore.

finally, we copy the snapshots to the correct dir in the datastore,
and clean up the temp dir

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:53:38 +02:00
Dominik Csapak
fa9507020a api2/tape/restore: refactor restore code into its own function
and create the 'email' and 'restore_owner' variable at the beginning,
so that we can reuse them and do not have to pass the sources of those
through too many functions

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:53:25 +02:00
Dietmar Maurer
1bff50afea tape locate_file: fix off by one error 2021-05-11 12:37:04 +02:00
Dominik Csapak
2d5d264f99 tape/pool_writer: do not unwrap on channel send
if the reader thread is already gone here, we panic here, resulting in
a nondescript error message, so simply ignore/warn in that case and
return gracefully

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 09:07:45 +02:00
Thomas Lamprecht
ea1458923e manager: acme plugin: auto-complete available DNS challenge types
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 15:55:49 +02:00
Thomas Lamprecht
e857f1fae8 completion: ACME plugin type: comment out http type for now, not useful
It may make sense in the future, e.g., if the built-in standalone
type is not enough, e.g., as HTTP**s**, HTTP 2 or even QUIC (HTTP 3)
is wanted in some setups, but for now there's no scenario where one
would profit from adding a new HTTP plugin, especially as it requires
the `data` property to be set, which makes no sense..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 15:50:08 +02:00
Thomas Lamprecht
3ec42e81b1 manager: acme plugin: remove ID completion helper from add command
we cannot add a plugin with an existing ID so this completion helper
is rather counterproductive...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 15:47:37 +02:00
Thomas Lamprecht
be1163acfe config: acme: drop now unused foreach_dns_plugin
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 14:41:44 +02:00
Thomas Lamprecht
d308dc8af7 acme: use proxmox-acme-plugins and load schema from there
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 14:41:12 +02:00
Thomas Lamprecht
60643023ad api: move AcmeChallengeSchema to acme types module
It will be reused in a later patch in another module which should not
depend on the actual API implementation (ugly and cyclic)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 14:39:07 +02:00
Thomas Lamprecht
875d53ef6c api: acme: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 11:56:38 +02:00
Wolfgang Bumiller
b41f9e9fec acme: fix bad nonce retry counter
Actually return the error on the 3rd try.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-10 11:52:04 +02:00
Dylan Whyte
a1b71c3c7d fix #3296: use proxy client to retrieve changelog
Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-05-10 08:48:52 +02:00