Commit Graph

3869 Commits

Author SHA1 Message Date
Dominik Csapak 03380db560 api2/tape: add api call to list media sets
we want a 'media-set' selector in the gui, this makes it
very easy to do and is not as costly as reusing the media list,
since we do not need to iterate over all media (e.g. unassigned)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-26 18:10:57 +02:00
Dominik Csapak c24cb13382 api: node/journal: fix parameter extraction of /nodes/node/journal
by extracting them via the api macro into the function signature

this fixes an issue, where giving 'since' and 'until' where not
used since we tried to extract them as 'str' while they were numbers.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-25 13:26:51 +02:00
Stefan Reiter 3a804a8a20 file-restore-daemon: limit concurrent download calls
While the issue with vsock packets starving kernel memory is mostly
worked around by the '64k -> 4k buffer' patch in
'proxmox-backup-restore-image', let's be safe and also limit the number
of concurrent transfers. 8 downloads per VM seems like a fair value.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Stefan Reiter 1fde4167ea file-restore-daemon: watchdog: add inhibit for long downloads
The extract API call may be active for more than the watchdog timeout,
so a simple ping is not enough.

This adds an "inhibit" API, which will stop the watchdog from completing
as long as at least one WatchdogInhibitor instance is alive. Keep one in
the download task, so it will be dropped once it completes (or errors).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Stefan Reiter 75f9f40922 file-restore-daemon: work around tokio DuplexStream bug
See this PR for more info: https://github.com/tokio-rs/tokio/pull/3756

As a workaround use a pair of connected unix sockets - this obviously
incurs some overhead, albeit not measureable on my machine. Once tokio
includes the fix we can go back to a DuplexStream for performance and
simplicity.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 11:56:43 +02:00
Thomas Lamprecht e9c2638f90 apt: fix removal of non-existant http-proxy config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-25 11:54:46 +02:00
Oguz Bektas 338c545f85 tasks: fix typos in API description
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-05-25 07:54:57 +02:00
Stefan Reiter e379b4a31c file-restore-daemon: disk: add RawFs bucket type
Used to specify a filesystem placed directly on a disk, without a
partition table inbetween. Detected by simply attempting to mount the
disk itself.

A helper "make_dev_node" is extracted to avoid code duplication.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 07:53:22 +02:00
Stefan Reiter 3d7ca2bdb9 file-restore-daemon: disk: allow arbitrary component count per bucket
A bucket might contain multiple (or 0) layers of components in its path
specification, so allow a mapping between bucket type strings and
expected component depth. For partitions, this is 1, as there is only
the partition number layer below the "part" node.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 07:53:22 +02:00
Stefan Reiter d34019e246 file-restore-daemon: disk: ignore "invalid fs" error
Mainly just causes log spam, we print a more useful error in the end if
all mounts fail anyway.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-05-25 07:53:22 +02:00
Thomas Lamprecht 64591e731e api: status: graceful-degrade when a datastore lookup fails
This can happen if the underlying storage failed, in which case we do
not want to fail the whole API call, as it should report the status
of all datastores. So rather add the error inline to the related
store entry and continue.

Allows to nicely visualize those stores in the gui.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-21 13:20:55 +02:00
Thomas Lamprecht 64e0786aa9 api: datastore status: refactor reused rrd get-data code into closure
Nicer and shorter than just using a variable for the common parameters

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-21 13:20:55 +02:00
Thomas Lamprecht 90761f0f62 api: datastore status: code cleanup, reduce indentation level
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-21 13:20:55 +02:00
Wolfgang Bumiller 1d781c5b20 update proxmox-http dependency
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-17 11:29:24 +02:00
Fabian Grünbichler 7d2be91bc9 move SimpleHttp to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:32:33 +02:00
Fabian Grünbichler 578895336a SimpleHttp: factor out product-specific bits
in preparation of moving the abstraction to proxmox_http

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:32:22 +02:00
Fabian Grünbichler 8c090937f5 move tools::http to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:31:54 +02:00
Fabian Grünbichler 4229633d98 move ProxyConfig to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:31:27 +02:00
Fabian Grünbichler 3ed7e87538 HttpsConnector: make keepalive configurable
it's the only PBS-specific part in there, so let's make it
product-agnostic before moving it off to proxmox-http.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:31:15 +02:00
Fabian Grünbichler 5b43cc4487 move MaybeTlsStream wrapper to proxmox_http
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:30:05 +02:00
Fabian Grünbichler 3241392117 refactor: move socket helper to proxmox crate
and constant to tools module.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:29:42 +02:00
Fabian Grünbichler c474a66b41 move websocket to new 'proxmox_http' crate
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:26:41 +02:00
Dominik Csapak f32791b4b2 api2/admin/datastore: add delete for groups
so that a user can delete a whole group at once, until now, the fastest
way for this was to prune to one snapshot, and delete that

code is basically a copy/paste from the snapshot delete, sans
the 'backup-time' parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-17 08:45:10 +02:00
Dominik Csapak d19010481d tape/test: repair tests after changing 'start_write_session'
i added a parameter and forgot to adapt the tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-14 10:01:54 +02:00
Dominik Csapak e953029e8f api2/tape/backup: add 'force-media-set' parameter to manual backup
so that a user can force a new media set, e.g. if he uses the
allocation policy 'continue', but wants to manually start a new
media-set.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-14 08:58:33 +02:00
Wolfgang Bumiller 126ccbcfa6 acme: improve errors when account loading fails
if the account does not exist, error with its name
if file loading fails, the error includes the full path
if the content fails to parse, show file & parse error
and in each case mention that it's about loading the acme account file

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-12 12:22:21 +02:00
Dietmar Maurer 440472cb32 correctly set apt proxy configuration 2021-05-12 12:19:24 +02:00
Dietmar Maurer 4ce7da516d reload cert inside command socket handler 2021-05-12 12:03:27 +02:00
Thomas Lamprecht 9fe4c79005 api: acme accounts: use name as worker ID
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 18:07:03 +02:00
Thomas Lamprecht f09f4d5fd5 config: acme: rust format
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 17:35:54 +02:00
Thomas Lamprecht 38b4f9b534 config: acme: fall-back to the "default" account
syncs behavior with both, the displayed state in the PBS
web-interface, and the behavior of PVE/PMG.

Without this a standard setup would result in a Error like:
> TASK ERROR: no acme client configured

which was pretty confusing, as the actual error was something else
(no account configured), and the web-interface showed "default" as
selected account, so a user had no idea what actually was wrong and
how to fix it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-11 17:33:07 +02:00
Wolfgang Bumiller fca1cef29f hot-reload proxy certificate when updating via the API
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller 45b8a0327f refactor send_command
- refactor the combinators,
- make it take a `&T: Serialize` instead of a Value, and
  allow sending the raw string via `send_raw_command`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller a723c08715 proxy: implement 'reload-certificate' command
to be used via the command socket

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller c381a162fb proxy: factor out tls acceptor creation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller b4931192c3 proxy: Arc usage cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller cc269b9ff9 proxy: "continue on error" for the accept call, too
as this gets rid of 2 levels of indentation

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller a5e3be4992 proxy: factor out accept_connection
no functional changes, moved code and named the channel's
type for more readability

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Fabian Grünbichler 85f4e834d8 client: use stderr for all fingerprint confirm msgs
an interactive client might still want machine-readable output on
stdout.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Fabian Grünbichler 065013ccec client: refactor verification callback
return a result with optional fingerprint instead of tuple, allowing
easy extraction of a meaningful error message.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Fabian Grünbichler 56d98ba966 client: improve fingerprint variable names
and pass as reference instead of cloning.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Fabian Grünbichler dda1b4fa44 fix #3391: improve mismatched fingerprint handling
if the expected fingerprint and the one returned by the server don't
match, print a warning and allow confirmation and proceeding if running
interactive.

previous:

$ proxmox-backup-client ...
Error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1915:

new:

$ proxmox-backup-client ...
WARNING: certificate fingerprint does not match expected fingerprint!
expected:    ac:cb:6a:bc:d6:b7:b4:77:3e:17:05:d6:b6:29:dd:1f:05:9c:2b:3a:df:84:3b:4d:f9:06:2c:be:da:06:52:12
fingerprint: ab:cb:6a:bc:d6:b7:b4:77:3e:17:05:d6:b6:29:dd:1f:05:9c:2b:3a:df:84:3b:4d:f9:06:2c:be:da:06:52:12
Are you sure you want to continue connecting? (y/n): n
Error: error trying to connect: error:1416F086:SSL routines:tls_process_server_certificate:certificate verify failed:../ssl/statem/statem_clnt.c:1915:

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-11 13:12:54 +02:00
Dominik Csapak 0ecdaa0dc0 bin/proxmox-tape: add optional snapshots to restore command
and add the appropriate completion helper

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:57:14 +02:00
Dominik Csapak 13f435caab tape/inventory: add completion helper for tape snapshots
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:56:55 +02:00
Dominik Csapak ff99780303 api2/tape/restore: add optional snapshots to 'restore'
this makes it possible to only restore some snapshots from a tape media-set
instead of the whole. If the user selects only a small part, this will
probably be faster (and definitely uses less space on the target
datastores).

the user has to provide a list of snapshots to restore in the form of
'store:type/group/id'
e.g. 'mystore:ct/100/2021-01-01T00:00:00Z'

we achieve this by first restoring the index to a temp dir, retrieving
a list of chunks, and using the catalog, we generate a list of
media/files that we need to (partially) restore.

finally, we copy the snapshots to the correct dir in the datastore,
and clean up the temp dir

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:53:38 +02:00
Dominik Csapak fa9507020a api2/tape/restore: refactor restore code into its own function
and create the 'email' and 'restore_owner' variable at the beginning,
so that we can reuse them and do not have to pass the sources of those
through too many functions

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 12:53:25 +02:00
Dietmar Maurer 1bff50afea tape locate_file: fix off by one error 2021-05-11 12:37:04 +02:00
Dominik Csapak 2d5d264f99 tape/pool_writer: do not unwrap on channel send
if the reader thread is already gone here, we panic here, resulting in
a nondescript error message, so simply ignore/warn in that case and
return gracefully

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-11 09:07:45 +02:00
Thomas Lamprecht ea1458923e manager: acme plugin: auto-complete available DNS challenge types
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 15:55:49 +02:00
Thomas Lamprecht e857f1fae8 completion: ACME plugin type: comment out http type for now, not useful
It may make sense in the future, e.g., if the built-in standalone
type is not enough, e.g., as HTTP**s**, HTTP 2 or even QUIC (HTTP 3)
is wanted in some setups, but for now there's no scenario where one
would profit from adding a new HTTP plugin, especially as it requires
the `data` property to be set, which makes no sense..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-10 15:50:08 +02:00