Commit Graph

5891 Commits

Author SHA1 Message Date
Wolfgang Bumiller
83771aa037 move tools::process_locker to pbs-tools
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 14:16:34 +02:00
Wolfgang Bumiller
95f9d67ce9 move UPID to pbs-api-types, add UPIDExt
pbs-server side related methods are added via the UPIDExt
trait

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 13:51:03 +02:00
Wolfgang Bumiller
314d360fcd buildsys: run tests on entire workspace by default
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 12:17:10 +02:00
Wolfgang Bumiller
f8a74456cc test fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 12:17:10 +02:00
Wolfgang Bumiller
4906bac10f linking fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 11:59:33 +02:00
Wolfgang Bumiller
86c831a5c3 fixup examples
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 11:49:42 +02:00
Wolfgang Bumiller
a5951b4f38 move manifest and backup_info to pbs-datastore
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 11:34:56 +02:00
Wolfgang Bumiller
f75292bd8d move tools::json to pbs-tools
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 11:34:56 +02:00
Wolfgang Bumiller
bfff4eaa7f move backup id related types to pbs-api-types
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 11:34:56 +02:00
Wolfgang Bumiller
067dc06dba add pbs-systemd: move string and unit handling there
the systemd config/unit parsing stays in pbs for now since
that's not usually required and uses our section config
parser

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 11:34:56 +02:00
Wolfgang Bumiller
18cdf20afc move tools::nom to pbs-tools
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 10:08:26 +02:00
Wolfgang Bumiller
e57841c442 move run_command to pbs-tools
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 10:04:05 +02:00
Wolfgang Bumiller
751f6b6148 move userid types to pbs-api-types
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 09:53:48 +02:00
Wolfgang Bumiller
3c430e9a55 move id and single line comment format to pbs-api-types
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 09:49:38 +02:00
Wolfgang Bumiller
155f657f6b move TaskState trait to pbs-datastore
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 09:24:39 +02:00
Wolfgang Bumiller
86fb38776b add pbs-api-types subcrate, move key_derivation
move key_derivation to pbs-datastore

pbs-api-types should only contain "basic" types which
* are usually required by clients
* don't depend on pbs-related code directly

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-07 09:04:09 +02:00
Wolfgang Bumiller
f323e90602 add pbs-datastore module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 15:11:52 +02:00
Wolfgang Bumiller
770a36e53a add pbs-tools subcrate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 15:10:37 +02:00
Wolfgang Bumiller
d420962fbc split out pbs-runtime module
These are mostly tokio specific "hacks" or "workarounds" we
only really need/want in our binaries without pulling it in
via our library crates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 14:52:25 +02:00
Wolfgang Bumiller
01fd2447b2 buildsys: don't use debcargo
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 14:50:46 +02:00
Wolfgang Bumiller
85beb7d875 tree-wide: switch to using mod.rs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 12:04:52 +02:00
Wolfgang Bumiller
af06decd1b split out pbs-buildcfg module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 12:00:14 +02:00
Wolfgang Bumiller
aceae32baa Cargo.toml: regroup imports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 11:48:28 +02:00
Thomas Lamprecht
74a4f9efc9 bump version to 2.0.1-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 23:14:54 +02:00
Thomas Lamprecht
fb1e7a86f4 ui: minimally increase font-size of product title and version
Similar like we did for Proxmox VE's manager. The main title and
version should stand a bit more out compared to simple nav/button
texts.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 23:13:33 +02:00
Thomas Lamprecht
dc99315cf9 ui: app: fix openID helper usage and rework style
one really does not need a if and an extra intermediate variable for
assigning a simple bool...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 23:12:08 +02:00
Thomas Lamprecht
34bd1109b0 bump version to 2.0.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:45:00 +02:00
Thomas Lamprecht
13a2445744 buildsys: docs: clean: also clean generated JS files
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:44:13 +02:00
Thomas Lamprecht
c968da789e acme: nit code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:44:13 +02:00
Stoiko Ivanov
3f84541412 fix #3496: acme: plugin: add sleep for dns propagation
the dns plugin config allow for a specified amount of time to wait for
the TXT record to be set and propagated through DNS.

This patch adds a sleep for this amount of time.
The log message was taken from the perl implementation in proxmox-acme
for consistency.

Tested with the powerdns plugin in my test setup.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-03 21:44:13 +02:00
Stoiko Ivanov
4d8bd03668 config: acme: make validation_delay crate public
we need the setting in acme::plugin.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-03 21:44:13 +02:00
Stoiko Ivanov
f9bd5e1691 acme: plugin: fix error message
extract_challenge is used by both dns-01 and http-01 challenges.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-03 21:44:13 +02:00
Thomas Lamprecht
ecd66ecaf6 restore daemon: use millisecond log resolution
During startup most of the stuff is happening in milliseconds (or
less), so the timestamp granularity of seconds made it hard to tell
if the previous command required 990ms or 1ms, which is quite the
difference in the restore daemon context.

Using micros seems not to bring too much additional information, a
millisecond is already an ok lower time resolution for logging, so
switch only to millis for now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:44:13 +02:00
Thomas Lamprecht
33d7292f29 restore daemon: create /run/proxmox-backup on startup
fixes file restore again.

The new Memcom tracking file lives in `/run/proxmox-backup` and is
always created on REST interaction, as CachedUserInfo uses it to
efficiently track config changes, and such a cache is used in each
REST handle_request.

Further, the Memcom infra expects the base run PBS dir to exists
already, which is an OK assumption to have, but in the file-restore
daemon we have a significantly more minimal environment, and the run
dir was simply not required there, even /run isn't a tmpfs yet.

Fixes fda19dcc6f ("fix CachedUserInfo by using a shared memory version counter")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:43:07 +02:00
Thomas Lamprecht
f4d371d2d2 REST: set error message extenesion for bad-request response log
We send it already to the user via the response body, but the
log_response does not has, nor wants to have FWIW, access to the
async body stream, so pass it through the ErrorMessageExtension
mechanism like we do else where.

Note that this is not only useful for PBS API proxy/daemon but also
the REST server of the file-restore daemon running inside the restore
VM, and it really is *very* helpful to debug things there..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht
835d0e5dd3 memcom: rustfmt + (trailing) whitespace cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht
9a06eb1618 file restore daemon: log about basic steps
to make the log more useful..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht
309e14ebb7 file restore daemon: reword warning about manual execution
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht
2d48533378 REST: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht
fffd6874e6 bump version to 2.0.0-2
only for the file-restore daemon, other packages where not uploaded!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 02:15:24 +02:00
Thomas Lamprecht
0ddd48f0b5 d/control: record breaks for older proxmox-backup-restore-image
> requires a Breaks on the old restore image (else the restore daemon
> crashes because of missing lock/LVM support).
- F.G., mailing list

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 02:13:56 +02:00
Stefan Reiter
cb590dbc07 file-restore-daemon/disk: add LVM (thin) support
Parses JSON output from 'pvs' and 'lvs' LVM utils and does two passes:
one to scan for thinpools and create a device node for their
metadata_lv, and a second to load all LVs, thin-provisioned or not.

Should support every LV-type that LVM supports, as we only parse LVM
tools and use 'vgscan --mknodes' to create device nodes for us.

Produces a two-layer BucketComponent hierarchy with VGs followed by LVs,
PVs are mapped to their respective disk node.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:31 +02:00
Stefan Reiter
6c4f762c49 file-restore-daemon/disk: ignore already-mounted error and prefix zpool
Prefix zpool mount paths to avoid clashing with other mount namespaces
(like LVM).

Also ignore "already-mounted" error and return it as success instead -
as we always assume that a mount path is unique, this is a safe
assumption, as nothing else could have been mounted here.

This fixes an issue where a mountpoint=legacy subvol might be available
on different disks, and thus have different Bucket instances that don't
share the mountpoint cache, which could lead to an error if the user
tried opening it multiple times on different disks.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:23 +02:00
Stefan Reiter
7a0afee391 file-restore-daemon/disk: fix component path errors
otherwise the path ends in an array ["foo", "bar"] instead of "foo/bar"

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:19 +02:00
Stefan Reiter
0dda883994 file-restore-daemon/disk: dedup BucketComponents and make size optional
To support nested BucketComponents, it is necessary to dedup them, as
otherwise two components like:
  /foo/bar
  /foo/baz
will result in /foo being shown twice at the first hierarchy.

Also make the size property based on index and optional, as for example
/foo in the example above might not have a size, and bar/baz might have
differing sizes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:13 +02:00
Fabian Grünbichler
c2e2078b3f openid: conditionally disable api endpoint
since it pulls in lots of additional linked libraries for all binaries
compiled as part of proxmox-backup. it can easily be re-enabled with
`--cfg openid` added to the RUSTFLAGS env variable.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:52:01 +02:00
Fabian Grünbichler
26a3450f19 openid: move helper from config to api2
it's not really needed in the config module, and this makes it easier to
disable the proxmox-openid dependency linkage as a stop-gap measure.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:52:01 +02:00
Thomas Lamprecht
324c069848 d/control: bump versioned dependency for proxmox-widget-toolkit to 3.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:50:12 +02:00
Thomas Lamprecht
bd4c5607ca d/control: commit update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:49:50 +02:00
Thomas Lamprecht
e1d85f1840 ui: login: cleanups, mostly openID related
similar to what was done in PVE.

 - factor out openid_login_param to widget-toolkit as
   getOpenIDRedirectionAuthorization and use it
 - use camel case to match our JS style guide and our framework (and
   basically the rest of the JS world)
 - minor cleanups like moving variable definition into the single if
   branch their used

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:46:24 +02:00