Commit Graph

5562 Commits

Author SHA1 Message Date
Stoiko Ivanov 3f84541412 fix #3496: acme: plugin: add sleep for dns propagation
the dns plugin config allow for a specified amount of time to wait for
the TXT record to be set and propagated through DNS.

This patch adds a sleep for this amount of time.
The log message was taken from the perl implementation in proxmox-acme
for consistency.

Tested with the powerdns plugin in my test setup.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-03 21:44:13 +02:00
Stoiko Ivanov 4d8bd03668 config: acme: make validation_delay crate public
we need the setting in acme::plugin.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-03 21:44:13 +02:00
Stoiko Ivanov f9bd5e1691 acme: plugin: fix error message
extract_challenge is used by both dns-01 and http-01 challenges.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2021-07-03 21:44:13 +02:00
Thomas Lamprecht ecd66ecaf6 restore daemon: use millisecond log resolution
During startup most of the stuff is happening in milliseconds (or
less), so the timestamp granularity of seconds made it hard to tell
if the previous command required 990ms or 1ms, which is quite the
difference in the restore daemon context.

Using micros seems not to bring too much additional information, a
millisecond is already an ok lower time resolution for logging, so
switch only to millis for now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:44:13 +02:00
Thomas Lamprecht 33d7292f29 restore daemon: create /run/proxmox-backup on startup
fixes file restore again.

The new Memcom tracking file lives in `/run/proxmox-backup` and is
always created on REST interaction, as CachedUserInfo uses it to
efficiently track config changes, and such a cache is used in each
REST handle_request.

Further, the Memcom infra expects the base run PBS dir to exists
already, which is an OK assumption to have, but in the file-restore
daemon we have a significantly more minimal environment, and the run
dir was simply not required there, even /run isn't a tmpfs yet.

Fixes fda19dcc6f ("fix CachedUserInfo by using a shared memory version counter")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:43:07 +02:00
Thomas Lamprecht f4d371d2d2 REST: set error message extenesion for bad-request response log
We send it already to the user via the response body, but the
log_response does not has, nor wants to have FWIW, access to the
async body stream, so pass it through the ErrorMessageExtension
mechanism like we do else where.

Note that this is not only useful for PBS API proxy/daemon but also
the REST server of the file-restore daemon running inside the restore
VM, and it really is *very* helpful to debug things there..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht 835d0e5dd3 memcom: rustfmt + (trailing) whitespace cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht 9a06eb1618 file restore daemon: log about basic steps
to make the log more useful..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht 309e14ebb7 file restore daemon: reword warning about manual execution
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht 2d48533378 REST: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 21:34:03 +02:00
Thomas Lamprecht fffd6874e6 bump version to 2.0.0-2
only for the file-restore daemon, other packages where not uploaded!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 02:15:24 +02:00
Thomas Lamprecht 0ddd48f0b5 d/control: record breaks for older proxmox-backup-restore-image
> requires a Breaks on the old restore image (else the restore daemon
> crashes because of missing lock/LVM support).
- F.G., mailing list

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 02:13:56 +02:00
Stefan Reiter cb590dbc07 file-restore-daemon/disk: add LVM (thin) support
Parses JSON output from 'pvs' and 'lvs' LVM utils and does two passes:
one to scan for thinpools and create a device node for their
metadata_lv, and a second to load all LVs, thin-provisioned or not.

Should support every LV-type that LVM supports, as we only parse LVM
tools and use 'vgscan --mknodes' to create device nodes for us.

Produces a two-layer BucketComponent hierarchy with VGs followed by LVs,
PVs are mapped to their respective disk node.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:31 +02:00
Stefan Reiter 6c4f762c49 file-restore-daemon/disk: ignore already-mounted error and prefix zpool
Prefix zpool mount paths to avoid clashing with other mount namespaces
(like LVM).

Also ignore "already-mounted" error and return it as success instead -
as we always assume that a mount path is unique, this is a safe
assumption, as nothing else could have been mounted here.

This fixes an issue where a mountpoint=legacy subvol might be available
on different disks, and thus have different Bucket instances that don't
share the mountpoint cache, which could lead to an error if the user
tried opening it multiple times on different disks.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:23 +02:00
Stefan Reiter 7a0afee391 file-restore-daemon/disk: fix component path errors
otherwise the path ends in an array ["foo", "bar"] instead of "foo/bar"

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:19 +02:00
Stefan Reiter 0dda883994 file-restore-daemon/disk: dedup BucketComponents and make size optional
To support nested BucketComponents, it is necessary to dedup them, as
otherwise two components like:
  /foo/bar
  /foo/baz
will result in /foo being shown twice at the first hierarchy.

Also make the size property based on index and optional, as for example
/foo in the example above might not have a size, and bar/baz might have
differing sizes.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Reviewed-By: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:54:13 +02:00
Fabian Grünbichler c2e2078b3f openid: conditionally disable api endpoint
since it pulls in lots of additional linked libraries for all binaries
compiled as part of proxmox-backup. it can easily be re-enabled with
`--cfg openid` added to the RUSTFLAGS env variable.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:52:01 +02:00
Fabian Grünbichler 26a3450f19 openid: move helper from config to api2
it's not really needed in the config module, and this makes it easier to
disable the proxmox-openid dependency linkage as a stop-gap measure.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-03 01:52:01 +02:00
Thomas Lamprecht 324c069848 d/control: bump versioned dependency for proxmox-widget-toolkit to 3.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:50:12 +02:00
Thomas Lamprecht bd4c5607ca d/control: commit update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:49:50 +02:00
Thomas Lamprecht e1d85f1840 ui: login: cleanups, mostly openID related
similar to what was done in PVE.

 - factor out openid_login_param to widget-toolkit as
   getOpenIDRedirectionAuthorization and use it
 - use camel case to match our JS style guide and our framework (and
   basically the rest of the JS world)
 - minor cleanups like moving variable definition into the single if
   branch their used

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:46:24 +02:00
Thomas Lamprecht 1ce1a5e5cc docs: installation: drop debian-release specific note
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:44:53 +02:00
Thomas Lamprecht 6f66a0ca71 docs: faq: update support table
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-03 01:44:09 +02:00
Thomas Lamprecht 62a5b3907b docs: initial update to repositories for bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-02 19:02:13 +02:00
Thomas Lamprecht 85b6c4ead4 ui: login: fix another bogus gettext usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-02 09:31:50 +02:00
Thomas Lamprecht a190979c04 ui: login: fix bogus gettext usage
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-02 08:40:14 +02:00
Dominik Csapak 4a489ae3de ui: dashboard/DataStoreStatistics: fix closing <i> tag
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-01 07:51:00 +02:00
Dominik Csapak 9ac8b73e07 tape/drive: fix logging when requesting media
we try to load the correct media in a loop until we find the correct tape.
when encountering an error or wrong tape, we want to log that (and send
an email if one is set) that requests the correct tape.

while trying to avoid printing the same errors more than once in a row,
we had at least one case (starting with an empty tape in the drive)
which would not print/send any tape request.

reworking that code to use a custom 'TapeRequest' enum, which contains
the state + error message, and a helper that prints and sends an email
when the state changes

this reduces the change check/log to a single variable, instead of 4
(tried, last_media_uuid, last_error, failure_reason)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-30 10:25:48 +02:00
Dietmar Maurer 414be8b675 tape: fix LTO locate_file for HP drives
Add test code to the first locate_file command, compute locate_offset.
Subsequent locate_file commands use that offset.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-30 09:08:58 +02:00
Dietmar Maurer fda19dcc6f fix CachedUserInfo by using a shared memory version counter 2021-06-30 08:54:30 +02:00
Dietmar Maurer cd975e5787 ui: implement OpenId login 2021-06-30 08:54:30 +02:00
Dietmar Maurer 3b7b1dfb8e api: add openid redirect/login API 2021-06-30 08:54:30 +02:00
Dietmar Maurer d8a47ec649 cleanup user/token is_active() check 2021-06-30 08:54:30 +02:00
Dietmar Maurer 252cd3b781 implement new helper is_active_user_id() 2021-06-30 08:54:30 +02:00
Dietmar Maurer 0decd11efb cli: add CLI to manage openid realms. 2021-06-30 08:54:30 +02:00
Dietmar Maurer b84d2592fb add API to manage openid realms 2021-06-30 08:54:30 +02:00
Dietmar Maurer 0219ba2cc5 check_acl_path: add /access/domains and /access/openid 2021-06-30 08:54:30 +02:00
Dietmar Maurer bbff6c4968 config: new domains.cfg to configure openid realm
Or other realmy types...
2021-06-30 08:54:30 +02:00
Dietmar Maurer bb88c6a29d depend on proxmox-openid-rs 2021-06-30 08:54:30 +02:00
Thomas Lamprecht a02466966d update enterprise repository to bullseye
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 19:57:50 +02:00
Thomas Lamprecht b0fc11804e d/changelog: add actual changelog for initial 2.0 build
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 19:40:44 +02:00
Thomas Lamprecht d9d81741e3 buildsys: switch to bullseye as upload dist target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 19:10:26 +02:00
Thomas Lamprecht 9678366102 bump version to 2.0.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 19:07:46 +02:00
Thomas Lamprecht a2c73c78dd buildsys: call dpkg-buildpackage directly in deb-all
else we may double-build

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 19:07:15 +02:00
Thomas Lamprecht c6a0e7d98e d/control: bump versioned dependency for ExtJS 7.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-06-28 19:05:58 +02:00
Dominik Csapak 85417b2a88 docs: build api-viewer from widget-toolkit-dev
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:26:40 +02:00
Dominik Csapak d738669066 docs: add Toolkit.js to lto-barcode
and generate a single js file for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:26:40 +02:00
Dominik Csapak 442d6da8fb docs: add Toolkit.js to prune simulator
from proxmox-widget-toolkit-dev and not as normal dependency,
else we would have to ship widget-toolkit on the wiki

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:26:40 +02:00
Dominik Csapak 62f10a01db docs/prune-simulator: remove displayField for Calendar Field
in extjs 7.0, specifying displayField overwrites the displayTpl,
which we want to use here, so remove it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-28 14:26:40 +02:00
Dominik Csapak 5667b76381 fix #3393 (again): pxar/create: try to read xattrs/fcaps/acls by default
we have a static list of filesystems and their capabilities regarding
file attributes and fs features (e.g. sockets/fifos/etc) which also
includes xattrs,acls and fcaps

if we did not know a filesystem by its magic number (for example cephfs),
we did not even attempt to read xattrs, etc.

this patch adds those flags by default to unknown filesystems, and
removes them when we encounter EOPNOTSUPP (to remove the number
of syscalls)

with this, we should be able to catch xattrs/acls/fcaps on all
(unknown) fs types that support them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-06-28 14:04:22 +02:00