Commit Graph

4252 Commits

Author SHA1 Message Date
Dominik Csapak 48910f9b0a api: zfs: create zpool with relatime=on
some operations (e.g. garbage collection/restore/etc.) are very read
intensive on the chunks, and having atime=on and relatime=off (zfs default)
makes those write intensive operations too. Additionally, 'ext4' defaults to
relatime, so also change the default for api-created zpools.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-12-16 10:56:04 +01:00
Dietmar Maurer d4877712f8 pbs-client: avoid mut self in http_client methods.
It is not necessary, so avoid it. The client can now be used
with multiple threads (without using a Mutex).

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-04 14:44:05 +01:00
Dominik Csapak 7549114c9f adapt compute_next_event to new signature
the 'utc' flag is now contained in the event itself and not given
as a flag to 'compute_next_event' anymore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-02 10:40:58 +01:00
Fabian Grünbichler f66d814792 fix broken format strings
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-01 12:46:37 +01:00
Dominik Csapak 68b6c1202c remove use of deprecated functions from proxmox-time
Depend on proxmox-time 1.1.1

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-01 07:23:18 +01:00
Wolfgang Bumiller e25982f24e remove unused identity macro
this is not required anymore by the sortable macro

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-29 11:24:02 +01:00
Wolfgang Bumiller e6e2927e72 update proxmox-tfa to 2.0
and fix still-very-bad updater usage in config api call...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-29 11:19:50 +01:00
Dietmar Maurer 577095e2f7 move pbs-tools/src/percent_encoding.rs to pbs-api-types/src/percent_encoding.rs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 11:48:52 +01:00
Dietmar Maurer 0df179c2b4 remove pbs-tools/src/cli.rs
Code is only used once in src/bin/proxmox_backup_debug/inspect.rs, so
move it into that file.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 08:33:10 +01:00
Dietmar Maurer 689ed51397 openid_login: improve error message for disabled users. 2021-11-25 07:29:33 +01:00
Dietmar Maurer 25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Dietmar Maurer c1e6efa8e1 sync job: correctly apply rate limit
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-23 09:42:18 +01:00
Dietmar Maurer d63db0863d proxmox-backup-manager traffic: render data human readable
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 09:07:05 +01:00
Dietmar Maurer 7a36833103 fix sync job regression test (add RateLimitConfig)
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 08:29:43 +01:00
Dietmar Maurer ca6e66aa5a Fingerprint: add new signature method
commit c42a54795d introcuded a bug by
using fp.to_string(). Replace this with fp.signature() which correctly
returns the full fingerprint instead of the short version.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 08:29:43 +01:00
Dietmar Maurer 2d5287fbbc use RateLimitConfig for HttpClient and pull
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 07:49:41 +01:00
Dietmar Maurer 6eb756bcab sync-job: add rate limit
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 07:49:41 +01:00
Dietmar Maurer 5647219049 pbs-api-types: split out type RateLimitConfig 2021-11-22 07:49:41 +01:00
Dietmar Maurer 193ec30c2b fix proxmox-backup-manager sync-job list
Property is called 'group-filter' (not 'groups').
2021-11-21 09:46:46 +01:00
Dominik Csapak 4fe77c36df api: traffic_control: add missing rename to 'kebab-case'
otherwise the 'delete' properties need underscores
(e.g. 'burst_in' instead of 'burst-in')

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-20 19:40:30 +01:00
Dietmar Maurer 118515dbd0 use HumanByte for traffic-control config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-20 19:35:24 +01:00
Thomas Lamprecht a58a5cf795 move HumanByte to pbs-abi-types crate
Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-20 19:35:24 +01:00
Dietmar Maurer 92a8f0bc82 depend on proxmox-async 0.2 2021-11-20 17:14:02 +01:00
Dietmar Maurer 9a1b24b6b1 use new proxmox-async crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 18:03:22 +01:00
Fabian Grünbichler ea67cd70c9 tfa: handle incompatible challenge data
by returning default data, in case the challenge data is not parseable.
this allows a new challenge to be started for the userid in question
without manual cleanup.

currently this can be triggered if an ongoing challenge created with
webauthn-rs 0.2.5 is stored in /run and attempted to be read
post-upgrade.

Reported-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-19 14:12:31 +01:00
Wolfgang Bumiller 281a5dd1fc cleanup unused re-exports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-19 12:49:46 +01:00
Fabian Grünbichler f5e2b4726d webauthn: correctly set origin when updating
with current proxmox-tfa this became a hard error, since origin and rp
are not both Strings anymore..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-19 11:58:17 +01:00
Dietmar Maurer d5790a9f27 use new proxmox-sys crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 11:06:35 +01:00
Dietmar Maurer 860eaec58f use proxmox::tools::fd::fd_change_cloexec from proxmox 0.15.3
Depend on proxmox 0.15.3

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 13:43:41 +01:00
Dietmar Maurer 26e949d5fe traffic-control api: return current traffic with config
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 12:24:33 +01:00
Stoiko Ivanov 7c2431d42c api: acme: fix typo
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Stoiko Ivanov 25c1420a12 config: acme: plugin: rustfmt
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Stoiko Ivanov c1a1e1ae8f api: config: acme: rustfmt
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Dietmar Maurer 10beed1199 openid: allow to configure scopes, prompt, ACRs and arbitrary username-claim values
- no longer set prompt to 'login' (makes auto-login possible)
- new prompt configuration
- allow arbitrary username-claim values

Depend on proxmox-openid 0.9.0.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 11:20:55 +01:00
Thomas Lamprecht 062edce27f group filter: rename CLI/API/Config "groups" option to "group-filter"
we even use that for basically all the related schema names, "groups"
allone is just rather not so telling, i.e., "groups" what?

While due to the additive nature of `group-filter` is not the best
possible name for passing multiple arguments on the CLI (the web-ui
can present this more UX-friendly anyway) due to possible confusion
about if the filter act like AND vs OR it can be documented and even
if a user is confused they still are safe on more being synced than
less. Also, the original param name wasn't really _that_ better in
that regards

Dietmar also suggested to use singular for the CLI option, while
there can be more they're passed over repeating the option, each with
a single filter.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak efd2713aa8 proxmox-tape: add groups filter to backup command
and add a completion handler to complete the backup groups

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak c8c5c7f571 fix #3533: tape backup: filter groups according to config
this fixes bug #3533, since now a user can backup a single datastore
on multiple tape media pools in parallel, e.g. vms on one pool, ct on
another.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak 91357c2034 tape backup jobs: add group filters to config/api
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak 097ccfe1d5 proxmox-tape: add missing 'notify-user' option to backup command
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler 61ef4ae8cb fix #sync.cfg/pull: don't remove by default
and convert existing (manually created/edited) jobs to the previous
default value of 'true'. the GUI has always set this value and defaults
to 'false'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler 1b52122a1f manager: render group filter properly
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler 1d9bc184f5 remote: add backup group scanning
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler 5f83d3f636 sync: add group filtering
like for manual pulls, but persisted in the sync job config and visible
in the relevant GUI parts.

GUI is read-only for now (and defaults to no filtering on creation), as
this is a rather advanced feature that requires a complex GUI to be
user-friendly (regex-freeform, type-combobox, remote group scanning +
selector with additional freeform input).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler 71e534631f pull: allow pulling groups selectively
without requiring workarounds based on ownership and limited
visibility/access.

if a group filter is set, remove_vanished will only consider filtered
groups for removal to prevent concurrent disjunct filters from trashing
eachother's synced groups.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler 6e9e6c7a54 pull/sync: extract passed along vars into struct
this is basically the sync job config without ID and some stuff
converted already, and a convenient helper to generate the http client
from it.

Suggested-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler e2e7560d5e pull: use BackupGroup consistently
instead of `GroupListItem`s. we convert it anyway, so might as well do
that at the start.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dietmar Maurer 15cc41b6cb proxmox-systemd: remove crate, use new proxmox-time 1.1.0 instead
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-17 13:07:51 +01:00
Wolfgang Bumiller 729bd1fd16 remove now unused serde_filter module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-17 09:50:08 +01:00
Wolfgang Bumiller 9407810fe1 switch tfa api to use proxmox-tfa::api
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-17 09:33:04 +01:00
Dietmar Maurer c42a54795d move fingerprint helpers from pbs-tools to pbs-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-17 07:07:40 +01:00