4369 Commits

Author SHA1 Message Date
Dominik Csapak
7c069e82d1 fix #3743: extract zfs dataset io stats from /proc/spl/kstat/zfs/POOL/objset-*
Recently, ZFS removed the pool global io stats from
/proc/spl/kstat/zfs/POOL/io with no replacement.

To gather stats about the datastores, access now the objset specific
entries there. To be able to make that efficient, cache a map of
dataset <-> obset ids, so that we do not have to parse all files each time.

We update the cache each time we try to get the info for a dataset
where we do not have a mapping.

We cannot update it on datastore add/remove since that happens in the
proxmox-backup daemon, while we need the info here in proxmox-backup-proxy.

Sadly with this we lose the io wait metric, but it seems that this is no
longer tracked in zfs at all, so nothing we can do for that.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-11 08:45:55 +01:00
Dietmar Maurer
b44483a853 datastore status: do not count empty groups
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-01-07 08:40:22 +01:00
Wolfgang Bumiller
ba857cbe68 tools::config: error on newlines in string values
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-01-05 10:04:04 +01:00
Hannes Laimer
e466526137 server: use configured email-from for sending mail
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-04 08:09:27 +01:00
Hannes Laimer
62222ed068 api2: make email-from updatable
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-04 08:09:27 +01:00
Hannes Laimer
f06b5283b0 config: add email-from to NodeConfig
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-04 08:05:34 +01:00
Fabian Grünbichler
645d52308b TimeSpan: parse via FromStr
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
7f6c169b25 use schema verify methods
the old, deprecated ones only forward to these anyway.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
6f1c26b083 tree-wide: is_ok/is_err()
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
3afecb8409 tree-wide: use is_empty() and similar
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
540fca5c9e tree-wide: cleanup manual map/flatten
found with clippy, best viewed with `-w` ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
8ff886773f view_task_result: remove unnecessary &mut
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
aa174e8e8a tree-wide: drop redundant clones
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
0a7f902e2a tape: multi-volume: fix overflow check
the part number cannot go above 255 at the moment, but if it ever gets
bumped to a bigger integer type this boundary wouldn't cause a
compile-error. explicitly checking for overflowing u8 makes this a bit
more future-proof, and shuts up clippy as well ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Fabian Grünbichler
9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Fabian Grünbichler
a0c69902c8 fix #3763: disable renegotiation
requires openssl crate with fix[0], like our packaged one.

0: https://github.com/sfackler/rust-openssl/pull/1584

Tested-by: Stoiko Ivanov s.ivanov@proxmox.com
Reviewed-by: Stoiko Ivanov s.ivanov@proxmox.com

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-27 09:09:26 +01:00
Wolfgang Bumiller
9fa3026a08 cleanup schema function calls
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-16 11:25:02 +01:00
Dominik Csapak
48910f9b0a api: zfs: create zpool with relatime=on
some operations (e.g. garbage collection/restore/etc.) are very read
intensive on the chunks, and having atime=on and relatime=off (zfs default)
makes those write intensive operations too. Additionally, 'ext4' defaults to
relatime, so also change the default for api-created zpools.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-12-16 10:56:04 +01:00
Dietmar Maurer
d4877712f8 pbs-client: avoid mut self in http_client methods.
It is not necessary, so avoid it. The client can now be used
with multiple threads (without using a Mutex).

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-04 14:44:05 +01:00
Dominik Csapak
7549114c9f adapt compute_next_event to new signature
the 'utc' flag is now contained in the event itself and not given
as a flag to 'compute_next_event' anymore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-02 10:40:58 +01:00
Fabian Grünbichler
f66d814792 fix broken format strings
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-01 12:46:37 +01:00
Dominik Csapak
68b6c1202c remove use of deprecated functions from proxmox-time
Depend on proxmox-time 1.1.1

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-01 07:23:18 +01:00
Wolfgang Bumiller
e25982f24e remove unused identity macro
this is not required anymore by the sortable macro

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-29 11:24:02 +01:00
Wolfgang Bumiller
e6e2927e72 update proxmox-tfa to 2.0
and fix still-very-bad updater usage in config api call...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-29 11:19:50 +01:00
Dietmar Maurer
577095e2f7 move pbs-tools/src/percent_encoding.rs to pbs-api-types/src/percent_encoding.rs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 11:48:52 +01:00
Dietmar Maurer
0df179c2b4 remove pbs-tools/src/cli.rs
Code is only used once in src/bin/proxmox_backup_debug/inspect.rs, so
move it into that file.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 08:33:10 +01:00
Dietmar Maurer
689ed51397 openid_login: improve error message for disabled users. 2021-11-25 07:29:33 +01:00
Dietmar Maurer
25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Dietmar Maurer
c1e6efa8e1 sync job: correctly apply rate limit
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-23 09:42:18 +01:00
Dietmar Maurer
d63db0863d proxmox-backup-manager traffic: render data human readable
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 09:07:05 +01:00
Dietmar Maurer
7a36833103 fix sync job regression test (add RateLimitConfig)
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 08:29:43 +01:00
Dietmar Maurer
ca6e66aa5a Fingerprint: add new signature method
commit c42a54795df332cd778a53ea500b901ae873c81d introcuded a bug by
using fp.to_string(). Replace this with fp.signature() which correctly
returns the full fingerprint instead of the short version.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 08:29:43 +01:00
Dietmar Maurer
2d5287fbbc use RateLimitConfig for HttpClient and pull
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 07:49:41 +01:00
Dietmar Maurer
6eb756bcab sync-job: add rate limit
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 07:49:41 +01:00
Dietmar Maurer
5647219049 pbs-api-types: split out type RateLimitConfig 2021-11-22 07:49:41 +01:00
Dietmar Maurer
193ec30c2b fix proxmox-backup-manager sync-job list
Property is called 'group-filter' (not 'groups').
2021-11-21 09:46:46 +01:00
Dominik Csapak
4fe77c36df api: traffic_control: add missing rename to 'kebab-case'
otherwise the 'delete' properties need underscores
(e.g. 'burst_in' instead of 'burst-in')

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-11-20 19:40:30 +01:00
Dietmar Maurer
118515dbd0 use HumanByte for traffic-control config
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-20 19:35:24 +01:00
Thomas Lamprecht
a58a5cf795 move HumanByte to pbs-abi-types crate
Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-20 19:35:24 +01:00
Dietmar Maurer
92a8f0bc82 depend on proxmox-async 0.2 2021-11-20 17:14:02 +01:00
Dietmar Maurer
9a1b24b6b1 use new proxmox-async crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 18:03:22 +01:00
Fabian Grünbichler
ea67cd70c9 tfa: handle incompatible challenge data
by returning default data, in case the challenge data is not parseable.
this allows a new challenge to be started for the userid in question
without manual cleanup.

currently this can be triggered if an ongoing challenge created with
webauthn-rs 0.2.5 is stored in /run and attempted to be read
post-upgrade.

Reported-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-19 14:12:31 +01:00
Wolfgang Bumiller
281a5dd1fc cleanup unused re-exports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-19 12:49:46 +01:00
Fabian Grünbichler
f5e2b4726d webauthn: correctly set origin when updating
with current proxmox-tfa this became a hard error, since origin and rp
are not both Strings anymore..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-11-19 11:58:17 +01:00
Dietmar Maurer
d5790a9f27 use new proxmox-sys crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 11:06:35 +01:00
Dietmar Maurer
860eaec58f use proxmox::tools::fd::fd_change_cloexec from proxmox 0.15.3
Depend on proxmox 0.15.3

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 13:43:41 +01:00
Dietmar Maurer
26e949d5fe traffic-control api: return current traffic with config
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 12:24:33 +01:00
Stoiko Ivanov
7c2431d42c api: acme: fix typo
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Stoiko Ivanov
25c1420a12 config: acme: plugin: rustfmt
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Stoiko Ivanov
c1a1e1ae8f api: config: acme: rustfmt
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00