Commit Graph

6065 Commits

Author SHA1 Message Date
Wolfgang Bumiller
fd7f760304 proxmox-rest-server: add missing 'derive' feature
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-11 13:57:48 +01:00
Thomas Lamprecht
af6fdb9d0d tools: disk: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-10 18:39:56 +01:00
Thomas Lamprecht
a1c906cb02 api: node/disk: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-10 13:12:15 +01:00
Fabian Grünbichler
dcf5a0f62d misc clippy fixes
the trivial ones ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 14:57:16 +01:00
Aaron Lauterer
bb9e503964 report: add tape, traffic control and disk infos
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-02-07 15:37:06 +01:00
Aaron Lauterer
b2fc573a62 report: move subscription info further up
This is something that is checked all the time. Having it further up
saves on scrolling and brings it into better alignment with PVE & PMG
regarding where in the report the info is located.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2022-02-07 15:37:06 +01:00
Matthias Heiserer
415da09826 node config: add english to translation enum for default-lang
Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-07 15:24:38 +01:00
Matthias Heiserer
5ffa68d2c4 api: node config: add default-lang integration
Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-07 15:24:38 +01:00
Thomas Lamprecht
e7668a3eea ui: webauthn: decrease upgrade frequency from 1s to 2.5s
this is nothing to important and nothing that'll get changed *that*
often, so 2.5s is more than enough.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-07 15:20:22 +01:00
Thomas Lamprecht
21898bb831 ui: webauthn: fix stopping store upgrades on destroy
`deactivate` is only triggered if we switch to a different tab on
the same navigation level, but if we switch to a completely different
component (e.g., fom `Options -> Others` to `Datastore foo`) we can
only work with the destroy event, use the before one as else we
cannot access the view controllers method anymore.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-07 15:19:47 +01:00
Dominik Csapak
7b944ff11a re-use PROXMOX_DEBUG env variable to control log level filter
So that we can make 'log::debug' messages actually appear in the
syslog.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-04 11:21:47 +01:00
Markus Frank
fce49eab30 fix #3856 hint parameter is not optional
For the API the parameter --hint is not optional. This patch fixes
the man page and cli command doesn't send an API call, if the
parameter does not exist.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2022-02-03 14:49:25 +01:00
Thomas Lamprecht
af35bc8b9c proxy: refactor gui-language logic
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-03 13:12:02 +01:00
Thomas Lamprecht
e5e48b01ad rest: add cookie_from_header helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-03 13:12:02 +01:00
Thomas Lamprecht
5d74f79643 proxy: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-03 13:12:02 +01:00
Matthias Heiserer
b0427dda76 docs: fix typo in tape backup
Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
2022-02-03 13:12:02 +01:00
Thomas Lamprecht
70ba718ce9 node config: avoid "allow" annotation
We rename those anyway for serialization so we do not need to bother
with spelling them in an non-idiomatic way just because i18n has it
like that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-03 13:12:02 +01:00
Matthias Heiserer
68811af9f9 fix #3103. node config: allow to configure default UI language
This language is only used if none is set in the cookies.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
2022-02-03 13:12:02 +01:00
Wolfgang Bumiller
163629e62e bump proxmox-acme-rs dependency to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-02 13:18:47 +01:00
Dominik Csapak
1993d98695 traffic-control: use SocketAddr from 'accept()'
instead of getting the 'peer_addr()' from the socket.
The advantage is that we must get this and thus can drop the mapping
from result -> option, and can drop the testing for None and a test case

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-31 09:58:14 +01:00
Dominik Csapak
127c5ac3a9 ui: datastore/Content: improve verification actions
verifying a single snapshot is now never skipped because of recent verify
verifying a group will now reverify after 29 days to be consistent
with the 'All OK (old)' display

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-27 15:31:55 +01:00
Dominik Csapak
7a1a5d206d verify: allow '0' days for reverification
and let it mean that we will always reverify

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-27 15:31:55 +01:00
Thomas Lamprecht
7a524f1048 bump version to 2.1.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 16:24:11 +01:00
Thomas Lamprecht
1d3b253721 README: update for bullseye
and start with a higher level for "h1" headlines

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 16:19:21 +01:00
Thomas Lamprecht
1f8b29f578 file restore: scale per-round delay up dynamically
Avoids latency for restore-VMs that are finished fast but not ready
yet the first round while not checking to often for slower ones, iow,
we assume that the start up distribution is looking like a chi-square
Χ² with k=3.

With 25*round we get at max 45 rounds totalling to 25.875 s delay and
1.125 max between-round delay, which still provides an ok reaction
time.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 16:12:58 +01:00
Thomas Lamprecht
48ce3d00a4 file restore: always wait up to 25s
the timeout for connecting may be much shorter if we get a response
(which doesn't needs to be Ok, e.g., "Connection refused"), so
instead of trying a fixed amount of 60 times lets try for 25s
independent of how often that will be then.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 16:04:43 +01:00
Dietmar Maurer
d91a0f9fc9 Set MMAP_THRESHOLD to a fixed value (128K)
glibc's malloc has a misguided heuristic to detect transient allocations that
will just result in allocation sizes below 32 MiB never using mmap.

That it turn means that those relatively big allocations are on the heap where
cleanup and returning memory to the OS is harder to do and easier to be blocked
by long living, small allocations at the top (end) of the heap.

Observing the malloc size distribution in a file-level backup run:

@size:
[0]                   14 |                                                    |
[1]                25214 |@@@@@                                               |
[2, 4)              9090 |@                                                   |
[4, 8)             12987 |@@                                                  |
[8, 16)            93453 |@@@@@@@@@@@@@@@@@@@@                                |
[16, 32)           30255 |@@@@@@                                              |
[32, 64)          237445 |@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@|
[64, 128)          32692 |@@@@@@@                                             |
[128, 256)         22296 |@@@@                                                |
[256, 512)         16177 |@@@                                                 |
[512, 1K)           5139 |@                                                   |
[1K, 2K)            3352 |                                                    |
[2K, 4K)             214 |                                                    |
[4K, 8K)            1568 |                                                    |
[8K, 16K)             95 |                                                    |
[16K, 32K)          3457 |                                                    |
[32K, 64K)          3175 |                                                    |
[64K, 128K)          161 |                                                    |
[128K, 256K)         453 |                                                    |
[256K, 512K)          93 |                                                    |
[512K, 1M)            74 |                                                    |
[1M, 2M)             774 |                                                    |
[2M, 4M)             319 |                                                    |
[4M, 8M)             700 |                                                    |
[8M, 16M)             93 |                                                    |
[16M, 32M)            18 |                                                    |

We see that all allocations will be on the heap, and that while most
allocations are small, the relatively few big ones will still make up most of
the RSS and if blocked from being released back to the OS result in much higher
peak and average usage for the program than actually required.

Avoiding the "dynamic" mmap-threshold increasement algorithm and fixing it at
the original default of 128 KiB reduces RSS size by factor 10-20 when running
backups. As with memory mappings other mappings or the heap can never block
freeing the memory fully back to the OS.

But, the drawback of using mmap is more wasted space for unaligned or small
allocation sizes, and the fact that the kernel allegedly zeros out the data
before giving it to user space. The former doesn't really matter for us when
using it only for allocations bigger than 128 KiB, and the latter is a
trade-off, using 10 to 20 times less memory brings its own performance
improvement possibilities for the whole system after all ;-)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 [ Thomas: added to comment & commit message + extra-empty-line fixes ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-26 14:10:54 +01:00
Thomas Lamprecht
3af17d8919 bump version to 2.1.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-21 10:48:42 +01:00
Dominik Csapak
98983a9dab pbs-tools: LruCache: implement Drop
this fixes the leaked memory for the cache, as we had only pointers
in the map/list which were freed, not the underlying chunks

moves the 'clear' implementation out of the trait bounds so that
Drop can reuse it

this is used e.g. for file download from a pxar

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-01-20 11:24:34 +01:00
Matthias Heiserer
e92df23806 docs: make external hyperlinks clickable
rustdoc lints detected that two external hyperlinks were not
clickable.

The short cut used is only available for internal links, otherwise
one needs to use the Markdown syntax, so either [Text](URL) or <URL>.

Signed-off-by: Matthias Heiserer <m.heiserer@proxmox.com>
[ T: commit message text width, mention markdown ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-18 15:54:33 +01:00
Fabian Grünbichler
5ee8dd784f ciphers: improve option naming
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-14 11:02:07 +01:00
Hannes Laimer
f37167aeff api2: make tls ciphers updatable
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-14 11:02:07 +01:00
Hannes Laimer
2eba3967b2 proxy: use ciphers from config if set
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-14 11:02:07 +01:00
Fabian Grünbichler
1d552d2dd5 ciphers: simplify API schema
these need to be checked (and are) via libssl anyway before persisting,
and newer versions might contain new ciphers/variants/... (and things
like @STRENGTH or @SECLEVEL=n were missing).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-14 11:02:07 +01:00
Hannes Laimer
1ec7f7e6f2 config: add tls ciphers to NodeConfig
for TLS 1.3 and for TLS <= 1.2

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-14 11:02:07 +01:00
Thomas Lamprecht
8ad9eb779e bump version to 2.1.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-12 16:49:33 +01:00
Fabian Grünbichler
18ba1b2249 api-types: relax NODENAME_SCHEMA
there isn't really a concept of 'nodes' in PBS (yet) anyway - and if
there ever is, it needs to be handled by the rest-server / specific API
endpoints (like in PVE), and not by the schema.

this allows dropping proxmox-sys from pbs-api-types (and thus nix and
some other transitive deps as well).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-12 15:42:58 +01:00
Fabian Grünbichler
e2e587e3c7 api-types: move RsaPubKeyInfo to pbs-client
it's the only thing requiring openssl in pbs-api-types, and it's only
used by the client to pretty-print the 'master' key, which is
client-specific.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-01-12 15:42:58 +01:00
Thomas Lamprecht
c10a6755f0 docs: fix some typos
the `congestion` typo has been mentioned in the forum:
https://forum.proxmox.com/threads/proxmox-backup-server-2-1-released.100240/#post-443370

fixed a few surrounding ones and ones that `codespell` found in
addition to that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-12 15:19:59 +01:00
Thomas Lamprecht
d43aca148f ui: sys config: add icons to tabs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-12 12:41:28 +01:00
Thomas Lamprecht
5dfe3b66ab ui: sys config: code cleanup/refactoring
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-12 12:41:28 +01:00
Thomas Lamprecht
50c0840146 ui: sys config: merge webauthn and general options into one tab
To much wasted space else.
Also rename "Options" to "Others", while it's not _that_ much better
it's slightly more intuitive than config -> options (which has some
redundancy)...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-12 12:41:25 +01:00
Thomas Lamprecht
2e02a859cf fix #3058: ui: improve remote edit UX by clarifying ID vs host
also fixup missing emptyText for fingerprint (adapted from PVE's PBS
storage addition) and code-style in surrounding areas a bit

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-01-12 09:38:59 +01:00
Dominik Csapak
64c075b6c2 ui: hide rrd chart for io delay if no io_ticks are returned
it makes no sense to show a completely empty graph

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-11 11:43:10 +01:00
Dominik Csapak
f27b6086b1 api/admin/datastore: rrd: do not include io_ticks for zfs datastores
since it is not possible to collect them, do not return them here either

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-11 11:42:09 +01:00
Dominik Csapak
7c069e82d1 fix #3743: extract zfs dataset io stats from /proc/spl/kstat/zfs/POOL/objset-*
Recently, ZFS removed the pool global io stats from
/proc/spl/kstat/zfs/POOL/io with no replacement.

To gather stats about the datastores, access now the objset specific
entries there. To be able to make that efficient, cache a map of
dataset <-> obset ids, so that we do not have to parse all files each time.

We update the cache each time we try to get the info for a dataset
where we do not have a mapping.

We cannot update it on datastore add/remove since that happens in the
proxmox-backup daemon, while we need the info here in proxmox-backup-proxy.

Sadly with this we lose the io wait metric, but it seems that this is no
longer tracked in zfs at all, so nothing we can do for that.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-11 08:45:55 +01:00
Dietmar Maurer
b44483a853 datastore status: do not count empty groups
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-01-07 08:40:22 +01:00
Wolfgang Bumiller
ba857cbe68 tools::config: error on newlines in string values
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-01-05 10:04:04 +01:00
Hannes Laimer
c772a4a683 ui: add new options tab under configuration
... and add from-email + move http-proxy there

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-04 08:24:17 +01:00
Hannes Laimer
e466526137 server: use configured email-from for sending mail
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-01-04 08:09:27 +01:00