Commit Graph

5763 Commits

Author SHA1 Message Date
Dietmar Maurer
c68fa58a59 remove proxmox-rrd-api-types crate, s/RRDTimeFrameResolution/RRDTimeFrame/
Because the types used inside the RRD have other requirements
than the API types:

- other serialization format
- the API may not support all RRD features

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
426dda0730 proxmox-rrd: extract_data: include values from current slot
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
eb37d4ece2 proxmox-rrd: remove dependency to proxmox-rrd-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
1198f8d4e6 proxmox-rrd: implement new CBOR based format
Storing much more data points now got get better graphs.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
4b709ade68 proxmox-backup-proxy: use tokio::task::spawn_blocking instead of block_in_place
allow the current thread to do some other work in-between

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
fa49d0fde9 RRD_CACHE: use a OnceCell instead of lazy_static
And initialize only with proxmox-backup-proxy. Other binaries dont need it.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Dietmar Maurer
1d44f175c6 proxmox-rrd: use a journal to reduce amount of bytes written
Append pending changes in a simple text based format that allows for
lockless appends as long as we stay below 4 KiB data per write.

Apply the journal every 30 minutes and on daemon startup.

Note that we do not ensure that the journal is synced, this is a
perfomance optimization we can make as the kernel defaults to
writeback in-flight data every 30s (sysctl vm/dirty_expire_centisecs)
anyway, so we lose at max half a minute of data on a crash, here one
should have in mind that we normally expose 1 minute as finest
granularity anyway, so not really much lost.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-13 13:36:02 +02:00
Wolfgang Bumiller
890b88cbef remove pbs-tools::ops::ControlFlow
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:36:40 +02:00
Wolfgang Bumiller
27709b49d5 pbs-config: drop default-features on proxmox-router dep
we don't need the 'cli' feature in there

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 13:11:08 +02:00
Dylan Whyte
7ccbce03d3 docs: language and formatting fixup
Some minor langague and formatting fixes to sections: Proxmox VE
Integration, pxar Command Line Tool, Managing Remotes, Maintenance
Tasks, Host System Administration, Network Management, and Technical
Overview.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-10-12 08:31:37 +02:00
Dylan Whyte
5fb852afed docs: backup-client: langauge and formatting fixup
also remove todo item for scheduling garbage collect with cron, and add
note about schedule configuration through proxmox-backup-manager/PBS GUI

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-10-12 08:30:39 +02:00
Dylan Whyte
60589e6066 docs: Update for new features/functionality
Update GUI section and GUI instructions to reflect current layout and
features

List OpenID connect in possible realms (user management)

Link Access Control section when referring to it (user management)

Include Tape roles in access control section

Minor formatting changes

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-10-12 08:28:29 +02:00
Dylan Whyte
717ce40612 docs: language and formatting fixup
Some minor changes to the sections: Introduction, Installation,
Terminology, GUI, Storage, and User Management

Mention tape backup in main features

Update epilog.rst with link for 'LXC'.
Remove FIXME from epilog.rst (I believe this was a note to repair
the not-yet-created pbs wiki link).

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-10-12 08:26:13 +02:00
Dominik Csapak
75442e813e api daemons: fix sending log-reopen command
send_command serializes everything so it cannot be used to send a
raw, optimized command. Normally that means we get an error like
> 'unable to parse parameters (expected json object)'
when used that way.

Switch over to send_raw_command which does not re-serializes the
command.

Fixes: 45b8a032 ("refactor send_command")
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-11 14:35:50 +02:00
Wolfgang Bumiller
853c55a049 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 12:08:57 +02:00
Wolfgang Bumiller
6ef1b649d9 update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Wolfgang Bumiller
e3f3359c86 bump proxmox dependency to 0.14.0 and proxmox-http to 0.5.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 11:18:22 +02:00
Dominik Csapak
0e1edf19b1 proxmox-backup-proxy: clean up old tasks when the task log was rotated
we maybe have old tasks when the task list was rotated, so clean them up

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-08 06:47:38 +02:00
Dominik Csapak
de55fff226 rest-server: add cleanup_old_tasks
this is a helper that removes task log files that are not referenced
by the task archive anymore

it gets the oldest task archive file, gets the first endtime (the
oldest) and removes all files in the taskdir where the mtime is older
than that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-08 06:38:52 +02:00
Dietmar Maurer
b3a67f1f14 proxmox-rrd: use correct directory options in create_rrdb_dir 2021-10-07 08:50:50 +02:00
Dietmar Maurer
3cc23ca6cc proxmox-rrd: cleanup error handling 2021-10-07 08:01:12 +02:00
Dietmar Maurer
3def6bfc64 proxmox-rrd: use log crate instead of eprintln, avoid duplicate logs 2021-10-06 18:19:22 +02:00
Dietmar Maurer
18e8bc17e4 proxmox-rrd: fix update (do not update) when time is in the past 2021-10-06 18:01:48 +02:00
Wolfgang Bumiller
f66d66aafe drop dynamic_index.rs duplicate in pbs-client
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-06 15:29:27 +02:00
Wolfgang Bumiller
7380c48dff pbs-tools::io::pipe: use nix Error type
there's no need to upgrade to anyhow::Error there already

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-06 15:28:58 +02:00
Dietmar Maurer
0191759316 proxmox-rrd: improve developer docs 2021-10-06 12:19:54 +02:00
Dietmar Maurer
dbc42e6f75 proxmox-rrd: remove serde dependency 2021-10-06 10:55:46 +02:00
Dietmar Maurer
d1c3bc5350 split out RRD api types into proxmox-rrd-api-types crate 2021-10-06 09:49:51 +02:00
Dietmar Maurer
a97301350f proxmox-rrd: use create_path instead of std::fs::create_dir_all
To ensure correct file ownership.
2021-10-06 08:37:14 +02:00
Dietmar Maurer
09340f28f5 move RRD code into proxmox-rrd crate 2021-10-06 08:13:28 +02:00
Thomas Lamprecht
20497c6346 bump version to 2.0.11-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-05 16:34:35 +02:00
Thomas Lamprecht
d0f7d0d9c1 d/changelog: fixup release
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-05 14:23:28 +02:00
Dietmar Maurer
608806e884 proxmox-rest-server: use new ServerAdapter trait instead of callbacks
Async callbacks are a PITA, so we now pass a single trait object which
implements check_auth and get_index.
2021-10-05 11:13:10 +02:00
Dietmar Maurer
48176b0a77 proxmox-rest-server: pass owned RestEnvironment to get_index
This way we avoid pointers with lifetimes.
2021-10-05 11:12:53 +02:00
Dietmar Maurer
3483a3b3a1 proxmox-rest-server: cleanup, access api_auth using a method 2021-10-05 11:12:53 +02:00
Wolfgang Bumiller
347e0d4c57 fix deprecated use of std::u64/... modules
integer primitive type modules are deprecated, use
associated constants instead

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-04 15:02:30 +02:00
Dominik Csapak
ae9b5c077a ui: datastore/Content: add empty text for no snapshots
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-04 10:28:10 +02:00
Dominik Csapak
747446eb50 ui: datastore/Content: reload in activate listener
when we trigger the first load before the panel was fully created,
there was no load mask for it (but the snapshots would "pop in" on load)

move the first reload into the 'activate' listener. this will be called
the every time a user opens the content tab of a datastore, so guard
it by a 'firstLoad' bool.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-04 10:28:10 +02:00
Thomas Lamprecht
e1c8c27f47 rest: daemon: group systemd FFI together
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-02 11:45:34 +02:00
Thomas Lamprecht
63cec1622a rest: daemon: sd notify: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-02 11:45:34 +02:00
Thomas Lamprecht
31142ef291 rest: daemon: sd notify barrier: avoid barging in between SystemdNotify enum and systemd_notify
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-02 11:45:34 +02:00
Thomas Lamprecht
058b4b9708 rest: daemon: sd notify barrier: allow caller to set timeout
else it's rather to subtle and not a nice interface considering that
we only want to have a thin wrapper for sd_notify_barrier..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-02 11:44:20 +02:00
Thomas Lamprecht
9a1330c72e rest: daemon: comment why using a systemd barrier is important for main PID handover
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-02 11:44:20 +02:00
Dominik Csapak
0a6df20986 rest-server/daemon: use sd_notify_barrier for service reloading
until now, we manually polled the systemd service state during a reload
so that the sd_notify messages get processed in the correct order
(RELOAD(old) -> MAINPID(old) -> READY(new))

with systemd >= 246 there is now 'sd_notify_barrier' which
blocks until systemd processed all prior messages

with that change, the daemon does not need to know the service name anymore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-10-02 11:44:20 +02:00
Dietmar Maurer
6680878b5c proxmox-rest-server: make get_index async 2021-10-01 09:38:10 +02:00
Dietmar Maurer
593043ed53 proxmox-rest-server: add comment why ApiService needs to be 'pub' 2021-10-01 08:35:51 +02:00
Dietmar Maurer
038f385089 proxmox-rest-server: make check_auth async 2021-10-01 07:53:59 +02:00
Dietmar Maurer
b914b94773 proxmox-rest-server: fix spelling errors 2021-10-01 06:43:30 +02:00
Dietmar Maurer
2194bc59c8 proxmox-rest-server: improve ApiService docs 2021-09-30 17:18:47 +02:00
Dietmar Maurer
a98a288e2d proxmox-rest-server: start module docs 2021-09-30 13:49:29 +02:00