Commit Graph

154 Commits

Author SHA1 Message Date
Wolfgang Bumiller d420962fbc split out pbs-runtime module
These are mostly tokio specific "hacks" or "workarounds" we
only really need/want in our binaries without pulling it in
via our library crates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 14:52:25 +02:00
Wolfgang Bumiller af06decd1b split out pbs-buildcfg module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 12:00:14 +02:00
Dominik Csapak 062cf75cdf proxmox-backup-proxy: fix leftover references on datastore removal
when we remove a datastore via api/cli, the proxy
has sometimes leftover references to that datastore in its
DATASTORE_MAP which includes an open filehandle on the
'.lock' file

this prevents unmounting/exporting the datastore even after removal,
only a reload/restart of the proxy did help

add a command to our command socket, which removes all non
configured datastores from the map, dropping the open filehandle

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-06-04 08:22:53 +02:00
Fabian Grünbichler 3241392117 refactor: move socket helper to proxmox crate
and constant to tools module.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-05-17 10:29:42 +02:00
Dietmar Maurer 4ce7da516d reload cert inside command socket handler 2021-05-12 12:03:27 +02:00
Wolfgang Bumiller 45b8a0327f refactor send_command
- refactor the combinators,
- make it take a `&T: Serialize` instead of a Value, and
  allow sending the raw string via `send_raw_command`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller a723c08715 proxy: implement 'reload-certificate' command
to be used via the command socket

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller c381a162fb proxy: factor out tls acceptor creation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller b4931192c3 proxy: Arc usage cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller cc269b9ff9 proxy: "continue on error" for the accept call, too
as this gets rid of 2 levels of indentation

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Wolfgang Bumiller a5e3be4992 proxy: factor out accept_connection
no functional changes, moved code and named the channel's
type for more readability

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-05-11 16:28:08 +02:00
Thomas Lamprecht 7a61f89e5a tape backup job: fix typo in error message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-05-03 12:01:23 +02:00
Dominik Csapak 368f4c5416 fix gathering io stats for zpools
if a datastore or root is not used directly on the pool dir
(e.g. the installer creates 2 sub datasets ROOT/pbs-1), info in
/proc/self/mountinfo returns not the pool, but the path to the
dataset, which has no iostats itself in /proc/spl/kstat/zfs/
but only the pool itself

so instead of not gathering data at all, gather the info from the
underlying pool instead. if one has multiple datastores on the same
pool those rrd stats will be the same for all those datastores now
(instead of empty) similar to 'normal' directories

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-12 13:35:38 +02:00
Thomas Lamprecht a21f9852fd enable tape backup by default
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-12 12:31:56 +02:00
Stefan Reiter 26858dba84 server/rest: add ApiAuth trait to make user auth generic
This allows switching the base user identification/authentication method
in the rest server. Will initially be used for single file restore VMs,
where authentication is based on a ticket file, not the PBS user
backend (PAM/local).

To avoid putting generic types into the RestServer type for this, we
merge the two calls "extract_auth_data" and "check_auth" into a single
one, which can use whatever type it wants internally.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Dietmar Maurer 40bf636b47 ui: enable experimental tape UI if tape.cfg exists 2021-03-03 09:02:02 +01:00
Dietmar Maurer 5830e5620d tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig 2021-02-25 09:44:31 +01:00
Dietmar Maurer 8513626b9f proxmox-backup-proxy: schedule tape backup jobs 2021-02-15 08:30:42 +01:00
Fabian Grünbichler ea368a06cd clippy: misc. fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
Fabian Grünbichler 17c7b46a69 clippy: use unwrap_or_default
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
Fabian Grünbichler 3b82f3eea5 clippy: avoid useless format!
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
Fabian Grünbichler 3984a5fd77 clippy: is_some/none/ok/err/empty
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
Fabian Grünbichler 0bfcea6a11 cleanup: remove unnecessary 'mut' and '.clone()'
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Fabian Grünbichler 0f860f712f tokio 1.0: update to new tokio-openssl interface
connect/accept are now happening on pinned SslStreams

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Fabian Grünbichler 7c66701366 tokio 1.0: use ReceiverStream from tokio-stream
to wrap a Receiver in a Stream. this will likely move back into tokio
proper once we have a std Stream..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Fabian Grünbichler 0a8d773ad0 tokio 1.0: delay -> sleep
almost the same thing, new name(s), no longer Unpin

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Wolfgang Bumiller 7f066a9b21 proxy: expose qrcodejs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Dominik Csapak d7c6ad60dd daemon: add hack for sd_notify
sd_notify is not synchronous, iow. it only waits until the message
reaches the queue not until it is processed by systemd

when the process that sent such a message exits before systemd could
process it, it cannot be associated to the correct pid

so in case of reloading, we send a message with 'MAINPID=<newpid>'
to signal that it will change. if now the old process exits before
systemd knows this, it will not accept the 'READY=1' message from the
child, since it rejects the MAINPID change

since there is no (AFAICS) library interface to check the unit status,
we use 'systemctl is-active <SERVICE_NAME>' to check the state until
it is not 'reloading' anymore.

on newer systemd versions, there is 'sd_notify_barrier' which would
allow us to wait for systemd to have all messages from the current
pid to be processed before acknowledging to the child, but on buster
the systemd version is to old...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-11 09:43:00 +01:00
Thomas Lamprecht 0eb0c4bd63 proxy: fix log message for auth log rotation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 16:34:03 +01:00
Dominik Csapak db8a606707 proxmox-backup-proxy: remove unnecessary alias
the basedir is already /usr/share/javascript/proxmox-backup/
so adding a subdir of that as alias is not needed

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-06 18:08:18 +01:00
Dietmar Maurer ad54df3178 get rid of backup@pam 2020-11-06 08:39:30 +01:00
Thomas Lamprecht fe7bdc9d29 proxy: also rotate auth.log file
no need for triggering re-open here, we always re-open that file.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 17:26:34 +01:00
Thomas Lamprecht 546b6a23df proxy: logrotate: do not serialize sending async log-reopen commands
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 17:26:34 +01:00
Dietmar Maurer ea93bea7bf proxy: log if there are too many open connections 2020-11-04 08:49:35 +01:00
Dietmar Maurer e1d367df47 proxy: use env PROXMOX_DEBUG to enable/disable debug output
We only print early connection errors when this env var is set.
2020-11-04 06:55:57 +01:00
Dietmar Maurer 71f413cd27 cleanup: use Arc to count open connections 2020-11-04 06:35:44 +01:00
Dietmar Maurer 48aa2b93b7 fix #3106: correctly queue incoming connections 2020-11-04 06:24:42 +01:00
Thomas Lamprecht a71a009313 proxy: drop now unused UPID import
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 21:08:38 +01:00
Dominik Csapak b6ba5acd29 proxmox-backup-proxy: use only jobstate for garbage_collection schedule
in case the garbage_collection errors out, we never set the in-memory
state, so if it failed, the last 'good' starttime was considered
for the schedule

this could lead to the job running every minute instead of the
correct schedule

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-02 21:08:38 +01:00
Dominik Csapak c724f65805 server/gc_job: add 'to_stdout'
we will use this for the manual api call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-02 21:08:38 +01:00
Thomas Lamprecht b7f2be5137 log rotate task: make task archive limits be binary based
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 19:53:30 +01:00
Thomas Lamprecht 72aa1834dc log rotate task: adapt internal jobstate ID, set worker one to None for now
as we have only one logrotate task currently..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 19:53:30 +01:00
Thomas Lamprecht fe4cc5b1a1 server: implement access log rotation with re-open via command socket
re-use the future we already have for task log rotation to trigger
it.

Move the FileLogger in ApiConfig into an Arc, so that we can actually
update it and REST using the new one.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 19:53:30 +01:00
Thomas Lamprecht 04b053d87e server: write main daemon PID to run directory
so that we can easily get the main PID of the last recently launched
daemon. Will be used to get the control socket of that one for access
lgo rotate in a future patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 19:50:24 +01:00
Thomas Lamprecht a68768cf31 server: use generalized commando socket for worker tasks commands
Allows to extend the use of that socket in the future, e.g., for log
rotate re-open signaling.

To reflect this we use a more general name, and change the commandos
to a more clear namespace.

Both are actually somewhat a breaking change, but the single real
world issue it should be able to cause is, that one won't be able to
stop task from older daemons, which still use the older abstract
socket name format.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-02 19:48:04 +01:00
Fabian Grünbichler 1cd951c93e proxy: fix warnings
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-30 12:49:43 +01:00
Dietmar Maurer 3b707fbb8f proxy: split out code to run garbage collection job 2020-10-30 11:01:45 +01:00
Dietmar Maurer b15751bf55 check_schedule cleanup: use &str instead of String
This way we can avoid many clone() calls.
2020-10-30 09:49:50 +01:00
Hannes Laimer 82c05b41fa proxy: extract commonly used logic for scheduling into new function
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-10-30 09:49:50 +01:00
Hannes Laimer b8d9079835 proxy: move prune logic into new file
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-10-30 09:49:50 +01:00