Commit Graph

4222 Commits

Author SHA1 Message Date
Stoiko Ivanov
7c2431d42c api: acme: fix typo
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Stoiko Ivanov
25c1420a12 config: acme: plugin: rustfmt
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Stoiko Ivanov
c1a1e1ae8f api: config: acme: rustfmt
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-18 11:32:22 +01:00
Dietmar Maurer
10beed1199 openid: allow to configure scopes, prompt, ACRs and arbitrary username-claim values
- no longer set prompt to 'login' (makes auto-login possible)
- new prompt configuration
- allow arbitrary username-claim values

Depend on proxmox-openid 0.9.0.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 11:20:55 +01:00
Thomas Lamprecht
062edce27f group filter: rename CLI/API/Config "groups" option to "group-filter"
we even use that for basically all the related schema names, "groups"
allone is just rather not so telling, i.e., "groups" what?

While due to the additive nature of `group-filter` is not the best
possible name for passing multiple arguments on the CLI (the web-ui
can present this more UX-friendly anyway) due to possible confusion
about if the filter act like AND vs OR it can be documented and even
if a user is confused they still are safe on more being synced than
less. Also, the original param name wasn't really _that_ better in
that regards

Dietmar also suggested to use singular for the CLI option, while
there can be more they're passed over repeating the option, each with
a single filter.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak
efd2713aa8 proxmox-tape: add groups filter to backup command
and add a completion handler to complete the backup groups

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak
c8c5c7f571 fix #3533: tape backup: filter groups according to config
this fixes bug #3533, since now a user can backup a single datastore
on multiple tape media pools in parallel, e.g. vms on one pool, ct on
another.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak
91357c2034 tape backup jobs: add group filters to config/api
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dominik Csapak
097ccfe1d5 proxmox-tape: add missing 'notify-user' option to backup command
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
61ef4ae8cb fix #sync.cfg/pull: don't remove by default
and convert existing (manually created/edited) jobs to the previous
default value of 'true'. the GUI has always set this value and defaults
to 'false'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
1b52122a1f manager: render group filter properly
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
1d9bc184f5 remote: add backup group scanning
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
5f83d3f636 sync: add group filtering
like for manual pulls, but persisted in the sync job config and visible
in the relevant GUI parts.

GUI is read-only for now (and defaults to no filtering on creation), as
this is a rather advanced feature that requires a complex GUI to be
user-friendly (regex-freeform, type-combobox, remote group scanning +
selector with additional freeform input).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
71e534631f pull: allow pulling groups selectively
without requiring workarounds based on ownership and limited
visibility/access.

if a group filter is set, remove_vanished will only consider filtered
groups for removal to prevent concurrent disjunct filters from trashing
eachother's synced groups.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
6e9e6c7a54 pull/sync: extract passed along vars into struct
this is basically the sync job config without ID and some stuff
converted already, and a convenient helper to generate the http client
from it.

Suggested-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Fabian Grünbichler
e2e7560d5e pull: use BackupGroup consistently
instead of `GroupListItem`s. we convert it anyway, so might as well do
that at the start.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-18 10:36:57 +01:00
Dietmar Maurer
15cc41b6cb proxmox-systemd: remove crate, use new proxmox-time 1.1.0 instead
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-17 13:07:51 +01:00
Wolfgang Bumiller
729bd1fd16 remove now unused serde_filter module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-17 09:50:08 +01:00
Wolfgang Bumiller
9407810fe1 switch tfa api to use proxmox-tfa::api
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-17 09:33:04 +01:00
Dietmar Maurer
c42a54795d move fingerprint helpers from pbs-tools to pbs-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-17 07:07:40 +01:00
Dietmar Maurer
24f9af9e0f add missing file from previous commit
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-14 18:49:29 +01:00
Dietmar Maurer
a0172d766b traffic-controls: add API/CLI to show current traffic
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-14 17:21:45 +01:00
Dietmar Maurer
09f999337a update to proxmox-http 0.5.4
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-14 08:27:45 +01:00
Dietmar Maurer
e3eb062c09 cached_traffic_control: fix regression tests
Avoid using shared memory in tests because of permission problems.
2021-11-14 08:05:40 +01:00
Dietmar Maurer
de21d4efdc implement rate limiter in shared memory
This kind of rate limiter can be used among several processes (as long
as all set the same rate/burst).
2021-11-13 17:49:38 +01:00
Dietmar Maurer
d5f58006d3 cached_traffic_control: use ShareableRateLimit trait object 2021-11-13 17:49:38 +01:00
Dietmar Maurer
cb80ffc1de pbs-config: use new SharedMemory helpers from proxmox-shared-memory crate
depend on proxmox-shared-memory crate.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-13 17:49:38 +01:00
Dietmar Maurer
bf013be1c4 create /var/lib/proxmox-bnackup at server startup
This was missing in previous patch...

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-12 08:12:18 +01:00
Dietmar Maurer
b935209584 fix directory permission problems
By carefully setting options on all create_path() calls,
and by creating "/var/lib/proxmox-backup" at api server startup.
2021-11-12 07:29:18 +01:00
Dietmar Maurer
e511e0e553 proxmox-backup-proxy: implement traffic control
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-10 10:15:40 +01:00
Dietmar Maurer
610150a4b4 implement a traffic control cache for fast rate control limiter lockups
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-10 10:15:40 +01:00
Dietmar Maurer
bfd12e871f Add traffic control configuration config with API
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-10 10:15:40 +01:00
Dietmar Maurer
68fd9ca6d6 openid_login: vertify that firstname, lastname and email fits our schema definitions
If not, we do not copy the values to our user.cfg.
2021-11-10 06:48:40 +01:00
Dietmar Maurer
4beb7d2dbe correctly lock remote config
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-06 17:35:10 +01:00
Dominik Csapak
9b1e2ae83c api: admin/datastore: reuse 'is_protected' implementation
we already have that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 12:54:54 +02:00
Dominik Csapak
8292d3d20e api2/admin/datastore: add get/set_protection
for gettin/setting the protected flag for snapshots (akin to notes)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 11:31:11 +02:00
Dominik Csapak
5cc7d89139 api2: datastore/delete_group: throw error for partially removed group
when a group could not be completely removed due to protected snapshot,
throw an error

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 11:31:07 +02:00
Dominik Csapak
343392613d pull_store/group: dont try remove locally protected snapshots
and log if a vanished groups could not be completely deleted if it
contains protected snapshots

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 11:31:04 +02:00
Dominik Csapak
02db72678f add protected info of snapshots to api and task logs
adds the info that a snapshot is protected to:
* snapshot list
* manual pruning (also dry-run)
* prune jobs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-28 11:30:51 +02:00
Dominik Csapak
9d42e0475b acme: interpret no TOS as accepted
some custom ACME endpoints do not have TOS, interpret this as
'the user has accepted the TOS', like we do for PVE.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-22 10:55:14 +02:00
Wolfgang Bumiller
0a33951e9e acme: new_account: prevent replacing existing accounts
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-22 08:35:24 +02:00
Dietmar Maurer
e0a19d3313 use new fsync parameter to replace_file and atomic_open_or_create
Depend on proxmox 0.15.0 and proxmox-openid 0.8.1

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-10-21 07:28:32 +02:00
Dominik Csapak
6b8329ee34 tape: simplify export_media_set for pool writer
our export code can handle if the tape is inside the drive, so unloading
it first does not have an benefit, it even makes the exporting slower,
since we first unload it into its original slot, and then moving it
to an import/export slot

so drop the code that unloads the tape from the drive, and let the
export code itself handle that

change the 'eject' into a 'rewind' and comment why we do that first

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-21 06:37:48 +02:00
Dietmar Maurer
934c8724e2 proxmox-rrd: add option to avoid page cache for load/save
use fadvice(.., POSIX_FADV_DONTNEED) for RRD files. We read those files only once,
and always rewrite them.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-10-19 11:17:09 +02:00
Dietmar Maurer
98eb435d90 proxmox-rrd: use syncfs after writing rrd files
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-10-19 11:17:09 +02:00
Dominik Csapak
ac2ca6c341 tape: improve export_media error message for not found tape
'export_media' can handle if the tape is in either a normal slot of the
library, or in the drive assigned to the current pool writer.
(because we need to lock the drive)

if it is, for some reason, in a different drive, the error message
 'media is not online'
could be slightly confusing for a user, since it would appear in the drive list

add the 'or a differen drive' to make it clearer

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-18 10:40:56 +02:00
Dietmar Maurer
fce7cd0d36 proxmox-rrd: avoild blocking readers while applying the journal
By using and extra RwLock<RRDMap> on the rrd data.
2021-10-15 09:22:07 +02:00
Dietmar Maurer
2e3f94e12f proxmox-rrd: make rrd load callback configurable 2021-10-14 11:41:26 +02:00
Dietmar Maurer
58f70bccbb proxmox-rrd: pass time and value to update function 2021-10-14 08:12:56 +02:00
Dietmar Maurer
fae4f6c509 cleanup: move rrd cache related code into extra file 2021-10-14 07:57:27 +02:00