Commit Graph

6149 Commits

Author SHA1 Message Date
Wolfgang Bumiller
38629c3961 move ChunkStream to pbs-client
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-20 10:52:21 +02:00
Thomas Lamprecht
513d019ac3 issue banner: avoid depending on proxmox crate for hostname
While this slightly duplicates code we just do not profit from the
central, lazy static variant here, as that is only really useful in
daemons to avoid doing frequent syscalls there.

proxmox just pull in far to much (e.g., tokio) and duplicating that
one line of simple code has no real maintenance cost, so just go for
that and use the nix crate directly.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-19 16:32:50 +02:00
Wolfgang Bumiller
3fa1b4b48c cleanup unused imports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 15:55:19 +02:00
Wolfgang Bumiller
a6eac535e4 Makefile: fix build.rs reference
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 15:54:53 +02:00
Wolfgang Bumiller
58a3fae773 move pxar binary to separate crate
and move its few remaining proxmox_backup deps out to
pbs-tools

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 15:53:43 +02:00
Wolfgang Bumiller
0889806a3c resolve some more client imports
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 15:03:24 +02:00
Wolfgang Bumiller
51ec8a3c62 move some api types to pbs-api-types
and resolve some imports in the client binary

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 15:01:03 +02:00
Wolfgang Bumiller
a12b1be728 move build.rs and friends to pbs-buildcfg
with this the main crate won't be re-compiled every time a
*binary* is modified

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 14:59:18 +02:00
Wolfgang Bumiller
4d04cd9ab9 comment on test output paths
cargo should be getting a new env var for this soon

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 14:24:13 +02:00
Wolfgang Bumiller
a3399f4337 doc and tests fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 14:16:28 +02:00
Wolfgang Bumiller
2b7f8dd5ea move client to pbs-client subcrate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 12:58:43 +02:00
Wolfgang Bumiller
72fbe9ffa5 move 'wait_for_local_worker' from client to server
this just made no sense in the client

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 10:44:44 +02:00
Wolfgang Bumiller
0be8bce718 d/control: fixup proxmox feature flags
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 10:09:43 +02:00
Wolfgang Bumiller
4805edc4ec move more tools for the client into subcrates
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 10:07:12 +02:00
Wolfgang Bumiller
9eb784076c move more helpers to pbs-tools
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 10:07:12 +02:00
Wolfgang Bumiller
b9c5cd8291 add proxmox-backup-banner binary crate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 10:07:12 +02:00
Wolfgang Bumiller
9008c0c177 bump proxmox-apt dependency
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-19 10:07:12 +02:00
Dominik Csapak
f027c2146e ui: datastore/Prune: improve title of group prune window
we are not actually pruning the whole datastore, but only the single
group, so set that as a title

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:42:30 +02:00
Dominik Csapak
afbf2e10f3 ui: datastore/Content: add 'Prune All' button
since the api call always starts a real worker, we cannot have a
preview. It would also be very hard to show that for all groups in a
non-confusing way. We reuse the pbsPruneInputPanel and add the dry-run
field there conditionally.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:42:09 +02:00
Dominik Csapak
9805207aa5 api: admin/datastore: add new 'prune-datastore' api call
to prune the whole datastore at once, with the given parameters.
We need a new api call since this can take a while and we need to start
a worker for this. The exisiting api call returns a list of removed/kept
snapshots and is synchronous.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:40:05 +02:00
Dominik Csapak
8e0b852f24 server/prune_job: add proper permission checks to 'prune_datastore'
checks for PRIV_DATASTORE_MODIFY, or else if the auth_id is the backup
owner, and skips the group if not.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:39:01 +02:00
Dominik Csapak
0052dc6d28 server/prune_job: add 'keep_all' logic to 'prune_datastore'
it is the same as when pruning single groups.
for prune_jobs, we never start the worker if there is no prune option set.
but if we want to call 'prune_datastore' from somewhere else, we
have to check it here again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:38:28 +02:00
Dominik Csapak
61f05679d2 server/prune_job: factor out 'prune_datastore'
we want to use that outside of a prune job

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:36:45 +02:00
Dominik Csapak
9751ef4b36 backup/datastore: refactor check_backup_owner there
and add a 'owns_backup' convenience function

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:36:02 +02:00
Dominik Csapak
0a240aaa9a api: admin/datastore: simplify prune api call
by using the api macro and reusing the PruneOptions from pbs-datastore

this means we can now drop the 'add_common_prune_prameters' macro

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:34:36 +02:00
Dominik Csapak
e0665a64bd client: simplify prune api method
by using the api macro on the async method and reusing the PruneOptions
from pbs-datastore with 'flatten: true'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:34:28 +02:00
Dominik Csapak
dc46aa9a00 pbs-datastore/prune: make PruneOptions an api type
so that we can reuse it from here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:34:18 +02:00
Dominik Csapak
ced694589d api-types: move PRUNE_SCHEMA_KEEP_* to pbs-api-types
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 11:26:09 +02:00
Dominik Csapak
6c053ffc89 tape: changer: sg_pt: make extra scsi request for dvcid
some libraries cannot handle a request with volume tags and DVCID set at
the same time.

So we make 2 separate requests and merge them, since we want to keep
the vendor/model/serial data.

to not overcomplicate the code, add another special type to ElementType

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-16 08:46:06 +02:00
Wolfgang Bumiller
9f5b57a348 buildsys: Prepare new way for path dependencies
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-15 09:56:32 +02:00
Wolfgang Bumiller
f1c4b8df34 features update
so we can drop default-features in proxmox for build-deps to
be more lean

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-15 09:56:05 +02:00
Fabian Grünbichler
269e274bb5 d/control: update proxmox b-d
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-07-14 13:51:38 +02:00
Dietmar Maurer
bfd357c5a1 depend on proxmox 0.11.6 (changed make_tmp_file() return type) 2021-07-14 13:37:26 +02:00
Dominik Csapak
9517a5759a fix #3526: correctly filter tasks with 'since' and 'until'
The previous assumption was that the Tasks returned by the Iterator are
sorted by the starttime, but that is not actually the case, and
could never have been, since we append the tasks into the log when
they are finished (not started) and running tasks are always iterated
first.

To correctly filter (and simplify the the api call) we forgo the
combinators, and use a for loop instead. This way we only have to do
the since/until checks only once per Task, but have to do the
start/limit counting ourselves.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-14 09:39:14 +02:00
Thomas Lamprecht
a5d51b0c4f docs: tape: drop technology preview admonitions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-13 16:48:05 +02:00
Stefan Reiter
d9822cd3cb fix #3515: file-restore-daemon: allow LVs/PVs with dash in name
LVM replaces any dashes '-' in an LV or PV name with two '--' for the
created device node in /dev/mapper/ to distinguish the seperating
character between the PV and LV name.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-13 12:07:51 +02:00
Stefan Reiter
66501529a2 file-restore: increase lock timeout on QEMU map
This lock is held during VM startup, so that multiple calls will not
start VMs twice. But this means that the timeout needs to incorporate
the time it might take a VM to boot, so increase it quite a bit.

This could previously lead to "interrupted system call" errors when
accessing backups with many disks.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-13 12:07:23 +02:00
Dominik Csapak
2072dede4a api2: tape: restore: add warning for list restore
if an error occurs, the snapshot dirs will already be created, and we
do not clean them up (some might already be finished).

Warn the user that they are not cleaned up.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-13 12:02:01 +02:00
Dominik Csapak
31c94d1645 chunk_store/insert_chunk: add more information to file errors
otherwise this context is missing in some tasks (e.g. tape restore)
and it is unclear where it came from

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-13 11:55:33 +02:00
Dietmar Maurer
9ee4c23833 tape: changer: sg_pt: always retry until timeout 2021-07-13 10:39:28 +02:00
Dominik Csapak
a14a1c7b90 ui: tape/BackupOverview: increase timeout for media-set content
a single catalog can be over 100MiB, and a media-set can have multiple
catalogs to read (no technical upper limit). On slow disks, this can
take much longer than 30 seconds (the default timeout).

The real solution would be to have some kind of index only for the gui
relevant part, e.g. a table in the beginning of the catalog, or
alternatively a seperate file with that info. Until we have such a
solution increase the timeout as a stopgap.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-13 09:44:17 +02:00
Thomas Lamprecht
9ef88578af bump version to 2.0.4-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-12 18:51:41 +02:00
Stefan Reiter
c4c4b5a3ef auth: 'crypt' is not thread safe
According to crypt(3):
"crypt places its result in a static storage area, which will be
overwritten by subsequent calls to crypt. It is not safe to call crypt
from multiple threads simultaneously."

This means that multiple login calls as a PBS-realm user can collide and
produce intermittent authentication failures. A visible case is for
file-restore, where VMs with many disks lead to just as many auth-calls
at the same time, as the GUI tries to expand each tree element on load.

Instead, use the thread-safe variant 'crypt_r', which places the result
into a pre-allocated buffer of type 'crypt_data'. The C struct is laid
out according to 'lib/crypt.h.in' and the man page mentioned above.

Use the opportunity and make both arguments to the rust 'crypt' function
take a &[u8].

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-07-12 18:38:48 +02:00
Dominik Csapak
0ed40b19c7 tape: changer: sg_pt: query element types separately
Some changers do not like the DVCID bit when querying non-drives,
this includes when querying 'all' elements.

To circumvent this, we query each type by itself (like mtx does it),
and only add the DVCID bit for drives (Data Transfer Elements).

Reported by a user in the forum:
https://forum.proxmox.com/threads/ibm-3584-ts3500-support.92291/

and limit to 1000 elements per request.
(Because some changers limit that request with the options we set)

instead of checking if the data len was equal to the allocation_len
for getting more data, we count the returned elements and compare
that with the number we requested

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-07-12 18:19:26 +02:00
Dietmar Maurer
a0cd0f9cec change tape drive lock path
New kernel has stricter checks on tmpfs with stick-bit on directories, so some
commands (i.e. proxmox-tape changer status) fails when executed as root, because
permission checks fails when locking the drive.

This patch move the drive locks to /run/proxmox-backup/drive-lock.

Note: This is incompatible to old locking mechmanism, so users may not
run tape backups during update (or running backup can fail).
2021-07-12 17:26:49 +02:00
Thomas Lamprecht
49e47c491b d/postinst: drop some legacy update handling
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-12 16:14:28 +02:00
Thomas Lamprecht
424d2d68d3 buildsys: try to avoid duplicate build due to "phony" docs dependency
Make docs target depend directly on the some docs-only required
binaries and add a new intermediate ".do-cargo-build" target that is
explicitly not a PHONY target.

That avoids one extra set of full builds.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-12 13:19:20 +02:00
Thomas Lamprecht
415690a0e7 bump version to 2.0.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-12 09:53:07 +02:00
Dominik Csapak
2c0abe9234 Revert "api: access: domains: add ExtraRealmInfo and RealmInfo structs"
This reverts commit da7ec1d2af.

not necessary, since we have the api in config/access/openid

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-12 09:53:07 +02:00
Dominik Csapak
2649c89358 Revert "api: access: domains: add get/create/update/delete domain call"
This reverts commit 5117cf4f17.

we already have that in api2/config/access

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-12 09:53:07 +02:00