Commit Graph

356 Commits

Author SHA1 Message Date
Thomas Lamprecht
e2aeff40eb tape: use inline variable in formats for code reduction
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:38:33 +02:00
Thomas Lamprecht
d20137e5a9 tree wide: typo fixes through codespell
Most, not all, found and fixes using `codespell -wci3 -L crate`

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-05 10:34:10 +02:00
Thomas Lamprecht
fbfb64a6b2 tree wide: clippy lint fixes
most (not all) where done automatically

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-06-02 15:59:55 +02:00
Dominik Csapak
8bec3ff691 tape/pool_writer: give proper types to 'contains_snapshot'
instead of a string. The underlying catalog implementation has to
care about how this is formatted, not the external caller

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-23 16:20:13 +02:00
Fabian Grünbichler
e49bd1e98f tape: media catalog: use match for magic check
like in other parts of the code

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 14:08:32 +02:00
Fabian Grünbichler
707c48ad46 tape: bump catalog/snapshot archive magic
the snapshot string format is not backwards compatible since it now has
an in-line namespace prefix. it's possible to select which magic to use
at the start of the backup, since a tape backup job knows whether it
operates on non-root namespaces up-front.

the MediaCatalog itself also has a similar incompatible change, but
there
- updating existing catalogs in-place
- not knowing what the catalog will contain in the future when initially
  creating/opening it
makes bumping the magic there harder. since the tape contents are
sufficiently guarded by the other two bumps, ignoring the
backwards-incomaptible change of the on-disk catalogs seems like an okay
tradeoff.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak
07ffb86451 api: tape/restore: add namespace mapping
by adding a new parameter 'namespaces', which contains a mapping
for a namespace like this:

store=datastore,source=foo,target=bar,max-depth=2

if source or target are omitted the root namespace is used for its value

this mapping can be given several times (on the cli) or as an array (via
api) to have mappings for multiple datastores

if a specific snapshot list is given simultaneously, the given snapshots
will be restored according to this mapping, or to the source namespace
if no mapping was found.

to do this, we reutilize the restore_list_worker, but change it so that
it does not hold a lock for the duration of the restore, but fails
if the snapshot does exist at the end. also the snapshot will now
be temporarily restored into the target datastore into the
'.tmp/<media-set-uuid>' folder.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 14:08:32 +02:00
Dominik Csapak
9c65e6ab4a tape: fix snapshot path in catalog and snapshot_archive
both used the 'Display' trait of pbs_datastore::BackupDir, which is not
intended to be serialized anywhere. Instead, manually format the path
using the print_ns_and_snapshot helper, and conversely, parse with
'parse_ns_and_snapshot'. to be a bit safer, change the register_snapshot
signature to take a BackupNamespace and BackupDir instead of a string.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 13:52:50 +02:00
Dominik Csapak
05b7175a56 tape: notify when arriving at end of media
when continuing a media set, we first move to the end of the tape and
start with the next (chunk) archive. If that takes long, the task logs
last line is 'moving to end of media' even if we already startet
writing. To make this less confusing, log that we arrived at the
end of the media.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-05-13 13:52:50 +02:00
Wolfgang Bumiller
db87d93efc make datastore BackupGroup/Dir ctors private
And use the api-types for their contents.

These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:56:23 +02:00
Thomas Lamprecht
4de1c42c20 tape: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 17:49:03 +02:00
Dominik Csapak
f26d7ca5c5 use io_format_err, io_bail, io_err_other from proxmox-lang
and move the comment from the local io_bail in pbs-client/src/pxar/fuse.rs
to the only use

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:13 +01:00
Dominik Csapak
dcd9c17fff tape/pool_writer: skip already backed up chunks in iterator
currently, the iterator goes over *all* chunks of the index, even
those already backed up by a previous snapshots in the same tape
backup. this is bad since for each iterator, we stat each chunk to
sort by inode number. so to avoid stat'ing the same chunks over
and over for consecutive snapshots, add a 'skip_fn' to the iterator
and in the pool writer and check the catalog_set if we can skip it

this means we can drop the later check for the catalog_set
(since we don't modify that here)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-02-18 10:40:41 +01:00
Fabian Grünbichler
645d52308b TimeSpan: parse via FromStr
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
aa174e8e8a tree-wide: drop redundant clones
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
0a7f902e2a tape: multi-volume: fix overflow check
the part number cannot go above 255 at the moment, but if it ever gets
bumped to a bigger integer type this boundary wouldn't cause a
compile-error. explicitly checking for overflowing u8 makes this a bit
more future-proof, and shuts up clippy as well ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Fabian Grünbichler
9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Dominik Csapak
7549114c9f adapt compute_next_event to new signature
the 'utc' flag is now contained in the event itself and not given
as a flag to 'compute_next_event' anymore

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-02 10:40:58 +01:00
Fabian Grünbichler
f66d814792 fix broken format strings
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-01 12:46:37 +01:00
Dominik Csapak
68b6c1202c remove use of deprecated functions from proxmox-time
Depend on proxmox-time 1.1.1

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-01 07:23:18 +01:00
Dietmar Maurer
25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Dietmar Maurer
ca6e66aa5a Fingerprint: add new signature method
commit c42a54795d introcuded a bug by
using fp.to_string(). Replace this with fp.signature() which correctly
returns the full fingerprint instead of the short version.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 08:29:43 +01:00
Dietmar Maurer
d5790a9f27 use new proxmox-sys crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 11:06:35 +01:00
Dietmar Maurer
15cc41b6cb proxmox-systemd: remove crate, use new proxmox-time 1.1.0 instead
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-17 13:07:51 +01:00
Dietmar Maurer
c42a54795d move fingerprint helpers from pbs-tools to pbs-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-17 07:07:40 +01:00
Dietmar Maurer
b935209584 fix directory permission problems
By carefully setting options on all create_path() calls,
and by creating "/var/lib/proxmox-backup" at api server startup.
2021-11-12 07:29:18 +01:00
Dietmar Maurer
e0a19d3313 use new fsync parameter to replace_file and atomic_open_or_create
Depend on proxmox 0.15.0 and proxmox-openid 0.8.1

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-10-21 07:28:32 +02:00
Dominik Csapak
6b8329ee34 tape: simplify export_media_set for pool writer
our export code can handle if the tape is inside the drive, so unloading
it first does not have an benefit, it even makes the exporting slower,
since we first unload it into its original slot, and then moving it
to an import/export slot

so drop the code that unloads the tape from the drive, and let the
export code itself handle that

change the 'eject' into a 'rewind' and comment why we do that first

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-21 06:37:48 +02:00
Dominik Csapak
ac2ca6c341 tape: improve export_media error message for not found tape
'export_media' can handle if the tape is in either a normal slot of the
library, or in the drive assigned to the current pool writer.
(because we need to lock the drive)

if it is, for some reason, in a different drive, the error message
 'media is not online'
could be slightly confusing for a user, since it would appear in the drive list

add the 'or a differen drive' to make it clearer

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-10-18 10:40:56 +02:00
Wolfgang Bumiller
6ef1b649d9 update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Dietmar Maurer
c95c1c83b0 move src/backup/snapshot_reader.rs to pbs_datastore crate 2021-09-27 09:58:20 +02:00
Dietmar Maurer
6d5d305d9d move src/backup/datastore.rs into pbs_datastore crate 2021-09-27 09:11:38 +02:00
Dietmar Maurer
619cd5cbcb cleanup WorkerTaskContext 2021-09-24 11:39:30 +02:00
Dietmar Maurer
1ec0d70d09 cleanup worker task logging
In order to avoid name conflicts with WorkerTaskContext

- renamed WorkerTask::log to WorkerTask::log_message

Note: Methods have different fuction signatures

Also renamed WorkerTask::warn to WorkerTask::log_warning for
consistency reasons.

Use the task_log!() and task_warn!() macros more often.
2021-09-24 10:34:11 +02:00
Dietmar Maurer
c8449217dc rename TaskState to WorkerTaskContext 2021-09-24 10:33:49 +02:00
Dietmar Maurer
b9700a9fe5 move worker_task.rs into proxmox-rest-server crate
Also moved pbs-datastore/src/task.rs to pbs-tools, which now depends on 'log'.
2021-09-24 10:28:17 +02:00
Dietmar Maurer
81867f0539 use UPID and systemd helpers from proxmox 0.13.4 2021-09-23 12:01:43 +02:00
Dietmar Maurer
84af82e8cf rename pbs-systemd to proxmox-systemd 2021-09-21 10:06:27 +02:00
Dietmar Maurer
ccc3896ff3 avoid type re-exports 2021-09-14 08:35:43 +02:00
Dietmar Maurer
cef5c72682 move src/tape/helpers/snapshot_reader.rs to src/backup/snapshot_reader.rs 2021-09-14 07:42:06 +02:00
Dietmar Maurer
048b43af24 split tape code into new pbs_tape workspace 2021-09-13 12:54:59 +02:00
Dietmar Maurer
67a5cf4714 fix regression tests 2021-09-10 12:45:06 +02:00
Dietmar Maurer
6227654ad8 more api type cleanups: avoid re-exports 2021-09-10 12:25:32 +02:00
Dietmar Maurer
5839c469c1 move tape_encryption_keys.rs to pbs_config workspace 2021-09-07 10:37:08 +02:00
Dietmar Maurer
bbdda58b35 moved key_derivation.rs from pbs_datastore to pbs-config/src/key_config.rs
Also moved pbs-datastore/src/crypt_config.rs to pbs-tools/src/crypt_config.rs.
We do not want to depend on pbs-api-types there, so I use [u8;32] instead of
Fingerprint.
2021-09-07 10:12:17 +02:00
Dominik Csapak
f46806414a tape/inventory: fix the tape tests as user, by mocking the lock
locking during the tests as regular user failed because we try to
chown to the backup user (which is not always possible).

Instead, do not lock at all, by implementing 'open_backup_lockfile' with
'create_mocked_lock'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-07 08:42:04 +02:00
Dietmar Maurer
1ce8e905ea move drive config to pbs_config workspace
Also moved the tape type definitions to pbs_api_types.
2021-09-03 09:10:18 +02:00
Dietmar Maurer
2121174827 start new pbs-config workspace
moved src/config/domains.rs
2021-09-02 12:58:20 +02:00
Wolfgang Bumiller
4c1b776168 another import cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-01 14:46:01 +02:00
Wolfgang Bumiller
42dad3abd3 fixup imports in tests and examples
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-01 12:32:21 +02:00