9c953dd260
tape: add code to write backup snapshot files (without chunks) to tape
2020-12-17 08:28:47 +01:00
3fbf2d2fcd
tape: cleanup MediaCatalog
2020-12-17 08:05:53 +01:00
e0af222ec3
KeyConfig: always calculate fingerprint
...
and warn if stored and calculated fingerprint don't match.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:52:55 +01:00
73b5011786
KeyConfig: add encrypt/decrypt test
...
the RSA key and the encryption key itself are hard-coded to avoid
stalling the test runs because of lack of entropy, they have no special
significance otherwise.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:47:45 +01:00
7137630d43
client: add 'import-with-master-key' command
...
to import an encrypted encryption key using a master key.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:46:24 +01:00
8acfd15d6e
key: move RSA-encryption to KeyConfig
...
since that is what gets encrypted, and not a CryptConfig.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:43:34 +01:00
48fbbfeb7e
fix #3197 : skip fingerprint check when restoring key
...
when restoring an encrypted key, the original one is obviously not
available to check the fingerprint with.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:37:54 +01:00
9990af3042
master key: store blob name in constant
...
since we will use it in more than one place.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:36:06 +01:00
fe6c19383b
tape: remove MediaLabelInfo, use MediaId instead
...
The additional content_uuid was quite useless...
2020-12-16 13:31:32 +01:00
9839d3f778
tape: improve docu
2020-12-16 12:43:51 +01:00
dd59e3c2a1
tape: improve docu
2020-12-16 12:23:52 +01:00
0b7432ae09
tape: add chunk archive reader/writer
2020-12-16 12:08:34 +01:00
c1c2c8f635
tape: cleanup MediaLocation type for direct use with API
2020-12-16 10:49:01 +01:00
42298d5896
tape: add magic number to identify media catalog files
2020-12-16 09:00:14 +01:00
205e187613
tape: add MediaCatalog implementation
2020-12-15 13:40:49 +01:00
a78348acbb
tape: rename DriveLabel to MediaLabel
2020-12-14 17:37:16 +01:00
410611b4f2
tape: improve file format docu
2020-12-14 17:29:57 +01:00
af07ec8f29
tape: minor code cleanup
2020-12-14 16:56:26 +01:00
3f803af00b
tape: scan - print more debug info
2020-12-14 13:16:18 +01:00
ac461bd651
tape: implement scan command (useful for debug)
2020-12-14 12:55:49 +01:00
ce955e1635
tape: implement eod cli command (debug tool)
2020-12-14 09:56:59 +01:00
e20d008c6a
tape: rename cli 'media media-destroy' toö 'media destroy'
2020-12-14 09:30:32 +01:00
fb657d8ee5
tape: implement destroy_media
2020-12-14 08:58:40 +01:00
fba0b77469
tape: add media api
2020-12-14 07:55:57 +01:00
b5c1296eaa
tape: make changer get_status async
2020-12-14 07:14:24 +01:00
065df12872
tape: split api type definitions for changers into extra file
2020-12-13 09:31:02 +01:00
7e1d4712b8
tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA
2020-12-13 09:22:08 +01:00
49c965a497
tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA
2020-12-13 09:18:16 +01:00
6fe9aedd0b
tape: correctly call Async handler in proxmox-tape
2020-12-12 09:58:47 +01:00
42cb9bd6a5
tape: avoid executor blocking in changer api
2020-12-12 09:45:08 +01:00
66dbe5639e
tape: avoid executor blocking in drive API
...
By using tokio::task::spawn_blocking().
2020-12-12 09:20:04 +01:00
663ef85992
tape: use WorkerTask for erase and rewind
2020-12-11 11:19:33 +01:00
e92c75815b
tape: split inventory api
...
inventory: sync, list labels with uuids,
update_inventory: WorkerTask, updates database
2020-12-11 10:42:29 +01:00
6dbad5b4b5
tape: run label commands as WorkerTask (threads)
2020-12-11 09:10:22 +01:00
bff7e3f3e4
tape: implement barcode-label-mdedia
2020-12-11 07:50:19 +01:00
83abc7497d
tape: implement inventory command
2020-12-11 07:39:28 +01:00
be1a8c94ae
fix build: add missing file
2020-12-10 13:40:20 +01:00
4606f34353
tape: implement read-label command
2020-12-10 13:20:39 +01:00
7bb720cb4d
tape: implement label command
2020-12-10 12:30:27 +01:00
c4d8542ec1
tape: add media pool handling
2020-12-10 11:41:35 +01:00
9700d5374a
tape: add media pool cli
2020-12-10 11:13:12 +01:00
05e90d6463
tape: add media pool config api
2020-12-10 10:52:27 +01:00
55118ca18e
tape: correctly sort drive api subdir
2020-12-10 10:09:12 +01:00
f70d8091d3
tape: implement option changer-drive-id
2020-12-10 09:09:06 +01:00
a3c709ef21
tape: cli cleanup - avoid api redefinition
2020-12-10 08:35:11 +01:00
4917f1e2d4
tape: implement delete property for drive update command
2020-12-10 08:25:46 +01:00
93829fc680
tape: cleanup load-slot api
2020-12-10 08:04:55 +01:00
5605ca5619
tape: cli cleanup - rename scana-for-* into scan
2020-12-10 07:58:45 +01:00
e49f0c03d9
tape: implement load-media command
2020-12-10 07:52:56 +01:00
0098b712a5
tape: implement eject
2020-12-09 17:50:48 +01:00
5fb694e8c0
tape: implement rewind
2020-12-09 17:43:38 +01:00
583a68a446
tape: implement erase media
2020-12-09 17:35:31 +01:00
e6604cf391
tape: add command line interface proxmox-tape
2020-12-09 13:00:20 +01:00
43cfb3c35a
tape: do not remove changer while still used
2020-12-09 12:55:54 +01:00
8a16c571d2
tape: add changer property to drive create api
2020-12-09 12:55:10 +01:00
314652a499
tape: set protected flag for configuration change api methods
2020-12-09 12:02:55 +01:00
6b68e5d597
client: move connect_to_localhost into client module
2020-12-09 11:59:50 +01:00
cafd51bf42
tape: add media state database
2020-12-09 11:21:56 +01:00
9b93c62044
remove unused descriptions from api macros
...
these are now a hard error in the api macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-09 10:55:18 +01:00
5d90860688
tape: expose basic tape/changer functionality at api2/tape/
2020-12-08 15:42:50 +01:00
5ba83ed099
tape: check digest on config update
2020-12-08 11:24:38 +01:00
50bf10ad56
tape: add changer configuration API
2020-12-08 09:04:56 +01:00
16d444c979
tape: add tape drive configuration API
2020-12-07 13:04:32 +01:00
fa9c9be737
tape: add tape device driver
2020-12-07 08:29:22 +01:00
2e7014e31d
tape: add BlockeReader/BlockedWriter streams
...
This is the basic format used to write data to tapes.
2020-12-06 12:09:55 +01:00
a84050c1f0
tape: add BlockHeader impl
2020-12-06 10:26:24 +01:00
7c9835465e
tape: add helpers to emulate tape read/write behavior
2020-12-06 09:41:16 +01:00
ec00200411
fix bug #3189 : fix change_password permission checks, run protected
2020-12-05 16:20:29 +01:00
b107fdb99a
tape: add tape changer support using 'mtx' command
2020-12-05 14:54:12 +01:00
7320e9ff4b
tape: add media invenotry
2020-12-05 12:54:15 +01:00
c4d2d54a6d
tape: define useful constants
2020-12-05 12:20:46 +01:00
1142350e8d
tape: add media pool config
2020-12-05 11:59:38 +01:00
d735b31345
tape: add tape read trait
2020-12-05 10:54:38 +01:00
e211fee562
tape: add tape write trait
2020-12-05 10:51:34 +01:00
8c15560b68
tape: add file format definitions
2020-12-05 10:45:08 +01:00
327e93711f
commit missing file: tape api type definitions
2020-12-04 16:00:52 +01:00
a076571470
tape support: add drive configuration
2020-12-04 15:42:32 +01:00
179145dc24
backup/datastore: move manifest locking to /run
...
this fixes the issue that on some filesystems, you cannot recursively
remove a directory when you hold a lock on a file inside (e.g. nfs/cifs)
it is not really backwards compatible (so during an upgrade, there
could be two daemons have the lock), but since the locking was
broken before (see previous patch) it should not really matter
(also it seems very unlikely that someone will trigger this)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 09:56:42 +01:00
6bd0a00c46
backup/datastore: really lock manifest on delete
...
'lock_manifest' returns a Result<File, Error> so we always got the result,
even when we did not get the lock, but we acted like we had.
bubble the locking error up
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 14:37:05 +01:00
f6e28f4e62
client/pull: log how many groups to pull were found
...
if no groups were found, the task log was very confusing as it
contained no real information why nothing was synced, e.g.:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
this patch simply logs how many groups were found and are about to be synced:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
found 0 groups to sync
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 07:22:50 +01:00
2260f065d4
cleanup: use extra file for StoreProgress
2020-12-01 06:34:33 +01:00
6eff8dec4f
cleanup: remove unnecessary StoreProgress clone()
2020-12-01 06:29:11 +01:00
7e25b9aaaa
verify: use same progress as pull
...
percentage of verified groups, interpolating based on snapshot count
within the group. in most cases, this will also be closer to 'real'
progress since added snapshots (those which will be verified) in active
backup groups will be roughly evenly distributed, while number of total
snapshots per group will be heavily skewed towards those groups which
have existed the longest, even though most of those old snapshots will
only be re-verified very infrequently.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:55 +01:00
f867ef9c4a
progress: add format variants
...
for iterating over a single group, or iterating just on the group level
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:12 +01:00
fc8920e35d
pull: factor out interpolated progress
...
and add group/snapshot count info.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:13:11 +01:00
7f3b0f67e7
remove BackupGroup::list_groups
...
BackupInfo::list_backup_groups is identical code-wise, and makes more
sense as entry point for listing groups.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:09:44 +01:00
844660036b
gc: don't limit index listing to same filesystem
...
WalkDir does not follow symlinks by default anyway, and this behaviour
is not documented anywhere. e.g., if a sysadmin mounts 'extra storage'
for some backup group or type (not knowing that only metadata is stored
in those directories), GC will ignore all the indices contained within
and happily garbage collect their chunks..
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:07:09 +01:00
efcac39d34
gc: remove duplicate variable
...
list_images already returns absolute paths, we don't need to prepend
anything.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:51 +01:00
cb4b721cb0
gc: log index files found outside of expected scheme
...
for safety reason, GC finds and marks all index files below the
datastore base path. as a result of regular operations, only index files
within the expected scheme of <TYPE>/<ID>/<TIMESTAMP> should exist.
add a small check + warning if the index list contains index files out
side of this expected scheme, so that an admin with shell access can
investigate.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:17 +01:00
7956877f14
gc: shorten progress messages
...
we have messages starting the phases anyway, and limit the number of
progress updates so that context remains available at all times.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:04:13 +01:00
43e60ceb41
file logger: remove test.log after test as well
...
and a doc formatting fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:13:21 +01:00
b760d8a23f
derive PartialEq for Userid
...
the manual implementation is equivalent
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:10:17 +01:00
2c1592263d
tiny clippy hint
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:03:43 +01:00
616533823c
don't enforce Vec and String in tools::join
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:56:59 +01:00
913dddea85
minor cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:56:21 +01:00
3530430365
tools avoid unnecessary copying of parameters/properties
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:53:49 +01:00
a4ba60be8f
minor cleanups
...
whitespace, formatting and superfluous lifetime annotations
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:47:31 +01:00
99e98f605c
network helpers: fix fd leak in get_network_interfaces
...
This one always leaked.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
935ee97b17
use fd_change_cloexec helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
6b9bfd7fe9
minor cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
Dietmar Maurer
Fabian Grünbichler
Wolfgang Bumiller
Dominik Csapak