proxmox-backup

Author	SHA1	Message	Date
Dominik Csapak	f24cbee77d	server/email_notifications: do not double html escape the default escape handler is handlebars::html_escape, but this are plain text emails and we manually escape them for the html part, so set the default escape handler to 'no_escape' this avoids double html escape for the characters: '&"<>' in emails Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-19 07:02:49 +01:00
Dominik Csapak	25b4d52dce	server/email_notifications: do not panic on template registration instead print an error and continue, the rendering functions will error out if one of the templates could not be registered if we `.unwrap()` here, it can lead to problems if the templates are not correct, i.e. we could panic while holding a lock, if something holds a mutex while this is called for the first time add a test to catch registration issues during package build Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-19 07:02:17 +01:00
Dominik Csapak	2729d134bd	tools/systemd/time: implement some Traits for TimeSpan namely * From<Duration> (to convert easily from duration to timespan) * Display (for better formatting) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-19 07:00:55 +01:00
Dietmar Maurer	32b75d36a8	tape: backup media catalogs	2021-03-19 06:58:46 +01:00
Dietmar Maurer	237314ad0d	tape: improve catalog consistency checks Try to check if we read the correct catalog by verifying uuid, media_set_uuid and seq_nr. Note: this changes the catalog format again.	2021-03-18 08:43:55 +01:00
Dominik Csapak	caf76ec592	tools/subscription: ignore ENOENT for apt auth config removal deleting a nonexistant file is hardly an error worth mentioning Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-17 20:12:58 +01:00
Dietmar Maurer	6ee3035523	tape: define magic number for catalog archives	2021-03-17 13:35:23 +01:00
Dietmar Maurer	b627ebbf40	tape: improve catalog parser	2021-03-17 11:29:23 +01:00
Dietmar Maurer	ef4bdf6b8b	tape: proxmox-tape media content - add 'store' attribute	2021-03-17 11:17:54 +01:00
Dietmar Maurer	54722acada	tape: store datastore name in tape archives and media catalog So that we can store multiple datastores on a single media set. Deduplication is now per datastore (not per media set).	2021-03-17 11:08:51 +01:00
Dietmar Maurer	0e2bf3aa1d	SnapshotReader: add self.datastore_name() helper	2021-03-17 10:16:34 +01:00
Dietmar Maurer	365126efa9	tape: PoolWriter - remove unnecessary move_to_eom	2021-03-17 10:16:34 +01:00
Dominik Csapak	7914e62b10	tools/zip: only add zip64 field when necessary if neither offset nor size exceeds 32bit, do not add the zip64 extension field Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-16 09:13:39 +01:00
Dominik Csapak	e36135031d	ui: tape/Restore: let the user choose an owner so that the tape backup can be restored as any user, given the current logged in user has the correct permission. Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-16 08:55:42 +01:00
Dominik Csapak	5a5ee0326e	proxmox-tape: add missing notify-user to 'proxmox-tape restore' Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-16 08:54:38 +01:00
Dietmar Maurer	776dabfb2e	tape: use MB/s for backup speed (to match drive speed specification)	2021-03-16 08:51:49 +01:00
Dietmar Maurer	5c4755ad08	tape: speedup backup by doing read/write in parallel	2021-03-16 08:51:49 +01:00
Dominik Csapak	7c1666289d	tools/zip: add missing start_disk field for zip64 extension it is not optional, even though we give the size explicitely Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-15 12:36:40 +01:00
Thomas Lamprecht	cded320e92	backup info: run rustfmt Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-03-14 19:18:35 +01:00
Wolfgang Bumiller	b31cdec225	update to pxar 0.10 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-03-12 10:48:09 +01:00
Wolfgang Bumiller	591b120d35	fix feature flag logic in pxar create Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-03-12 10:17:51 +01:00
Dietmar Maurer	e8913fea12	tape: write_chunk_archive - do not consume partially written chunk at EOT So that it is re-written to the next tape.	2021-03-12 07:14:50 +01:00
Dominik Csapak	ef0ea4ba05	server/worker_task: improve endtime for unknown tasks instead of always using the starttime, use the last timestamp from the log this way, one can see when the task was aborted without having to read the log Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-11 09:56:12 +01:00
Dominik Csapak	2892624783	tape/send_load_media_email: move to server/email_notifications and reuse 'send_job_status_mail' there so that we get consistent formatted mails from pbs (e.g. html part and author) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-11 09:56:12 +01:00
Dietmar Maurer	2c10410b0d	tape: improve backup task log	2021-03-11 08:43:13 +01:00
Thomas Lamprecht	d1d74c4367	typo fixes all over the place found and semi-manually replaced by using: codespell -L mut -L crate -i 3 -w Mostly in comments, but also email notification and two occurrences of misspelled 'reserved' struct member, which where not used and cargo build did not complain about the change, soo ... Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-03-10 16:39:57 +01:00
Dietmar Maurer	5ef4c7bcd3	tape: fix scsi volume_statistics and cartridge_memory for quantum drives	2021-03-10 14:13:48 +01:00
Dietmar Maurer	6396bace3d	tape: improve backup task log (show percentage)	2021-03-10 10:59:13 +01:00
Dietmar Maurer	713a128adf	tape: improve backup task log format	2021-03-10 09:54:51 +01:00
Dietmar Maurer	affc224aca	tape: read_tape_mam - pass correct allocation len	2021-03-10 09:24:38 +01:00
Dietmar Maurer	6f82d32977	tape: cleanup - remove wrong inline comment	2021-03-10 08:11:51 +01:00
Dominik Csapak	2a06e08618	api2/tape/backup: continue on vanishing snapshots when we do a prune during a tape backup, do not cancel the tape backup, but continue with a warning the task still fails and prompts the user to check the log Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-09 10:20:54 +01:00
Dietmar Maurer	1057b1f5a5	tape: lock artificial "__UNASSIGNED__" pool to avoid races	2021-03-09 10:00:26 +01:00
Dietmar Maurer	af76234112	tape: improve MediaPool allocation by sorting tapes by ctime and label_text	2021-03-09 08:33:21 +01:00
Dominik Csapak	1d7fcbece8	tape/linux_tape: remove debug prints Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-08 14:29:19 +01:00
Dominik Csapak	8703a68a31	tape/backup: add success/error notifications Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-08 14:29:06 +01:00
Dietmar Maurer	9ce2481a69	tape: clean - fix if cleaning tape is already loaded	2021-03-08 09:08:55 +01:00
Dietmar Maurer	d95c74c6e7	tape: test tape alert flags after drive clean	2021-03-08 09:08:55 +01:00
Dietmar Maurer	5ca5f8daf3	pmt: add tape-alert-flags command	2021-03-08 07:37:05 +01:00
Dominik Csapak	c9793d47f9	api2/tape: add notify_user to backup(-jobs) and restore api calls so that a user can be given that will be notified for manual intervention (e.g. inserting a tape) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-05 17:20:37 +01:00
Dominik Csapak	9152a0077f	config/tape_job: add notify-user config to tape jobs Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>	2021-03-05 12:51:25 +01:00
Dietmar Maurer	0b90c67fb4	tape: avoid Updatable for now	2021-03-05 12:51:25 +01:00
Dietmar Maurer	b4975d3102	tape: finish api permission checks	2021-03-05 11:49:07 +01:00
Dietmar Maurer	ee33795b72	acl: use /tape/device instead of /tape/drive and /tape/changer	2021-03-05 10:06:19 +01:00
Dietmar Maurer	90e16be3ae	tape: improve PoolWriter logging Log reason why we allocate a new media set.	2021-03-05 09:59:38 +01:00
Dietmar Maurer	cf90a369e2	cleanup: rename token_user into auth_id_filter	2021-03-05 08:36:18 +01:00
Dietmar Maurer	cd5d6103ea	tape: fix media_expire_time	2021-03-04 16:58:24 +01:00
Dietmar Maurer	1bed3aedc8	tape: fix wrong media set expire time	2021-03-04 12:38:28 +01:00
Dietmar Maurer	ab77d660cc	tape: improve media status in list_media	2021-03-04 10:17:10 +01:00
Dominik Csapak	78593b5b5c	tape/drive: improve error and email handling for requesting a tape load Try once first to load the correct tape before sending an email to insert the correct one. This way, the admin does not get a mail if the correct tape is already inserted. Also include the error we got that prompted the email to insert the tape. This means that if the admin gets prompted to insert e.g. "FOO" but inserts "BAR", he'll get an email that the wrong tape is inserted. Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-03 16:09:39 +01:00
Dominik Csapak	831c43c91b	config/tfa: webauthn: disallow registering a token twice by adding the existing credential id to the 'excludeCredentials' list this prevents the browser from registering a token twice, which lets authentication fail on some browser/token combinations (e.g. onlykey/solokey+chromium) while is seems this is currently a bug in chromium, in a future spec update the underlying behaviour should be better defined, making this an authenticator bug also explicitly catch registering errors and show appropriate error messages 0: https://bugs.chromium.org/p/chromium/issues/detail?id=1087642 Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-03 14:05:05 +01:00
Dominik Csapak	7f37cacfac	config/tfa: set UserVerificationPolicy to Discouraged the current default is 'Preferred', which is not really useful, as the (web) client can simply change this to discouraged, since the webauthn_rs crate does not verify the 'user_verified' bit of the response in that case setting this to 'Required' is not really useful either at the moment, since a user can have a mix of different authenticators that may or may not support user verification there is ongoing discussion in the crate how to handle that[0] we could probably expose this setting(discouraged/required) to the user/admin and save it to the credential and allow only registering credentials of the same type or filter them out on login (i.e. if there is an authenticator that can handle userVerification, require it) in any case, the current default is not helpful for security, but makes loggin in harder, since the key will by default want to verify the user 0: https://github.com/kanidm/webauthn-rs/pull/49 Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-03 14:05:05 +01:00
Thomas Lamprecht	3bb7e62e88	zpool parsers: small code cleanup Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-03-03 14:05:05 +01:00
Thomas Lamprecht	3b060167f6	zpool parsers: fixup test value Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-03-03 14:05:05 +01:00
Dietmar Maurer	8a76e71129	tape: impl access permission for media content list	2021-03-03 13:16:15 +01:00
Dietmar Maurer	396fd747a6	tape: impl access permissions for tape jobs	2021-03-03 12:48:06 +01:00
Dietmar Maurer	16bd08b297	tape: add acl path /tape/job/{id}	2021-03-03 12:48:06 +01:00
Dietmar Maurer	ccdf327ac8	tape: impl access permissions for encryption key config	2021-03-03 12:21:55 +01:00
Dietmar Maurer	8cd63df0dc	tape: impl api permissions for drive/changer/pool configuration	2021-03-03 12:10:00 +01:00
Dietmar Maurer	b90cb34fd6	complete_acl_path: add more paths	2021-03-03 11:09:54 +01:00
Dietmar Maurer	d6c1e12c06	add roles to implement tape backup permissions	2021-03-03 11:09:02 +01:00
Dietmar Maurer	40bf636b47	ui: enable experimental tape UI if tape.cfg exists	2021-03-03 09:02:02 +01:00
Dominik Csapak	347cde827b	dashboard: fix datastore full estimate for not changing usage If the usage of a datastore did not change, we did not return an estimate. The ui interpreted this as 'not enough data', but it should actually be 'never'. Fixing this by always setting the estimate first to 0 and overwriting if we successfully calculated one, and checking for 'undefined' in the ui. Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-02 17:51:13 +01:00
Dominik Csapak	8bf5769382	api2/tape/drive: add state to DriveListEntry Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-03-02 12:51:19 +01:00
Dietmar Maurer	db04d10d14	tape: add comment to media pool config	2021-03-02 11:42:58 +01:00
Dietmar Maurer	1d14c31658	Revert "tape: also abort backup/restore on server shutdown" This reverts commit `9bd81bb384`. Turns out this is not really a good idea.	2021-03-02 08:00:10 +01:00
Dietmar Maurer	9bd81bb384	tape: also abort backup/restore on server shutdown	2021-03-02 07:19:31 +01:00
Thomas Lamprecht	d64226efee	disks/zfs: fix test input and enforce check for config key we test for the config key in the API so it makes sense to have as test here too. Actually it would be better if we'd have a expect Value defined here and enforce that it matches, but better than nothing. Fix the input for test 1, where tabs got replaced by spaces, as else it fails Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-03-01 17:36:16 +01:00
Thomas Lamprecht	2440eaa2df	disks/zfs: pull common test code out in helper Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-03-01 14:48:35 +01:00
Dietmar Maurer	1552d9699c	tape: add 'Restore Key' button	2021-03-01 11:11:38 +01:00
Dietmar Maurer	16f9ea6708	tape: restore - do not check if pool is configured	2021-03-01 09:43:23 +01:00
Dietmar Maurer	d984a9acf0	tape: add Inventory.media_list() helper	2021-03-01 09:30:45 +01:00
Dietmar Maurer	955f4aefcd	tape: show media inventory/content, even if there is no pool configuration	2021-03-01 09:21:11 +01:00
Dietmar Maurer	3832911d50	docs: remove backup protocol api dumps We now have the api-viewer for that.	2021-02-26 19:00:08 +01:00
Dietmar Maurer	08ec39be0c	tape: add api to set media status	2021-02-26 10:44:07 +01:00
Dietmar Maurer	25350f3370	tape: set media status if we detect damaged medium at start of backup	2021-02-26 09:00:50 +01:00
Dietmar Maurer	0023cfa385	tape: fix 'proxmox-tape backup' cli command	2021-02-26 08:51:24 +01:00
Dietmar Maurer	5830e5620d	tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig	2021-02-25 09:44:31 +01:00
Wolfgang Bumiller	3554fe6480	update to schema changes in proxmox Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-24 14:50:08 +01:00
Dietmar Maurer	0dadf66dc7	tape: implement backup using latest-only flag	2021-02-24 13:51:53 +01:00
Thomas Lamprecht	a941bbd0c9	client: raise HTTP_TIMEOUT to 120s As 20s is really not that high, especially for loaded setups one is connected to through a spotty network (looking at you ÖBB railnet) and gets latency spikes of 5 - 10s for some minutes at a time.. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-02-24 11:57:21 +01:00
Dietmar Maurer	21e3ed3449	tape: add 'latest-only' property to backup job config	2021-02-24 11:19:12 +01:00
Dominik Csapak	52d8db7925	api2/config/tape_backup_job: fix duplicate id parameter since the PUT api call is using the 'Updater', the 'id' parameter is already encoded in there, tripping up the api verify tests with 'Duplicate keys found in AllOf schema: id' "fixing" it by removing the explicit id from the api call and taking it from the Updater (and failing if it does not exists there; even though that should never happen) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-23 17:23:47 +01:00
Dominik Csapak	875d375d7a	api2/types: ArchiveEntry: fix inverse 'leaf' value a 'leaf' node is every file except directories, so we have to reverse the logtic here this fixes the pxar.didx browser in the web ui Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-23 17:23:47 +01:00
Dominik Csapak	cba167b874	pxar/extract: if possible create files sparesly instead of filling them with zeroes this fixes an issue where we could not restore a container with large sparse files in the backup (e.g. a 10GiB sparse file in a container with a 8GiB disk) if the last operation of the copy was a seek, we need to truncate the file to the correct size (seek beyond filesize does not change it) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-23 14:44:59 +01:00
Dominik Csapak	4961404c7c	api2/config/tape_backup_job: enable update api call Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-23 13:17:49 +01:00
Dietmar Maurer	3fbf2311e4	tape: improve backup task logging	2021-02-23 12:58:44 +01:00
Dietmar Maurer	043018cfbe	doc: fix wrong api method description	2021-02-22 12:10:34 +01:00
Dietmar Maurer	2037d9af03	api-viewer: show permissions	2021-02-22 12:01:24 +01:00
Dietmar Maurer	7f07991035	docs: improve MEDIA_SET_NAMING_TEMPLATE_SCHEMA description	2021-02-22 09:50:16 +01:00
Dietmar Maurer	18ce01caff	tape: use correct schema for changer path (SCSI_CHANGER_PATH_SCHEMA)	2021-02-22 09:43:13 +01:00
Dietmar Maurer	5bc8e80a99	cleanup: cleanup use clause (avoid *)	2021-02-22 09:33:29 +01:00
Dietmar Maurer	6252df4c18	docs: fix EXPORT_SLOT_LIST_SCHEMA description	2021-02-22 09:05:07 +01:00
Dietmar Maurer	451856d21d	api-viewer: fix path for upgraded HTTP/2 proptocls	2021-02-22 08:38:27 +01:00
Dietmar Maurer	aa30663ca5	docgen: improve regex format	2021-02-21 16:14:11 +01:00
Dietmar Maurer	8616a4afe5	docgen: dump string format (property strings)	2021-02-21 15:54:14 +01:00
Dietmar Maurer	bc2358319b	docgen: improve api schema dump	2021-02-21 12:00:06 +01:00
Dietmar Maurer	0bf4b81370	docs: api-viewer - add backup and restore APIs	2021-02-21 10:38:33 +01:00
Dietmar Maurer	fee0fe5422	docgen: implement api schema generation (for api-viewer) Just a start - not complete jet.	2021-02-21 09:06:58 +01:00
Dietmar Maurer	86d9f4e733	tape: extend MediaChange trait to return MtxStatus	2021-02-20 10:23:16 +01:00
Dietmar Maurer	3f16f1b006	tape: update changer status inside ScsiMediaChange implementation	2021-02-20 09:56:27 +01:00
Dietmar Maurer	cbd9899389	tape: update changer status cache after load/unload	2021-02-20 09:06:17 +01:00
Dietmar Maurer	cd44fb8d84	tape: create tmp dirs early at server startup	2021-02-19 18:10:02 +01:00
Dietmar Maurer	4188fd59a0	tape: cache changer state	2021-02-19 16:48:19 +01:00
Dominik Csapak	926d05ef0b	api2/tape/{backup, restore}, proxmox-tape: set device state set the drive device state everywhere we lock it, so that we know what it currently does Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:16:30 +01:00
Dominik Csapak	8be48ddfc7	api2/tape/changer: add drive state to changer status output if we can find the drive in the config and it has a state Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:15:15 +01:00
Dominik Csapak	41e66bfaf6	api2/tape/drive: wrap some api calls in run_drive_blocking_task those calls could also block, so we have to run them in a blocking tokio task, as to not block the current thread nice side effect is that we now also update the state for that drive in those instances Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:14:08 +01:00
Dominik Csapak	47a7241410	api2/tape/drive: use run_drive_blocking_task where possible Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:13:36 +01:00
Dominik Csapak	54c77b3d62	api2/tape/drive: add wrapper for tokio::task::spawn_blocking similar to the worker wrapper, lock, write status, run code, unset status Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:13:21 +01:00
Dominik Csapak	a1c5575308	api2/tape/drive: use 'run_drive_worker' where possible Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:11:04 +01:00
Dominik Csapak	a44c934b5d	api2/tape/drive: add run_drive_worker wrapper a wrapper for locking, starting the worker and writing the state Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:08:27 +01:00
Dominik Csapak	546d2653ee	tape/drive: add get/set status functions simply writes into/reads from a file in /run, we will use this for writing the upid (or potential other states) per drive Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:05:09 +01:00
Dominik Csapak	33c06b3388	tape/drive: add test_device_path_lock this makes it possible to detect if the drive was locked in a non-blocking manner Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-19 10:03:30 +01:00
Dietmar Maurer	1917ea3ce1	cleanup: use serde(flatten) for VerificationJobStatus, improve code reuse	2021-02-19 09:50:25 +01:00
Dietmar Maurer	70842b9ef2	cleanup: use serde(flatten) for SyncJobStatus, improve code reuse	2021-02-19 09:36:39 +01:00
Dietmar Maurer	7690a8e7bd	api: list tape backup jobs with scheduling status	2021-02-19 09:02:13 +01:00
Dietmar Maurer	1860208560	add helper to compute job scheduling state	2021-02-19 08:58:50 +01:00
Dietmar Maurer	1689296d46	api: add new type JobScheduleStatus	2021-02-19 08:23:41 +01:00
Dietmar Maurer	9bbd83b1f2	tape: media_list API - allow to update online status for a single changer	2021-02-18 10:59:33 +01:00
Dietmar Maurer	d0647e5a02	tape: use worker tasks for media load/unload	2021-02-18 09:04:51 +01:00
Stefan Reiter	f1a83e9759	client: extract common functions to proxmox_client_tools module ...including common schemata, connect(), extract_*() and completion functions. For later use with proxmox-file-restore binary. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 10:13:10 +01:00
Stefan Reiter	6afb60abf5	asyncify pxar create_archive ...to take advantage of the aio::Encoder from the pxar create. Rather straightforward conversion, but does require getting rid of references in the Archiver struct, and thus has to be given the Mutex for the catalog directly. The callback is boxed. archive_dir_contents can call itself recursively, and thus needs to return a boxed future. Users are adjusted, namely PxarBackupStream is converted to use an Abortable future instead of a thread so it supports async in its handler function, and the pxar bin create_archive is converted to an async API function. One test case is made to just use 'block_on'. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-17 09:24:20 +01:00
Dominik Csapak	a42212fc1e	pxar/extract: add extract_sub_dir to extract some subdirectory of a pxar into a given target this will be used in the client Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 08:32:41 +01:00
Dominik Csapak	2e21948156	api2/admin/datastore: refactor create_zip into pxar/extract we will reuse that code in the client, so we need to move it to where we can access it from the client Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> [clippy fixes] Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 08:32:41 +01:00
Dominik Csapak	5279ee745f	api2/admin/datastore: accept "/" as path for root makes more sense than sending "root'" Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 08:32:41 +01:00
Dominik Csapak	227501c063	api2/admin/datastore: refactor list_dir_content in catalog_reader we will reuse that later in the client, so we need it somewhere we can use from there Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> [add strongly typed ArchiveEntry and put api code into helpers.rs] Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 08:32:41 +01:00
Stefan Reiter	89d25b1931	client: add VsockClient to connect to virtio-vsock VMs Currently useful only for single file restore, but kept generic enough to use any compatible API endpoint over a virtio-vsock[0,1] interface. VsockClient is adapted and slimmed down from HttpClient. A tower-compatible VsockConnector is implemented, using a wrapped UnixStream as transfer. The UnixStream has to be wrapped in a custom struct to implement 'Connection', Async{Read,Write} are simply forwarded directly to the underlying stream. [0] https://www.man7.org/linux/man-pages/man7/vsock.7.html [1] https://wiki.qemu.org/Features/VirtioVsock Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 08:15:43 +01:00
Stefan Reiter	b57c0dbe30	rest: implement tower service for UnixStream This allows anything that can be represented as a UnixStream to be used as transport for an API server (e.g. virtio sockets). A tower service expects an IP address as it's peer, which we can't reliably provide for unix socket based transports, so just fake one. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2021-02-17 07:50:35 +01:00
Dietmar Maurer	8b910bb6bc	tape: add regression test for compute_media_state	2021-02-17 06:57:29 +01:00
Dietmar Maurer	3cdd1a3424	tape: erase_media - automatically load media (with tape libraries).	2021-02-17 06:55:53 +01:00
Dominik Csapak	be61c56c21	api2/tape/drive: add optional label-text to erase if given, erases the tape only iff the inserted tape contains that label used to safeguard tape erasing from ui for standalone drives Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-17 06:55:53 +01:00
Dietmar Maurer	dbaef7d04d	tape: add regression test for alloc_writable_media	2021-02-17 06:55:49 +01:00
Dietmar Maurer	159100b944	tape: ui - avoid changer status query in backup content panel	2021-02-15 12:17:36 +01:00
Dietmar Maurer	41a8db3576	tape: implement API to run tape backup jobs	2021-02-15 11:09:30 +01:00
Dietmar Maurer	fe291ab794	cleanup: tape_backup_job.rs - do not use #[allow(non_camel_case_types)]	2021-02-15 10:41:04 +01:00
Dietmar Maurer	adb65b9889	tape: fix proxmox-tape pool config command	2021-02-15 08:58:05 +01:00
Dietmar Maurer	8513626b9f	proxmox-backup-proxy: schedule tape backup jobs	2021-02-15 08:30:42 +01:00
Dietmar Maurer	7ca0ba4515	tape: add tape-job.cfg manual page	2021-02-15 07:05:39 +01:00
Dietmar Maurer	42200c405a	fix api sort order	2021-02-15 06:35:45 +01:00
Dietmar Maurer	be327dbccd	tape: add tape backup job configuration	2021-02-14 10:35:51 +01:00
Dietmar Maurer	7b1bf4c098	tape: erase_media - automatically erase catalog and inventory	2021-02-12 11:30:34 +01:00
Dominik Csapak	7c9fb570cc	api2/types/tape/media_location: fix deserialization of MediaLocation::Vault Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-11 14:23:54 +01:00
Dominik Csapak	56d22c66c0	api2/tape/media: add 'move' api call Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-11 14:23:27 +01:00
Dominik Csapak	85cdc4f371	proxmox-tape: change lookup_drive_name to extract_drive_name in most uses, we want to remove the drive from the param afterwards where we don't, we already overwrite it with the result of this function this fixes some commands (like 'proxmox-tape read-label --drive foo') that failed with: parameter 'drive': duplicate parameter. Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-11 14:22:50 +01:00
Dietmar Maurer	4a874665eb	proxmox-backup-manger verify-job CLI Add missing command line interface to manage verification jobs.	2021-02-11 14:06:44 +01:00
Dietmar Maurer	6f6b69946e	docs: use type_text for config::acl::Role The list of roles is simply too long, so we use type_text to generate reasonable docs.	2021-02-11 13:47:01 +01:00
Dietmar Maurer	5b7f44555e	docs: add manual page for verification.cfg	2021-02-11 13:40:11 +01:00
Dietmar Maurer	2ca396c015	docs: add manual page for acl.cfg	2021-02-11 13:13:34 +01:00
Dietmar Maurer	8f02db04f9	docs: use new type_text option for calendar events	2021-02-11 11:51:22 +01:00
Dietmar Maurer	9f35e44681	fix proxmox_backup_manager remote list: s/userid/auth-id/	2021-02-11 11:31:20 +01:00
Dietmar Maurer	6279b8f5a5	docs: add manukal page for remote.cfg	2021-02-11 11:25:15 +01:00
Dietmar Maurer	3084232cb5	depend on proxmox 0.10.5	2021-02-11 10:49:16 +01:00
Dietmar Maurer	2322a980d0	docs: add utility binary to generate docs	2021-02-10 11:05:02 +01:00
Wolfgang Bumiller	7f9d8438ab	tfa: derive WebauthnConfigUpdater via api macro Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-10 10:47:47 +01:00
Wolfgang Bumiller	befd95a90a	pxar: improve error messages Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-09 13:09:00 +01:00
Wolfgang Bumiller	ab6cd4229b	pxar: skip write_attr_fd if they are equal Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-09 13:09:00 +01:00
Wolfgang Bumiller	9213744ecb	fix #3026 : pxar: metadata: apply flags _after_ updating mtime otherwise we'll fail to update the mtime of immutable files Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-09 13:08:49 +01:00
Dominik Csapak	afcf8b3ed6	api2/tape/drive: add missing locks for some drive actions else a backup/restore can easily be disrupted Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-09 08:46:03 +01:00
Fabian Grünbichler	15d2c7786e	client: move test output files to target/testout Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-08 09:52:16 +01:00
Dietmar Maurer	fbf8779388	doc: improve pmt/pmtx manual pages	2021-02-06 16:50:56 +01:00
Dietmar Maurer	ced7838de4	tape: fix path for test files	2021-02-06 09:32:50 +01:00
Fabian Grünbichler	2f26b8668a	client: track key source, print when used to avoid confusing messages about using encryption keys when restoring plaintext backups, or about loading master keys when they are not actually used for the current operation. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 09:07:31 +01:00
Fabian Grünbichler	9432838914	client: refactor crypto_parameter handling pull out the crypt-mode to logically group arms and make the whole mess a bit more "human-parsable". Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 09:06:28 +01:00
Fabian Grünbichler	1a89a7794e	client: extend tests for master key handling Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 09:05:01 +01:00
Fabian Grünbichler	c0a87c12fb	client: allow passing specific master key it's needed for PVE's LXC integration, and might be interesting for other more special usage scenarios as well. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 09:04:42 +01:00
Fabian Grünbichler	c6a7ea0a2f	client: refactor keyfile_parameters no semantic changes intended Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 09:03:03 +01:00
Fabian Grünbichler	5bb057e5a2	client: add test for keyfile_parameters this will get more complex soon, so add test to document current behaviour. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 08:58:15 +01:00
Fabian Grünbichler	2924b37d6d	key: rustfmt module Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 08:56:19 +01:00
Fabian Grünbichler	42c0f784e2	key: add show-master-pubkey command and print public key when generating/importing.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 08:55:21 +01:00
Fabian Grünbichler	05f17d1ec4	key: make 'default' master key explicit Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-06 08:52:11 +01:00
Dominik Csapak	f197c286d5	tape/inventory: fix missing parameter in error message Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-05 16:12:18 +01:00
Dietmar Maurer	4c209d6b10	install pmt binary	2021-02-05 12:42:20 +01:00
Dietmar Maurer	25aa55b5f5	tape: correctly lock the tape device	2021-02-05 10:50:21 +01:00
Dietmar Maurer	cdf39e62b3	tape: MediaPool - replace use_offline_media with changer_name This way, we can improve location_is_available, because we only consider media from that changer as available.	2021-02-04 10:15:18 +01:00
Dietmar Maurer	b81e37f6ab	tape: improve code reuse	2021-02-04 09:39:16 +01:00
Dietmar Maurer	ddebbb52fd	tape: fix tests for BlockedReader	2021-02-04 08:54:54 +01:00
Dietmar Maurer	983e929e25	tape: add multi volume reader/writer implementations We currently do not use it. Added anaways, to show the possibility.	2021-02-04 08:36:35 +01:00
Dietmar Maurer	f47e035721	tape: cleanup - move tape file readers/writers into src/tape/file_formats folder	2021-02-04 07:59:37 +01:00
Dietmar Maurer	a80d72f999	tape: allow to abort restore tasks	2021-02-04 07:05:43 +01:00
Dietmar Maurer	8de9a9917f	cleanup: use task_log macro	2021-02-04 06:55:18 +01:00
Wolfgang Bumiller	fa016c1697	HttpsConnector: use hostname instead of URL again fixes connecting to hosts with valid certificates without a pinned fingerprint this was accidentally changed in the tokio-1.0 updates apparently Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com> Fixes: `0f860f712f` ("tokio 1.0: update to new tokio-openssl interface")	2021-02-03 15:18:18 +01:00
Dietmar Maurer	7d2c156eb1	tape: BlockedReader - always consume EOF	2021-02-03 13:25:59 +01:00
Dietmar Maurer	b22e8c3632	tape: add media pool regression tests	2021-02-03 10:23:04 +01:00
Dietmar Maurer	7929292618	tape: add regresion test for media state	2021-02-03 09:34:31 +01:00
Dietmar Maurer	0d4e4cae7f	tape: improve pmt command line completion	2021-02-03 08:54:12 +01:00
Dominik Csapak	50a4797fb1	api2/types/tape/media: add media_set_ctime to MediaContentEntry to be able to better sort in the ui Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-02 14:45:54 +01:00
Dietmar Maurer	cc2a0b12f8	test: define tape tests as submodule	2021-02-02 14:38:15 +01:00
Dietmar Maurer	988e8de122	tape: set correct ownership on lock file	2021-02-02 14:18:57 +01:00
Dietmar Maurer	2f8809c6bc	test: src/tape/inventory.rs - avoid chown when running tests	2021-02-02 13:43:16 +01:00
Dietmar Maurer	8937c65951	tape: add pmt stoptions/stsethoptions/stclearoptions	2021-02-02 08:58:02 +01:00
Dietmar Maurer	6c6ad82d90	tape: add pmt setblk	2021-02-02 07:19:54 +01:00
Dietmar Maurer	d0f11b66f7	thape: add read_tapedev_options, display driver options with status command	2021-02-02 06:40:40 +01:00
Dietmar Maurer	ca953d831f	cleanup: remove MT_ST_ prefix from SetDrvBufferOptions	2021-02-01 17:54:53 +01:00
Fabian Grünbichler	01c023d50f	paperkey: rustfmt Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-01 17:05:40 +01:00
Fabian Grünbichler	c2113a405e	paperkey: simplify block generation the chunk-iterator already does exactly what we want here.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-01 17:05:32 +01:00
Fabian Grünbichler	5dae81d199	paperkey: allow RSA keys without passphrase some users might want to store the plain version of their master key for long-term storage and rely on physical security instead of a passphrase to protect the paper key. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-02-01 17:05:22 +01:00
Wolfgang Bumiller	6248e51797	change half-ticket time range from -120..240 to -60..600 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-02-01 15:13:11 +01:00
Dietmar Maurer	19e4a36c70	tape: do not use drive.open() within pmt Do not fail if no media is loaded. Inportant for load command.	2021-02-01 12:39:50 +01:00
Dietmar Maurer	90769e5694	tape: add pmt lock/unlock	2021-02-01 12:18:55 +01:00
Dietmar Maurer	b8cbe5d65b	tape: fix tape alert flag decoding	2021-02-01 12:18:55 +01:00
Dietmar Maurer	d690d14568	tape: add pmt bsr/fsr	2021-02-01 10:39:04 +01:00
Dietmar Maurer	85ef624440	tape: add pmt asf	2021-02-01 10:32:21 +01:00
Dietmar Maurer	e995996290	tape: pmt - fix count parameter schema	2021-02-01 10:21:25 +01:00
Dietmar Maurer	8e6ad4301d	tape: add pmt fsfm/bsfm, pass count as arg_param	2021-02-01 10:18:18 +01:00
Dominik Csapak	1399c592d1	garbage_collection: only ignore 'missing chunk' errors with the fix for #2909 (improving handling missing chunks), we changed from bailing to warning during a garbage collection when updating the atime of a chunk. but, updating the atime can not only fail when the chunk is missing, but also on other occasions, e.g. no permissions or more importantly, no space left on the device. in that case, the atime of a valid and used chunk cannot be updated, and the second sweep of the gc will remove that chunk. [0] is a real world example of that happening. instead, only warn on really missin chunks, and bail on all other errors. 0: https://forum.proxmox.com/threads/pbs-server-full-two-days-later-almost-empty.83274/ Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-02-01 09:18:59 +01:00
Dietmar Maurer	9883b54cba	tape: remove drive from pool config	2021-02-01 09:14:28 +01:00
Dietmar Maurer	83b8949a98	tape: add pmt weof	2021-01-31 17:33:07 +01:00
Dietmar Maurer	28f60e5291	cleanup: avoid compiler warnings	2021-01-31 17:02:55 +01:00
Dietmar Maurer	1f31d06f48	tape: add pmt bsf	2021-01-31 17:00:15 +01:00
Dietmar Maurer	2f2e83c890	tape: add pmt fsf	2021-01-31 16:54:16 +01:00
Dietmar Maurer	b22c618734	tape: add pmt erase	2021-01-31 16:34:10 +01:00
Dietmar Maurer	1e041082bb	tape: add pmt command line tool Experimental, not installed by now.	2021-01-31 16:19:53 +01:00
Dietmar Maurer	b5b99a52cd	tape: API type cleanup, use serde flatten to derive types	2021-01-30 09:36:54 +01:00
Dietmar Maurer	9586ce2f46	tape: move scan_drives API code to correct file	2021-01-30 08:03:17 +01:00
Dietmar Maurer	c297835b01	tape: proxmox-tape - use API instead of direct functions calls	2021-01-29 11:49:11 +01:00
Dietmar Maurer	e68269fcaf	tape: proxmox-tape inventory: call API	2021-01-29 11:21:57 +01:00
Dietmar Maurer	5243df4712	tape: proxmox-tape - use API instead of direct functions calls	2021-01-29 10:50:11 +01:00
Dietmar Maurer	4470eba551	cleanup: factor out common client code to view task log/result	2021-01-29 10:10:04 +01:00
Dietmar Maurer	1f2c4713ef	tape: improve backup task abort behaviour	2021-01-29 09:23:39 +01:00
Dietmar Maurer	a6c16894ff	worker_task: log something when we receive an abort request	2021-01-29 09:22:37 +01:00
Dietmar Maurer	271764deb9	tape: make it possible to abort tape backup tasks (check_abort) Also use task_log makro instead of worker.log.	2021-01-29 09:07:55 +01:00
Dietmar Maurer	52f7a73009	display_task_log: make it possible to abort tasks with CTRL-C	2021-01-29 09:06:15 +01:00
Dominik Csapak	bdb6e6b83f	api2/reader: asyncify the reader worker task this way, the code is much more readable Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-29 06:59:25 +01:00
Dietmar Maurer	41dacd5d3d	tape: use worker task for eject-media api	2021-01-28 16:49:08 +01:00
Dietmar Maurer	eb1dfb02b5	tape: proxmox-tape - use api for erase-media and rewind	2021-01-28 16:36:10 +01:00
Dietmar Maurer	bdb62b20a3	tape: media_pool config api - set protected flags where required	2021-01-28 15:42:32 +01:00
Dietmar Maurer	f2ca03d7d0	cleanup: avoid compiler warning	2021-01-28 15:32:21 +01:00
Dominik Csapak	00ac86c31b	tape/drive/linux_tape: fix and refactor usage of sg-tape-cmd when executing this code as non-root, we use sg-tape-cmd (a setuid binary) to execute various ioctls on the tape device we give the command the open tape device fd as stdin, but did not dup it, so the std::process:Stdio handle closed it on drop, which let subsequent operation on that file fail (since it was closed) fix it by dup'ing it before giving it to the command, and also refactor the calling code, so that we do not forget to do this Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:24:32 +01:00
Dominik Csapak	627d000098	tape: change changer-drive-id to changer-drivenum because it changed in the config Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:11:22 +01:00
Dominik Csapak	4be4736603	tape/changer: refactor marking of import/export slots from config we did this for 'mtx', but missed it for the sg_pt_changer code refactor it into the MtxStatus strut, and call it from both code paths Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:10:55 +01:00
Dominik Csapak	2da7aca8e8	tape/changer: add vendor/model to DriveStatus Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:10:31 +01:00
Dominik Csapak	b03ec281bf	api2/config/{drive, changer}: prevent adding same device multiple times this check is not perfect since there are often multiple device nodes per drive/changer, but from the scan api we should return always the same, so for an api user this should be enough Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:03:56 +01:00
Dominik Csapak	cef4654ff4	api2/tape/drive: change methods of some api calls from put to get makes more sense to have retrieving api calls as get instead of put Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:02:52 +01:00
Dominik Csapak	f45dceeb73	api2/tape/drive: add load_media as api call code was already there, just add it as api call Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:02:13 +01:00
Dominik Csapak	18262a88c9	api2/tape/changer: add changer filter to list_drives api call so that an api user can get the drives belonging to a changer without having to parse the config listing themselves Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 15:01:41 +01:00
Dietmar Maurer	87f4be7998	tape: use api to run proxmox-tape backup	2021-01-28 14:56:42 +01:00
Dietmar Maurer	d737adc6be	tape: rename changer_drive_id to changer_drivenum	2021-01-28 11:29:59 +01:00
Dominik Csapak	5fdaecf6f4	api2/tape/drive: reorganize drive api similar to the changers, create a listing at /tape/drive and put the specific api calls below that move the scan api call up one level remove the status info from the config listing Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 11:15:17 +01:00
Dominik Csapak	d8792b88ef	api2/types/tape/drive: add changer_drivenum so that an api user can see which drive belongs to which drivenum of a changer for ones with multiple drives Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-28 11:14:28 +01:00
Dietmar Maurer	651a61f559	pmtx: implement scan command	2021-01-27 12:40:51 +01:00
Dietmar Maurer	89ccb125d1	tape: use 36 byte Inquiry (recommended size)	2021-01-27 12:35:28 +01:00
Dietmar Maurer	887f1cb90c	cleanup: move scan changers API implementation	2021-01-27 09:58:16 +01:00
Dietmar Maurer	16b4d78400	tape: rename retry_command to execute_scsi_command, make retry a flag	2021-01-27 09:34:24 +01:00
Dietmar Maurer	ec8d9c6b80	tape: repeat changer scsi command until successful	2021-01-27 08:59:10 +01:00
Dietmar Maurer	49c2d1dcad	sgutils2: use sg_get_asc_ascq_str to produce error messages	2021-01-27 06:56:11 +01:00
Dietmar Maurer	d0f51651f9	sgutils2: add ASC codes from tandeberg docs	2021-01-26 18:54:08 +01:00
Dietmar Maurer	481ccf16a5	sgutils2: further improve error messages	2021-01-26 15:19:43 +01:00
Dietmar Maurer	a223458753	sgutils2: support RequestSense Descriptor format	2021-01-26 13:38:16 +01:00
Dominik Csapak	e1740f3f01	tape/changer/mtx: add mtx parser test Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-26 12:51:26 +01:00
Dominik Csapak	740dc9d1d4	api2/tape/changer: reorganize api add a changer listing here (copied from api2/config/changer) and put the status and transfer api calls below that puts the changer scan into the top level tape api and removes the (now redundant) info from the config api path Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-26 12:47:34 +01:00
Dominik Csapak	bbf01b644c	tape: fix typos Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-26 12:39:54 +01:00
Dietmar Maurer	c81c46c336	sgutils2: improve error messages	2021-01-26 12:24:58 +01:00
Dietmar Maurer	c3747b93c8	tape: add new command line tool "pmtx" Also improve sgutil2 error reporting	2021-01-26 11:57:15 +01:00
Dominik Csapak	6864fd0149	server/worker_task: improve newline handling in upid_read_status improves upid_read_status with: * ignore multiple newlines at the end * remove all code that could panic (array index access) the one place where we access with '[pos+1..]' is ok since we explicitely test the len of the vector, this is done to let rust optimize away the range checks, so it cannot panic Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2021-01-26 10:48:15 +01:00
Wolfgang Bumiller	340c0bf9e3	pxar: don't clone patterns unnecessarily The options struct has no Drop handler and is passed by-move so we can partially move out of it. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-26 10:24:18 +01:00
Fabian Grünbichler	4d104cd4d8	clippy: more misc fixes Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:55 +01:00
Fabian Grünbichler	367c0ff7c6	clippy: allow api functions with many arguments some of those can be reduced/cleaned up when we have updater support in the api macro. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:52 +01:00
Fabian Grünbichler	9c26a3d61a	verify: factor out common parameters all the verify methods pass along the following: - task worker - datastore - corrupt and verified chunks might as well pull that out into a common type, with the added bonus of now having a single point for construction instead of copying the default capacaties in three different modules.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:49 +01:00
Fabian Grünbichler	93e3581ce7	derive/impl and use Default for some structs and revamp HttpClientOptions with two constructors for the common use cases Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:45 +01:00
Fabian Grünbichler	f4e52bb27d	authid: make Tokenname(Ref) derive Eq it's needed to derive Hash, and we always compare Authids or their Userid components, never just the Tokenname part anyway.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:40 +01:00
Fabian Grünbichler	72064fd0df	pxar: extract PxarExtractOptions same as PxarCreateOptions, but for extraction/restore rather than create. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:36 +01:00
Fabian Grünbichler	77486a608e	pxar: factor out PxarCreateOptions containing the CLI parameters that are mostly passed-through from the client to our pxar archive creation wrapper in pxar::create Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:32 +01:00
Fabian Grünbichler	e97025ab02	pxar: typedef on_error as ErrorHandler Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:26 +01:00
Fabian Grünbichler	e43b9175c0	client: factor out UploadOptions to reduce function signature complexity. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:18 +01:00
Fabian Grünbichler	9cc1415ef5	systemd/time: extract Time/DateSpec structs could be pulled up into CalendarEvent if desired.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:54:13 +01:00
Fabian Grünbichler	bd215dc0e4	async index reader: typedef ReadFuture Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:53:58 +01:00
Fabian Grünbichler	12e874cef0	allow complex Futures in tower_service impl Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:53:55 +01:00
Fabian Grünbichler	6d233161b0	client: refactor catalog upload spawning by pulling out Result type into separate struct Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:53:51 +01:00
Fabian Grünbichler	905a570489	broadcast_future: refactor broadcast/future binding into its own, private struct. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:53:48 +01:00
Fabian Grünbichler	432fe44187	report: type-alias function call tuple to make clippy happy. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-26 09:53:43 +01:00
Wolfgang Bumiller	51b938496d	tools::sgutils2: name fixup it's not a box anymore Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 15:05:52 +01:00
Wolfgang Bumiller	b7f9b25e4d	tools::sgutils2: use NonNull Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 14:56:10 +01:00
Wolfgang Bumiller	fe61280b6b	tools::sgutils2: extern 'C' and import ordering Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 14:54:25 +01:00
Wolfgang Bumiller	68c087d578	tools::sgutils2: don't transmute to a Box Otherwise we run the drop handler for the scsi pt object AND the box itself, which shouldn't even work as it should be doing a double-free (unless the library does some kind of reference counting in which case this should simply crash later on?) anyway, let's make a wrapper simply called `SgPt` containing the pointer from `construct_scsi_pt_obj()` Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 14:48:27 +01:00
Wolfgang Bumiller	d6bf87cab7	tools::sgutils2: const correctness Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 14:33:45 +01:00
Dietmar Maurer	2b96a43879	tape: cleanup - use ScsiMediaChange trait instead of mtx_status()	2021-01-25 13:25:22 +01:00
Dietmar Maurer	697c41c584	tape: add/use rust scsi changer implementation using libsgutil2	2021-01-25 13:14:07 +01:00
Dietmar Maurer	a2379996e6	sgutils2: add scsi_inquiry command	2021-01-25 13:14:07 +01:00
Fabian Grünbichler	29077d95db	http-client: further clippy cleanups Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:54 +01:00
Fabian Grünbichler	dbd00a57b0	http-client: fix typoed ticket cache condition which was even copy-pasted once without noticing. found with clippy. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:51 +01:00
Fabian Grünbichler	d08cff51a4	rework GC traversal error handling the error message don't make sense with an empty default Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:48 +01:00
Fabian Grünbichler	3e461dec1c	apt: let api handle optional bool with default one less FIXME :) Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:46 +01:00
Fabian Grünbichler	4d08e25913	clippy: rewrite ifs with identical return values Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:43 +01:00
Fabian Grünbichler	43313c2ee7	clippy: rewrite comparison chains chunk_stream one can be collapsed, since split == split_to with at set to buffer.len() anyway. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:39 +01:00
Fabian Grünbichler	81b2a87232	clippy: fix Mutex with unused value Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:36 +01:00
Fabian Grünbichler	3d8cd0ced7	clippy: add is_empty() when len() is implemented Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-25 11:41:32 +01:00
Dietmar Maurer	7c78d54231	sgutils: allow command which does not transfer any data	2021-01-24 15:19:43 +01:00
Dietmar Maurer	f9d71e8b17	sgutils2: allow to set custom timeouts	2021-01-24 14:54:30 +01:00
Dietmar Maurer	0107fd323c	cleanup: avoid compiler warnings	2021-01-23 17:34:26 +01:00
Dietmar Maurer	979dccc7ec	tape: avoid error when clearing encryption key Simply ignore clear request when sg_spin_data_encryption_caps fails. Assume those are tapes without hardware encryption support.	2021-01-23 10:20:43 +01:00
Dietmar Maurer	bf78f70885	improve code docs in api2 Note: API methos should be declared pub, so that they show up in the generated docu.	2021-01-22 15:57:42 +01:00
Dietmar Maurer	0d916ac531	tape: add media pool config code docs	2021-01-22 12:01:46 +01:00
Dietmar Maurer	d4ab407045	tape: add drive config code docs	2021-01-22 11:51:36 +01:00
Dietmar Maurer	45212a8c78	fix mtx parser bug: s/strip_suffix/strip_prefix/	2021-01-22 11:00:56 +01:00
Dietmar Maurer	64b83c3d70	tape: implement paperkey command for tape encryption keys	2021-01-22 09:56:14 +01:00
Dietmar Maurer	639a6782bd	paperkey: move code to src/tools/paperkey.rs	2021-01-22 09:42:59 +01:00
Dietmar Maurer	5f34d69bcc	tape: add volume-statistics api/command	2021-01-22 08:45:35 +01:00
Dietmar Maurer	337ff5a3cc	tape: add estimated medium wearout to status	2021-01-22 08:06:25 +01:00
Dietmar Maurer	8e6459a818	tape: set encryption key on restore	2021-01-22 07:26:42 +01:00
Dietmar Maurer	aff3e16194	tape: add code docs to src/config/tape_encryption_keys.rs	2021-01-21 18:23:07 +01:00
Dietmar Maurer	9372c0787d	renamed src/tape/sgutils2.rs -> src/tools/sgutils2.rs	2021-01-21 17:57:17 +01:00
Dietmar Maurer	83fb2da53e	tape: move MediaCatalog magic number into struct (doc cleanup)	2021-01-21 17:48:07 +01:00
Dietmar Maurer	645a044bf6	tape: further hierarchy improvements	2021-01-21 17:25:32 +01:00
Dietmar Maurer	37796ff73f	tape: change code hierarchy to improve docs	2021-01-21 17:12:01 +01:00
Dietmar Maurer	e1fdcb1678	tape: do not export/doc low level libsgutils2 bindings	2021-01-21 16:38:24 +01:00
Dietmar Maurer	edda5039d4	tape: improve code docs	2021-01-21 13:19:07 +01:00
Dietmar Maurer	1c86893d95	cleanup: always compute fingerprint in KeyConfig constructors	2021-01-21 11:56:54 +01:00
Dietmar Maurer	d543587d34	Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup	2021-01-21 10:56:52 +01:00
Dietmar Maurer	780bc4cad2	tape: try to set encryption key with read-label command	2021-01-21 10:31:49 +01:00
Dietmar Maurer	18bd6ba13d	tape: restore_key - always update key, even if there is already an entry	2021-01-21 10:31:49 +01:00
Dietmar Maurer	4dafc513cc	tape: fix file permissions for tape encryptiuon keys	2021-01-21 10:31:49 +01:00
Dietmar Maurer	7acd5c5659	cleanup: remove missleading wording from code docs	2021-01-21 10:31:49 +01:00
Dietmar Maurer	8428063d9e	cleanup: KeyConfig::decrypt - show password hint on error	2021-01-21 10:31:49 +01:00
Dietmar Maurer	f490dda05a	tape: use type Uuid instead of String	2021-01-21 10:31:49 +01:00
Dietmar Maurer	2b191385ea	tape: use specialized encryption key per media-set	2021-01-21 10:31:49 +01:00
Dietmar Maurer	bc228e5eaf	api: add types for UUIDs	2021-01-20 17:16:46 +01:00
Fabian Grünbichler	8be65e34de	clippy: replace transmute with &* Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:41:02 +01:00
Fabian Grünbichler	d967d8f1a7	clippy: remove drop(&..) it does nothing. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:41:02 +01:00
Fabian Grünbichler	50deb0d3f8	clippy: use is_null to check for null pointers Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:41:02 +01:00
Fabian Grünbichler	1d928b25fe	clippy: remove some unnecessary reference taking Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	f2f81791d1	clippy: fix for_kv_map and allow it in the one case where the entry loop is intended, but the code is not yet implemented fully. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	382f10a0cc	clippy: fix/allow needless_range_loop Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	0d2133db98	clippy: use while let loops Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	09faa9ee95	clippy: pass &str/&[..] instead of &String/&Vec Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	ccec086e25	clippy: remove unnecessary &mut Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	05725ac9a4	clippy: remove unnecessary `let` binding Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	96b7483138	clippy: remove/replace needless explicit lifetimes Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	81281d04a4	clippy: fix/allow identity_op Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	e062ebbc29	clippy: us *_or_else with function calls Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	b92cad0938	clippy: convert single match to if let Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	ea368a06cd	clippy: misc. fixes Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	3f48cdb380	clippy: don't pass along unit value make it explicit. this whole section should probably be re-written with select! Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	17c7b46a69	clippy: use unwrap_or_default Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	a375df6f4c	clippy: use copied/cloned instead of map Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	a3775bb4e8	clippy: shorten assignments Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	1e0c6194b5	clippy: fix option_as_ref_deref Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	a6bd669854	clippy: use matches! Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	6334bdc1c5	clippy: collapse nested ifs Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	3b82f3eea5	clippy: avoid useless format! Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	38556bf60d	clippy: remove explicit returns Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	d8d8af9826	clippy: use chars / byte string literals Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	3984a5fd77	clippy: is_some/none/ok/err/empty Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:54 +01:00
Fabian Grünbichler	397356096a	clippy: remove needless bool literals Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:23:52 +01:00
Fabian Grünbichler	365915da9a	clippy: use strip_prefix instead of manual stripping it's less error-prone (off-by-one!) Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:22:59 +01:00
Fabian Grünbichler	87152fbac6	clippy: drop redundant 'static lifetime those declarations are already const/static.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:22:59 +01:00
Fabian Grünbichler	22a9189ee0	clippy: remove unnecessary closures Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:22:59 +01:00
Fabian Grünbichler	4428818412	clippy: remove unnecessary clones and from::<T>(T) Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:22:59 +01:00
Fabian Grünbichler	47ea98e0e3	clippy: collapse/rework nested ifs no semantic changes (intended). Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-20 16:22:59 +01:00
Dietmar Maurer	6dd0513546	tape: allocate new media set when pool encryption key changes	2021-01-20 15:43:39 +01:00
Dietmar Maurer	8abe51b71d	improve code docs	2021-01-20 15:43:19 +01:00
Dietmar Maurer	69b8bc3bfa	tape: implemenmt show key Moved API types Kdf and KeyInfo to src/api2/types/mod.rs.	2021-01-20 15:43:19 +01:00
Dietmar Maurer	301b8aa0a5	tape: implement change-passphrase for tape encryption keys	2021-01-20 15:43:19 +01:00
Dietmar Maurer	e5b6c93323	tape: add --kdf parameter to create key api	2021-01-20 15:43:19 +01:00
Dietmar Maurer	9a045790ed	cleanup KeyConfig	2021-01-20 15:43:19 +01:00
Dietmar Maurer	82a103c8f9	add "password hint" to KeyConfig	2021-01-20 15:43:19 +01:00
Dietmar Maurer	feb1645f37	tape: generate random encryptions keys and store key_config on media	2021-01-19 11:20:07 +01:00
Dietmar Maurer	8ca37d6a65	cleanup: factor out decrypt_key_config	2021-01-19 11:20:07 +01:00
Wolfgang Bumiller	9b6bddb24c	tfa: remove/empty description for recovery keys While the user chosen description is not allowed to be empty, we do leave it empty for recovery keys, as a "dummy description" makes little sense... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-18 15:20:39 +01:00
Wolfgang Bumiller	ad5cee1d22	tfa: add 'created' timestamp to entries Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-18 14:06:12 +01:00
Wolfgang Bumiller	ca1060862e	tfa: remember recovery indices and tell the client which keys are still available rather than just yes/no/low Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-18 13:51:23 +01:00
Dietmar Maurer	8a0046f519	tape: implement encrypted backup - simple version This is just a proof of concept, only storing the encryption key fingerprint inside the media-set label.	2021-01-18 13:38:22 +01:00
Dietmar Maurer	84cbdb35c4	implement FromStr for Fingerprint	2021-01-18 13:38:22 +01:00
Dietmar Maurer	1e93fbb5c1	tape: add encrypt property to media pool configuration	2021-01-18 13:38:22 +01:00
Dietmar Maurer	619554af2b	tape: clear encryption key before writing labels We always write labels unencrypted.	2021-01-18 13:38:22 +01:00
Dietmar Maurer	d5a48b5ce4	tape: add hardware encryption key managenent api	2021-01-18 13:38:22 +01:00
Fabian Grünbichler	44de5bcc00	pull: add error context for initial group list call otherwise the user is confronted with a generic error like "permission check failed" with no indication that it refers to a request made to the remote PBS instance.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-18 06:51:05 +01:00
Fabian Grünbichler	e2956c605d	pull: rustfmt Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-18 06:50:23 +01:00
Dietmar Maurer	b22b6c2299	tape: encryption scsi command cleanup	2021-01-16 18:24:04 +01:00
Dietmar Maurer	90950c9c20	tape: add scsi commands to control drive hardware encryption	2021-01-16 15:59:05 +01:00
Dietmar Maurer	0c5b9e7820	tape: sgutils2.rs - add do_out_command() Make it possible to run commands that writes data.	2021-01-16 15:59:05 +01:00
Wolfgang Bumiller	a670b99db1	tfa: add webauthn configuration API entry points Currently there's not yet a node config and the WA config is somewhat "tightly coupled" to the user entries in that changing it can lock them all out, so for now I opted for fewer reorganization and just use a digest of the canonicalized config here, and keep it all in the tfa.json file. Experimentally using the flatten feature on the methods with an`Updater` struct similar to what the api macro is supposed to be able to derive on its own in the future. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-15 15:19:52 +01:00
Wolfgang Bumiller	aefd74197a	bakckup::manifest: use tools::json for canonical representation Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-15 15:19:52 +01:00
Wolfgang Bumiller	9ff747ef50	add tools::json for canonical json generation moving this from backup::manifest, no functional changes Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-15 15:19:52 +01:00
Dietmar Maurer	a08a198577	tape: do not abort backup if tape drive does not support tape-alert-flags	2021-01-15 11:43:17 +01:00
Oguz Bektas	6bbe49aa14	access: restrict password changes on @pam realm to superuser for behavior consistency with `update_user` Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2021-01-15 08:49:22 +01:00
Oguz Bektas	5aa1019010	access: limit editing pam credentials to superuser modifying @pam users credentials should be only possible for root@pam, otherwise it can have unintended consequences. also enforce the same limit on user creation (except self_service check, since it makes no sense during user creation) Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2021-01-15 08:49:22 +01:00
Fabian Grünbichler	29a59b380c	proxmox 0.10: adapt to moved ParameterSchema Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	0bfcea6a11	cleanup: remove unnecessary 'mut' and '.clone()' Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	89e9134a3f	hyper: use new hyper::upgrade the old Body::on_upgrade method is no more Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	b5a202acb6	tokio 1.0: update to new Signal interface Signal does not yet re-implement Stream (and is not yet wrapped in tokio-stream either). see https://github.com/tokio-rs/tokio/pull/3383 Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	0f860f712f	tokio 1.0: update to new tokio-openssl interface connect/accept are now happening on pinned SslStreams Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	7c66701366	tokio 1.0: use ReceiverStream from tokio-stream to wrap a Receiver in a Stream. this will likely move back into tokio proper once we have a std Stream.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	585e90c0de	tokio: adapt to 1.0 process:Child changes Child itself is no longer a Future, but it has a new wait() async fn that does the same thing Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	5c852d5b82	tokio: adapt to 1.0 runtime changes enter() now returns a guard, and the builder got revamped to make the choice between MT and current thread explicit. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	484172b5f8	tokio 1.0: AsyncRead/Seek with ReadBuf Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	d148958b67	proxmox 0.10: use tokio::time::timeout directly TimeoutFutureExt is no more Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	0a8d773ad0	tokio 1.0: delay -> sleep almost the same thing, new name(s), no longer Unpin Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Fabian Grünbichler	427d90e6c1	update to tokio 1.0 and various related crates Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2021-01-14 16:01:33 +01:00
Thomas Lamprecht	1a0b410554	manager: user/token list: fix rendering 0 (never) expire date Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2021-01-14 13:59:08 +01:00
Dietmar Maurer	2d50a6192f	tape: sg-tape-cmd - add more ways to specify devices	2021-01-14 13:05:26 +01:00
Dietmar Maurer	781da7f6f0	tape: add --inventorize flag to read-label API/CLI	2021-01-14 11:51:23 +01:00
Dietmar Maurer	25e464c5ce	tape: MediaPool - allow to allocate free tapes	2021-01-13 14:25:51 +01:00
Dietmar Maurer	8446fbca85	tape: rename changer_id to label_text	2021-01-13 13:26:59 +01:00
Dietmar Maurer	0bce2118e7	tape: improve docu	2021-01-12 16:37:23 +01:00
Dietmar Maurer	6543214dde	tape: MediaListEntry - add ctime	2021-01-12 12:01:21 +01:00
Dietmar Maurer	a484c9cf96	tape: automatically reload tapes inside autoloader We always automatically unload tapes to free library slots, so it should not happen that an ejected tape resides inside the drive. This is just a safe guard to handle the situation in case it happens ... You can manually produce the situation by ejecting a tape without unloading: mt -f /dev/nst0 eject Note: Our "proxmox-tape eject" does automatic unload	2021-01-12 09:49:05 +01:00
Dietmar Maurer	5654d8ceba	tape: make eject/export more reliable, improve logging	2021-01-12 09:16:16 +01:00
Dietmar Maurer	31cf625af5	tape: improve backup logs	2021-01-11 13:23:12 +01:00
Dietmar Maurer	93be18ffd2	tape: fix tape alert flag values	2021-01-11 13:23:12 +01:00
Wolfgang Bumiller	ad0ed40a59	api: return "invalid" as CSRF token for partial tickets So that old clients don't `unwrap` a `None` value. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:13 +01:00
Wolfgang Bumiller	7ad33e8052	tfa: use UNAUTHORIZED http status in password check to trigger our 3s delay in the rest handler Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:13 +01:00
Wolfgang Bumiller	abfe0c0e70	tfa: fixup for challenge file split Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:13 +01:00
Wolfgang Bumiller	f22dfb5ece	tfa: remove tfa user when a user is deleted Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:10 +01:00
Wolfgang Bumiller	4bda51688b	tfa: improve user existence check Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	eab25e2f33	tfa: allow deletion of entries of non-existent users Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	94bd11bae2	typo fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	759af9f00c	tfa api: return types and 'pub' structs/methods Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	f58e5132aa	tfa: entry access/iteration cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	d831846706	tfa: r#type parameter name Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	1fc9ac0433	tfa: _entry api method name suffix consistency Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:23:03 +01:00
Wolfgang Bumiller	7f066a9b21	proxy: expose qrcodejs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:22:32 +01:00
Wolfgang Bumiller	027ef213aa	api: tfa management and login Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:22:32 +01:00
Wolfgang Bumiller	dc1fdd6267	config: add tfa configuration Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:22:32 +01:00
Wolfgang Bumiller	96918252e5	buildcfg: add rundir helper macro Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:22:32 +01:00
Wolfgang Bumiller	014dc5f9d7	tools: add create_run_dir helper Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:22:32 +01:00
Wolfgang Bumiller	59e94227af	add tools::serde_filter submodule can be used to perform filtering at parse time Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2021-01-11 10:22:32 +01:00
Dietmar Maurer	6638c034d2	tape: remove unused eject_on_unload method	2021-01-10 16:20:18 +01:00
Dietmar Maurer	04df41cec1	tape: more MediaChange cleanups Try to provide generic implementation for complex operations: - unload_to_free_slot - load_media - export media - clean drive - online_media_changer_ids	2021-01-10 15:32:52 +01:00
Dietmar Maurer	483da89d03	tape: improve export media to directly export from drive, add CLI	2021-01-10 13:44:44 +01:00
Dietmar Maurer	c92e3832bf	tape: cleanup: s/transfer/transfer_media/, avoid compiler warnings	2021-01-10 12:18:30 +01:00
Dietmar Maurer	edb90f6afa	tape: backup - implement export-media-set option	2021-01-10 11:59:55 +01:00
Dietmar Maurer	0057f0e580	tape: MediaChange - add transfer, implement export	2021-01-10 11:51:09 +01:00
Dietmar Maurer	e6217b8b36	tape: renamed src/tape/changer/linux_tape.rs -> src/tape/changer/mtx.rs	2021-01-10 10:07:40 +01:00
Dietmar Maurer	6fe16039b9	tape: simplify media changer implementation - new struct MtxMediaChanger	2021-01-10 10:02:01 +01:00
Dietmar Maurer	42967bf185	tape: backup - implement --eject-media option	2021-01-09 15:17:03 +01:00
Dietmar Maurer	5843268c47	tape: abort backup when we detect critical tape alert flags	2021-01-09 12:34:00 +01:00
Dietmar Maurer	7273ba3de2	tape: change default media set naming template to "%c"	2021-01-09 10:51:51 +01:00
Dietmar Maurer	0bf1c314da	tape: show catalog status in media list	2021-01-09 10:24:48 +01:00
Dietmar Maurer	c7926d8e8c	tape: split MediaSet into extra file	2021-01-09 08:54:58 +01:00
Dietmar Maurer	df69a4fc59	tape: implement drive clean	2021-01-08 11:32:56 +01:00
Dietmar Maurer	25d3965769	tape: correctly skip cleaning tapes (not regular tapes)	2021-01-08 09:16:42 +01:00
Dietmar Maurer	879569d73f	tape: changer transfer - make name parameter optional	2021-01-07 17:09:47 +01:00
Dietmar Maurer	b63f833d36	tape: fix paramater name - s/slot/source-slot/	2021-01-07 15:39:25 +01:00
Dietmar Maurer	482c6e33dd	tape: changer status command: make changer name optional	2021-01-07 15:12:19 +01:00
Dietmar Maurer	46a1863f88	tape: improve MediaChange trait We expose the whole MtxStatus, and we can load/store from/to specified slot numbers.	2021-01-07 14:26:43 +01:00
Dietmar Maurer	d5035c5600	tape: mtx_status - consider new export-slots property	2021-01-06 11:53:33 +01:00
Dietmar Maurer	38ae42b11a	tape: changer - add export-slot config	2021-01-06 11:06:50 +01:00
Dietmar Maurer	c4b2b9ab41	tape: only query volume stats if we can read MAM	2021-01-06 09:20:36 +01:00
Dietmar Maurer	ef942e04c2	tape: add function to classify tape-alert-flags	2021-01-05 17:23:30 +01:00
Dietmar Maurer	b40ab10d38	tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus	2021-01-05 13:43:17 +01:00
Dietmar Maurer	f8ccbfdedd	tape: implement read_volume_statistics	2021-01-05 12:58:18 +01:00
Dietmar Maurer	470f1c798a	tape: status - show thape alert flags	2021-01-04 13:15:30 +01:00
Dietmar Maurer	5c012b392a	tape: use LP 12h TapeAlert Response to query tape alert flags	2021-01-04 13:14:02 +01:00
Dietmar Maurer	165b641c1d	tape: changer status - show full slots (for cartridge without barcode)	2021-01-04 12:06:05 +01:00
Dietmar Maurer	66e42bec05	tape: further PoolWriter cleanups	2021-01-03 12:08:40 +01:00
Dietmar Maurer	c503ea7045	tape: cleanup - rename 'info' to 'media_id' Second try.	2021-01-03 11:38:00 +01:00
Dietmar Maurer	745ec187ce	Revert "tape: cleanup - rename 'info' to 'media_id'" This reverts commit `f046313c0e`. media_id is already use as parameter, so this commit is totally buggy.	2021-01-03 11:14:58 +01:00
Dietmar Maurer	f046313c0e	tape: cleanup - rename 'info' to 'media_id'	2021-01-03 10:37:42 +01:00
Dietmar Maurer	74595b8821	tape: sg-tape-cmd tape-alert-flags	2021-01-03 10:09:43 +01:00
Dietmar Maurer	c9fdd142a4	tape: commit missing file	2021-01-02 13:39:34 +01:00
Dietmar Maurer	abaa6d0ac9	tape: decode TapeAlertFlags in cartridge-memory command	2021-01-02 10:55:30 +01:00
Dietmar Maurer	cfae8f0656	tape: merge MediaStateDatabase into Inventory	2021-01-01 16:15:13 +01:00
Dietmar Maurer	54f4ecd46a	tape: implement MediaPool flag to consider offline media For standalone tape drives.	2021-01-01 10:03:59 +01:00
Dietmar Maurer	b9b4b31284	tape: add basic restore api/command	2020-12-31 10:26:48 +01:00
Dietmar Maurer	b4772d1c43	tape: new inventory helper - lookup_media_set_pool	2020-12-31 10:03:17 +01:00
Fabian Grünbichler	08ac90f920	api: allow tokens to list users their owner, or all if they have the appropriate privileges. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-31 08:29:49 +01:00
Fabian Grünbichler	13f5863561	api: improve error messages for restricted endpoints the old variant attempted to parse a tokenid as userid and returned the cryptic parsing error to the client, which is rather confusing. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-31 08:29:09 +01:00
Dietmar Maurer	81764111fe	tape: media_change - log all errors	2020-12-30 19:17:18 +01:00
Dietmar Maurer	cb022525ff	tape: only log to stdout in CLI environment	2020-12-30 19:01:39 +01:00
Dietmar Maurer	75656a78c6	tape: improve inline docu	2020-12-30 17:28:33 +01:00
Dietmar Maurer	284eb5daff	tape: cleanup/simplify media_change code	2020-12-30 17:16:57 +01:00
Dietmar Maurer	ff58c51919	tape: improve media request/load	2020-12-30 13:09:28 +01:00
Dietmar Maurer	12299b333b	tape: set minimal media label length to 2	2020-12-30 10:15:02 +01:00
Dietmar Maurer	b017bbc441	tape: add restore code, implement catalog api/command	2020-12-30 09:48:18 +01:00
Dietmar Maurer	9e8c0d2e33	tape: cleanup - remove debug messages	2020-12-30 08:41:30 +01:00
Dietmar Maurer	250c29edd2	tape: correctly sort media api entries	2020-12-29 12:09:51 +01:00
Dietmar Maurer	c431659d05	cleanup: remove debug output	2020-12-29 11:59:57 +01:00
Dietmar Maurer	a33389c391	tape: implement media content list api	2020-12-29 11:58:26 +01:00
Dietmar Maurer	3460565414	tape: create the MediaCatalog when we label a tape	2020-12-29 10:55:20 +01:00
Dietmar Maurer	26b62138ee	cleanup: disable debug message when we detect a stopped worker task	2020-12-29 10:53:16 +01:00
Dietmar Maurer	afb0220642	tape: cleanup LinuxDriveStatus - make density optional	2020-12-29 09:10:30 +01:00
Dietmar Maurer	0993923ed5	tape: factor out get_drive_and_media_status	2020-12-29 08:39:06 +01:00
Dietmar Maurer	e0362b0d0f	tape: correctly parse mtx import/export slots	2020-12-28 13:32:56 +01:00
Dietmar Maurer	b27c32821c	tape: install new sg-tape-cmd setuid binary	2020-12-28 11:10:25 +01:00
Dietmar Maurer	76b15a035f	tape: MediaCatalog: write magic number before content	2020-12-26 11:05:25 +01:00
Dietmar Maurer	eb8feb1281	tape: add LTO1 to TapeDensity	2020-12-26 10:48:32 +01:00
Dietmar Maurer	fc6ce9835b	tape: fix non-rewinding tape device check	2020-12-25 15:38:29 +01:00
Dietmar Maurer	8ae9f4efc2	tape: minor cleanups	2020-12-25 13:45:26 +01:00
Dietmar Maurer	c9d13b0fc4	tape: expose check_tape_is_linux_tape_device	2020-12-24 15:51:49 +01:00
Dietmar Maurer	bfacc1d8c3	tape: cleanup - factor out open_linux_tape_device	2020-12-24 11:24:45 +01:00
Dietmar Maurer	5ae86dfaa1	tape: return media usage info with status command	2020-12-23 11:24:34 +01:00
Dietmar Maurer	dbe7e556b0	tape: implement binding for libsgutils2 So that we can read cartridge memory without calling "sg_raw". In future, we may need further low level command to control the tape..	2020-12-23 09:44:53 +01:00
Stefan Reiter	4799280ccd	http_client: add timeouts for critical connects Use timeout futures for sections that might hang in certain error conditions. This is mostly intended to be used as a safeguard, not a first line of defense - i.e. best-effort avoidance of total hangs. Not every future used for the HttpClient/H2Client is changed, only those where a quick response is to be expected. For example, the response reading futures are left alone, so data transfer is never capped with timeout, only the initial server connect. It is also used for upgrading to H2 connections, as that can take a long time on overloaded servers. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2020-12-22 13:31:10 +01:00
Dietmar Maurer	cb80d900b3	tape: add drive status api	2020-12-22 10:42:22 +01:00
Dietmar Maurer	ee01737e87	tape: rename 'mam' api to 'cartridge-memory'	2020-12-22 09:27:34 +01:00
Dietmar Maurer	2012825913	depend on proxmox 0.9.0	2020-12-22 08:52:24 +01:00
Wolfgang Bumiller	b2362a1207	adaptions for proxmox 0.9 and proxmox-api-macro 0.3 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-12-22 07:31:05 +01:00
Dietmar Maurer	1e20f819d5	tape: add command to read cartridge memory (MAM) Thsi add an additional dependency to sg3-utils (small).	2020-12-21 12:12:33 +01:00
Dietmar Maurer	9aa58f0143	cleanup: rename mtfsf into forward_space_count_files	2020-12-18 16:57:49 +01:00
Dietmar Maurer	8835664653	tape: add tape backup api	2020-12-18 15:32:12 +01:00
Dietmar Maurer	d37da6b7fc	tape: add PoolWriter	2020-12-18 15:27:44 +01:00
Dietmar Maurer	b9ee86efe1	tape: use SnapshotReader to create snapshot archive	2020-12-18 12:11:29 +01:00
Dietmar Maurer	d108b610fd	tape: fix write_media_set_label - move to correct position	2020-12-18 12:11:29 +01:00
Dominik Csapak	0ec79339f7	tools/daemon: improve reload behaviour it seems that sometimes, the child process signal gets handled before the parent process signal. Systemd then ignores the childs signal (finished reloading) and only after going into reloading state because of the parent. this will never finish. Instead, wait for the state to change to 'reloading' after sending that signal in the parent, an only fork afterwards. This way we ensure that systemd knows about the reloading before actually trying to do it. Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Tested-By: Fabian Ebner <f.ebner@proxmox.com>	2020-12-18 10:30:37 +01:00
Dietmar Maurer	2afdc7f27d	tape: MediaPool::with_config() - remove name parameter Not required, because config already contains the pool name.	2020-12-18 08:14:24 +01:00
Dietmar Maurer	26aa9aca40	tape: return current_file_number as u64	2020-12-18 07:44:50 +01:00
Dominik Csapak	3e2984bcb9	tools/process_locker: Decrement writer count in drop handler of ProcessLockSharedGuard. We use a counter to determine if we can unlock the file again, but we never actually decremented the writer count, so we held the lock forever. This fixes the issue that we could not start a garbage collect after a reload, as long as the old process is still running, even when that process has no active backup anymore but another long running task (e.g. file download, terminal, etc.). Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-12-18 07:15:08 +01:00
Fabian Grünbichler	a7a5406c32	acl: rustfmt module Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-18 07:07:01 +01:00
Fabian Grünbichler	4f727a783e	acl: reformat privileges for better readability, and tell rustfmt to leave those definitions alone. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-18 07:05:45 +01:00
Fabian Grünbichler	23dc68fdea	acl: add docs and adapt visibility document all public things, add some doc links and make some previously-public things only available for test cases or within the crate: previously public, now private: - AclTreeNode::extract_user_roles (we have extract_roles()) - AclTreeNode::extract_group_roles (same) - AclTreeNode::delete_group_role (exists on AclTree) - AclTreeNode::delete_user_role (same) - AclTreeNode::insert_group_role (same) - AclTreeNode::insert_user_role (same) - AclTree::write_config (we have save_config()) - AclTree::load (we have config()/cached_config()) previously public, now crate-internal: - AclTree::from_raw (only used by tests) - split_acl_path (used by some test binaries) Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-18 07:05:11 +01:00
Dietmar Maurer	b532dd00c4	tape: add helper to read snapshot contents - lock the snapshot for reading - use openat to open files - provides an iterator over all chunks	2020-12-17 13:07:52 +01:00
Fabian Grünbichler	c01742855a	KeyConfig: bail on wrong fingerprint instead of just logging the error. this should never happen in practice unless someone is messing with the keyfile, in which case, it's better to abort. update tests accordingly (wrong fingerprint should fail, no fingerprint should get the expected one). Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 11:27:06 +01:00
Dietmar Maurer	9c953dd260	tape: add code to write backup snapshot files (without chunks) to tape	2020-12-17 08:28:47 +01:00
Dietmar Maurer	3fbf2d2fcd	tape: cleanup MediaCatalog	2020-12-17 08:05:53 +01:00
Fabian Grünbichler	e0af222ec3	KeyConfig: always calculate fingerprint and warn if stored and calculated fingerprint don't match. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 06:52:55 +01:00
Fabian Grünbichler	73b5011786	KeyConfig: add encrypt/decrypt test the RSA key and the encryption key itself are hard-coded to avoid stalling the test runs because of lack of entropy, they have no special significance otherwise. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 06:47:45 +01:00
Fabian Grünbichler	7137630d43	client: add 'import-with-master-key' command to import an encrypted encryption key using a master key. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 06:46:24 +01:00
Fabian Grünbichler	8acfd15d6e	key: move RSA-encryption to KeyConfig since that is what gets encrypted, and not a CryptConfig. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 06:43:34 +01:00
Fabian Grünbichler	48fbbfeb7e	fix #3197 : skip fingerprint check when restoring key when restoring an encrypted key, the original one is obviously not available to check the fingerprint with. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 06:37:54 +01:00
Fabian Grünbichler	9990af3042	master key: store blob name in constant since we will use it in more than one place. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-17 06:36:06 +01:00
Dietmar Maurer	fe6c19383b	tape: remove MediaLabelInfo, use MediaId instead The additional content_uuid was quite useless...	2020-12-16 13:31:32 +01:00
Dietmar Maurer	9839d3f778	tape: improve docu	2020-12-16 12:43:51 +01:00
Dietmar Maurer	dd59e3c2a1	tape: improve docu	2020-12-16 12:23:52 +01:00
Dietmar Maurer	0b7432ae09	tape: add chunk archive reader/writer	2020-12-16 12:08:34 +01:00
Dietmar Maurer	c1c2c8f635	tape: cleanup MediaLocation type for direct use with API	2020-12-16 10:49:01 +01:00
Dietmar Maurer	42298d5896	tape: add magic number to identify media catalog files	2020-12-16 09:00:14 +01:00
Dietmar Maurer	205e187613	tape: add MediaCatalog implementation	2020-12-15 13:40:49 +01:00
Dietmar Maurer	a78348acbb	tape: rename DriveLabel to MediaLabel	2020-12-14 17:37:16 +01:00
Dietmar Maurer	410611b4f2	tape: improve file format docu	2020-12-14 17:29:57 +01:00
Dietmar Maurer	af07ec8f29	tape: minor code cleanup	2020-12-14 16:56:26 +01:00
Dietmar Maurer	3f803af00b	tape: scan - print more debug info	2020-12-14 13:16:18 +01:00
Dietmar Maurer	ac461bd651	tape: implement scan command (useful for debug)	2020-12-14 12:55:49 +01:00
Dietmar Maurer	ce955e1635	tape: implement eod cli command (debug tool)	2020-12-14 09:56:59 +01:00
Dietmar Maurer	e20d008c6a	tape: rename cli 'media media-destroy' toö 'media destroy'	2020-12-14 09:30:32 +01:00
Dietmar Maurer	fb657d8ee5	tape: implement destroy_media	2020-12-14 08:58:40 +01:00
Dietmar Maurer	fba0b77469	tape: add media api	2020-12-14 07:55:57 +01:00
Dietmar Maurer	b5c1296eaa	tape: make changer get_status async	2020-12-14 07:14:24 +01:00
Dietmar Maurer	065df12872	tape: split api type definitions for changers into extra file	2020-12-13 09:31:02 +01:00
Dietmar Maurer	7e1d4712b8	tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA	2020-12-13 09:22:08 +01:00
Dietmar Maurer	49c965a497	tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA	2020-12-13 09:18:16 +01:00
Dietmar Maurer	6fe9aedd0b	tape: correctly call Async handler in proxmox-tape	2020-12-12 09:58:47 +01:00
Dietmar Maurer	42cb9bd6a5	tape: avoid executor blocking in changer api	2020-12-12 09:45:08 +01:00
Dietmar Maurer	66dbe5639e	tape: avoid executor blocking in drive API By using tokio::task::spawn_blocking().	2020-12-12 09:20:04 +01:00
Dietmar Maurer	663ef85992	tape: use WorkerTask for erase and rewind	2020-12-11 11:19:33 +01:00
Dietmar Maurer	e92c75815b	tape: split inventory api inventory: sync, list labels with uuids, update_inventory: WorkerTask, updates database	2020-12-11 10:42:29 +01:00
Dietmar Maurer	6dbad5b4b5	tape: run label commands as WorkerTask (threads)	2020-12-11 09:10:22 +01:00
Dietmar Maurer	bff7e3f3e4	tape: implement barcode-label-mdedia	2020-12-11 07:50:19 +01:00
Dietmar Maurer	83abc7497d	tape: implement inventory command	2020-12-11 07:39:28 +01:00
Dietmar Maurer	be1a8c94ae	fix build: add missing file	2020-12-10 13:40:20 +01:00
Dietmar Maurer	4606f34353	tape: implement read-label command	2020-12-10 13:20:39 +01:00
Dietmar Maurer	7bb720cb4d	tape: implement label command	2020-12-10 12:30:27 +01:00
Dietmar Maurer	c4d8542ec1	tape: add media pool handling	2020-12-10 11:41:35 +01:00
Dietmar Maurer	9700d5374a	tape: add media pool cli	2020-12-10 11:13:12 +01:00
Dietmar Maurer	05e90d6463	tape: add media pool config api	2020-12-10 10:52:27 +01:00
Dietmar Maurer	55118ca18e	tape: correctly sort drive api subdir	2020-12-10 10:09:12 +01:00
Dietmar Maurer	f70d8091d3	tape: implement option changer-drive-id	2020-12-10 09:09:06 +01:00
Dietmar Maurer	a3c709ef21	tape: cli cleanup - avoid api redefinition	2020-12-10 08:35:11 +01:00
Dietmar Maurer	4917f1e2d4	tape: implement delete property for drive update command	2020-12-10 08:25:46 +01:00
Dietmar Maurer	93829fc680	tape: cleanup load-slot api	2020-12-10 08:04:55 +01:00
Dietmar Maurer	5605ca5619	tape: cli cleanup - rename scana-for-* into scan	2020-12-10 07:58:45 +01:00
Dietmar Maurer	e49f0c03d9	tape: implement load-media command	2020-12-10 07:52:56 +01:00
Dietmar Maurer	0098b712a5	tape: implement eject	2020-12-09 17:50:48 +01:00
Dietmar Maurer	5fb694e8c0	tape: implement rewind	2020-12-09 17:43:38 +01:00
Dietmar Maurer	583a68a446	tape: implement erase media	2020-12-09 17:35:31 +01:00
Dietmar Maurer	e6604cf391	tape: add command line interface proxmox-tape	2020-12-09 13:00:20 +01:00
Dietmar Maurer	43cfb3c35a	tape: do not remove changer while still used	2020-12-09 12:55:54 +01:00
Dietmar Maurer	8a16c571d2	tape: add changer property to drive create api	2020-12-09 12:55:10 +01:00
Dietmar Maurer	314652a499	tape: set protected flag for configuration change api methods	2020-12-09 12:02:55 +01:00
Dietmar Maurer	6b68e5d597	client: move connect_to_localhost into client module	2020-12-09 11:59:50 +01:00
Dietmar Maurer	cafd51bf42	tape: add media state database	2020-12-09 11:21:56 +01:00
Wolfgang Bumiller	9b93c62044	remove unused descriptions from api macros these are now a hard error in the api macro Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-12-09 10:55:18 +01:00
Dietmar Maurer	5d90860688	tape: expose basic tape/changer functionality at api2/tape/	2020-12-08 15:42:50 +01:00
Dietmar Maurer	5ba83ed099	tape: check digest on config update	2020-12-08 11:24:38 +01:00
Dietmar Maurer	50bf10ad56	tape: add changer configuration API	2020-12-08 09:04:56 +01:00
Dietmar Maurer	16d444c979	tape: add tape drive configuration API	2020-12-07 13:04:32 +01:00
Dietmar Maurer	fa9c9be737	tape: add tape device driver	2020-12-07 08:29:22 +01:00
Dietmar Maurer	2e7014e31d	tape: add BlockeReader/BlockedWriter streams This is the basic format used to write data to tapes.	2020-12-06 12:09:55 +01:00
Dietmar Maurer	a84050c1f0	tape: add BlockHeader impl	2020-12-06 10:26:24 +01:00
Dietmar Maurer	7c9835465e	tape: add helpers to emulate tape read/write behavior	2020-12-06 09:41:16 +01:00
Dietmar Maurer	ec00200411	fix bug #3189 : fix change_password permission checks, run protected	2020-12-05 16:20:29 +01:00
Dietmar Maurer	b107fdb99a	tape: add tape changer support using 'mtx' command	2020-12-05 14:54:12 +01:00
Dietmar Maurer	7320e9ff4b	tape: add media invenotry	2020-12-05 12:54:15 +01:00
Dietmar Maurer	c4d2d54a6d	tape: define useful constants	2020-12-05 12:20:46 +01:00
Dietmar Maurer	1142350e8d	tape: add media pool config	2020-12-05 11:59:38 +01:00
Dietmar Maurer	d735b31345	tape: add tape read trait	2020-12-05 10:54:38 +01:00
Dietmar Maurer	e211fee562	tape: add tape write trait	2020-12-05 10:51:34 +01:00
Dietmar Maurer	8c15560b68	tape: add file format definitions	2020-12-05 10:45:08 +01:00
Dietmar Maurer	327e93711f	commit missing file: tape api type definitions	2020-12-04 16:00:52 +01:00
Dietmar Maurer	a076571470	tape support: add drive configuration	2020-12-04 15:42:32 +01:00
Dominik Csapak	179145dc24	backup/datastore: move manifest locking to /run this fixes the issue that on some filesystems, you cannot recursively remove a directory when you hold a lock on a file inside (e.g. nfs/cifs) it is not really backwards compatible (so during an upgrade, there could be two daemons have the lock), but since the locking was broken before (see previous patch) it should not really matter (also it seems very unlikely that someone will trigger this) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-12-03 09:56:42 +01:00
Dominik Csapak	6bd0a00c46	backup/datastore: really lock manifest on delete 'lock_manifest' returns a Result<File, Error> so we always got the result, even when we did not get the lock, but we acted like we had. bubble the locking error up Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-12-02 14:37:05 +01:00
Dominik Csapak	f6e28f4e62	client/pull: log how many groups to pull were found if no groups were found, the task log was very confusing as it contained no real information why nothing was synced, e.g.: Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee' Sync datastore 'local-datastore' from 'remote/datastore' sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end TASK OK this patch simply logs how many groups were found and are about to be synced: Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee' Sync datastore 'local-datastore' from 'remote/datastore' found 0 groups to sync sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end TASK OK Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-12-02 07:22:50 +01:00
Dietmar Maurer	2260f065d4	cleanup: use extra file for StoreProgress	2020-12-01 06:34:33 +01:00
Dietmar Maurer	6eff8dec4f	cleanup: remove unnecessary StoreProgress clone()	2020-12-01 06:29:11 +01:00
Fabian Grünbichler	7e25b9aaaa	verify: use same progress as pull percentage of verified groups, interpolating based on snapshot count within the group. in most cases, this will also be closer to 'real' progress since added snapshots (those which will be verified) in active backup groups will be roughly evenly distributed, while number of total snapshots per group will be heavily skewed towards those groups which have existed the longest, even though most of those old snapshots will only be re-verified very infrequently. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:22:55 +01:00
Fabian Grünbichler	f867ef9c4a	progress: add format variants for iterating over a single group, or iterating just on the group level Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:22:12 +01:00
Fabian Grünbichler	fc8920e35d	pull: factor out interpolated progress and add group/snapshot count info. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:13:11 +01:00
Fabian Grünbichler	7f3b0f67e7	remove BackupGroup::list_groups BackupInfo::list_backup_groups is identical code-wise, and makes more sense as entry point for listing groups. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:09:44 +01:00
Fabian Grünbichler	844660036b	gc: don't limit index listing to same filesystem WalkDir does not follow symlinks by default anyway, and this behaviour is not documented anywhere. e.g., if a sysadmin mounts 'extra storage' for some backup group or type (not knowing that only metadata is stored in those directories), GC will ignore all the indices contained within and happily garbage collect their chunks.. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:07:09 +01:00
Fabian Grünbichler	efcac39d34	gc: remove duplicate variable list_images already returns absolute paths, we don't need to prepend anything. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:06:51 +01:00
Fabian Grünbichler	cb4b721cb0	gc: log index files found outside of expected scheme for safety reason, GC finds and marks all index files below the datastore base path. as a result of regular operations, only index files within the expected scheme of <TYPE>/<ID>/<TIMESTAMP> should exist. add a small check + warning if the index list contains index files out side of this expected scheme, so that an admin with shell access can investigate. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:06:17 +01:00
Fabian Grünbichler	7956877f14	gc: shorten progress messages we have messages starting the phases anyway, and limit the number of progress updates so that context remains available at all times. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-12-01 06:04:13 +01:00
Wolfgang Bumiller	43e60ceb41	file logger: remove test.log after test as well and a doc formatting fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 14:13:21 +01:00
Wolfgang Bumiller	b760d8a23f	derive PartialEq for Userid the manual implementation is equivalent Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 14:10:17 +01:00
Wolfgang Bumiller	2c1592263d	tiny clippy hint Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 14:03:43 +01:00
Wolfgang Bumiller	616533823c	don't enforce Vec and String in tools::join Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 13:56:59 +01:00
Wolfgang Bumiller	913dddea85	minor cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 13:56:21 +01:00
Wolfgang Bumiller	3530430365	tools avoid unnecessary copying of parameters/properties Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 13:53:49 +01:00
Wolfgang Bumiller	a4ba60be8f	minor cleanups whitespace, formatting and superfluous lifetime annotations Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 13:47:31 +01:00
Wolfgang Bumiller	99e98f605c	network helpers: fix fd leak in get_network_interfaces This one always leaked. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-11-30 11:25:53 +01:00

... 10 11 12 13 14 ...

4141 Commits