proxmox-backup

Author	SHA1	Message	Date
Fabian Grünbichler	1b1110581a	manifest: revert canonicalization to old behaviour JSON keys MUST be quoted. this is a one-time break in signature validation for backups created with the broken canonicalization code. QEMU backups are not affected, as libproxmox-backup-qemu never linked the broken versions. Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>	2020-07-20 11:37:53 +02:00
Thomas Lamprecht	449e4a66fe	tools/xattr: a char from C is not universally a rust i8 Make it actually do the correct cast by using `libc::c_char`. Fixes issues when building on other platforms, e.g., the aarch64 client only build on Arch Linux ARM I tested in my free time. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-07-19 19:46:27 +02:00
Thomas Lamprecht	217c22c754	server: add path value to NOT_FOUND http error Especially helpful for requests not coming from browsers (where the URL is normally easy to find out). Makes it easier to detect if one triggered a request with an old client, or so.. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-07-16 12:46:51 +02:00
Dominik Csapak	ac5e9e770b	catalog_shell: add exit command it is nice to have a command to exit from the shell instead of only allowing ctrl+d or ctrl+c the api method is just for documentation/help purposes and does nothing by itself, the real logic is directly in the read loop Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-15 12:19:57 +02:00
Wolfgang Bumiller	b25deec0be	pxar: .pxarexclude: absolute paths and byte based paths Change the .pxarexclude parser to byte based parsing with `.split(b'\n')` instead of `.lines()`, to not panic on non-utf8 paths. Specially deal with absolute paths by prefixing them with the current directory. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-15 11:55:48 +02:00
Wolfgang Bumiller	cdf1da2872	tools: add strip_ascii_whitespace for byte slices Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-15 11:55:48 +02:00
Thomas Lamprecht	3cfc56f5c2	cached user info: check_privs: print privilege path in error message As else this is really user unfriendly, and it not printing it has no advantage. If one doesn't wants to leak resource existence they just need to always check permissions before checking if the requested resource exists, if that's not done one can leak information also without getting the path returned (as the system will either print "resource doesn't exists" or "no permissions" respectively) Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-07-15 08:55:58 +02:00
Wolfgang Bumiller	757d0ccc76	warning fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-14 10:37:14 +02:00
Wolfgang Bumiller	032cd1b862	pxar: restore file attributes, improve errors and use the correct integer types for these operations Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-14 10:25:45 +02:00
Dietmar Maurer	f58233a73a	src/backup/data_blob_reader.rs: avoid unwrap() - return error instead	2020-07-10 11:28:19 +02:00
Dominik Csapak	bfcef26a99	api2/node/status: add fingerprint and rename get_usage to get_status (since its not usage only anymore) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-10 11:08:53 +02:00
Dominik Csapak	ec01eeadc6	refactor CertInfo to tools we want to reuse some of the functionality elsewhere Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-10 11:08:53 +02:00
Wolfgang Bumiller	660a34892d	update proxmox crate to 0.2.0 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-10 11:08:27 +02:00
Dietmar Maurer	d86034afec	src/bin/proxmox_backup_client/catalog.rs: fix keyfile handling	2020-07-10 10:36:45 +02:00
Dietmar Maurer	62593aba1e	src/backup/manifest.rs: fix signature (exclude 'signature' property)	2020-07-10 10:36:45 +02:00
Wolfgang Bumiller	0eaef8eb84	client: show key path when creating/changing default key Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-10 09:58:24 +02:00
Thomas Lamprecht	e39974afbf	client: add simple version command Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-07-10 09:34:07 +02:00
Dietmar Maurer	dde18bbb85	proxmox-backup-client benchmark: improve output format	2020-07-10 09:13:52 +02:00
Dietmar Maurer	a40e1b0e8b	src/server/rest.rs: avoid compiler warning	2020-07-10 09:13:52 +02:00
Stoiko Ivanov	c687da9e8e	datastore: chown base dir on creation When creating a new datastore the basedir is only owned by the backup user if it did not exist beforehand (create_path chowns only if it creates the directory), and returns false if it did not create the directory). This improves the experience when adding a new datastore on a fresh disk or existing directory (not owned by backup) - backups/pulls can be run instead of terminating with EPERM. Tested on my local testinstall with a new disk, and a existing directory: Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>	2020-07-09 18:20:16 +02:00
Dietmar Maurer	4327a8462a	proxmox-backup-client benchamrk: add more speed tests	2020-07-09 17:07:22 +02:00
Dietmar Maurer	e193544b8e	src/server/rest.rs: disable debug logs	2020-07-09 16:18:14 +02:00
Dietmar Maurer	323b2f3dd6	proxmox-backup-client benchmark: add --verbose flag	2020-07-09 16:16:39 +02:00
Dominik Csapak	fae11693f0	fix cross process task listing it does not make sense to check if the worker is running if we already have an endtime and state our 'worker_is_active_local' heuristic returns true for non process-local tasks, so we got 'running' for all tasks that were not started by 'our' pid and were still running Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-09 15:30:52 +02:00
Wolfgang Bumiller	bf7e2a4648	simpler lost+found pattern the **/ is not required and currently also mistakenly doesn't match /lost+found which is probably buggy on the pathpatterns crate side and needs fixing there Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-09 14:06:42 +02:00
Dominik Csapak	3ec99affc8	get_disks: don't fail on zfs_devices zfs does not have to be installed, so simply log an error and continue, users still get an error when clicking directly on ZFS Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-09 13:47:31 +02:00
Dominik Csapak	a9649ddc44	disks/zpool_status: add test for pool with special character Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-09 13:37:31 +02:00
Dominik Csapak	4f9096a211	disks/zpool_list: allow some more characters for pool list not exhaustive of what zfs allows (space is missing), but this can be done easily without problems Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-09 13:37:31 +02:00
Dominik Csapak	c3a4b5e2e1	zpool_list: add tests for special pool names those names are allowed for zpools these will fail for now, but it will be fixed in the next commit Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-09 13:37:31 +02:00
Dominik Csapak	7957fabff2	api: add ZPOOL_NAME_SCHEMA and regex poolnames can containe spaces and some other special characters Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-07-09 13:37:31 +02:00
Wolfgang Bumiller	20a4e4e252	minor optimization to 'to_canonical_json' * don't clone hash keys, just use references * we don't need a String, stick to Vec<u8> and use serde_json::to_writer to avoid a temporary strings altogether Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-09 13:32:11 +02:00
Dietmar Maurer	4459ffe30e	src/backup/manifest.rs: add default toömake it compatible with older backus	2020-07-09 13:25:38 +02:00
Dietmar Maurer	dfa517ad6c	src/backup/manifest.rs: rename into_string -> to_string And do not consume self.	2020-07-09 11:28:05 +02:00
Dietmar Maurer	3dacedce71	src/backup/manifest.rs: use serde_json::from_value() to deserialize data Also modified from_data compute signature ditectly from json.	2020-07-09 09:50:28 +02:00
Dietmar Maurer	b53f637914	src/backup/manifest.rs: cleanup signature generation	2020-07-09 09:20:49 +02:00
Dietmar Maurer	2107a5aebc	src/backup/manifest.rs: include signature inside the manifest This is more flexible, because we can choose what fileds we want to sign.	2020-07-08 16:23:26 +02:00
Dietmar Maurer	3638341aa4	src/backup/file_formats.rs: remove signed chunks We can include signature in the manifest instead (patch will follow).	2020-07-08 16:23:26 +02:00
Wolfgang Bumiller	0351f23ba4	client: introduce --keyfd parameter This is a more convenient way to pass along the key when creating encrypted backups of unprivileged containers in PVE where the unprivileged user namespace cannot access `/etc/pve/priv`. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 13:56:38 +02:00
Dietmar Maurer	c1ff544eff	src/backup/crypt_config.rs - compute_digest: make it more secure	2020-07-08 12:53:04 +02:00
Wolfgang Bumiller	05389a0109	more xdg cleanup and encryption parameter improvements Have a single common function to get the BaseDirectories instance and a wrapper for `find()` and `place()` which wrap the error with some context. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 10:57:28 +02:00
Wolfgang Bumiller	b65390ebc9	client: xdg usage: place() vs find() place() is used when creating a file, as it will create intermediate directories, only use it when actually placing a new file. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 10:57:28 +02:00
Dietmar Maurer	3bad3e6e52	src/client/backup_writer.rs - upload_stream: add crypt_mode	2020-07-08 10:43:28 +02:00
Wolfgang Bumiller	24be37e3f6	client: fix schema to include --crypt-mode parameter Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 10:09:15 +02:00
Wolfgang Bumiller	1008a69a13	pxar: less confusing logic Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 09:58:29 +02:00
Wolfgang Bumiller	521a0acb2e	DataStore::load_manifest: also return CryptMode Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 09:19:53 +02:00
Wolfgang Bumiller	3b66040de6	add DataBlob::crypt_mode and move use statements up Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 09:19:53 +02:00
Wolfgang Bumiller	af3a0ae7b1	remove CryptMode::sign_only special method Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-08 09:19:53 +02:00
Dietmar Maurer	4e36f78438	src/backup/manifest.rs: support old encrypted property Just to avoid confusion.	2020-07-08 08:52:27 +02:00
Wolfgang Bumiller	f28d9088ed	introduce a CryptMode enum This also replaces the recently introduced --encryption parameter on the client with a --crypt-mode parameter. This can be "none", "encrypt" or "sign-only". Note that this introduces various changes in the API types which previously did not take the above distinction into account properly: Both `BackupContent` and the manifest's `FileInfo`: lose `encryption: Option<bool>` gain `crypt_mode: Option<CryptMode>` Within the backup manifest itself, the "crypt-mode" property will always be set. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-07 15:24:19 +02:00
Dietmar Maurer	f91d58e157	src/tools/runtime.rs: implement get_runtime_with_builder	2020-07-07 10:11:04 +02:00
Dietmar Maurer	7443a6e092	src/client/remote_chunk_reader.rs: implement clone for RemoteChunkReader	2020-07-07 07:34:58 +02:00
Wolfgang Bumiller	96ee857752	client: add --encryption boolen parameter This can be used to explicitly disable encryption even if a default key file exists in ~/.config. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-06 14:36:04 +02:00
Wolfgang Bumiller	887018bb79	client: use default encryption key if it is available Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-06 14:36:04 +02:00
Wolfgang Bumiller	9696f5193b	client: move key management into separate module and use api macro for methods and Kdf type Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-06 14:36:04 +02:00
Wolfgang Bumiller	e13c4f66bb	minor style & whitespace fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-07-06 10:55:25 +02:00
Thomas Lamprecht	08cb2038bd	api: disks: indentation fixup Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-07-03 17:59:30 +02:00
Dietmar Maurer	9de69cdb1a	src/bin/proxmox_backup_client/catalog.rs: split out catalog code	2020-07-03 16:45:47 +02:00
Dietmar Maurer	e9764238df	make ReadChunk not require mutable self. That way we can reduce lock contentions because we lock for much shorter times.	2020-07-03 07:37:29 +02:00
Dietmar Maurer	cc7995ac40	src/bin/proxmox_backup_client/task.rs: split out task command	2020-07-02 18:04:29 +02:00
Dietmar Maurer	43abba4b4f	src/bin/proxmox_backup_client/mount.rs: split out mount code	2020-07-02 17:49:59 +02:00
Dietmar Maurer	caea8d611f	proxmox-backup-client: add benchmark command This is just a start, We need to add more useful things here...	2020-07-02 14:01:57 +02:00
Wolfgang Bumiller	7d0754a6d2	pxar: fixup 'vanished-file' logic a bit Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-30 14:41:42 +02:00
Wolfgang Bumiller	5afa0755ea	pxar: fix missing newlines in warnings Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-30 14:37:20 +02:00
Dietmar Maurer	2162e2c15d	src/api2/admin/datastore.rs: avoid slash in UPID strings	2020-06-30 13:11:22 +02:00
Dietmar Maurer	4059285649	fix typo	2020-06-29 12:59:25 +02:00
Dietmar Maurer	2e079b8bf2	partially revert commit `1f82f9b7b5` do it backwards compatible. Also, code was wrong because FixedIndexWriter still computed old style csums...	2020-06-29 12:44:45 +02:00
Wolfgang Bumiller	a8e2940ff3	pxar: deal with files changing size during archiving Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-26 11:49:51 +02:00
Dominik Csapak	4eb4e94918	fix test output field separator for pools is always a tab when using -H Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-26 10:31:11 +02:00
Dietmar Maurer	817bcda848	src/backup/verify.rs: do not stop on server shutdown This is a read-only task, so there is no need to stop.	2020-06-26 09:45:59 +02:00
Dominik Csapak	f6de2c7359	WorkerTask: add warnings and count them so that we have one level more between errors and OK Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-26 09:42:11 +02:00
Dominik Csapak	402c8861d8	fix typo Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-26 09:12:29 +02:00
Dominik Csapak	3f683799a8	improve 'debug' parameter instead of checking on '1' or 'true', check that it is there and not '0' and 'false'. this allows using simply https://foo:8007/?debug instead of https://foo:8007/?debug=1 Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-26 09:12:14 +02:00
Dietmar Maurer	1f82f9b7b5	src/backup/index.rs: add compute_csum And use it for fixed and dynamic index. Please note that this changes checksums for fixed indexes, so restore older backups will fails now (not backward compatible).	2020-06-26 09:00:34 +02:00
Dietmar Maurer	fe3e65c3ea	src/api2/backup.rs: call register_chunk in previous download api	2020-06-26 08:22:46 +02:00
Dietmar Maurer	fdaab0df4e	src/backup/index.rs: add chunk_info method	2020-06-26 08:14:45 +02:00
Dietmar Maurer	b957aa81bd	update backup api for incremental backup Signed-off-by: Dietmar Maurer <dietmar@proxmox.com> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2020-06-26 07:17:08 +02:00
Dietmar Maurer	8ea00f6e49	allow to abort verify jobs And improve job description rendering on gui.	2020-06-25 12:56:36 +02:00
Stefan Reiter	e22f4882e7	extract create_download_response API helper and put it into a new "api2::helpers" module. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2020-06-25 11:57:37 +02:00
Dietmar Maurer	355c055e81	src/bin/proxmox-backup-manager.rs: implement verify	2020-06-24 13:35:21 +02:00
Dietmar Maurer	c2009e5309	src/api2/admin/datastore.rs: add verify api	2020-06-24 13:35:21 +02:00
Dietmar Maurer	23f74c190e	src/backup/backup_info.rs: impl Display for BackupGroup	2020-06-24 13:35:21 +02:00
Wolfgang Bumiller	a6f8728339	update to pxar 0.1.9, update ReadAt implementations Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-24 11:57:12 +02:00
Stefan Reiter	facd9801cf	add incremental backup support To support incremental backups (where not all chunks are sent to the server), a new parameter "reuse-csum" is introduced on the "create_fixed_index" API call. When set and equal to last backups' checksum, the backup writer clones the data from the last index of this archive file, and only updates chunks it actually receives. In incremental mode some checks usually done on closing an index cannot be made, since they would be inaccurate. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2020-06-24 10:01:25 +02:00
Dietmar Maurer	21302088de	remove debug println	2020-06-24 09:15:13 +02:00
Stefan Reiter	8268c9d161	fix overflow panic during upload if only data chunks are registered (high chance during incremental backup), then chunk_count might be one lower then upload_stat.count because of the zero chunk being unconditionally uploaded but not used. Thus when subtracting the two, an overflow would occur. In general, don't let the client make the server panic, instead just set duplicates to 0. Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>	2020-06-24 09:07:22 +02:00
Dominik Csapak	b91b7d9ffd	api2/node/disks/zfs: check if default zfs mount path exists and if it does bail, because otherwise we would get an error on mounting and have a zpool that is not imported and disks that are used Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-24 07:22:39 +02:00
Dominik Csapak	d33d8f4e6a	api2/admin/datastore: add pxar-file-download api call streams a file from a pxar file of an unencrypted backup Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-24 07:21:15 +02:00
Dominik Csapak	5b1cfa01f1	api2/admin/datastore: add 'catalog' api call returns the dir listing of the given filepath of the backup snapshot the filepath has to be base64 encoded or 'root' Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-24 07:16:12 +02:00
Dominik Csapak	05d18b907a	add From<&DirEntryAttribute to CatalogEntryType and make it pub(crate) we want to get a string representation of the DirEntryAttribute like 'f' for file, etc. and since we have such a mapping already in the CatalogEntryType, use that Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-24 07:08:50 +02:00
Dominik Csapak	e44fe0c9f5	derive Clone for the LocalChunkReader this will be necessary for accessing local pxar behind didx files Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-24 07:07:28 +02:00
Dominik Csapak	4cf0ced950	add LocalDynamicReadAt mostly copied from BufferedDynamicReadAt from proxmox-backup-client but the reader is wrapped in an Arc in addition to the Mutex we will use this for local access to a pxar behind a didx file Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-24 07:05:31 +02:00
Dietmar Maurer	60f9a6ea8f	src/backup/datastore.rs: add new helpers to load blobs and verify chunks	2020-06-24 06:58:14 +02:00
Dietmar Maurer	1090fd4424	src/backup/data_blob.rs: cleanup - improve code reuse	2020-06-24 06:56:48 +02:00
Dietmar Maurer	92c3fd2e22	src/backup/chunk_store.rs: allow to read name() This is helpful for logging ...	2020-06-24 06:54:21 +02:00
Dietmar Maurer	d6d3b353be	cleanup: implement FromStr for BackupGroup	2020-06-23 08:16:56 +02:00
Dietmar Maurer	a67f7d0a07	cleanup: implement FromStr for BackupDir	2020-06-23 08:09:52 +02:00
Dietmar Maurer	c8137518fe	src/bin/proxmox_backup_manager/disk.rs: add renderer for wearout So that we display the same value as the GUI.	2020-06-23 07:44:09 +02:00
Dominik Csapak	cbef49bf4f	remove absolute paths when executing binaries we set the paths manually, so this is ok Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-23 07:09:06 +02:00
Dominik Csapak	0b99e5aebc	remove debug prints Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-23 06:33:58 +02:00
Dominik Csapak	f386f512d0	add AsyncReaderStream and replace AsyncIndexReader's stream implementation with that Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-23 06:33:31 +02:00
Dietmar Maurer	3ddb14889a	src/tools/daemon.rs: reopen STDOUT/STDERR journald streams to get correct PID in logs	2020-06-22 13:06:53 +02:00
Wolfgang Bumiller	c08fac4d69	tools::daemon: sync with child after MainPid message Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-22 10:58:04 +02:00
Wolfgang Bumiller	c40440092d	tools: add socketpair helper Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-22 10:49:56 +02:00
Wolfgang Bumiller	dc2ef2b54f	tools::daemon: fetch exe name in the beginning We get the path to our executable via a readlink() on "/proc/self/exe", which appends a " (deleted)" during package reloads. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-22 10:31:54 +02:00
Dietmar Maurer	f28cfb322a	avoid compiler warnings	2020-06-20 07:24:02 +02:00
Dietmar Maurer	3bbe291c51	zpool_status.rs - indented_list_to_tree: do not set name property This is no necessary. We only touch/set 'children' and 'leaf' properties.	2020-06-20 07:19:25 +02:00
Dietmar Maurer	42d19fdf69	src/api2/node/disks/zfs.rs: always set pool name	2020-06-20 07:15:32 +02:00
Dietmar Maurer	215968e033	src/tools/disks/zpool_status.rs: add 'leaf' attribute to root node, rename 'prev' into 'parent'	2020-06-20 06:49:06 +02:00
Dietmar Maurer	eddd1a1b9c	src/tools/disks/zpool_status.rs: move use clause top of file	2020-06-20 06:17:22 +02:00
Dietmar Maurer	d2ce211899	fixup for previous commit	2020-06-20 06:15:26 +02:00
Dietmar Maurer	1cb46c6f65	src/tools/disks/zpool_status.rs - cleanup: use struct StackItem instead of tuple	2020-06-19 18:58:57 +02:00
Dietmar Maurer	5d88c3a1c8	src/tools/disks/zpool_status.rs: remove unnecessary checks Thos things can never happen, so simply use unwrap().	2020-06-19 18:27:39 +02:00
Dietmar Maurer	07fb504943	src/tools/disks/zpool_status.rs: simplify code by using serde_json::to_value	2020-06-19 17:51:13 +02:00
Dietmar Maurer	f675c5e978	src/tools/disks/zpool_status.rs - add all attributes to the tree	2020-06-19 16:55:28 +02:00
Wolfgang Bumiller	4e37d9ce67	add general indented_list_to_tree implementation Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-19 14:37:40 +02:00
Wolfgang Bumiller	e303077132	lru_cache: restrict and annotate Send impl Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-19 09:37:34 +02:00
Dominik Csapak	6ef9bb59eb	api2/admin/datastore: add download-decoded endpoint similar to 'download', but streams the decoded file when it is not encrypted Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 08:39:15 +02:00
Dominik Csapak	eeaa2c212b	impl Sync for DataBlobReader this is safe for the reason explained in the comment Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 08:37:47 +02:00
Dominik Csapak	4a3adc3de8	add AsyncIndexReader implements AsyncRead as well as Stream for an IndexFile and a store that implements AsyncReadChunk we can use this to asyncread or stream the content of a FixedIndex or DynamicIndex Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 08:32:33 +02:00
Dominik Csapak	abdb976340	add Display trait to BackupDir Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 08:28:35 +02:00
Dominik Csapak	3b62116ce6	implement AsyncReadChunk for LocalChunkReader same as the sync ReadChunk but uses tokio::fs::read instead of file_get_contents Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 07:54:23 +02:00
Dominik Csapak	1c090810f5	api2/admin/datastore/snapshos: show encrypted and size info per file Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 07:39:56 +02:00
Dominik Csapak	e181d2f6da	add encrypted info to Manifest we want to save if a file of a backup is encrypted, so that we can * show that info on the gui * can later decide if we need to decrypt the backup Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 07:35:39 +02:00
Dominik Csapak	16021f6ab7	use the existing async method for read_raw_chunk does the same, except the manual drop, but thats handled there by letting the value go out of scope Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 07:23:17 +02:00
Dominik Csapak	ba694720fc	api2/admin/datastore: log stream error during file download the client cannot get an error during an chunked http transfer, so at least log it server side Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 06:58:55 +02:00
Dominik Csapak	bde8e243cf	remove unsafe copy code copy_nonoverlapping is basically a memcpy which can also be done via copy_from_slice which is not unsafe (copy_from_slice uses copy_nonoverlapping internally) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-19 06:56:15 +02:00
Dietmar Maurer	3352ee5656	parse_zpool_status_field: handle tabs without copying input	2020-06-18 19:40:01 +02:00
Dietmar Maurer	b29cbc414d	parse_zpool_status_vdev: consider tabs as 8 spaces	2020-06-18 18:38:56 +02:00
Dietmar Maurer	026dc1d11f	src/api2/node/disks/zfs.rs: add zpool_details api	2020-06-18 15:04:46 +02:00
Dietmar Maurer	9438aca6c9	src/tools/disks/zpool_status.rs: improve parser	2020-06-18 14:55:22 +02:00
Dietmar Maurer	547f0c97e4	src/tools/nom.rs: new helper parse_complete_line() for single line parsers Like parse_complete(), but generates simpler error messages.	2020-06-18 12:57:55 +02:00
Dietmar Maurer	177a2de992	src/tools/nom.rs: move nom helpers into separate file	2020-06-18 12:41:13 +02:00
Dietmar Maurer	0686b1f4db	src/tools/disks/zpool_list.rs: split code into separate file	2020-06-18 10:31:07 +02:00
Dietmar Maurer	0727e56a06	src/tools/disks/zpool_status.rs: parse zpool status output	2020-06-18 10:23:15 +02:00
Dietmar Maurer	2fd3d57490	src/tools/disks/zfs.rs: rename ZFSPoolStatus into ZFSPoolInfo, fix error message	2020-06-17 09:08:26 +02:00
Dietmar Maurer	3f851d1321	src/api2/node/disks/directory.rs: add early check if disk is unused	2020-06-17 08:31:11 +02:00
Dietmar Maurer	1aef491e24	src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks	2020-06-17 08:07:54 +02:00
Dietmar Maurer	d0eccae37d	avoid compiler warning	2020-06-17 08:07:42 +02:00
Dietmar Maurer	a34154d900	src/tools/disks/zfs.rs: cleanup parse_pool_header	2020-06-17 07:47:11 +02:00
Dietmar Maurer	c2cc32b4dd	src/tools/disks/zfs.rs: add more parser tests	2020-06-17 07:38:19 +02:00
Dietmar Maurer	46405fa35d	src/tools/disks/zfs.rs: add comment	2020-06-17 07:14:26 +02:00
Dietmar Maurer	66af7f51bc	src/tools/disks/zfs.rs: make zfs list parser private	2020-06-17 07:00:54 +02:00
Dietmar Maurer	c72ccd4e33	src/tools/disks/zfs.rs: add regression tests for parse_zfs_list	2020-06-16 18:14:35 +02:00
Dietmar Maurer	902b2cc278	src/tools/disks/zfs.rs: simplify code	2020-06-16 17:51:17 +02:00
Wolfgang Bumiller	8ecd7c9c21	move api dump binaries back to src/bin for package building they're required for docs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 14:48:01 +02:00
Dietmar Maurer	fb5a066500	src/api2/node/disks.rs: expose directory api	2020-06-16 13:36:32 +02:00
Wolfgang Bumiller	d19c96d507	move test binaries to examples/ These aren't installed and are only used for manual testing, so there's no reason to force them to be built all the time. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 13:32:24 +02:00
Dietmar Maurer	929a13b357	src/api2/node/disks/zfs.rs: add zpool api	2020-06-16 13:25:53 +02:00
Dietmar Maurer	36c65ee0b0	src/tools/disks/zfs.rs: cleanup (rename usage properties) And allow to parse zpool list output without -v flag.	2020-06-16 13:25:53 +02:00
Dietmar Maurer	3378fd9fe5	src/tools/disks/zfs.rs: parse more infos (dedup, fragmentation, health)	2020-06-16 13:25:53 +02:00
Dietmar Maurer	58c51cf3d9	avoid compiler warnings	2020-06-16 13:25:53 +02:00
Dietmar Maurer	5509b199fb	use new run_command helper	2020-06-16 13:25:53 +02:00
Wolfgang Bumiller	bb59df9134	catalog: don't panic on invalid file mtimes Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 11:25:54 +02:00
Wolfgang Bumiller	2564b0834f	fix file timestamps in catalog Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 11:25:31 +02:00
Wolfgang Bumiller	9321bbd1f5	pxar: fix missing subdirectories in catalogs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 11:04:56 +02:00
Wolfgang Bumiller	4264e52220	reuse some extractor code in catalog shell Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 10:54:54 +02:00
Wolfgang Bumiller	6988b29bdc	use O_EXCL when creating files during extraction Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 10:33:27 +02:00
Wolfgang Bumiller	98c54240e6	pxar: make extractor state more reusable Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 10:32:13 +02:00
Wolfgang Bumiller	d30c192589	AsyncReadChunk: require Send Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-16 09:50:29 +02:00
Dietmar Maurer	ac7513e368	src/tools.rs: add setup_safe_path_env()	2020-06-15 10:38:30 +02:00
Dietmar Maurer	fbbcd85839	src/api2/node/disks/directory.rs: implement add-datastore feature	2020-06-15 10:01:50 +02:00
Wolfgang Bumiller	7a6b549270	dynamic index: make it hard to mess up endianess Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-15 09:19:35 +02:00
Wolfgang Bumiller	0196b9bf5b	remove unnecessary .into Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 14:57:58 +02:00
Dominik Csapak	20b3094bcb	api2/status: add task list endpoint for now mostly copy/paste from nodes/nodename/tasks (without the parameters) but we should replace the 'read_task_list' with a method that gives us the tasks since some timestamp so that we can get a longer list of tasks than for the node (we could of course embed this then in the nodes/node/task api call and remove this again as long as the api is not stable) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 14:50:17 +02:00
Dominik Csapak	df528ee6fa	implement From<TaskListInfo> for TaskListItem and use it where its convenient Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 14:50:12 +02:00
Wolfgang Bumiller	57e50fb906	use new Mmap helper for dynamic index Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 13:57:56 +02:00
Wolfgang Bumiller	3d571d5509	some internal combinator-influenced api cleanup The download methods used to take the destination by value and return them again, since this was required when using combinators before we had `async fn`. But this is just an ugly left-over now. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 11:46:42 +02:00
Wolfgang Bumiller	8e6e18b77c	client: make dump_image async, use async chunk reader Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 11:40:18 +02:00
Wolfgang Bumiller	4d16badf6f	add an AsyncReadChunk trait Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 11:38:21 +02:00
Wolfgang Bumiller	a609cf210e	more cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 11:01:04 +02:00
Wolfgang Bumiller	1498659b4e	cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 10:59:34 +02:00
Wolfgang Bumiller	4482f3fe11	pxar, acl: cleanup acl helper usage use NixPath for Acl::set_file to avoid memduping the c string Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 10:52:18 +02:00
Dominik Csapak	5d85847f91	client: only start catalog upload if we have one else we start a dynamic writer and never close it, leading to a backup error this fixes an issue with backing up vm templates (and possibly vms without disks) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 10:38:25 +02:00
Dominik Csapak	476b4acadc	BackupEnvironment: do not set finished flag prematurely we check if all dynamic_writers are closed and if the backup contains any valid files, we can only mark the backup finished after those checks, else the backup task gets marked as OK, even though it is not finished and no cleanups run Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 10:37:52 +02:00
Wolfgang Bumiller	cf1bd08131	pxar: fcaps in fuse Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-12 10:37:48 +02:00
Dominik Csapak	ec8f042459	api2/status: use new rrd::extract_cached_data and drop the now unused extract_lists function this also fixes a bug, where we did not add the datastore to the list at all when there was no rrd data Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 13:31:16 +02:00
Dominik Csapak	431cc7b185	rrd: move creation of serde value into api there is now a 'extract_cached_data' which just returns the data of the specified field, and an api function that converts a list of fields to the correct serde value this way we do not have to create a serde value in rrd/cache.rs (makes for a better interface) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 13:31:14 +02:00
Dominik Csapak	e693818afc	refactor time functions to tools Signed-off-by: Dominik Csapak <d.csapak@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 13:31:10 +02:00
Wolfgang Bumiller	3d68536fc2	pxar: support .pxareclude files, error report updates Report vanished files (instead of erroring out on them), also only warn about files inaccessible due to permissions instead of bailing out. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 12:22:18 +02:00
Wolfgang Bumiller	26e78a2efb	downgrade some FIXMEs to TODOs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 11:09:23 +02:00
Wolfgang Bumiller	5444fa940b	turn pxar::flags into bitflags, pxar::Flags Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 11:05:53 +02:00
Dietmar Maurer	d4f2397d4c	add api to format disks and create datastores	2020-06-10 11:03:36 +02:00
Wolfgang Bumiller	fab2413741	catalog: remove unused SenderWriter Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 10:42:42 +02:00
Dietmar Maurer	669c137fec	src/tools/systemd.rs: implement daemon_reload, start_unit, stop_unit and enable_unit	2020-06-10 08:56:04 +02:00
Wolfgang Bumiller	fc6047fcb1	pxar: don't skip list+found by default This used to be default-off and was accidentally set to on-by-default with the pxar crate update. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 08:53:10 +02:00
Wolfgang Bumiller	3014088684	pxar: sort .pxareclude-cli file and fix its mode Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-10 08:49:14 +02:00
Dietmar Maurer	144006fade	src/tools.rs: add new run_command helper	2020-06-10 07:16:47 +02:00
Dietmar Maurer	b9cf6ee797	src/tools/systemd/types.rs: add Mount config	2020-06-09 18:47:10 +02:00
Wolfgang Bumiller	cdde66d277	statistics: covariance(): avoid allocation Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-09 13:57:27 +02:00
Wolfgang Bumiller	239e49f927	pxar: create .pxarexclude-cli file Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-09 13:17:59 +02:00
Dominik Csapak	bda48e04da	api2: add status/datastore-usages api call returns a list of the datastores and their usages, a list of usages of the past month (for the gui) and an estimation of when its full (using the linear regression) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-09 12:20:25 +02:00
Dominik Csapak	ba97479848	add statistics module provides some basic statistics functions (sum, mean, etc.) and a function to return the parameters of the linear regression of two variables implemented using num_traits to be more flexible for the types Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-09 12:19:51 +02:00
Dominik Csapak	6cad8ce4ce	rrd: add 'extract_lists' this is an interface to simply get the Vec<Option<f64>> out of rrd without going through serde values we return a list of timestamps and a HashMap with the lists we could find (otherwise it is not in the map) if no lists could be extracted, the time list is also empty Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-09 12:19:06 +02:00
Dominik Csapak	33070956af	let disk_usage return StorageStatus and use it for datastores/nodes disk_usage returned the same values as defined in StorageStatus, so simply use that with that we can replace the logic of the datastore status with that function and also use it for root disk usage of the nodes Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-09 12:05:39 +02:00
Dietmar Maurer	da84cc52f4	src/tools/systemd.rs: implement escape_unit and unescape_unit	2020-06-09 11:52:06 +02:00
Wolfgang Bumiller	2179359f40	move src/pxar.rs -> src/pxar/mod.rs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-09 10:49:59 +02:00
Dietmar Maurer	9bb161c881	src/tools/disks.rs: add create_single_linux_partition and create_file_system	2020-06-08 17:43:01 +02:00
Dietmar Maurer	297e600730	cleanup comment	2020-06-08 17:43:01 +02:00
Dietmar Maurer	ed7b3a7de2	src/tools/disks.rs: add get_fs_uuid helper	2020-06-08 17:43:01 +02:00
Dietmar Maurer	0f358204bd	src/tools/disks.rs: add helper to list partitions	2020-06-08 17:43:01 +02:00
Dietmar Maurer	ca6124d5fa	src/tools/disks.rs: make helpers pub So that I can use them with my test code.	2020-06-08 17:43:01 +02:00
Wolfgang Bumiller	7eacdc765b	pxar: split assert_relative_path the check for a single component is only required in the dir stack atm Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-08 15:02:52 +02:00
Wolfgang Bumiller	c443f58b09	switch to external pxar and fuse crates Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-08 13:56:58 +02:00
Wolfgang Bumiller	1e3d9b103d	xattr: make xattr_name_fcaps public Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-08 13:56:58 +02:00
Wolfgang Bumiller	386990ba09	tools: add file_get_non_comment_lines Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-08 13:56:58 +02:00
Dietmar Maurer	bc853b028f	src/tools/disks.rs: cleanup, remove unused DiskUse bitflag type	2020-06-08 09:43:07 +02:00
Dietmar Maurer	d406de299b	src/tools/disks.rs: use dev_t to index zfs/lvm device sets	2020-06-08 09:01:34 +02:00
Dietmar Maurer	dfb31de8f0	proxmox_backup_manager disk list: display gpt column	2020-06-08 07:35:44 +02:00
Dietmar Maurer	7c3aa258f8	src/tools/disks/zfs.rs: allow empty zpool list output	2020-06-08 07:23:04 +02:00
Dietmar Maurer	044055062c	src/tools/disks.rs: new helper to reread partition table	2020-06-08 07:22:06 +02:00
Dietmar Maurer	2b388026f8	src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node	2020-06-08 07:20:59 +02:00
Dietmar Maurer	707974fdb3	src/api2/node/disks.rs: implement initgpt API	2020-06-07 10:30:34 +02:00
Dietmar Maurer	9069debcd8	src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA	2020-06-07 07:20:25 +02:00
Dietmar Maurer	fa2bdc1309	src/config/acl.rs: add /system/disks to valid acl paths	2020-06-06 15:48:15 +02:00
Dietmar Maurer	8e40aa63c1	src/bin/proxmox-backup-manager.rs: add disk subcommand	2020-06-06 15:40:28 +02:00
Dietmar Maurer	d2522b2db6	src/tools/disks.rs: fix disk size, add completion helper	2020-06-06 15:39:25 +02:00
Dietmar Maurer	ce8e3de401	move disks api to /node/<node>/disks	2020-06-06 14:43:36 +02:00
Dietmar Maurer	7fa2779559	src/api2/disks.rs: implement smart api	2020-06-06 12:23:11 +02:00
Dietmar Maurer	042afd6e52	src/tools/disks.rs: new helper disk_by_name()	2020-06-06 12:22:38 +02:00
Dietmar Maurer	ff30caeaf8	src/api2/disks.rs - list-disks: add usage-type filter	2020-06-06 11:48:58 +02:00
Dietmar Maurer	553cd12ba6	src/api2/disks.rs: start disks api	2020-06-06 11:38:47 +02:00
Dietmar Maurer	de1e1a9d95	src/tools/disks.rs: use api macro so that we can use those types with the api	2020-06-06 11:37:24 +02:00
Dietmar Maurer	91960d6162	src/tools/disks.rs - get_disks: query smart status	2020-06-06 09:18:20 +02:00
Dietmar Maurer	4c24a48eb3	src/tools/disks/smart.rs: use model.to_string_lossy() to simplify code	2020-06-06 09:05:22 +02:00
Dietmar Maurer	484e761dab	src/tools/disks/smart.rs: try to get correct wearout for ATA devices	2020-06-06 09:01:15 +02:00
Dietmar Maurer	059b7a252e	src/tools/disks/smart.rs - get_smart_data: use &Disk instead of &str So that we can query other device infos easily (model, vendor, ..)	2020-06-06 08:24:58 +02:00
Dietmar Maurer	eb80aac288	src/tools/disks/smart.rs: parse output from smartctl	2020-06-05 18:30:06 +02:00
Dietmar Maurer	c26aad405f	src/tools/disks.rs: implement get_disks (similar to the one in PVE::Diskmanage) But no ceph support for now. Also no support for old cciss block devices.	2020-06-05 10:33:53 +02:00
Dietmar Maurer	f03a0e509e	src/tools/disks.rs; use correct subdir to check holders	2020-06-05 10:33:53 +02:00
Dietmar Maurer	4c1e8855cc	src/tools/disks.rs: fix disk type detection, remove newline from vendor string	2020-06-05 08:09:52 +02:00
Thomas Lamprecht	a720894ff0	rrd: fix off-by-one in save interval calculation Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-06-04 10:30:47 +02:00
Dominik Csapak	a95a3fb893	fix csum calculation of not 'chunk_size' aligned images the last chunk does not have to be as big as the chunk_size, just use the already available 'chunk_end' function which does the correct thing this fixes restoration of images whose sizes are not a multiple of 'chunk_size' as well Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-04 10:18:30 +02:00
Dietmar Maurer	620911b426	src/tools/disks/lvm.rs: implement get_lvm_devices()	2020-06-04 09:12:19 +02:00
Dietmar Maurer	5c264c8d80	src/tools/disks.rs: add/use get_partition_type_info	2020-06-04 07:48:22 +02:00
Dominik Csapak	8d78589969	improve display of 'next run' for sync jobs if the last sync job is too far in the past (or there was none at all for now) we run it at the next iteration, so we want to show that we now calculate the next_run by using either the real last endtime as time or 0 then in the frontend, we check if the next_run is < now and show 'pending' (we do it this way also for replication on pve) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-04 07:03:54 +02:00
Dominik Csapak	eed8a5ad79	tools/systemd/time: fix compute_next_event for weekdays two things were wrong here: * the range (x..y) does not include y, so the range (day_num+1..6) goes from (day_num+1) to 5 (but sunday is 6) * WeekDays.bits() does not return the 'day_num' of that day, but the bit value (e.g. 64 for SUNDAY) but was treated as the index of the day of the week to fix this, we drop the map to WeekDays and use the 'indices' directly this patch makes the test work again Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-04 07:02:33 +02:00
Dominik Csapak	538b9c1c27	systemd/time: add tests for all weekdays this fails for now Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-06-04 07:02:23 +02:00
Thomas Lamprecht	55919bf141	verify_file: add missing closing parenthesis in error message Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-06-03 19:10:01 +02:00
Dietmar Maurer	456ad0c478	src/tools/disks/zfs.rs: add parser for zpool list output	2020-06-03 12:16:08 +02:00
Dietmar Maurer	c48aa39f3b	src/bin/proxmox-backup-client.rs: implement quite flag	2020-06-03 10:11:37 +02:00
Thomas Lamprecht	2d32fe2c04	client restore: don't add server file ending if already specified If one executes a client command like # proxmox-backup-client files <snapshot> --repository ... the files shown have already the '.fidx' or '.blob' file ending, so if a user would just copy paste that one the client would always add .blob, and the server would not find that file. So avoid adding file endings if it is already a known OK one. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-06-03 07:03:55 +02:00
Thomas Lamprecht	dc155e9bd7	client restore: factor out archive/type parsing will be extended in a next patch. Also drop a dead else branch, can never get hit as we always add .blob as fallback Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-06-03 07:03:12 +02:00
Dietmar Maurer	4e14781aec	fix typo	2020-06-03 06:59:43 +02:00
Wolfgang Bumiller	a595f0fee0	client: improve connection/new fingerprint query Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-06-02 10:40:31 +02:00
Thomas Lamprecht	add5861e8d	typo fixes all over the place Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-05-30 16:39:08 +02:00
Dietmar Maurer	1610c45a86	src/client/pull.rs: also download client.log.blob	2020-05-30 14:51:33 +02:00
Dietmar Maurer	b2387eaa45	avoid compiler warnings	2020-05-30 14:05:33 +02:00
Dietmar Maurer	96d65fbcd0	cleanup: define/use const for predefined blob file names.	2020-05-30 14:04:15 +02:00
Dietmar Maurer	7cc3473a4e	src/client/backup_specification.rs: split code into extra file	2020-05-30 10:54:38 +02:00
Dietmar Maurer	4856a21836	src/client/pull.rs: more verbose logging	2020-05-30 08:12:43 +02:00
Dominik Csapak	04b0ca8b59	add owner to group and snapshot listings while touching it, make columns and tbar in DataStoreContent.js declarative members and remove the (now) unnecessary initComponent Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-30 07:24:12 +02:00
Dominik Csapak	d43f86f3f3	api2: add admin/sync endpoint this returns the list of syncjobs with status, as opposed to config/sync (which is just the config) also adds an api call where users can run the job manually under /admin/sync/$ID/run Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 11:31:32 +02:00
Dominik Csapak	997d7e19fc	config/sync: add SyncJobStatus Struct/Schema contains the config + status Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 11:29:39 +02:00
Dominik Csapak	c67b1fa72f	syncjob: change worker type for sync jobs 'sync' is used for manually pulling a remote datastore changing it for a scheduled sync to 'syncjob' so that we can differentiate between both types of syncs this also adds a seperate task description for it Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 11:28:04 +02:00
Dominik Csapak	268687ddf0	api2/pull: refactor priv checking and creating pull parameters we want to reuse those in the api call for manually running a sync job Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 11:27:43 +02:00
Dominik Csapak	426c1e353b	api2/config/sync: fix id parameter 'name' is not the correct parameter for get/post Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 11:24:54 +02:00
Dominik Csapak	2888b27f4c	create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 11:24:25 +02:00
Dietmar Maurer	934f5bb8ac	src/bin/proxmox-backup-proxy.rs: cleanup, move code to src/tools/disks.rs And simplify find_mounted_device by using stat.st_dev	2020-05-29 11:13:36 +02:00
Dominik Csapak	9857472211	fix removing of remotes we have to save the remote config after removing the section Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-29 10:48:26 +02:00
Dietmar Maurer	013fa7bbcb	rrd: reduce io by saving data only once a minute	2020-05-29 09:16:13 +02:00
Dietmar Maurer	a8d7033cb2	src/bin/proxmox-backup-proxy.rs: add test if last prune job is still running	2020-05-29 08:06:48 +02:00
Dietmar Maurer	04ad7bc436	src/bin/proxmox-backup-proxy.rs: test if last sync job is still running	2020-05-29 08:06:48 +02:00
Dietmar Maurer	77ebbefc1a	src/server/worker_task.rs: make worker_is_active_local pub	2020-05-29 08:06:48 +02:00
Dietmar Maurer	750252ba2f	src/tools/systemd/time.rs: add test for "daily" schedule	2020-05-29 07:52:09 +02:00
Dietmar Maurer	dc58194ebe	src/bin/proxmox-backup-proxy.rs: use correct id to lookup sync jobs	2020-05-29 07:50:59 +02:00
Thomas Lamprecht	090decbe76	BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names The repo URL consists of * optional userid * optional host * datastore name All three have defined regex or format, but none of that is used, so for example not all valid datastore names are accepted. Move definition of the regex over to api2::types where we can access all required regexes easily. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-05-29 06:29:23 +02:00
Thomas Lamprecht	c32186595e	api2::types: factor out USER_ID regex allows for better reuse in a next patch Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-05-29 06:27:38 +02:00
Dietmar Maurer	c94e1f655e	rrd stats: improve io delay stats	2020-05-28 19:12:13 +02:00
Dietmar Maurer	91e5bb49f5	src/bin/proxmox-backup-proxy.rs: simplify code and gather all stats for the root disk	2020-05-28 12:30:54 +02:00
Dietmar Maurer	547e3c2f6c	src/tools/disks/zfs.rs: use wtime + rtime (wait + run time)	2020-05-28 11:45:34 +02:00
Dietmar Maurer	25c550bc28	src/bin/proxmox-backup-proxy.rs: gather zpool io stats	2020-05-28 10:09:13 +02:00
Dietmar Maurer	0146133b4b	src/tools/disks/zfs.rs: helper to read zfs pool io stats	2020-05-28 10:07:52 +02:00
Dietmar Maurer	3eeba68785	depend on proxmox 0.1.38, use new fs helper functions	2020-05-28 10:06:44 +02:00
Dominik Csapak	f5056656b2	use the sync id for the scheduled sync worker task this way, multiple sync jobs with the same local store, can get scheduled Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-28 06:26:03 +02:00
Dominik Csapak	8c87743642	fix 'remove_vanished' cli arg again since the target side wants this to be a boolean and serde interprets a None Value as 'null' we have to only add this when it is really set via cli Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-28 06:25:30 +02:00
Dominik Csapak	05d755b282	fix inserting of worker tasks when starting a new task, we do two things to keep track of tasks (in that order): * updating the 'active' file with a list of tasks with 'update_active_workers' * updating the WORKER_TASK_LIST the second also updates the status of running tasks in the file by checking if it is still running by checking the WORKER_TASK_LIST since those two things are not locked, it can happend that we update the file, and before updating the WORKER_TASK_LIST, another thread calls update_active_workers and tries to get the status from the task log, which won't have any data yet so the status is 'unknown' (we do not update that status ever, likely for performance reasons, so we have to fix this here) by switching the order of the two operations, we make sure that only tasks reach the 'active' file which are inserted in the WORKER_TASK_LIST Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-28 06:24:42 +02:00
Dietmar Maurer	143b654550	src/tools.rs - command_output: add parameter to check exit code	2020-05-27 07:25:39 +02:00
Dietmar Maurer	97fab7aa11	src/tools.rs: new helper to handle command_output (std::process::Output)	2020-05-27 06:53:25 +02:00
Dominik Csapak	40dc103103	fix cli pull api call there is no 'delete' parameter, only 'remove-vanished', so fix that Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 12:39:19 +02:00
Dominik Csapak	de4db62c57	remotes: save passwords as base64 to avoid having arbitrary characters in the config (e.g. newlines) note that this breaks existings configs Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 12:38:06 +02:00
Dietmar Maurer	1a0d3d11d2	src/api2/admin/datastore.rs: add rrd api	2020-05-26 12:26:14 +02:00
Dietmar Maurer	8c03041a2c	src/bin/proxmox-backup-proxy.rs: gather block device stats on datastore	2020-05-26 11:20:59 +02:00
Dietmar Maurer	3fcc4b4e5c	src/tools/disks.rs: add helper to read block device stats	2020-05-26 11:20:22 +02:00
Dietmar Maurer	3ed07ed2cd	src/tools/disks.rs: export read_sys	2020-05-26 09:49:13 +02:00
Dominik Csapak	83fd4b3b1b	remote: try to use Struct for api with a catch: password is in the struct but we do not want it to return via the api, so we only 'serialize' it when the string is not empty (this can only happen when the format is not checked by us, iow. when its returned from the api) and setting it manually to "" when we return remotes from the api this way we can still use the type but do not return the password Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 08:55:07 +02:00
Dominik Csapak	5dcdcea293	api2/config/remote: remove password from read_remote Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 08:49:12 +02:00
Dominik Csapak	99f443c6ae	api2/config/remote: lock and use digest for removal Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 08:48:45 +02:00
Dominik Csapak	4f966d0592	api2/config/remote: use rpcenv for digest for read_remote Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 08:48:28 +02:00
Dominik Csapak	db0c228719	config/remote: add 'name' to Remote struct and use it as section id, like with User Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-26 08:48:05 +02:00
Dietmar Maurer	309ef20d6d	src/bin/proxmox-backup-proxy.rs: simplify code	2020-05-25 16:20:32 +02:00
Dietmar Maurer	d0833a70f7	src/bin/proxmox-backup-proxy.rs: gather datastore usage stats	2020-05-25 16:20:32 +02:00
Dietmar Maurer	d8dc281992	www/DataStoreStatus.js: display loadavg stats	2020-05-25 11:54:15 +02:00
Dietmar Maurer	2c66a590c0	src/bin/proxmox-backup-proxy.rs: gather iowait stats	2020-05-25 11:54:15 +02:00
Dietmar Maurer	485841da2c	src/bin/proxmox-backup-proxy.rs: gather loadavg stats	2020-05-25 11:40:20 +02:00
Dietmar Maurer	3e930f2bdc	www/DataStoreStatus.js: display root disk stats	2020-05-25 11:34:24 +02:00
Dietmar Maurer	dd15c0aa3b	src/bin/proxmox-backup-proxy.rs: gather root disk stats	2020-05-25 11:10:07 +02:00
Dietmar Maurer	c1b24fbf0b	www/DataStoreStatus.js: display swap stats	2020-05-25 10:39:54 +02:00
Dietmar Maurer	3f23b17298	src/rrd/rrd.rs: do not wrap error and return ErrorKind::NotFound	2020-05-25 10:30:04 +02:00
Dietmar Maurer	c25c9d8dd1	src/bin/proxmox-backup-proxy.rs: gather swap usage stats	2020-05-25 10:25:58 +02:00
Dietmar Maurer	84dc6adcc1	src/rrd/cache.rs: display/log error when RRD load fails	2020-05-25 10:18:53 +02:00
Dietmar Maurer	0c4344650d	src/rrd/rrd.rs: store/verify magic number	2020-05-25 09:21:54 +02:00
Dietmar Maurer	4f9513996c	src/bin/proxmox-backup-proxy.rs: use block_in_place for rrd update	2020-05-25 08:30:59 +02:00
Dietmar Maurer	736edc7a7e	src/rrd/rrd.rs: implement DST_COUNTER	2020-05-25 08:14:30 +02:00
Dietmar Maurer	2b55de407e	src/rrd/rrd.rs: correctly compute derived values use f64 for time.	2020-05-25 07:02:04 +02:00
Dietmar Maurer	a608806f65	www/DataStoreStatus.js: display netin/netout	2020-05-24 19:02:35 +02:00
Dietmar Maurer	8f0cec2642	src/bin/proxmox-backup-proxy.rs: gather netin/netout stats	2020-05-24 19:02:35 +02:00
Dietmar Maurer	0ed9a2b3ae	src/config/network.rs: implement is_physical_nic() helper	2020-05-24 19:02:35 +02:00
Dietmar Maurer	58edd33d2b	src/rrd/rrd.rs: implement DST_DERIVE	2020-05-24 19:02:35 +02:00
Dietmar Maurer	4fb05fde17	src/rrd/rrd.rs: restructure whole code	2020-05-24 16:51:28 +02:00
Dietmar Maurer	daca4f7888	src/rrd/rrd.rs: reduce size by using f64:NAN as UNKNOWN	2020-05-24 09:09:09 +02:00
Dietmar Maurer	4e6585839b	src/rrd/rrd.rs: simplify an fix old value deletion	2020-05-24 06:44:06 +02:00
Dietmar Maurer	8c981ae379	rrd: fix display interval, try to avoid numeric errors	2020-05-23 16:03:43 +02:00
Dietmar Maurer	803ab12ad4	rrd: simplify code	2020-05-23 15:37:17 +02:00
Dietmar Maurer	a4a3f7ca5e	rrd: pack multiple rrd values into th estat list	2020-05-23 14:03:44 +02:00
Dietmar Maurer	a2f862eed6	add experimental rrd api to get cpu stats	2020-05-23 11:50:53 +02:00
Dietmar Maurer	eaeda365e0	start gathering stats using new rrd module	2020-05-23 10:43:08 +02:00
Dietmar Maurer	6359dc891a	add simple rrd implementation	2020-05-23 10:42:48 +02:00
Dietmar Maurer	07ad6470ca	src/client/pull.rs: split out pull related code	2020-05-22 08:04:20 +02:00
Dietmar Maurer	a6160cdfeb	src/bin/proxmox-backup-proxy.rs: schedule sync jobs	2020-05-22 07:50:59 +02:00
Dietmar Maurer	183125d576	src/api2/pull.rs: aquire try_shared_chunk_store_lock inside pull_store	2020-05-22 07:24:17 +02:00
Dietmar Maurer	a3016d6583	proxmox-backup-manager: add sync-job cli	2020-05-21 11:44:45 +02:00
Dietmar Maurer	b29d046e89	proxmox-backup-manager: split out cert.rs	2020-05-21 11:22:20 +02:00
Dietmar Maurer	380bd7df97	proxmox-backup-manager: split out datastore.rs	2020-05-21 11:14:34 +02:00
Dietmar Maurer	ea6f404e55	proxmox-backup-manager: split out dns.rs	2020-05-21 11:10:58 +02:00
Dietmar Maurer	a35a211d9e	proxmox-backup-manager: split out network.rs	2020-05-21 11:08:38 +02:00
Dietmar Maurer	53e14507c1	proxmox-backup-manager: split out acl.rs	2020-05-21 10:56:46 +02:00
Dietmar Maurer	6fa39e53e0	proxmox-backup-manager: split out users.rs	2020-05-21 10:53:06 +02:00
Dietmar Maurer	a220a4564a	roxmox-backup-manager: start splitting command into several files	2020-05-21 10:46:07 +02:00
Dietmar Maurer	6f652b1b3a	rename 'job' to 'sync'	2020-05-21 10:29:25 +02:00
Dietmar Maurer	b1d4edc769	src/api2/config/job.rs: add job api	2020-05-21 10:16:35 +02:00
Dietmar Maurer	b4900286ce	src/config/jobs.rs: use SectionConfig for jobs	2020-05-21 10:16:35 +02:00
Dietmar Maurer	c681885227	src/bin/proxmox-backup-manager.rs: format output of show commands	2020-05-20 16:47:37 +02:00
Dietmar Maurer	ee8b464466	src/tools/systemd.rs: avoid compiler warnings	2020-05-20 16:47:08 +02:00
Dominik Csapak	ce55db66d6	proxmox-backup-manager: add show command for remote and datastore to show the data for a single item Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 16:33:07 +02:00
Dominik Csapak	2882c881e9	api2/access/acl: add path and exact parameter to list_acl so that we can get only a subset of the acls, filtered by the backed also return the digest here Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 13:44:36 +02:00
Dominik Csapak	12e3895399	api2/access/acl: make update_acl a protected api call since we want to set the owner of the acl config to 'root' which is only possible when using a protected api call Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 13:22:41 +02:00
Dominik Csapak	11b6391c83	add 'exact' parameter to extract_acl_node_data so that we can return acls for a single path Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 13:22:10 +02:00
Dominik Csapak	b05672579e	api2/roles: change return field of role to roleid to be compatible with the pve api with this, we can reuse the ui parts (RoleSelector) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 13:21:47 +02:00
Dominik Csapak	5160c0e986	api2/acl: add privs array to roles so that an admin can see which roles have which privileges Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 13:21:37 +02:00
Dominik Csapak	1ad9dd08f4	acls: use constnamemap macro for privileges Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 13:21:28 +02:00
Dietmar Maurer	25829a879b	src/bin/proxmox-backup-proxy.rs: schedule prune jobs	2020-05-20 13:00:53 +02:00
Dietmar Maurer	872062ee9f	src/config/datastore.rs_ change prune types from i64 to u64	2020-05-20 13:00:13 +02:00
Dietmar Maurer	67f7ffd0db	src/config/datastore.rs: add prune settings	2020-05-20 11:29:59 +02:00
Dietmar Maurer	0fafac2492	src/api2/access/user.rs: remove useless description The description is not used at all if we refer to a type.	2020-05-20 11:27:58 +02:00
Dietmar Maurer	49ff10921c	src/api2/types.rs: define PRUNE_SCHEMA_KEEP_*	2020-05-20 10:13:38 +02:00
Dietmar Maurer	479e4932b5	src/tools/systemd/parse_time.rs: improve error message	2020-05-20 09:43:16 +02:00
Dietmar Maurer	dd7a7eae8f	src/bin/proxmox-backup-manager.rs: add completion helper for gc-schedule	2020-05-20 09:42:51 +02:00
Dietmar Maurer	8545480a31	src/bin/proxmox-backup-proxy.rs: add simple task scheduler for garbage collection	2020-05-20 08:59:45 +02:00
Dietmar Maurer	d6c28ddf84	src/tools/systemd/time.rs: export parse/verify	2020-05-20 08:38:39 +02:00
Dietmar Maurer	42fdbe5112	src/config/datastore.rs: add gc-schedule property	2020-05-20 08:38:10 +02:00
Dominik Csapak	9c5c383bff	user: create default root user as typed struct we added a userid attribute to the User struct, but missed that we created the default user without that attribuet via the json! macro which lead to a runtime panic on the deserialization by using the struct directly, such errors will be caught by the compiler in the future with this change, we can remove the serde_json import here Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-20 06:09:08 +02:00
Dietmar Maurer	7d4e362993	depend on proxmox 0.1.32, src/api2/access/user.rs: simplify code	2020-05-19 12:58:46 +02:00
Dominik Csapak	522c0da0a0	use new 'id_property' for user::User and use it in api calls this allows us to return a user::User (or Vec<> of it) instead of a generic serde value Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-19 09:33:56 +02:00
Dominik Csapak	16c75c580b	adapt to changes of SectionConfigPlugin it requires not an Option<String> for the optional id_property Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-05-19 09:28:45 +02:00
Dietmar Maurer	07ce44a633	avoid compiler warnings	2020-05-19 07:03:41 +02:00
Dietmar Maurer	e8d1da6a15	depend on proxmox 0.1.31 - use Value to store result metadata	2020-05-18 09:57:35 +02:00
Dietmar Maurer	7a314d18f7	src/tools/systemd/parse_time.rs: check max values	2020-05-16 13:13:50 +02:00
Dietmar Maurer	2d08c97ae2	CalendarEvent - compute_next_value: use change tracking to avoid repeated testing	2020-05-16 10:32:27 +02:00
Dietmar Maurer	50ce1f987d	CalendarEvent - compute_next_value: support seconds	2020-05-16 10:21:24 +02:00
Dietmar Maurer	d1a5ffdf78	src/tools/systemd/tm_editor.rs: new helper class	2020-05-16 10:09:41 +02:00
Dietmar Maurer	99baf7afcc	CalendarEvent: test and fix repeated values	2020-05-16 07:43:51 +02:00
Dietmar Maurer	fed270bf3f	CalendarEvent: speedup/simplify repetition tests	2020-05-16 07:09:53 +02:00
Dietmar Maurer	e05b637c73	src/tools/systemd/parse_time.rs: move parser into separate file	2020-05-16 06:53:15 +02:00
Dietmar Maurer	2ee6b3fdb9	src/tools/systemd/time.rs: implement compute_next_event	2020-05-16 06:33:03 +02:00
Dietmar Maurer	f3a96b2cdb	renamed: src/tools/systemd/parser.rs -> src/tools/systemd/config.rs	2020-05-16 06:32:28 +02:00
Dietmar Maurer	a260c74a12	src/tools/systemd/time.rs: add helpers to compute CalendarEvents	2020-05-15 17:55:54 +02:00
Dietmar Maurer	30f577248b	src/api2/node/time.rs: avoid custom unsafe readlink implementations	2020-05-15 06:50:07 +02:00
Dietmar Maurer	00491c0230	src/tools/systemd/parser.rs: use different setups for service and timer files, code cleanup	2020-05-14 13:55:13 +02:00
Dietmar Maurer	2ebdbac1c4	depend on nom, add parser for systemd calendar enents and time span	2020-05-14 12:18:30 +02:00
Dietmar Maurer	f486e9e50e	add systemd configuration file parser/writer, start job configuration	2020-05-12 13:07:49 +02:00
Dietmar Maurer	65dab0266c	proxmox-backup-manager: add completion helper for port list	2020-05-08 17:28:04 +02:00
Dietmar Maurer	525008f7ad	proxmox-backup-manager - network list: render ports/slaves And render interface name as first comumn.	2020-05-08 16:07:23 +02:00
Dietmar Maurer	5bef0f43da	src/config/network.rs - check_bridge_ports: correctly check vlan ports	2020-05-08 15:51:47 +02:00
Dietmar Maurer	0f6bdbb01f	src/config/network.rs - write_config: add more consistency checks	2020-05-08 14:31:38 +02:00
Dietmar Maurer	a4ccb46176	src/config/network.rs: avoid duplicate port usage	2020-05-08 11:15:00 +02:00
Dietmar Maurer	80bf084876	src/config/network.rs: do not combine entries It is unclear when and how to write combined entries ...	2020-05-08 10:20:57 +02:00
Dietmar Maurer	db5672e83e	src/config/network.rs: always write bridge_ports and bond_slaves So that we can reliable detect the interface type.	2020-05-08 09:58:03 +02:00
Dietmar Maurer	86a5d56c4e	proxmox-backup-manager: add network create command	2020-05-08 09:55:56 +02:00
Dietmar Maurer	3dd27a3bf8	src/api2/node/network.rs: add protected flag to revert	2020-05-08 09:30:25 +02:00
Dietmar Maurer	3aedb73816	src/api2/node/network.rs: pass bridge_ports and slaves a property strings To make it compatible with pve.	2020-05-08 08:49:17 +02:00
Dietmar Maurer	bab5d18c3d	src/config/network.rs: implement bond_mode and rename bond_slaves to slaves to make it compatible with pve.	2020-05-07 14:07:45 +02:00
Dietmar Maurer	c2ffc68554	src/api2/node/network.rs: cleanup - factor out check_duplicate_gateway	2020-05-07 11:26:30 +02:00
Dietmar Maurer	9651833130	src/api2/node/network.rs: allow to create bridge and bond	2020-05-07 11:09:12 +02:00
Dietmar Maurer	7b22acd0c2	src/config/network.rs: make it compatible with pve and depend on proxmox 0.1.26	2020-05-07 09:28:25 +02:00
Dietmar Maurer	5751e49566	src/server/worker_task.rs: implement and use status command	2020-05-07 09:27:33 +02:00
Dietmar Maurer	197de83ffa	src/server/command_socket.rs: do not abort loop on client errors, allow backup gid	2020-05-07 09:27:33 +02:00
Wolfgang Bumiller	10effc9849	add tools/disks.rs (work in progress...) Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-05-05 10:14:42 +02:00
Dietmar Maurer	99641a6bbb	garbage_collect: call fail_on_abort to abort GV when requested.	2020-05-05 09:06:34 +02:00
Dietmar Maurer	74f7240b8d	src/bin/proxmox-backup-client.rs: add human readable date to prune list	2020-05-05 07:33:58 +02:00
Dietmar Maurer	db1e061dcb	src/bin/proxmox-backup-client.rs: correctly format prune result list.	2020-05-05 06:45:37 +02:00
Dietmar Maurer	b080583ba8	src/bin/proxmox-backup-manager.rs: improve user list output	2020-05-01 16:22:50 +02:00
Dietmar Maurer	ed751dc2ab	src/api2/node/status.rs: rework api, implement reboot and shutdown	2020-04-30 11:52:40 +02:00
Dietmar Maurer	ca9dfe5fa4	src/api2/node/tasks.rs: use api macro features for default values	2020-04-30 11:51:56 +02:00
Dietmar Maurer	720af9f69b	src/api2/node/tasks.rs: allow users to list/access there own tasks	2020-04-30 10:05:50 +02:00
Dietmar Maurer	f1490da82a	use resonable acl paths (fixup)	2020-04-30 09:32:13 +02:00
Dietmar Maurer	74c08a5782	use reasonable acl paths	2020-04-30 09:30:00 +02:00
Dietmar Maurer	bd88dc4116	cached_config: avoid parsing non-existent files multiple times	2020-04-30 07:04:23 +02:00
Dominik Csapak	f9e3b1104e	change index to templates using handlebars using a handlebars instance in ApiConfig, to cache the templates as long as possible, this is currently ok, as the index template can only change when the whole package changes if we split this in the future, we have to trigger a reload of the daemon on gui package upgrade (so that the template gets reloaded) Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-04-29 17:05:53 +02:00
Dietmar Maurer	bc0d03885c	use proxmox 0.1.25, use new EnumEntry feature	2020-04-29 13:01:24 +02:00
Dietmar Maurer	b9f2f761bb	avoid problems with missing acl.cfg and user.cfg	2020-04-29 10:40:42 +02:00
Dietmar Maurer	30fb602578	src/api2/admin/datastore.rs - get_datastore_list: only return name and comment We dont want to leak the full configuration to users with limited access permission. Please use the api2::config::datastore api to get the full configuration.	2020-04-29 09:21:34 +02:00
Dietmar Maurer	0a00f6e01c	src/api2/config/datastore.rs_ add delete property to update method	2020-04-29 09:09:59 +02:00
Dietmar Maurer	30003baaa4	src/api2/config/remote.rs: fix white space	2020-04-29 09:09:39 +02:00
Dietmar Maurer	5211705ff1	src/api2/config/remote.rs: add delete parameter to update method	2020-04-29 09:04:17 +02:00
Dietmar Maurer	ec67af9af3	src/api2/pull.rs: require Datastore.Prune if delete flag is set.	2020-04-29 07:19:32 +02:00
Dietmar Maurer	8247db5b39	src/config/acl.rs: introduice privileges and roles for remotes	2020-04-29 07:03:44 +02:00
Dietmar Maurer	409f44247b	fix api2::types::ACL_ROLE_SCHEMA make sure we list all roles ...	2020-04-28 13:25:02 +02:00
Dietmar Maurer	dd335b77f5	src/config/acl.rs - fix regression tests	2020-04-28 11:16:15 +02:00
Dietmar Maurer	6f6aa95abb	add Datastore.Backup, Datastore.PowerUser and Datastore.Reader role	2020-04-28 11:07:25 +02:00
Dietmar Maurer	54552dda59	implemnt backup ownership, improve datastore access permissions	2020-04-28 10:22:25 +02:00
Dietmar Maurer	1347b1152d	src/config/cached_user_info.rs - lookup_privs: correctly handle superuser	2020-04-27 13:22:03 +02:00
Dietmar Maurer	d00e1a216f	src/config/acl.rs: introduce more/better datastore privileges	2020-04-27 07:13:50 +02:00
Dietmar Maurer	9c7fe29dfc	src/config/acl.rs: rtename PRTIV_DATASTORE_ALLOCATE to PRIV_DATASTORE_MODIFY	2020-04-27 06:50:35 +02:00
Dietmar Maurer	14627d671a	src/bin/proxmox-backup-manager.rs: add dns sub command Also improved the DNS api, added a --delete option.	2020-04-26 08:23:23 +02:00
Dietmar Maurer	76227a6acd	src/bin/proxmox-backup-manager.rs: fix node parameter handling	2020-04-25 17:20:22 +02:00
Dietmar Maurer	26d9aebc28	move src/api2/config/network.rs to src/api2/node/network.rs So that we have the same api path for network config as pve.	2020-04-25 17:00:38 +02:00
Dietmar Maurer	1ca540a63b	src/config/network.rs: auto-add lo, and implement a few regression tests	2020-04-24 12:57:11 +02:00
Wolfgang Bumiller	9094186a57	xattr: cleanup: don't use pxar types in the API Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 11:23:48 +02:00
Wolfgang Bumiller	27a3decbfe	xattr: api cleanup Make `flistxattr()` return a `ListXAttr` helper which provides an iterator over `&CStr`. This exposes the property that xattr names are a zero-terminated string without simply being an opaque "byte vector". Using &[u8] as a type here is too lax. Also let `fgetxattr` take a `CStr`. While this may be a burden on the caller, we usually already have zero-terminated strings on the call site. Currently we only use this method coming from `flistxattr` after all. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 10:56:52 +02:00
Wolfgang Bumiller	9af76ef075	xattr: use checked_mul to increase size Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 10:56:52 +02:00
Wolfgang Bumiller	00ec8d1685	tools: pub use Fd from proxmox crate Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 10:56:52 +02:00
Dietmar Maurer	fd7c0979b4	src/bin/proxmox-backup-manager.rs: implement netwerk revert	2020-04-24 10:45:49 +02:00
Dietmar Maurer	c67bc9c35c	src/bin/proxmox-backup-manager.rs: new command to show pending network changes	2020-04-24 10:27:43 +02:00
Dietmar Maurer	3181f9b625	src/bin/proxmox-backup-manager.rs: only show pending changes with "text" format	2020-04-24 10:16:57 +02:00
Dietmar Maurer	2eefd9aee1	src/config/network.rs: implement network reload, set "changes" attribute	2020-04-24 09:55:46 +02:00
Dietmar Maurer	8a6b86b8a7	src/config/network.rs: use a simple String for comments	2020-04-24 07:46:08 +02:00
Dietmar Maurer	96d9478668	src/config/network/parser.rs: corectly detect vanished interfaces	2020-04-24 07:26:54 +02:00
Dietmar Maurer	10a9be45bd	src/api2/config/network.rs: implement update/delete comments	2020-04-23 16:08:35 +02:00
Dietmar Maurer	5f60a58fd5	src/config/network.rs; support interface comments, cleanups	2020-04-23 15:54:30 +02:00
Dietmar Maurer	659c3be3d5	src/config/network.rs: avoid newline after family options	2020-04-23 11:30:41 +02:00
Dietmar Maurer	5e4e88e83f	src/api2/config/network.rs: implement update/delete for bridge_ports and bond_slaves	2020-04-23 11:21:27 +02:00
Dietmar Maurer	339965d720	src/api2/config/network.rs: only allow one default gateway	2020-04-23 10:37:40 +02:00
Dietmar Maurer	c38b4bb8b2	src/config/network.rs: do not allow to change interface type	2020-04-23 09:43:38 +02:00
Dietmar Maurer	42fbe91a34	src/config/network.rs: parse bond-slaves	2020-04-23 09:31:10 +02:00
Dietmar Maurer	1d9a68c2fc	src/config/network.rs: parse bridge-ports	2020-04-23 09:24:17 +02:00
Dietmar Maurer	02269f3dba	src/config/network.rs: introduce NetworkInterfaceType	2020-04-23 08:45:03 +02:00
Dietmar Maurer	d5ca9bd5df	src/config/network.rs: cleanup (new helper combine_entry)	2020-04-23 07:54:12 +02:00
Dietmar Maurer	02e36d96ad	src/config/network.rs: write changes to interfaces.new	2020-04-23 07:19:29 +02:00
Dietmar Maurer	2c18efd902	src/config/network.rs: use a single mtu setting (instead of mtu_v4 and mtu_v6)	2020-04-23 07:07:14 +02:00
Dietmar Maurer	4cb6bd894c	src/bin/proxmox-backup-manager.rs: improve network list output format	2020-04-23 06:44:55 +02:00
Dietmar Maurer	b1564af25a	src/bin/proxmox-backup-manager.rs: format datastore list output	2020-04-22 17:37:20 +02:00
Dietmar Maurer	bf004ecd87	src/bin/proxmox-backup-manager.rs: format network list output	2020-04-22 17:14:52 +02:00
Dietmar Maurer	f1026a5aa9	src/api2/config/network.rs: allow to update 'auto' flag	2020-04-22 16:46:46 +02:00
Dietmar Maurer	3fce3bc36e	src/config/network/parser.rs: parse MTU settings	2020-04-22 13:44:51 +02:00
Dietmar Maurer	f8e7ac686a	src/config/network.rs: only save attriubutes used by configuration method	2020-04-22 12:42:09 +02:00
Dietmar Maurer	c016482c7a	src/api2/config/network.rs: implement delete property	2020-04-22 12:19:31 +02:00
Dietmar Maurer	27f2c23049	src/api2/config/network.rs: allow to update configuration method	2020-04-22 11:32:36 +02:00
Dietmar Maurer	df6bb03d0e	src/api2/config/network.rs: improve network api	2020-04-22 10:54:07 +02:00
Dietmar Maurer	e2d940b949	src/config/network/parser.rs: remove debug println	2020-04-22 10:53:26 +02:00
Dietmar Maurer	0c226bc173	src/config/network/helper.rs: fix CIDR regex	2020-04-22 10:52:31 +02:00
Dietmar Maurer	76cf5208cf	src/api2/types.rs: add schemas for IP/CIDR	2020-04-22 10:28:53 +02:00
Dietmar Maurer	2ea7bf1b3d	src/api2/config/datastore.rs_ fix method docs	2020-04-22 08:53:16 +02:00
Dietmar Maurer	8b57cd4441	src/config/network.rs: remove netmask support rely on cidr instead.	2020-04-22 08:45:13 +02:00
Dietmar Maurer	68da20bf62	src/api2/types.rs: define NETWORK_INTERFACE_NAME_SCHEMA	2020-04-21 17:54:52 +02:00
Dietmar Maurer	c357260d09	src/config/network.rs: move type definitions to src/api2/types.rs	2020-04-21 17:25:05 +02:00
Dietmar Maurer	7e02d08cd0	rename ConfigMethod to NetworkConfigMethod	2020-04-21 17:17:57 +02:00
Dietmar Maurer	ca0e534796	src/api2/config/network.rs: start network configuration api	2020-04-21 14:28:26 +02:00
Dietmar Maurer	904e988667	src/config/network.rs: impleement load/save	2020-04-21 12:55:33 +02:00
Dietmar Maurer	3f129233be	src/config/network.rs: add Interface flags 'exists' and 'active'	2020-04-21 11:46:56 +02:00
Dietmar Maurer	a9bb491e35	src/config/network.rs: cleanup autostart flag handling	2020-04-21 11:06:22 +02:00
Dietmar Maurer	1ec7f8a0dd	src/config/network/helper.rs: new helper get_network_interfaces()	2020-04-21 10:32:54 +02:00
Dietmar Maurer	92310d585c	src/config/network.rs: simplify code	2020-04-20 18:10:15 +02:00
Dietmar Maurer	f34d4401f7	src/config/network.rs: read/write /etc/network/interfaces Start implementing a recursive descent parser.	2020-04-20 14:15:57 +02:00
Dietmar Maurer	6e695960ca	src/config/cached_user_info.rs: cache it up to 5 seconds	2020-04-18 08:49:20 +02:00
Dietmar Maurer	365f0f720c	fix permission tests using non-uri parameters We nood to do those tests inside the fuction body instead...	2020-04-18 08:23:04 +02:00
Dietmar Maurer	a737179eb4	src/config/cached_user_info.rs: new check_privs helper	2020-04-18 08:09:34 +02:00
Dietmar Maurer	bb072ba49c	src/api2/access.rs: cleanup	2020-04-18 07:28:25 +02:00
Dietmar Maurer	ff329f970b	src/api2/types.rs: use anyhow::Error in test cases	2020-04-18 07:05:31 +02:00
Wolfgang Bumiller	f7d4e4b506	switch from failure to anyhow Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-17 18:43:30 +02:00
Dietmar Maurer	404d78c41e	src/api2/pull.rs: add access permission	2020-04-17 15:27:04 +02:00
Dietmar Maurer	1bfc1efa50	src/api2/subscription.rs: add access permissions	2020-04-17 15:14:28 +02:00
Dietmar Maurer	73ce1d1146	src/api2/reader.rs: add access permissions	2020-04-17 15:01:56 +02:00
Dietmar Maurer	70e5f2461d	src/api2/config/remote.rs: add access permissions	2020-04-17 14:57:26 +02:00
Dietmar Maurer	c0ef209aeb	src/api2/config/datastore.rs: impl digest check for delete, add access permissions	2020-04-17 14:51:29 +02:00
Dietmar Maurer	9f9f7eefa3	src/api2/backup.rs: add access permissions	2020-04-17 14:40:20 +02:00
Dietmar Maurer	bb34b58910	src/api2/admin/datastore.rs: add access permissions - first try We need to refine this later (introduce backup owner concept?)	2020-04-17 14:36:27 +02:00
Dietmar Maurer	5972def5ec	acl: change path "storage" to "datastore"	2020-04-17 14:15:44 +02:00
Dietmar Maurer	aa90ced3bf	src/api2/access/role.rs: use schema ACL_ROLE_SCHEMA	2020-04-17 14:14:06 +02:00
Dietmar Maurer	ca257c8097	move type defs from src/api2/access/acl.rs to src/api2/types.rs	2020-04-17 14:13:15 +02:00
Dietmar Maurer	3fff55b293	src/api2/access/role.rs: new api to list roles	2020-04-17 14:03:24 +02:00
Dietmar Maurer	4f66423fcc	src/api2/access/user.rs: add access permissions	2020-04-17 11:04:36 +02:00
Dietmar Maurer	d4f020f4c5	src/api2/access/user.rs: add access permissions	2020-04-17 10:08:45 +02:00
Dietmar Maurer	d28ddb8e04	src/api2/access/acl.rs: add access permissions	2020-04-17 10:03:09 +02:00
Dietmar Maurer	83b6a7cf71	src/api2/node/tasks.rs: use api macro, implement access permissions	2020-04-16 17:47:21 +02:00
Dietmar Maurer	e4681f9f71	src/api2/node/syslog.rs: add access permissions	2020-04-16 17:08:19 +02:00
Dietmar Maurer	b5037fa8ed	src/api2/node/status.rs: add access permissions	2020-04-16 17:05:09 +02:00
Dietmar Maurer	9989d2c4e9	src/server/rest.rs: reduce delay for permission error to 500ms	2020-04-16 12:56:34 +02:00
Dietmar Maurer	1cf7bbf412	src/api2/node/services.rs: add access permissions	2020-04-16 12:47:16 +02:00
Dietmar Maurer	68ed0c629d	src/api2/node/journal.rs: add access permissions	2020-04-16 12:47:16 +02:00
Dietmar Maurer	4b40148caa	start impl. access permissions	2020-04-16 12:47:16 +02:00
Dietmar Maurer	423e656163	src/config/cached_user_info.rs: new helper class	2020-04-16 10:05:16 +02:00
Dietmar Maurer	109d7817cd	src/config/user.rs - cached_config: do not store/return digest	2020-04-15 11:35:57 +02:00
Dietmar Maurer	5354511fd0	src/config/acl.rs: implement cached_config	2020-04-15 11:30:47 +02:00
Dietmar Maurer	bd098a7f77	src/api2/node/dns.rs: use api macro (cleanup)	2020-04-15 10:09:18 +02:00
Dietmar Maurer	8d048af2bf	acl: improve NoAccess handling	2020-04-15 08:11:43 +02:00
Dietmar Maurer	9a328319dd	pxar extract: remove pattern from arg_param, add target instead	2020-04-15 06:41:37 +02:00
Dietmar Maurer	7e3d2e5b41	pxar create: remove exclude from arg_param	2020-04-15 06:31:46 +02:00
Dietmar Maurer	9c06f6c292	fix previous commit - use result.	2020-04-14 17:48:10 +02:00
Dietmar Maurer	9f4e47dd93	acl update: check path	2020-04-14 17:23:48 +02:00
Dietmar Maurer	d83175dd69	acl update: check if user exist.	2020-04-14 13:46:27 +02:00
Dietmar Maurer	68ccdf09a4	src/config/user.rs: implement user config cache	2020-04-14 13:45:45 +02:00
Dietmar Maurer	9765092ede	acl api: implement update	2020-04-14 10:16:49 +02:00
Dietmar Maurer	ed3e60ae69	start ACL api	2020-04-13 11:09:44 +02:00
Dietmar Maurer	a83eab3c4d	acl: use BTreeMap and BTreeSet to avoid sort()	2020-04-12 17:13:53 +02:00
Dietmar Maurer	0815ec7e65	acl: implement roles(), add regression tests.	2020-04-12 13:06:50 +02:00
Dietmar Maurer	5c6cdf9815	add acl config	2020-04-11 12:24:26 +02:00
Dietmar Maurer	b88f9c5b1e	PASSWORD_SCHEMA: set max_length to 1024 (for tickets)	2020-04-09 13:35:58 +02:00
Dietmar Maurer	879546aff6	api: add default property to domain list	2020-04-09 13:35:08 +02:00
Dietmar Maurer	73b40e9b46	api: correctly sort access subdirmap	2020-04-09 13:34:07 +02:00
Dietmar Maurer	708db4b3ae	api: add list_domains	2020-04-09 11:36:45 +02:00
Dietmar Maurer	685e13347e	api: move config/user to access/users, implement change_password To make it similar to the pve api	2020-04-09 10:21:24 +02:00
Dietmar Maurer	7d817b0358	implement auth framework	2020-04-08 14:06:15 +02:00
Dietmar Maurer	579728c641	add user configiguration	2020-04-08 14:06:15 +02:00
Dietmar Maurer	cf459b1982	gc: log pending removals	2020-04-06 09:50:40 +02:00
Dietmar Maurer	dda7015497	prune api: return a usable result (we run synchronous anyways)	2020-04-01 12:24:28 +02:00
Dietmar Maurer	4299ca727c	src/server/rest.rs: use correct formatter	2020-03-26 12:54:20 +01:00
Dominik Csapak	6650a242fb	rewrite future select in upgrade_to_backup_protocol using select macro and handle all ok/err cases with at least logging Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-03-18 11:33:59 +01:00
Dietmar Maurer	ca26db1140	delete unused src/storage/config.rs	2020-03-02 13:31:38 +01:00
Dietmar Maurer	90c5239d46	use SectionConfig from proxmox 0.1.18-1	2020-03-02 12:52:11 +01:00
Dietmar Maurer	18deda403d	src/bin/proxmox-backup-client.rs - list_groups: render full snapshot path for last-backup column	2020-02-28 09:09:22 +01:00
Dietmar Maurer	6882b3716c	src/bin/proxmox-backup-manager.rs: avoid compiler warning	2020-02-28 08:52:12 +01:00
Christian Ebner	32d192a952	catalog: shell: Use the new logic including resolving symlinks for catalog and remove the old unused code. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:51:01 +01:00
Christian Ebner	fee5528e59	catalog: shell: introduce new CatalogPathStack to navigate in catalog shell. This is basically a rewrite of the current logic for navigating the catalog, but in addition allows to follow symlinks. Following symlinks introduces the issue that generation of canonical paths (needed in the actual pxar archive) is more complex, as symlinks have to be resolved and loops avoided. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:50:43 +01:00
Christian Ebner	4145c36749	catalog: add missing function documentation. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:50:08 +01:00
Christian Ebner	c2f9149461	catalog: introduce is_symlink() to check if DirEntry is a symlink. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:49:55 +01:00
Christian Ebner	11ee5c0563	catalog: derive PartEq for equality checks of two DirEntry's. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:49:39 +01:00
Christian Ebner	536683e73b	src/backup/dynamic_index.rs: Add LruCache for chunks. In order to improve non-sequential reads of chunks as e.g. in FUSE. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:47:16 +01:00
Christian Ebner	35ddf0b419	catalog: shell: Introduce clear-selected command. 'clear-selected' allows to clear all the match patterns from the list of patterns for a subsequent restore. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:40:55 +01:00
Christian Ebner	8e464141cf	catalog: shell: Improve list-selected command. 'list-selected' now shows the filenames matching the patterns for a restore instead of the patterns themselfs. The patterns can be displayed by passing the '--pattern' flag. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:40:42 +01:00
Christian Ebner	351b913d1e	pxar::decoder: Return the correct symlink target size. This eliminates also repeated calls to readlink in fuse, which occur when the preallocated buffer to store the symlink target path is to small. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:38:56 +01:00
Christian Ebner	189996cf4a	proxmox-backup-client: expose exclude match patterns to cli. Allows to pass exclude match patterns to the 'backup' command. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 07:56:09 +01:00
Dietmar Maurer	ac3faaf5c0	src/bin/proxmox-backup-manager.rs: cleanup output handling	2020-02-28 07:42:36 +01:00
Dietmar Maurer	4939255fb4	src/tools/format.rs: move output rendering code to this new file	2020-02-28 07:30:35 +01:00
Dietmar Maurer	c81b2b7c70	src/bin/proxmox-backup-client.rs - list_groups: use format_and_print_result_full() Depend on proxmox 0.1.16-1	2020-02-27 13:32:11 +01:00
Dietmar Maurer	c2043614f7	src/bin/proxmox-backup-client.rs: use new output formath helpers from proxmox 0.15.1-1	2020-02-27 12:41:15 +01:00
Dietmar Maurer	e23f586344	src/bin/proxmox-backup-client.rs - status: improve alignment	2020-02-27 11:38:09 +01:00
Dietmar Maurer	be2425ff85	src/bin/proxmox-backup-client.rs - status: do not print header	2020-02-27 11:31:05 +01:00
Dietmar Maurer	f24fc1166b	src/bin/proxmox-backup-client.rs - list_snapshots: use format_and_print_result_full() Depend on proxmox v0.1.14.	2020-02-27 11:28:44 +01:00
Christian Ebner	af934f8cf6	tools::lru_cache: Improve access() and insert() by using HashMap::entry(). entry() allows to lookup the position where and entry belongs and update/insert it in the HashMap more efficiently than get_mut() and insert(). Details: https://gankra.github.io/blah/hashbrown-insert/ In addition, use the struct LinkedList and remove the outdated code. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:56:25 +01:00
Christian Ebner	e3ab9a383c	tools::lru_cache: Separate LinkedList part of LruCache into own struct. In order to keep the separation more aparent and avoid mut borrow conflics. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:55:48 +01:00
Christian Ebner	75c2ee7bab	tools::lru_cache: Make key generic. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:55:16 +01:00
Christian Ebner	03f779c6f5	catalog: shell: Improve output of stat command. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:53:17 +01:00
Dietmar Maurer	ea5f547fdc	src/bin/proxmox-backup-client.rs - list_snapshot_files: use format_and_print_result_full()	2020-02-26 13:49:47 +01:00
Dietmar Maurer	390c5bdde4	src/bin/proxmox-backup-client.rs - status: use format_and_print_result_full	2020-02-26 13:29:00 +01:00
Dietmar Maurer	03ac286c80	src/bin/proxmox-backup-manager.rs: use crate::tools::runtime::block_on()	2020-02-26 11:37:38 +01:00
Dietmar Maurer	dd04383bb8	src/tools/runtime.rs: call OPENSSL_thread_stop to avoid race with openssl cleanup handlers	2020-02-26 10:38:55 +01:00
Dietmar Maurer	ca2dbb8af1	src/client/backup_writer.rs: also print archive size	2020-02-25 14:12:29 +01:00
Wolfgang Bumiller	0bcbb5434e	cleanup unused module Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-25 11:38:27 +01:00
Christian Ebner	495b238bc1	pxar: Change text of warning for files matched by exclude patterns. In addition to the .pxarexclude files, glob match patterns can be passed to pxar also via cli parameters. Therefore the warning is rephrased to be more ambiguous. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-25 10:47:24 +01:00
Dietmar Maurer	6da73c823f	src/client/backup_writer.rs: prefix upload stats with archive name	2020-02-24 13:24:46 +01:00
Dietmar Maurer	e02c3d461f	src/client/backup_writer.rs: implement verbose options	2020-02-24 12:48:40 +01:00
Dietmar Maurer	ee5fe978e2	src/client/backup_writer.rs: reduce verbosity	2020-02-24 11:33:00 +01:00
Dietmar Maurer	8ce49a76da	src/backup/backup_info.rs: fix SNAPSHOT_PATH_REGEX and GROUP_PATH_REGEX	2020-02-18 13:16:35 +01:00
Wolfgang Bumiller	9e003074cb	runtime: fix blocking strategy: - do not "double"-block_in_place() (it may not be nested) - do not call block_in_place() in non-worker threads is_in_tokio() isn't sufficient, we need to actually know that we're in a worker-thread, so we do this by remembering that we're blocking. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-17 09:48:22 +01:00
Wolfgang Bumiller	1283d58ca9	runtime: remove IN_TOKIO thread local variable tokio now has Handle::try_current() allowing us to generally check for a tokio runtime even if spawned by someone else Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-13 08:56:38 +01:00
Dietmar Maurer	a6e3da987c	src/client/http_client.rs: do not store fingerprints passed via options in cache	2020-02-11 11:54:43 +01:00
Dietmar Maurer	1bd6f32b43	src/client/http_client.rs: always convert fingerprint to lowercase	2020-02-11 11:42:03 +01:00
Dietmar Maurer	d1c657276a	src/client/http_client.rs: remove useless password_env	2020-02-11 11:42:03 +01:00
Dietmar Maurer	a05c0c6ff6	src/bin/proxmox-backup-client.rs: allows to pass fingerprint via env vars	2020-02-11 11:42:03 +01:00
Thomas Lamprecht	b69b8af26b	s/pbsbanner/proxmox-backup-banner/ + move to /usr/lib We want to avoid pbs if possible and also avoid placing internal binaries, not intended for human direct use, in /bin or /sbin paths. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-02-10 14:43:28 +01:00
Thomas Lamprecht	e2b5e75a3e	pbsbanner: use match and also return error Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-02-07 18:30:55 +01:00
Thomas Lamprecht	274b0c7bb7	add login banner service Modeled after the one from PVE, but using rust instead of perl for resolving the nodename and writing to /etc/issue Behavior differs a bit. We write all non-loopback addresses to this file, as the gui accepts connections from them all, so limiting it to the first one is not really sensible. Further an error to resolve, or only getting loopback addresses won't write out an empty /etc/issue file, but a note about the error at the place where the address would be displayed. Named it "pbsbanner", not "proxmox-backup-banner" as it's rather an internal tool anyway and mirrors pvebanner, pmgbanner Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-02-06 22:19:22 +01:00
Christian Ebner	ba050e3788	catalog: shell: Allow two or more successive slashes in path. Two or more successive slashes should be allowed and treated as a single slash. We also do not treat two successive slashes at the beginning of a path any different. Details are found here: https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap04.html#tag_04_11 Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-05 09:32:29 +01:00
Christian Ebner	9ff9a00572	pxar::fuse: cleanup: Remove unused code. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-03 13:43:11 +01:00
Christian Ebner	3e69abef02	pxar::fuse: Refactor lookup in order to cache accessed entries. Context::find_goodbye_entry() is removed and incorporated into the lookup callback in order to take advantage of the entry_cache and since it is only used inside this callback. All entries read on lookup are cached. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-03 13:43:06 +01:00
Christian Ebner	63698e720c	pxar: Improve read performance for fuse. By storing the payload start offset in the `DirectoryEntry` and passing this information to `Decoder::read()`, the payload can be read directly and a repeated re-reading of the entry information is avoided. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-03 12:08:03 +01:00
Dietmar Maurer	032d3ad80f	src/bin/proxmox-backup-client.rs: do not use api macro for prune So that we can use add_common_prune_prameters!() macro again.	2020-01-31 10:31:36 +01:00
Dietmar Maurer	b8a192e3b0	src/config/remote.rs: do not serialize empty option	2020-01-31 09:09:24 +01:00
Dietmar Maurer	c43b36bcb6	src/section_config.rs - write: improve error message	2020-01-31 09:08:42 +01:00
Dietmar Maurer	501f4fa220	depend on proxmox 0.1.13, use new tty helpers from there	2020-01-31 08:16:00 +01:00
Christian Ebner	74616f63b0	proxmox-backup-client: add missing prune options to api definition. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-31 07:01:56 +01:00
Dietmar Maurer	93fbb4ef0a	update proxmox dependency to 0.1.12	2020-01-31 06:59:17 +01:00
Dietmar Maurer	43592f290d	src/bin/proxmox-backup-manager.rs: remove stale debug comment	2020-01-30 17:59:13 +01:00
Dietmar Maurer	9e165b5cad	src/bin/proxmox-backup-manager.rs - list remotes: do not use client, call directly	2020-01-30 17:57:37 +01:00
Dietmar Maurer	9894469e89	src/bin/proxmox-backup-manager.rs: use new texdt table formatter	2020-01-30 13:31:20 +01:00
Dietmar Maurer	8528fce874	src/api2/node/tasks.rs; make list_tasks public	2020-01-30 13:31:20 +01:00
Dietmar Maurer	70fa1aa3fc	fixup	2020-01-30 13:31:20 +01:00
Dietmar Maurer	5eeea607ae	src/section_config.rs - convert_to_array: add option to skip properties	2020-01-30 13:26:46 +01:00
Christian Ebner	93a9eb284e	pxar::fuse: Do not return fcaps value on listxattr. listxattr must only return the name list, no extended attribute values. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:35:09 +01:00
Christian Ebner	5d2158e89a	pxar::fuse: return ACL names in listxattr if present Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:35:03 +01:00
Christian Ebner	72677fb0a5	pxar::fuse: add support to read ACLs. ACLs are stored separately in the pxar archive. This implements the functionality needed to read the ACLs and return them as extended attributes in the getxattr callback. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:35:00 +01:00
Christian Ebner	0502ce6da3	tools::acl: Add helpers to create extended attribute buffer. This helpers are used to construct the extended attributes values from the ACLs stored in the pxar archive. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:34:57 +01:00
Christian Ebner	bcf0d452c9	pxar: move acl helper functions to src/tools/acl.rs They are not only needed by the pxar::sequential_decoder but also for the fuse xattr impl, so it makes more sense to have them there. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:34:54 +01:00
Wolfgang Bumiller	6f763ae673	tools: LruCache: add dropcheck marker See https://doc.rust-lang.org/nomicon/phantom-data.html Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:18:27 +01:00
Dietmar Maurer	99384f7933	src/api2/types.rs: define and use struct TaskListItem	2020-01-28 11:23:08 +01:00
Dietmar Maurer	2c4b303c62	src/api2/node/tasks.rs: use api macro for list_tasks	2020-01-28 11:23:08 +01:00
Christian Ebner	3e56c4abd7	pxar::fuse: Introduce LRU caching for goodbye table and directory entries. The goodbye table of directory entries is cached in a LRU cache to speed up subsequent accesses. This is especially important for directories with many entries, as then the readdirplus callback is called repeatedly because of the limited reply buffer size. `DirectoryEntry`s are cached for subsequent access in their own LRU cache, independent of the goodbye tables. In order to avoid borrow conflicts, the `Context` provides a fn as_mut_refs as well as a fn run_with_context_refs. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-28 10:06:53 +01:00
Christian Ebner	cc552754c8	pxar::fuse: refactor fn stat By passing `&DirectoryEntry` to stat, the function interface is simplified. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-28 10:06:51 +01:00
Dietmar Maurer	d08bc483db	use proxmox 0.1.9 with new cli command helpers	2020-01-27 18:08:43 +01:00
Dietmar Maurer	99d863d770	src/client/http_client.rs: new password_env option	2020-01-27 10:42:37 +01:00
Dietmar Maurer	5030b7cea4	src/client/http_client.rs: new prefix options	2020-01-27 10:20:55 +01:00
Dietmar Maurer	5a74756c15	src/client/http_client.rs: implement fingerprint cache	2020-01-25 15:37:34 +01:00
Dietmar Maurer	d59dbeca1b	src/client/http_client.rs: new struct HttpClientOptions	2020-01-25 12:18:00 +01:00
Dietmar Maurer	6afbe1d846	src/config/remote.rs: add fingerprint	2020-01-25 09:49:45 +01:00
Dietmar Maurer	dcb8db66d9	src/api2/types.rs: add CERT_FINGERPRINT_SHA256_SCHEMA	2020-01-25 09:49:45 +01:00
Christian Ebner	90c815bfe2	src/bin/proxmox-backup-client.rs: fix another typo in property description Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-24 13:46:08 +01:00
Christian Ebner	5eee6d89b6	src/bin/proxmox-backup-client.rs: fix typo in property description Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-24 13:45:22 +01:00
Christian Ebner	04350b4c08	tools: add access method to LruCache This will return a mutable reference just like get_mut, but on a cache miss it will get and insert the missing value via the fetch method provided via the Cacher trait. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-24 09:32:09 +01:00
Dietmar Maurer	4191018c79	src/api2/admin/datastore.rs - download_file: use BACKUP_ARCHIVE_NAME_SCHEMA for file-name	2020-01-23 13:49:30 +01:00
Dietmar Maurer	a92830dc39	src/api2/types.rs: define and use api type GarbageCollectionStatus	2020-01-23 13:40:12 +01:00

... 10 11 12 13 14 ...

2915 Commits