proxmox-backup

Author	SHA1	Message	Date
Dietmar Maurer	b9f2f761bb	avoid problems with missing acl.cfg and user.cfg	2020-04-29 10:40:42 +02:00
Dietmar Maurer	30fb602578	src/api2/admin/datastore.rs - get_datastore_list: only return name and comment We dont want to leak the full configuration to users with limited access permission. Please use the api2::config::datastore api to get the full configuration.	2020-04-29 09:21:34 +02:00
Dietmar Maurer	0a00f6e01c	src/api2/config/datastore.rs_ add delete property to update method	2020-04-29 09:09:59 +02:00
Dietmar Maurer	30003baaa4	src/api2/config/remote.rs: fix white space	2020-04-29 09:09:39 +02:00
Dietmar Maurer	5211705ff1	src/api2/config/remote.rs: add delete parameter to update method	2020-04-29 09:04:17 +02:00
Dietmar Maurer	ec67af9af3	src/api2/pull.rs: require Datastore.Prune if delete flag is set.	2020-04-29 07:19:32 +02:00
Dietmar Maurer	8247db5b39	src/config/acl.rs: introduice privileges and roles for remotes	2020-04-29 07:03:44 +02:00
Dietmar Maurer	409f44247b	fix api2::types::ACL_ROLE_SCHEMA make sure we list all roles ...	2020-04-28 13:25:02 +02:00
Dietmar Maurer	dd335b77f5	src/config/acl.rs - fix regression tests	2020-04-28 11:16:15 +02:00
Dietmar Maurer	6f6aa95abb	add Datastore.Backup, Datastore.PowerUser and Datastore.Reader role	2020-04-28 11:07:25 +02:00
Dietmar Maurer	54552dda59	implemnt backup ownership, improve datastore access permissions	2020-04-28 10:22:25 +02:00
Dietmar Maurer	1347b1152d	src/config/cached_user_info.rs - lookup_privs: correctly handle superuser	2020-04-27 13:22:03 +02:00
Dietmar Maurer	d00e1a216f	src/config/acl.rs: introduce more/better datastore privileges	2020-04-27 07:13:50 +02:00
Dietmar Maurer	9c7fe29dfc	src/config/acl.rs: rtename PRTIV_DATASTORE_ALLOCATE to PRIV_DATASTORE_MODIFY	2020-04-27 06:50:35 +02:00
Dietmar Maurer	14627d671a	src/bin/proxmox-backup-manager.rs: add dns sub command Also improved the DNS api, added a --delete option.	2020-04-26 08:23:23 +02:00
Dietmar Maurer	76227a6acd	src/bin/proxmox-backup-manager.rs: fix node parameter handling	2020-04-25 17:20:22 +02:00
Dietmar Maurer	26d9aebc28	move src/api2/config/network.rs to src/api2/node/network.rs So that we have the same api path for network config as pve.	2020-04-25 17:00:38 +02:00
Dietmar Maurer	1ca540a63b	src/config/network.rs: auto-add lo, and implement a few regression tests	2020-04-24 12:57:11 +02:00
Wolfgang Bumiller	9094186a57	xattr: cleanup: don't use pxar types in the API Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 11:23:48 +02:00
Wolfgang Bumiller	27a3decbfe	xattr: api cleanup Make `flistxattr()` return a `ListXAttr` helper which provides an iterator over `&CStr`. This exposes the property that xattr names are a zero-terminated string without simply being an opaque "byte vector". Using &[u8] as a type here is too lax. Also let `fgetxattr` take a `CStr`. While this may be a burden on the caller, we usually already have zero-terminated strings on the call site. Currently we only use this method coming from `flistxattr` after all. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 10:56:52 +02:00
Wolfgang Bumiller	9af76ef075	xattr: use checked_mul to increase size Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 10:56:52 +02:00
Wolfgang Bumiller	00ec8d1685	tools: pub use Fd from proxmox crate Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-24 10:56:52 +02:00
Dietmar Maurer	fd7c0979b4	src/bin/proxmox-backup-manager.rs: implement netwerk revert	2020-04-24 10:45:49 +02:00
Dietmar Maurer	c67bc9c35c	src/bin/proxmox-backup-manager.rs: new command to show pending network changes	2020-04-24 10:27:43 +02:00
Dietmar Maurer	3181f9b625	src/bin/proxmox-backup-manager.rs: only show pending changes with "text" format	2020-04-24 10:16:57 +02:00
Dietmar Maurer	2eefd9aee1	src/config/network.rs: implement network reload, set "changes" attribute	2020-04-24 09:55:46 +02:00
Dietmar Maurer	8a6b86b8a7	src/config/network.rs: use a simple String for comments	2020-04-24 07:46:08 +02:00
Dietmar Maurer	96d9478668	src/config/network/parser.rs: corectly detect vanished interfaces	2020-04-24 07:26:54 +02:00
Dietmar Maurer	10a9be45bd	src/api2/config/network.rs: implement update/delete comments	2020-04-23 16:08:35 +02:00
Dietmar Maurer	5f60a58fd5	src/config/network.rs; support interface comments, cleanups	2020-04-23 15:54:30 +02:00
Dietmar Maurer	659c3be3d5	src/config/network.rs: avoid newline after family options	2020-04-23 11:30:41 +02:00
Dietmar Maurer	5e4e88e83f	src/api2/config/network.rs: implement update/delete for bridge_ports and bond_slaves	2020-04-23 11:21:27 +02:00
Dietmar Maurer	339965d720	src/api2/config/network.rs: only allow one default gateway	2020-04-23 10:37:40 +02:00
Dietmar Maurer	c38b4bb8b2	src/config/network.rs: do not allow to change interface type	2020-04-23 09:43:38 +02:00
Dietmar Maurer	42fbe91a34	src/config/network.rs: parse bond-slaves	2020-04-23 09:31:10 +02:00
Dietmar Maurer	1d9a68c2fc	src/config/network.rs: parse bridge-ports	2020-04-23 09:24:17 +02:00
Dietmar Maurer	02269f3dba	src/config/network.rs: introduce NetworkInterfaceType	2020-04-23 08:45:03 +02:00
Dietmar Maurer	d5ca9bd5df	src/config/network.rs: cleanup (new helper combine_entry)	2020-04-23 07:54:12 +02:00
Dietmar Maurer	02e36d96ad	src/config/network.rs: write changes to interfaces.new	2020-04-23 07:19:29 +02:00
Dietmar Maurer	2c18efd902	src/config/network.rs: use a single mtu setting (instead of mtu_v4 and mtu_v6)	2020-04-23 07:07:14 +02:00
Dietmar Maurer	4cb6bd894c	src/bin/proxmox-backup-manager.rs: improve network list output format	2020-04-23 06:44:55 +02:00
Dietmar Maurer	b1564af25a	src/bin/proxmox-backup-manager.rs: format datastore list output	2020-04-22 17:37:20 +02:00
Dietmar Maurer	bf004ecd87	src/bin/proxmox-backup-manager.rs: format network list output	2020-04-22 17:14:52 +02:00
Dietmar Maurer	f1026a5aa9	src/api2/config/network.rs: allow to update 'auto' flag	2020-04-22 16:46:46 +02:00
Dietmar Maurer	3fce3bc36e	src/config/network/parser.rs: parse MTU settings	2020-04-22 13:44:51 +02:00
Dietmar Maurer	f8e7ac686a	src/config/network.rs: only save attriubutes used by configuration method	2020-04-22 12:42:09 +02:00
Dietmar Maurer	c016482c7a	src/api2/config/network.rs: implement delete property	2020-04-22 12:19:31 +02:00
Dietmar Maurer	27f2c23049	src/api2/config/network.rs: allow to update configuration method	2020-04-22 11:32:36 +02:00
Dietmar Maurer	df6bb03d0e	src/api2/config/network.rs: improve network api	2020-04-22 10:54:07 +02:00
Dietmar Maurer	e2d940b949	src/config/network/parser.rs: remove debug println	2020-04-22 10:53:26 +02:00
Dietmar Maurer	0c226bc173	src/config/network/helper.rs: fix CIDR regex	2020-04-22 10:52:31 +02:00
Dietmar Maurer	76cf5208cf	src/api2/types.rs: add schemas for IP/CIDR	2020-04-22 10:28:53 +02:00
Dietmar Maurer	2ea7bf1b3d	src/api2/config/datastore.rs_ fix method docs	2020-04-22 08:53:16 +02:00
Dietmar Maurer	8b57cd4441	src/config/network.rs: remove netmask support rely on cidr instead.	2020-04-22 08:45:13 +02:00
Dietmar Maurer	68da20bf62	src/api2/types.rs: define NETWORK_INTERFACE_NAME_SCHEMA	2020-04-21 17:54:52 +02:00
Dietmar Maurer	c357260d09	src/config/network.rs: move type definitions to src/api2/types.rs	2020-04-21 17:25:05 +02:00
Dietmar Maurer	7e02d08cd0	rename ConfigMethod to NetworkConfigMethod	2020-04-21 17:17:57 +02:00
Dietmar Maurer	ca0e534796	src/api2/config/network.rs: start network configuration api	2020-04-21 14:28:26 +02:00
Dietmar Maurer	904e988667	src/config/network.rs: impleement load/save	2020-04-21 12:55:33 +02:00
Dietmar Maurer	3f129233be	src/config/network.rs: add Interface flags 'exists' and 'active'	2020-04-21 11:46:56 +02:00
Dietmar Maurer	a9bb491e35	src/config/network.rs: cleanup autostart flag handling	2020-04-21 11:06:22 +02:00
Dietmar Maurer	1ec7f8a0dd	src/config/network/helper.rs: new helper get_network_interfaces()	2020-04-21 10:32:54 +02:00
Dietmar Maurer	92310d585c	src/config/network.rs: simplify code	2020-04-20 18:10:15 +02:00
Dietmar Maurer	f34d4401f7	src/config/network.rs: read/write /etc/network/interfaces Start implementing a recursive descent parser.	2020-04-20 14:15:57 +02:00
Dietmar Maurer	6e695960ca	src/config/cached_user_info.rs: cache it up to 5 seconds	2020-04-18 08:49:20 +02:00
Dietmar Maurer	365f0f720c	fix permission tests using non-uri parameters We nood to do those tests inside the fuction body instead...	2020-04-18 08:23:04 +02:00
Dietmar Maurer	a737179eb4	src/config/cached_user_info.rs: new check_privs helper	2020-04-18 08:09:34 +02:00
Dietmar Maurer	bb072ba49c	src/api2/access.rs: cleanup	2020-04-18 07:28:25 +02:00
Dietmar Maurer	ff329f970b	src/api2/types.rs: use anyhow::Error in test cases	2020-04-18 07:05:31 +02:00
Wolfgang Bumiller	f7d4e4b506	switch from failure to anyhow Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-04-17 18:43:30 +02:00
Dietmar Maurer	404d78c41e	src/api2/pull.rs: add access permission	2020-04-17 15:27:04 +02:00
Dietmar Maurer	1bfc1efa50	src/api2/subscription.rs: add access permissions	2020-04-17 15:14:28 +02:00
Dietmar Maurer	73ce1d1146	src/api2/reader.rs: add access permissions	2020-04-17 15:01:56 +02:00
Dietmar Maurer	70e5f2461d	src/api2/config/remote.rs: add access permissions	2020-04-17 14:57:26 +02:00
Dietmar Maurer	c0ef209aeb	src/api2/config/datastore.rs: impl digest check for delete, add access permissions	2020-04-17 14:51:29 +02:00
Dietmar Maurer	9f9f7eefa3	src/api2/backup.rs: add access permissions	2020-04-17 14:40:20 +02:00
Dietmar Maurer	bb34b58910	src/api2/admin/datastore.rs: add access permissions - first try We need to refine this later (introduce backup owner concept?)	2020-04-17 14:36:27 +02:00
Dietmar Maurer	5972def5ec	acl: change path "storage" to "datastore"	2020-04-17 14:15:44 +02:00
Dietmar Maurer	aa90ced3bf	src/api2/access/role.rs: use schema ACL_ROLE_SCHEMA	2020-04-17 14:14:06 +02:00
Dietmar Maurer	ca257c8097	move type defs from src/api2/access/acl.rs to src/api2/types.rs	2020-04-17 14:13:15 +02:00
Dietmar Maurer	3fff55b293	src/api2/access/role.rs: new api to list roles	2020-04-17 14:03:24 +02:00
Dietmar Maurer	4f66423fcc	src/api2/access/user.rs: add access permissions	2020-04-17 11:04:36 +02:00
Dietmar Maurer	d4f020f4c5	src/api2/access/user.rs: add access permissions	2020-04-17 10:08:45 +02:00
Dietmar Maurer	d28ddb8e04	src/api2/access/acl.rs: add access permissions	2020-04-17 10:03:09 +02:00
Dietmar Maurer	83b6a7cf71	src/api2/node/tasks.rs: use api macro, implement access permissions	2020-04-16 17:47:21 +02:00
Dietmar Maurer	e4681f9f71	src/api2/node/syslog.rs: add access permissions	2020-04-16 17:08:19 +02:00
Dietmar Maurer	b5037fa8ed	src/api2/node/status.rs: add access permissions	2020-04-16 17:05:09 +02:00
Dietmar Maurer	9989d2c4e9	src/server/rest.rs: reduce delay for permission error to 500ms	2020-04-16 12:56:34 +02:00
Dietmar Maurer	1cf7bbf412	src/api2/node/services.rs: add access permissions	2020-04-16 12:47:16 +02:00
Dietmar Maurer	68ed0c629d	src/api2/node/journal.rs: add access permissions	2020-04-16 12:47:16 +02:00
Dietmar Maurer	4b40148caa	start impl. access permissions	2020-04-16 12:47:16 +02:00
Dietmar Maurer	423e656163	src/config/cached_user_info.rs: new helper class	2020-04-16 10:05:16 +02:00
Dietmar Maurer	109d7817cd	src/config/user.rs - cached_config: do not store/return digest	2020-04-15 11:35:57 +02:00
Dietmar Maurer	5354511fd0	src/config/acl.rs: implement cached_config	2020-04-15 11:30:47 +02:00
Dietmar Maurer	bd098a7f77	src/api2/node/dns.rs: use api macro (cleanup)	2020-04-15 10:09:18 +02:00
Dietmar Maurer	8d048af2bf	acl: improve NoAccess handling	2020-04-15 08:11:43 +02:00
Dietmar Maurer	9a328319dd	pxar extract: remove pattern from arg_param, add target instead	2020-04-15 06:41:37 +02:00
Dietmar Maurer	7e3d2e5b41	pxar create: remove exclude from arg_param	2020-04-15 06:31:46 +02:00
Dietmar Maurer	9c06f6c292	fix previous commit - use result.	2020-04-14 17:48:10 +02:00
Dietmar Maurer	9f4e47dd93	acl update: check path	2020-04-14 17:23:48 +02:00
Dietmar Maurer	d83175dd69	acl update: check if user exist.	2020-04-14 13:46:27 +02:00
Dietmar Maurer	68ccdf09a4	src/config/user.rs: implement user config cache	2020-04-14 13:45:45 +02:00
Dietmar Maurer	9765092ede	acl api: implement update	2020-04-14 10:16:49 +02:00
Dietmar Maurer	ed3e60ae69	start ACL api	2020-04-13 11:09:44 +02:00
Dietmar Maurer	a83eab3c4d	acl: use BTreeMap and BTreeSet to avoid sort()	2020-04-12 17:13:53 +02:00
Dietmar Maurer	0815ec7e65	acl: implement roles(), add regression tests.	2020-04-12 13:06:50 +02:00
Dietmar Maurer	5c6cdf9815	add acl config	2020-04-11 12:24:26 +02:00
Dietmar Maurer	b88f9c5b1e	PASSWORD_SCHEMA: set max_length to 1024 (for tickets)	2020-04-09 13:35:58 +02:00
Dietmar Maurer	879546aff6	api: add default property to domain list	2020-04-09 13:35:08 +02:00
Dietmar Maurer	73b40e9b46	api: correctly sort access subdirmap	2020-04-09 13:34:07 +02:00
Dietmar Maurer	708db4b3ae	api: add list_domains	2020-04-09 11:36:45 +02:00
Dietmar Maurer	685e13347e	api: move config/user to access/users, implement change_password To make it similar to the pve api	2020-04-09 10:21:24 +02:00
Dietmar Maurer	7d817b0358	implement auth framework	2020-04-08 14:06:15 +02:00
Dietmar Maurer	579728c641	add user configiguration	2020-04-08 14:06:15 +02:00
Dietmar Maurer	cf459b1982	gc: log pending removals	2020-04-06 09:50:40 +02:00
Dietmar Maurer	dda7015497	prune api: return a usable result (we run synchronous anyways)	2020-04-01 12:24:28 +02:00
Dietmar Maurer	4299ca727c	src/server/rest.rs: use correct formatter	2020-03-26 12:54:20 +01:00
Dominik Csapak	6650a242fb	rewrite future select in upgrade_to_backup_protocol using select macro and handle all ok/err cases with at least logging Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>	2020-03-18 11:33:59 +01:00
Dietmar Maurer	ca26db1140	delete unused src/storage/config.rs	2020-03-02 13:31:38 +01:00
Dietmar Maurer	90c5239d46	use SectionConfig from proxmox 0.1.18-1	2020-03-02 12:52:11 +01:00
Dietmar Maurer	18deda403d	src/bin/proxmox-backup-client.rs - list_groups: render full snapshot path for last-backup column	2020-02-28 09:09:22 +01:00
Dietmar Maurer	6882b3716c	src/bin/proxmox-backup-manager.rs: avoid compiler warning	2020-02-28 08:52:12 +01:00
Christian Ebner	32d192a952	catalog: shell: Use the new logic including resolving symlinks for catalog and remove the old unused code. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:51:01 +01:00
Christian Ebner	fee5528e59	catalog: shell: introduce new CatalogPathStack to navigate in catalog shell. This is basically a rewrite of the current logic for navigating the catalog, but in addition allows to follow symlinks. Following symlinks introduces the issue that generation of canonical paths (needed in the actual pxar archive) is more complex, as symlinks have to be resolved and loops avoided. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:50:43 +01:00
Christian Ebner	4145c36749	catalog: add missing function documentation. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:50:08 +01:00
Christian Ebner	c2f9149461	catalog: introduce is_symlink() to check if DirEntry is a symlink. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:49:55 +01:00
Christian Ebner	11ee5c0563	catalog: derive PartEq for equality checks of two DirEntry's. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:49:39 +01:00
Christian Ebner	536683e73b	src/backup/dynamic_index.rs: Add LruCache for chunks. In order to improve non-sequential reads of chunks as e.g. in FUSE. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:47:16 +01:00
Christian Ebner	35ddf0b419	catalog: shell: Introduce clear-selected command. 'clear-selected' allows to clear all the match patterns from the list of patterns for a subsequent restore. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:40:55 +01:00
Christian Ebner	8e464141cf	catalog: shell: Improve list-selected command. 'list-selected' now shows the filenames matching the patterns for a restore instead of the patterns themselfs. The patterns can be displayed by passing the '--pattern' flag. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:40:42 +01:00
Christian Ebner	351b913d1e	pxar::decoder: Return the correct symlink target size. This eliminates also repeated calls to readlink in fuse, which occur when the preallocated buffer to store the symlink target path is to small. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 08:38:56 +01:00
Christian Ebner	189996cf4a	proxmox-backup-client: expose exclude match patterns to cli. Allows to pass exclude match patterns to the 'backup' command. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-28 07:56:09 +01:00
Dietmar Maurer	ac3faaf5c0	src/bin/proxmox-backup-manager.rs: cleanup output handling	2020-02-28 07:42:36 +01:00
Dietmar Maurer	4939255fb4	src/tools/format.rs: move output rendering code to this new file	2020-02-28 07:30:35 +01:00
Dietmar Maurer	c81b2b7c70	src/bin/proxmox-backup-client.rs - list_groups: use format_and_print_result_full() Depend on proxmox 0.1.16-1	2020-02-27 13:32:11 +01:00
Dietmar Maurer	c2043614f7	src/bin/proxmox-backup-client.rs: use new output formath helpers from proxmox 0.15.1-1	2020-02-27 12:41:15 +01:00
Dietmar Maurer	e23f586344	src/bin/proxmox-backup-client.rs - status: improve alignment	2020-02-27 11:38:09 +01:00
Dietmar Maurer	be2425ff85	src/bin/proxmox-backup-client.rs - status: do not print header	2020-02-27 11:31:05 +01:00
Dietmar Maurer	f24fc1166b	src/bin/proxmox-backup-client.rs - list_snapshots: use format_and_print_result_full() Depend on proxmox v0.1.14.	2020-02-27 11:28:44 +01:00
Christian Ebner	af934f8cf6	tools::lru_cache: Improve access() and insert() by using HashMap::entry(). entry() allows to lookup the position where and entry belongs and update/insert it in the HashMap more efficiently than get_mut() and insert(). Details: https://gankra.github.io/blah/hashbrown-insert/ In addition, use the struct LinkedList and remove the outdated code. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:56:25 +01:00
Christian Ebner	e3ab9a383c	tools::lru_cache: Separate LinkedList part of LruCache into own struct. In order to keep the separation more aparent and avoid mut borrow conflics. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:55:48 +01:00
Christian Ebner	75c2ee7bab	tools::lru_cache: Make key generic. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:55:16 +01:00
Christian Ebner	03f779c6f5	catalog: shell: Improve output of stat command. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-27 06:53:17 +01:00
Dietmar Maurer	ea5f547fdc	src/bin/proxmox-backup-client.rs - list_snapshot_files: use format_and_print_result_full()	2020-02-26 13:49:47 +01:00
Dietmar Maurer	390c5bdde4	src/bin/proxmox-backup-client.rs - status: use format_and_print_result_full	2020-02-26 13:29:00 +01:00
Dietmar Maurer	03ac286c80	src/bin/proxmox-backup-manager.rs: use crate::tools::runtime::block_on()	2020-02-26 11:37:38 +01:00
Dietmar Maurer	dd04383bb8	src/tools/runtime.rs: call OPENSSL_thread_stop to avoid race with openssl cleanup handlers	2020-02-26 10:38:55 +01:00
Dietmar Maurer	ca2dbb8af1	src/client/backup_writer.rs: also print archive size	2020-02-25 14:12:29 +01:00
Wolfgang Bumiller	0bcbb5434e	cleanup unused module Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-25 11:38:27 +01:00
Christian Ebner	495b238bc1	pxar: Change text of warning for files matched by exclude patterns. In addition to the .pxarexclude files, glob match patterns can be passed to pxar also via cli parameters. Therefore the warning is rephrased to be more ambiguous. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-02-25 10:47:24 +01:00
Dietmar Maurer	6da73c823f	src/client/backup_writer.rs: prefix upload stats with archive name	2020-02-24 13:24:46 +01:00
Dietmar Maurer	e02c3d461f	src/client/backup_writer.rs: implement verbose options	2020-02-24 12:48:40 +01:00
Dietmar Maurer	ee5fe978e2	src/client/backup_writer.rs: reduce verbosity	2020-02-24 11:33:00 +01:00
Dietmar Maurer	8ce49a76da	src/backup/backup_info.rs: fix SNAPSHOT_PATH_REGEX and GROUP_PATH_REGEX	2020-02-18 13:16:35 +01:00
Wolfgang Bumiller	9e003074cb	runtime: fix blocking strategy: - do not "double"-block_in_place() (it may not be nested) - do not call block_in_place() in non-worker threads is_in_tokio() isn't sufficient, we need to actually know that we're in a worker-thread, so we do this by remembering that we're blocking. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-17 09:48:22 +01:00
Wolfgang Bumiller	1283d58ca9	runtime: remove IN_TOKIO thread local variable tokio now has Handle::try_current() allowing us to generally check for a tokio runtime even if spawned by someone else Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-13 08:56:38 +01:00
Dietmar Maurer	a6e3da987c	src/client/http_client.rs: do not store fingerprints passed via options in cache	2020-02-11 11:54:43 +01:00
Dietmar Maurer	1bd6f32b43	src/client/http_client.rs: always convert fingerprint to lowercase	2020-02-11 11:42:03 +01:00
Dietmar Maurer	d1c657276a	src/client/http_client.rs: remove useless password_env	2020-02-11 11:42:03 +01:00
Dietmar Maurer	a05c0c6ff6	src/bin/proxmox-backup-client.rs: allows to pass fingerprint via env vars	2020-02-11 11:42:03 +01:00
Thomas Lamprecht	b69b8af26b	s/pbsbanner/proxmox-backup-banner/ + move to /usr/lib We want to avoid pbs if possible and also avoid placing internal binaries, not intended for human direct use, in /bin or /sbin paths. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-02-10 14:43:28 +01:00
Thomas Lamprecht	e2b5e75a3e	pbsbanner: use match and also return error Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-02-07 18:30:55 +01:00
Thomas Lamprecht	274b0c7bb7	add login banner service Modeled after the one from PVE, but using rust instead of perl for resolving the nodename and writing to /etc/issue Behavior differs a bit. We write all non-loopback addresses to this file, as the gui accepts connections from them all, so limiting it to the first one is not really sensible. Further an error to resolve, or only getting loopback addresses won't write out an empty /etc/issue file, but a note about the error at the place where the address would be displayed. Named it "pbsbanner", not "proxmox-backup-banner" as it's rather an internal tool anyway and mirrors pvebanner, pmgbanner Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-02-06 22:19:22 +01:00
Christian Ebner	ba050e3788	catalog: shell: Allow two or more successive slashes in path. Two or more successive slashes should be allowed and treated as a single slash. We also do not treat two successive slashes at the beginning of a path any different. Details are found here: https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap04.html#tag_04_11 Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-05 09:32:29 +01:00
Christian Ebner	9ff9a00572	pxar::fuse: cleanup: Remove unused code. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-03 13:43:11 +01:00
Christian Ebner	3e69abef02	pxar::fuse: Refactor lookup in order to cache accessed entries. Context::find_goodbye_entry() is removed and incorporated into the lookup callback in order to take advantage of the entry_cache and since it is only used inside this callback. All entries read on lookup are cached. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-03 13:43:06 +01:00
Christian Ebner	63698e720c	pxar: Improve read performance for fuse. By storing the payload start offset in the `DirectoryEntry` and passing this information to `Decoder::read()`, the payload can be read directly and a repeated re-reading of the entry information is avoided. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-02-03 12:08:03 +01:00
Dietmar Maurer	032d3ad80f	src/bin/proxmox-backup-client.rs: do not use api macro for prune So that we can use add_common_prune_prameters!() macro again.	2020-01-31 10:31:36 +01:00
Dietmar Maurer	b8a192e3b0	src/config/remote.rs: do not serialize empty option	2020-01-31 09:09:24 +01:00
Dietmar Maurer	c43b36bcb6	src/section_config.rs - write: improve error message	2020-01-31 09:08:42 +01:00
Dietmar Maurer	501f4fa220	depend on proxmox 0.1.13, use new tty helpers from there	2020-01-31 08:16:00 +01:00
Christian Ebner	74616f63b0	proxmox-backup-client: add missing prune options to api definition. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-31 07:01:56 +01:00
Dietmar Maurer	93fbb4ef0a	update proxmox dependency to 0.1.12	2020-01-31 06:59:17 +01:00
Dietmar Maurer	43592f290d	src/bin/proxmox-backup-manager.rs: remove stale debug comment	2020-01-30 17:59:13 +01:00
Dietmar Maurer	9e165b5cad	src/bin/proxmox-backup-manager.rs - list remotes: do not use client, call directly	2020-01-30 17:57:37 +01:00
Dietmar Maurer	9894469e89	src/bin/proxmox-backup-manager.rs: use new texdt table formatter	2020-01-30 13:31:20 +01:00
Dietmar Maurer	8528fce874	src/api2/node/tasks.rs; make list_tasks public	2020-01-30 13:31:20 +01:00
Dietmar Maurer	70fa1aa3fc	fixup	2020-01-30 13:31:20 +01:00
Dietmar Maurer	5eeea607ae	src/section_config.rs - convert_to_array: add option to skip properties	2020-01-30 13:26:46 +01:00
Christian Ebner	93a9eb284e	pxar::fuse: Do not return fcaps value on listxattr. listxattr must only return the name list, no extended attribute values. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:35:09 +01:00
Christian Ebner	5d2158e89a	pxar::fuse: return ACL names in listxattr if present Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:35:03 +01:00
Christian Ebner	72677fb0a5	pxar::fuse: add support to read ACLs. ACLs are stored separately in the pxar archive. This implements the functionality needed to read the ACLs and return them as extended attributes in the getxattr callback. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:35:00 +01:00
Christian Ebner	0502ce6da3	tools::acl: Add helpers to create extended attribute buffer. This helpers are used to construct the extended attributes values from the ACLs stored in the pxar archive. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:34:57 +01:00
Christian Ebner	bcf0d452c9	pxar: move acl helper functions to src/tools/acl.rs They are not only needed by the pxar::sequential_decoder but also for the fuse xattr impl, so it makes more sense to have them there. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:34:54 +01:00
Wolfgang Bumiller	6f763ae673	tools: LruCache: add dropcheck marker See https://doc.rust-lang.org/nomicon/phantom-data.html Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-29 09:18:27 +01:00
Dietmar Maurer	99384f7933	src/api2/types.rs: define and use struct TaskListItem	2020-01-28 11:23:08 +01:00
Dietmar Maurer	2c4b303c62	src/api2/node/tasks.rs: use api macro for list_tasks	2020-01-28 11:23:08 +01:00
Christian Ebner	3e56c4abd7	pxar::fuse: Introduce LRU caching for goodbye table and directory entries. The goodbye table of directory entries is cached in a LRU cache to speed up subsequent accesses. This is especially important for directories with many entries, as then the readdirplus callback is called repeatedly because of the limited reply buffer size. `DirectoryEntry`s are cached for subsequent access in their own LRU cache, independent of the goodbye tables. In order to avoid borrow conflicts, the `Context` provides a fn as_mut_refs as well as a fn run_with_context_refs. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-28 10:06:53 +01:00
Christian Ebner	cc552754c8	pxar::fuse: refactor fn stat By passing `&DirectoryEntry` to stat, the function interface is simplified. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-28 10:06:51 +01:00
Dietmar Maurer	d08bc483db	use proxmox 0.1.9 with new cli command helpers	2020-01-27 18:08:43 +01:00
Dietmar Maurer	99d863d770	src/client/http_client.rs: new password_env option	2020-01-27 10:42:37 +01:00
Dietmar Maurer	5030b7cea4	src/client/http_client.rs: new prefix options	2020-01-27 10:20:55 +01:00
Dietmar Maurer	5a74756c15	src/client/http_client.rs: implement fingerprint cache	2020-01-25 15:37:34 +01:00
Dietmar Maurer	d59dbeca1b	src/client/http_client.rs: new struct HttpClientOptions	2020-01-25 12:18:00 +01:00
Dietmar Maurer	6afbe1d846	src/config/remote.rs: add fingerprint	2020-01-25 09:49:45 +01:00
Dietmar Maurer	dcb8db66d9	src/api2/types.rs: add CERT_FINGERPRINT_SHA256_SCHEMA	2020-01-25 09:49:45 +01:00
Christian Ebner	90c815bfe2	src/bin/proxmox-backup-client.rs: fix another typo in property description Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-24 13:46:08 +01:00
Christian Ebner	5eee6d89b6	src/bin/proxmox-backup-client.rs: fix typo in property description Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-24 13:45:22 +01:00
Christian Ebner	04350b4c08	tools: add access method to LruCache This will return a mutable reference just like get_mut, but on a cache miss it will get and insert the missing value via the fetch method provided via the Cacher trait. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-24 09:32:09 +01:00
Dietmar Maurer	4191018c79	src/api2/admin/datastore.rs - download_file: use BACKUP_ARCHIVE_NAME_SCHEMA for file-name	2020-01-23 13:49:30 +01:00
Dietmar Maurer	a92830dc39	src/api2/types.rs: define and use api type GarbageCollectionStatus	2020-01-23 13:40:12 +01:00
Dietmar Maurer	dfc58d47ad	src/api2/admin/datastore.rs: use api macro for start_garbage_collection	2020-01-23 13:01:21 +01:00
Dietmar Maurer	1dc117bbba	src/api2/types.rs: define and use struct StorageStatus	2020-01-23 12:51:30 +01:00
Dietmar Maurer	09b1f7b202	src/api2/types.rs: define struct BackupContent, and use it with list_snapshot_files	2020-01-23 11:17:38 +01:00
Dietmar Maurer	51534c8de9	src/backup/manifest.rs: check if manifest contains files	2020-01-23 11:16:12 +01:00
Dietmar Maurer	68a6a0ee03	src/api2/admin/datastore.rs: use api macro for delete_snapshot	2020-01-23 10:16:45 +01:00
Dietmar Maurer	6abce6c2bb	src/backup/datastore.rs: remove_backup_(group/dir) - return Error instead of io::Error	2020-01-23 10:14:46 +01:00
Dietmar Maurer	c425bdc9b2	src/api2/pull.rs: delete fanished snapshots	2020-01-23 09:58:57 +01:00
Dietmar Maurer	8a1d68c8b9	src/backup/datastore.rs: improve error messages	2020-01-23 09:58:14 +01:00
Dietmar Maurer	18cc66ee85	src/api2/pull.rs: use new Datastore::last_successful_backup helper Do not fail if local group does not exists.	2020-01-22 15:06:28 +01:00
Dietmar Maurer	41b373eced	src/backup/datastore.rs: new helpers last_successful_backup: Returns the time of the last successful backup group_path: Returns the absolute path for a backup_group snapshot_path: Returns the absolute path for a backup_dir	2020-01-22 15:05:47 +01:00
Thomas Lamprecht	1ca037d81c	followup: commit all changes Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-01-22 14:14:52 +01:00
Thomas Lamprecht	44b65b4ec7	api/datastore: do not allow updating path for now It's a bit dangerous as it points to all the saved backups, so they would be seemingly lost after updating the path. Follow our logic from other products, e.g. in PVE we do not allow to update the backing path/location of a storage either for similar reasons. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2020-01-22 13:55:37 +01:00
Christian Ebner	1685c2e341	tools: LRU cache to use for pxar FUSE implementation. Implements a cache with least recently used cache replacement policy. Internally the state is tracked by a HashMap (for fast access) and a doubly linked list (for the access order). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-22 13:11:42 +01:00
Dietmar Maurer	f1d99e3f6a	src/bin/proxmox-backup-client.rs: use a std channel to write the catalog	2020-01-22 12:49:08 +01:00
Dietmar Maurer	02141b4d9b	src/client/pxar_backup_stream.rs: use a channel instead of a pipe	2020-01-22 11:37:16 +01:00
Dietmar Maurer	dcd033a53c	src/tools/std_channel_writer.rs: new StdChannelWriter helper class	2020-01-22 11:33:57 +01:00
Wolfgang Bumiller	9ea4bce444	bump proxmox crate to 0.1.7 The -sys, -tools and -api crate have now been merged into the proxmx crate directly. Only macro crates are separate (but still reexported by the proxmox crate in their designated locations). When we need to depend on "parts" of the crate later on we'll just have to use features. The reason is mostly that these modules had inter-dependencies which really make them not independent enough to be their own crates. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-21 13:48:37 +01:00
Dietmar Maurer	3f06d6fbbd	use proxmox_backup::tools::runtime::main to run async run_cli_command	2020-01-21 11:34:45 +01:00
Wolfgang Bumiller	077ac6f8b3	runtime: actually park on Pending... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-21 10:37:16 +01:00
Wolfgang Bumiller	2d51569396	runtime: custom block_on implementation futures::executor::block_on does not allow nesting, but we need that... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-21 10:24:52 +01:00
Wolfgang Bumiller	650e052299	runtime: drop now not required Send and static restrictions Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-20 14:09:24 +01:00
Wolfgang Bumiller	d973aa827c	introduce new runtime tokio helpers Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-20 13:12:40 +01:00
Christian Ebner	aac9dbf635	pxar::fuse: remove open, opendir and releasedir callbacks They are not needed as there is no internal state which needs to be tracked. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-20 12:56:21 +01:00
Christian Ebner	a8aff3535d	pxar: Include symlink target in DirectoryEntry This allows to read the target path of a symbolic link in the Decoder::read_directory_entry() function and stores it in the DirectoryEntry. By this the Decoder::read_link() function becomes obsolete and is therefore removed. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-20 12:56:10 +01:00
Dietmar Maurer	138910bcd4	src/client/pxar_backup_stream.rs: use std:🧵:Builder to name the thread To make debugging easier.	2020-01-18 12:21:22 +01:00
Dietmar Maurer	4b4eba0b9e	src/api2/pull.rs: implement delete flag for vanished groups	2020-01-17 11:24:55 +01:00
Dietmar Maurer	11d89239c3	src/backup/backup_info.rs: new list_groups helper	2020-01-17 10:42:03 +01:00
Dietmar Maurer	b31c8019d7	src/api2/types.rs: define and use GroupListItem	2020-01-17 10:17:18 +01:00
Christian Ebner	2bbbade367	pxar: Refactor fuse and remove unused code. By ambiguously using the Decoder::read_directory_entry() the code is simplified and reading of the DirectoryEntry is concentrated into Context::run_in_context(). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-17 09:40:39 +01:00
Christian Ebner	95c9460c4a	pxar::fuse: Always use start offset of entries as inode. Previously it was disciminated based on the entry mode. For directories, the inode was the offset of the corresponding goodbye tail mark while for all others it was the offset of the filename. By simply using the start offset as calculated from the corresponding goodbye table entry (which yields the archive offset of the filename), the code is simplified and the more ambiguous read_directory_entry() function can be used. The disatvantage of this approach is the need to keep track of the start and end offsets for each entry, as the end offset is needed in order to access the goodbye table of directory entries. The root node still has to be treated special, as it's inode is 1 as per fuse definition and it has no filename as per the pxar file format definition. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-17 09:40:39 +01:00
Christian Ebner	6744440714	pxar::decoder: refactor Decoder::lookup() Use Decoder::read_directory_entry() instead of Decoder::attributes() as this already returns the needed DirectoryEntry. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-17 09:40:39 +01:00
Dietmar Maurer	f357390c15	renamed: src/config/remotes.rs -> src/config/remote.rs And use 'remote' instead of 'remotes' everywhere.	2020-01-16 14:32:06 +01:00
Dietmar Maurer	d2dd827877	src/backup/catalog.rs - SenderWriter: use tokio::task::block_in_place Make sure we do not block the executor.	2020-01-16 14:24:15 +01:00
Dietmar Maurer	94609e2380	src/api2/pull.rs: use reference to remote configuration	2020-01-16 13:55:19 +01:00
Dietmar Maurer	625d19621f	src/client/remote_chunk_reader.rs: use tokio::task::block_in_place to avoid blocking Else we block the current executor.	2020-01-16 13:55:19 +01:00
Wolfgang Bumiller	6a0dc4a5bc	bump proxmox crate version Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-16 11:40:23 +01:00
Wolfgang Bumiller	619495b25e	fix use of deprecated items Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-16 08:58:54 +01:00
Wolfgang Bumiller	18c0df4c92	import ordering cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2020-01-16 08:51:52 +01:00
Christian Ebner	ef2d7f2f96	pxar::fuse: remove readdir implementation By not implementing readdir but only readdirplus, the FUSE_CAP_READDIRPLUS flag is set while the FUSE_CAP_READDIRPLUS_AUTO flag is not set. Thereby the kernel will issue only readdirplus calls. Documentation at: https://libfuse.github.io/doxygen/fuse-3_88_80_2include_2fuse__common_8h.html#a9b90333ad08d0e1c2ed0134d9305ee87 As the expensive part for accessing and reading the attributes is seeking and decoding each directory entry, it is usefull to force readdirplus calls. By this a struct `EntryParam` is returned for each entry, therebye avoiding a subsequent lookup call. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-15 14:44:53 +01:00
Dietmar Maurer	3be839c61c	src/bin/proxmox-backup-manager.rs: add completion function for update commands	2020-01-15 12:42:13 +01:00
Dietmar Maurer	002a191abf	src/api2/config: check digest	2020-01-15 12:27:05 +01:00
Dietmar Maurer	347834df25	src/api2/config: correctly lock files	2020-01-15 11:57:12 +01:00
Oguz Bektas	1c79e1017f	services: fix indentation Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2020-01-15 10:55:04 +01:00
Oguz Bektas	cbe01dc5c7	tools: add read_and_verify_password helper add a helper to perform some basic checks on password prompts. - verification (asks for a 2nd time) - check length also use the new helper where password input in tty is taken to reduce duplicate code. this helper should be used when creating keys, changing passphrases etc. note: this helper can be extended later on to provide better checks for password strength. Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2020-01-15 10:52:57 +01:00
Christian Ebner	ea5061979b	pxar::fuse: Avoid macro to generate filler function, use closure instead. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-15 10:25:16 +01:00
Dietmar Maurer	f3ec5dae7f	src/api2/config: improve api schema by adding return type	2020-01-14 14:55:58 +01:00
Dietmar Maurer	ddc526623f	src/bin/proxmox-backup-manager.rs: add datastore update cli	2020-01-14 14:47:26 +01:00
Dietmar Maurer	c5799e4043	src/api2/config/datastore.rs: improve api, implement update and read	2020-01-14 14:45:56 +01:00
Dietmar Maurer	08195ac823	src/api2/config/remotes.rs: improve api, implement update and read	2020-01-14 14:20:16 +01:00
Dietmar Maurer	16f04b9d79	src/section_config.rs: add lookup_json() helper	2020-01-14 14:19:22 +01:00
Dietmar Maurer	71805bb7ec	src/api2/config/remotes.rs: fix variable name -> s/remote/id/	2020-01-14 13:20:13 +01:00
Dietmar Maurer	d0187a51a9	src/section_config.rs - convert_to_array: optionally add digest datastore::config() -> also return digest remotes::config() -> also return digest	2020-01-14 12:57:03 +01:00
Dietmar Maurer	4566303b05	src/section_config.rs - write: improve error message	2020-01-14 12:02:46 +01:00
Dietmar Maurer	f51420ba1f	src/section_config.rs: do not allow control characters	2020-01-14 11:41:48 +01:00
Dietmar Maurer	7e7b781a18	src/api2/types.rs: also define PASSWORD_FORMAT and use it correctly	2020-01-14 11:32:02 +01:00
Dietmar Maurer	da4a15a351	src/api2/types.rs: define and use PASSWORD_REGEX	2020-01-14 11:22:42 +01:00
Dietmar Maurer	ff620a3d1a	src/api2/types.rs: add regression test for PROXMOX_USER_ID_SCHEMA	2020-01-13 17:26:43 +01:00
Dietmar Maurer	ae62c4fe59	src/api2/types.rs: fix regex patterns	2020-01-13 14:38:31 +01:00
Dietmar Maurer	163dc16c0b	src/api2/types.rs: define PROXMOX_AUTH_REALM_SCHEMA and PROXMOX_USER_ID_SCHEMA And try to use nbew schemas with config api...	2020-01-13 14:18:19 +01:00
Dietmar Maurer	b25f313d66	src/api2/types.rs: define HOSTNAME_FORMAT, DNS_NAME_FORMAT and DNS_NAME_OR_IP_FORMAT	2020-01-13 13:10:52 +01:00
Dietmar Maurer	8aea35fcff	src/config/*: add #[serde(skip_serializing_if="Option::is_none")] to optinal comment	2020-01-13 12:14:14 +01:00
Dietmar Maurer	454c13edce	src/api2/types.rs: define SINGLE_LINE_COMMENT_SCHEMA	2020-01-13 12:02:13 +01:00
Dietmar Maurer	167971ed49	src/api2/types.rs: define REMOTE_ID_SCHEMA here	2020-01-13 11:47:07 +01:00
Christian Ebner	6de36b94b1	pxar::fuse: impl readdirplus callback readdirplus returns the entries together with their `EntryParam`, so subsequent lookups for each of the entries are avoided. In order to reduce code duplication, the code for filling the reply buffer is moved into a macro. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-13 11:17:34 +01:00
Christian Ebner	caa201d895	pxar::fuse: Get rid of broken cacheing Remove the current caching of attrs and goodbye tables as it is broken anyway. This will be replaced with a LRU cache. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-13 11:16:06 +01:00
Oguz Bektas	6d20a29d73	fix typo for function name load_and_decrypt_key Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2020-01-13 11:14:47 +01:00
Dietmar Maurer	50af953e1b	src/section_config.rs: make set_data generic	2020-01-11 11:09:27 +01:00
Dietmar Maurer	688fbe07a1	cleanup config api, add remotes config cli interface	2020-01-11 10:42:09 +01:00
Dietmar Maurer	9e9bc6525e	src/config/datastore.rs: define DataStoreConfig using api macro	2020-01-11 09:18:42 +01:00
Dietmar Maurer	ca44172404	src/api2/config/remotes.rs: cleanup - avoid compiler warnings	2020-01-10 19:25:26 +01:00
Christian Ebner	c7257c5539	pxar::fuse: refactor HashMap to store child parent inode relation. By moving the HashMap into `Context`, the use of lazy_static as well as the additional Mutex can be avoided (`Context` is already guarded by a Mutex). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-10 13:45:58 +01:00
Christian Ebner	90fc97af6a	pxar::decoder::Decoder: include xattrs and payload size in `DirectoryEntry`. By reading and including xattrs and payload size in struct `DirectoryEntry`, the tuple of return types is avoided and the code is simpler. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-10 13:45:40 +01:00
Christian Ebner	6fc053ed85	pxar: encoder: limit number of max entries held at once in memory during archive creation. Limit the total number of entries and therefore the approximate memory consumption instead of doing this on a per directory basis as it was previously. This makes more sense as it limits not only the width but also the depth of the directory tree. Further, instead of hardcoding this value, allow to pass this information as additional optional parameter 'entires-max'. By this, creation of the archive with directories containing a large number of entries is possible. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-10 13:45:08 +01:00
Dietmar Maurer	141304d64e	src/api2/config/remotes.rs: new API to configure remotes	2020-01-10 13:28:15 +01:00
Dietmar Maurer	331b869d96	src/bin/proxmox-backup-manager.rs: implement shell completion for remote datastore name	2020-01-10 11:58:31 +01:00
Dietmar Maurer	eb506c830c	rename src/api2/sync.rs -> src/api2/pull.rs Use "pull" instead of "sync", because this also indicates a direction (like "sync from").	2020-01-10 11:15:16 +01:00
Dietmar Maurer	a81af92f9d	src/section_config.rs: implement generic lookup	2020-01-09 17:35:44 +01:00
Dietmar Maurer	0eb0e02451	src/bin/proxmox-backup-manager.rs: implement cli sync command	2020-01-09 14:52:29 +01:00
Dietmar Maurer	a575320657	src/config/remotes.rs: implement SectionConfig for remote hosts	2020-01-09 14:51:02 +01:00
Christian Ebner	ecbc62261c	pxar: encoder: warn on lacking read permissions instead of fail. If during creation of the archive, files/dirs with lacking read permissions are encountered, the user is displayed a warning and the archive is created without including the file/dir. Previously this resulted in an error and the archive creation failed. In order to implement this also for the .pxarexclude files, the Error type of MatchPattern::from_file() and MatchPattern::from_line() was adopted accordingly. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-09 10:41:19 +01:00
Christian Ebner	88cee60bad	pxar: encoder: fix incorrect goodbye table entries and update displayed path. If nodes are excluded by feature flags, they must not appear in the goodbye table. This is fixed by continuing with the next entry in the for loop. Further the relative path buffer is now poped in order to correctly display the path. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-09 10:39:41 +01:00
Christian Ebner	ec906dcf19	src/pxar/encoder.rs: avoid use of macro and initialize Vec with known capacity. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2020-01-09 10:38:58 +01:00
Dietmar Maurer	71da3d6a1b	src/api2/types.rs: fix SnapshotListItem schema definition	2020-01-09 10:36:19 +01:00
Dietmar Maurer	de8ec04123	src/api2/sync.rs: implement remote sync	2020-01-08 14:03:52 +01:00
Dietmar Maurer	27c9affb23	src/bin/proxmox-backup-client.rs: cleanup - factor out api_datastore_latest_snapshot()	2020-01-07 15:18:36 +01:00
Dietmar Maurer	42af4b8f06	src/bin/proxmox-backup-client.rs: cleanup - factor out api_datastore_list_snapshots()	2020-01-07 15:07:57 +01:00
Dietmar Maurer	af9d4afc66	src/bin/proxmox-backup-client.rs: use SnapshotListItem	2020-01-07 13:25:25 +01:00
Dietmar Maurer	fc189b198c	src/api2/types.rs: define and use new api type SnapshotListItem	2020-01-07 12:52:52 +01:00
Dietmar Maurer	8f14e8fe4c	src/backup/backup_info.rs: implement last_successful_backup() A backup is considered successful if there is a manifest file.	2020-01-06 11:35:22 +01:00
Dietmar Maurer	247a8ca5bb	src/backup/manifest.rs: impl TryFrom DataBlob	2020-01-05 16:28:18 +01:00
Dietmar Maurer	7759eef552	src/backup/datastore.rs: implement cleanup_backup_dir()	2020-01-05 15:15:12 +01:00
Dietmar Maurer	5e1d1bead1	src/client/backup_reader.rs: make compute_file_csum() public	2020-01-05 14:50:54 +01:00
Dietmar Maurer	3743dee6de	src/client/http_client.rs: allow access to AuthInfo fields	2020-01-05 10:31:19 +01:00
Dietmar Maurer	3758b398cd	src/backup/read_chunk.rs: add read_raw_chunk() Allow to read chunk data without decoding.	2020-01-02 13:29:10 +01:00
Dietmar Maurer	2585a8a4e2	src/backup/chunk_store.rs: implement cond_touch_chunk() This will be used by backup sync to test if a chunk already exists.	2020-01-02 13:26:28 +01:00
Dietmar Maurer	1cf5178ac5	src/backup/datastore.rs: new helper try_shared_chunk_store_lock()	2020-01-02 11:00:33 +01:00
Dietmar Maurer	1e8da0a789	src/backup/manifest.rs: new helper archive_type()	2019-12-31 15:23:41 +01:00
Dietmar Maurer	94f085fab5	src/bin/proxmox-backup-proxy.rs: remove unused use clause	2019-12-30 11:50:24 +01:00
Dietmar Maurer	22be470d90	src/bin/proxmox-backup-api.rs: create cert here	2019-12-27 13:44:21 +01:00
Dietmar Maurer	ca6119551d	src/client/http_client.rs - start_h2_connection: correctly return error during upgrade	2019-12-27 13:41:31 +01:00
Wolfgang Bumiller	a546a8a072	proxy/api: await running workers before shutdown Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-27 12:50:33 +01:00
Dietmar Maurer	f8fd5095d8	src/bin/proxmox-backup-proxy.rs: create self signed cert at startup.	2019-12-27 11:21:32 +01:00
Dietmar Maurer	afddffc7ec	proxmox-backup-manager.rs: remove wrong fixme	2019-12-27 10:17:06 +01:00
Dietmar Maurer	e739a8d8b4	src/bin/proxmox-backup-manager.rs: impl. cert info	2019-12-23 13:06:43 +01:00
Dietmar Maurer	550e0d8870	src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate Without calling external openssl binary.	2019-12-22 17:35:06 +01:00
Wolfgang Bumiller	5e62d19cfb	fix create_datastore parameters: comment was declared as optional but unwrap()ed, causing panics when not using it Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-20 12:30:25 +01:00
Dietmar Maurer	645995634a	src/api2/config/datastore.rs - create: pass uid and gid instead of User	2019-12-20 09:23:58 +01:00
Dietmar Maurer	e67770d496	src/backup/chunk_store.rs - create: pass User instead of CreateOptions	2019-12-20 09:11:40 +01:00
Thomas Lamprecht	bca9093520	api/ui: datastore: allow to set simple comment for now forbid all control characters[0] in the comment value, the section config writer cannot cope with newlines in the value, it writes them out literally, allowing "injection" or breaking the whole config. In the webinterface use also a textfield, not a textarea. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-19 17:58:01 +01:00
Wolfgang Bumiller	afdcfb5bc9	let ChunkStore::create take CreateOptions Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-19 13:14:49 +01:00
Dietmar Maurer	47d4712152	src/bin/proxmox-backup-manager.rs: add task management cli	2019-12-19 11:27:36 +01:00
Dietmar Maurer	d6c4a1198a	src/bin/proxmox-backup-client.rs: use async fn	2019-12-19 11:20:59 +01:00
Dietmar Maurer	f74a03da1f	remove tools::getpwnam_ugid, impl. crate::backup::backup_user() And use new nix::unistd::User struct.	2019-12-19 10:20:13 +01:00
Dietmar Maurer	8bf4559b4e	update to nix 0.16	2019-12-19 09:29:44 +01:00
Dietmar Maurer	769f8c9998	src/bin/proxmox-backup-manager.rs: connect to daemon to execute GC Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>	2019-12-19 08:45:27 +01:00
Dietmar Maurer	25de1c8005	src/client/backup_repo.rs: impl. new()	2019-12-19 08:44:38 +01:00
Dietmar Maurer	94913f35ca	src/bin/proxmox-backup-client.rs - start_garbage_collection: add missing output-format parameter	2019-12-19 07:57:53 +01:00
Dietmar Maurer	868c585219	src/backup/datastore.rs: avoid divide by zero	2019-12-19 07:14:23 +01:00
Dietmar Maurer	217170e11e	src/server/worker_task.rs: set worker thread name This gives better logs on thread panic...	2019-12-19 07:07:39 +01:00
Dietmar Maurer	7e210bd0b4	src/backup/chunk_store.rs: create lock file with correct owner	2019-12-19 06:55:53 +01:00
Thomas Lamprecht	2337df7b3d	api2: add very basic 'status/usage' endpoint for nodes For returning the nodes basic "usage status", for now one gets memory and CPU utilization. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-18 17:24:32 +01:00
Dietmar Maurer	bce023c57c	src/tools.rs: fix typo	2019-12-18 14:29:29 +01:00
Dietmar Maurer	0b97bc6158	src/backup/chunk_store.rs: use proxmox::tools::fs::create_path	2019-12-18 12:26:43 +01:00
Dietmar Maurer	f8ec1473be	src/config/datastore.rs: use backup gid instead of uid	2019-12-18 12:21:44 +01:00
Wolfgang Bumiller	feaa1ad35f	replace file_set_contents with replace_file Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-18 11:16:04 +01:00
Dietmar Maurer	a66e7920dc	src/api2/config/datastore.rs: set protected flags for create/delete API	2019-12-18 10:52:45 +01:00
Dietmar Maurer	f0db500808	src/api2/config/datastore.rs: fix api docu	2019-12-18 10:48:19 +01:00
Dietmar Maurer	424766bc3b	src/config/datastore.rs: change file owner/permissions owner(root) => read and write group(backup) => read only	2019-12-18 10:41:58 +01:00
Oguz Bektas	c07b458cac	datastore: set correct owner on datastore.cfg since manager runs as root, we have to specify backup:root owner while saving config. Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2019-12-17 16:59:10 +01:00
Oguz Bektas	14f1e63067	chunk_store: create parent directories 'datastore create storename /path/to/dir/that/may/not/exist' should work. Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>	2019-12-17 15:39:42 +01:00
Christian Ebner	ebfb2df22b	binary_search_tree: add additional doctest for search_binary_tree_by() Make sure a start indexes larger or equal to the array size results in a return value of `None`. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 15:37:02 +01:00
Christian Ebner	48f6d67764	binary_search_tree: fix off by one error in search_binary_tree_by() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 14:57:34 +01:00
Christian Ebner	38d9a69875	catalog: decoder and shell: fix incorrect logic in find matching The find matching was incorrectly performed starting from the parent directroy and not as intended from the entries of the parent directory. Further, the match pattern passed from the catalog shell contains the absolute path of the search entry point as prefix, so find() must always start from the archive root. This is because the match pattern has to be stored in the selected list for a subsequent restore-selected command in the shell. All matching paths are shown as absolute paths with all contents in the subdir, equal to what would be restored by the given pattern. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 14:57:34 +01:00
Dietmar Maurer	81cc71c0d7	use proxmox-mini-journalreader to display syslog	2019-12-17 14:06:48 +01:00
Dietmar Maurer	20197f7cf5	src/api2/node/syslog.rs: use api macro	2019-12-17 13:26:49 +01:00
Thomas Lamprecht	9c01e73c2f	ui: add "Documentation" button to main view Similar to PVE and PMG, for quick access when one has the basic webinterface open anyway. Should move to the "proxmoxHelpButton" once we have an onlineHelp mapping to the docs. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-17 11:22:34 +01:00
Thomas Lamprecht	2d694f8f1f	add CSS file for PBS ExtJS6 basic ui some fitting rules copied over from PVE's ext6-pve.css file. simply place it in the css subfolder where the proxmox-backup-gui.js file is hosted and add a "css/" alias for that directory, the formatter gets use the right content type with that. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-17 11:20:32 +01:00
Christian Ebner	25cdd0e0a1	catalog_shell: impl find for catalog shell. Implements the find command which allows to find and select files for subsequent restore. Files selected for restore are now stored in a Vec instead of a HashSet. This is needed, since instead of the full paths for each file, selected files are now identified by a list of match pattern, where ordering matters. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 11:12:34 +01:00
Christian Ebner	90dfd0a71d	catalog: impl find() for file search by match pattern. find() iterates over the file tree and matches each node against a list of match patterns provided at function call. For each matching node, a callback function with the current directroy stack is called. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 11:12:34 +01:00
Christian Ebner	a333b4ccea	pxar: match_pattern: impl invert() to switch the match type of a MatchPattern. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 11:12:34 +01:00
Christian Ebner	48af80b3ab	pxar: match_pattern: impl traits needed for MatchPattern to be comparable. In order to check if two MatchPattern are identical, which will be used to avoid duplicates. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 11:12:34 +01:00
Christian Ebner	255bb86030	pxar: match_pattern: refactor MatchPattern and introduce MatchPatternSlice. The MatchPattern impl heavily used copies and therefore was inefficient regarding memory management. This patch intoduces MatchPatternSlice as struct to avoid copies and perform the same pattern matching functionality. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-17 11:12:34 +01:00
Wolfgang Bumiller	dc08934563	drop Cancellable future in favor of abortable futures-0.3 has a futures::future::abortable() function which does the exact same, returns an Abortable future with an AbortHandle providing an abort() method. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-17 10:52:07 +01:00
Wolfgang Bumiller	8554ac5ec3	fix broadcast_future test case We used to await all the futures via the runtime's shutdown method, which doesn't exist anymore, so await all the join handles instead. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-17 10:47:12 +01:00
Thomas Lamprecht	ba3eb88d95	generate authkey: public part needs to be readable by backup group else the API proxy cannot use it and fails to run.. Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-17 10:17:03 +01:00
Thomas Lamprecht	9c18e93522	handle_static_file_download: move from and_then to await Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-17 08:56:55 +01:00
Thomas Lamprecht	4e8cff1c9d	cipherbench: align/format output nicely Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-17 08:49:34 +01:00
Thomas Lamprecht	4832f53ab5	cipherbench: be conform with rfc7539, use 12 byte nonce else we get an error from this call, using a 16 byte (128 bit) nonce is currently only supported by the still in draft XChaCha20-Poly1305, not the current default specified by RFC 7539[0], which uses a 12 byte (96 bit) nonce. Fixes the following error: > thread 'main' panicked at 'called `Result::unwrap()` on an `Err` > value: ErrorStack([])', src/libcore/result.rs:1165:5 [0]: https://tools.ietf.org/html/rfc7539 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>	2019-12-17 08:49:07 +01:00
Dietmar Maurer	a47a02ae62	src/bin/proxmox-backup-client.rs: use #[api] macro	2019-12-16 13:49:18 +01:00
Wolfgang Bumiller	78d5436040	client: use api macro for catalog_shell async fn Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-16 11:24:44 +01:00
Dietmar Maurer	87c423751a	src/bin/proxmox-backup-client.rs: use ApiHandler::Async for prune and status	2019-12-16 11:00:43 +01:00
Dietmar Maurer	8a8a470316	src/bin/proxmox-backup-client.rs: use new ApiHandler::Async	2019-12-16 10:06:26 +01:00
Dietmar Maurer	b9799012cf	src/pxar/decoder.rs: require Send	2019-12-16 10:01:51 +01:00
Dietmar Maurer	bb084b9c91	api2: update for latest proxmox-api changes - rename ApiFuture into ApiResponseFuture - impl. ApiHandler::Async	2019-12-16 10:01:51 +01:00
Dietmar Maurer	236a396aa1	src/api2/admin/datastore.rs - prune: log retention options	2019-12-14 16:32:16 +01:00
Dietmar Maurer	503995c767	src/api2/admin/datastore.rs: fix prune worker id	2019-12-14 16:05:21 +01:00
Wolfgang Bumiller	f374ba4c0b	update tests Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-13 12:42:02 +01:00
Wolfgang Bumiller	b5c9a706ce	fix cancellable future Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-13 12:10:04 +01:00
Wolfgang Bumiller	8a1028e0e7	update to url/percent-encoding to 2.1 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-13 11:57:01 +01:00
Wolfgang Bumiller	db0cb9ce0b	update a chunk of stuff to the hyper release Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-13 11:24:41 +01:00
Dietmar Maurer	3f1020b732	src/bin/proxmox-backup-client.rs: implement task stop	2019-12-13 09:43:08 +01:00
Dietmar Maurer	1ae5677de0	src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT	2019-12-12 14:15:40 +01:00
Dietmar Maurer	d0adf270fb	src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA	2019-12-12 12:37:11 +01:00
Dietmar Maurer	66c49c21c3	src/api2/types.rs: factor out DATASTORE_SCHEMA	2019-12-11 13:45:23 +01:00
Dietmar Maurer	567d3e00fb	src/api2/node/tasks.rs: new filter "store" to filter tasks for one store	2019-12-11 12:53:34 +01:00
Christian Ebner	d3dbe52f37	pxar: match_pattern: refactor match_filename The match_filename() in sequentail_decoder and encoder are moved to be static functions of MatchPattern. This allows to reuse the code also in the catalog find implementation. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-11 10:55:01 +01:00
Christian Ebner	f084505ec5	src/backup/catalog_shell.rs: sort output of list-selected Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-11 10:54:20 +01:00
Dietmar Maurer	6d04612955	src/api2/node/tasks.rs: add "running" filter to list only running tasks	2019-12-11 10:52:27 +01:00
Dietmar Maurer	8560fe3e5f	src/api2/node/tasks.rs - extract_upid: avoid duplicate error message	2019-12-10 13:45:56 +01:00
Dietmar Maurer	5830c20560	src/bin/proxmox-backup-client.rs: add simple task management cli	2019-12-10 13:43:53 +01:00
Dietmar Maurer	73e57f244e	src/backup/datastore.rs - garbage-collection: fix average chunk size	2019-12-10 11:25:29 +01:00
Dietmar Maurer	d105176f45	src/bin/proxmox-backup-client.rs: cleanup - factor out view_task_result()	2019-12-09 18:00:22 +01:00
Dietmar Maurer	48ef3c3327	cli: avoid useless .into() This needs latest proxmox-api updates.	2019-12-09 17:40:34 +01:00
Dietmar Maurer	eb7e2ee00b	src/bin/proxmox-backup-client.rs: remove unnecessary .to_owned()	2019-12-09 17:11:48 +01:00
Dietmar Maurer	e5f7def40e	src/bin/proxmox-backup-client.rs - garbage-collection: display task log	2019-12-09 13:29:23 +01:00
Dietmar Maurer	0ec9e1b010	src/bin/proxmox-backup-client.rs: impl. and use complete_pxar_archive_name	2019-12-09 13:14:32 +01:00
Dietmar Maurer	1c6ad6ef90	src/bin/proxmox-backup-client.rs: impl. catalog sub command catalog dump catalog shell	2019-12-09 13:07:25 +01:00
Dietmar Maurer	5a0b484bdd	src/client/task_log.rs: move display_task_log into extra file	2019-12-08 11:27:15 +01:00
Dietmar Maurer	3b03abfe14	src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune And use display_task_log on the client side.	2019-12-08 10:59:47 +01:00
Dietmar Maurer	ec137a99c6	src/bin/proxmox-backup-client.r - display_task_log: use string range instead of as_bytes	2019-12-08 10:25:26 +01:00
Dietmar Maurer	b73ff941f6	src/bin/proxmox-backup-client.r - display_task_log: add new strip_date parameter	2019-12-08 10:22:02 +01:00
Dietmar Maurer	163e9bbe91	src/bin/proxmox-backup-client.rs: new display task log helper New helper to display worker task logs. Use it for prune (experimental).	2019-12-07 16:11:26 +01:00
Dietmar Maurer	6b508dd563	src/server/formatter.rs: impl. new result attribute "active"	2019-12-07 15:29:42 +01:00
Dietmar Maurer	102d8d4136	src/backup/prune.rs: implement --keep-hourly	2019-12-07 11:23:33 +01:00
Dietmar Maurer	052c4aa160	src/backup/prune.rs: prune - correctly use iso_week year	2019-12-06 19:41:38 +01:00
Christian Ebner	fa0b950070	pxar: fuse: add support for list- and getxattr Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-06 16:44:15 +01:00
Christian Ebner	25ad4cbf63	pxar: fuse: cache goodbye table for each directory on opendir and release it on releasedir Cache not only the goodbye table for the last directory but for each opened directory. The opendir fuse callback will fill the cache with the goodbye table and releasedir will remove it from the cache. This should reduce the number of chuncks fetched from the server in some cases. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-06 16:43:59 +01:00
Christian Ebner	02491b8fc6	pxar: fuse: use binary search instead of linear search in goodbye table. The goodbye table is layed out as binary search tree based on the hash, so use this to be more efficient when looking up a hash in the table for directories with a large number of entries. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-06 16:42:01 +01:00
Dietmar Maurer	7074a0b339	src/bin/proxmox-backup-client.rs: white space cleanup	2019-12-06 13:35:59 +01:00
Dietmar Maurer	74fa81b89c	src/bin/proxmox-backup-client.rs: implement prune --dry-run	2019-12-06 13:35:25 +01:00
Dietmar Maurer	1f604f1a5a	src/api2/admin/datastore.rs: add new api test_prune()	2019-12-06 13:06:56 +01:00
Dietmar Maurer	0ab08ac980	src/api2/admin/datastore.rs: cleanup - move schema defininition in front of rust method	2019-12-06 12:36:41 +01:00
Dietmar Maurer	1a0678b601	src/api2/admin/datastore.rs: remove prune parameters from status call Not idea why they are added there.	2019-12-06 12:32:53 +01:00
Dietmar Maurer	9e3f008804	src/backup/prune.rs: add new helper keeps_something()	2019-12-06 12:28:31 +01:00
Dietmar Maurer	d75fbb0a21	src/bin/dump-catalog-shell-cli.rs: generate catalog shell docs	2019-12-06 09:54:13 +01:00
Dietmar Maurer	55c3cb69cd	rename catalog_shell_api() into catalog_shell_cli()	2019-12-06 09:42:05 +01:00
Dietmar Maurer	ecbaa38fa4	src/backup/catalog_shell.rs: factor out catalog_shell_api() In order to extract documentation ...	2019-12-06 09:35:47 +01:00
Christian Ebner	cca67dc0c3	src/backup/readline.rs: removed in favor of new readline impl Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-06 09:03:04 +01:00
Christian Ebner	6934c6fe77	src/backup/catalog_shell.rs: adapt to use API Schema definition and rustyline This major refactoring of the catalog based shell utilizes the new API macro and the API Schema as well as rustyline instead of the old GNU readline C API. The code now has these 3 main components: * The `Shell` which handles the readline loop via rustyline. * The shell functions defined via the API macro. * The `Context` which holds catalog and decoder instances. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-06 09:02:44 +01:00
Dietmar Maurer	9b78352188	src/backup/prune.rs: define new struct PruneOptions	2019-12-06 08:56:27 +01:00
Dietmar Maurer	92acbd69f7	src/backup/backup_info.rs: remove old prune code	2019-12-06 08:29:27 +01:00
Dietmar Maurer	408434360b	src/backup/prune.rs: add PruneMark::KeepPartial Use separate mark for partial backup - dont mess them up with regular ones.	2019-12-06 08:15:32 +01:00
Dietmar Maurer	a8c8366cfa	src/backup/prune.rs: factor out code into remove_incomplete_snapshots()	2019-12-06 08:12:08 +01:00
Dietmar Maurer	dc18849156	src/backup/prune.rs: moved prune related code into extra file	2019-12-06 08:06:21 +01:00
Dietmar Maurer	6f47dd8a0f	src/backup/backup_info.rs - prune: improve algorythm	2019-12-05 18:41:28 +01:00
Dietmar Maurer	8f0b4c1f90	src/api2/admin/datastore.rs - prune: log info about all snapshots	2019-12-05 15:35:11 +01:00
Dietmar Maurer	9ce42759ec	src/backup/backup_info.rs - compute_prune_list: remove unfinished backups	2019-12-05 11:18:10 +01:00
Dietmar Maurer	2c034f8d0a	src/backup/backup_info.rs: improve prune algorithm	2019-12-05 08:55:19 +01:00
Dietmar Maurer	aeeac29bb1	src/backup/backup_info.rs: new compute_prune_list helper So that we can write regression tests for this.	2019-12-04 15:49:11 +01:00
Dietmar Maurer	3d229a4a01	Revert "test another api macro use" This reverts commit `a9aa52e6a8`. Because we do not want to use macros for the backup protocol for now. And because it crashes backup tasks for some unknown reason.	2019-12-04 14:18:11 +01:00
Dietmar Maurer	4f1e40a29e	src/api2/admin/datastore.rs - read_backup_index: add manifest with size So that the client can display the size of the index file.	2019-12-04 10:03:52 +01:00
Wolfgang Bumiller	a9aa52e6a8	test another api macro use The api macro now supports hyphens in parameter names and referencing externally defined `Schema`s, so here's an example. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-12-03 13:41:40 +01:00
Christian Ebner	de0aa53174	pxar: decoder: fix incorrect read for hardlinks. Decoders read must check if the file is a hardlink and read data from the corresponding offset if so. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-03 12:08:23 +01:00
Christian Ebner	b7e18004a2	pxar: decoder: return original name in attributes() The returned filename should be the one of the file given at the offset, not of the one the hardlink points to. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-12-03 12:08:07 +01:00
Dietmar Maurer	7eea56ca00	use cli from proxmox::api	2019-12-02 11:56:29 +01:00
Dietmar Maurer	524b22df35	src/cli/completion.rs - get_help_completion: fix for "help -" expansion	2019-12-02 11:11:31 +01:00
Dietmar Maurer	e8e9bae4d1	src/bin/completion.rs: add echo command	2019-12-02 11:04:40 +01:00
Dietmar Maurer	19d7a59236	src/bin/completion.rs: use api macros	2019-12-02 10:57:19 +01:00
Dietmar Maurer	f9a51322cb	src/cli/command.rs: use insert_help()	2019-12-02 08:52:08 +01:00
Dietmar Maurer	e3e72546dd	src/cli/command.rs: cleanup, remove unused top_def parameter	2019-12-02 08:47:03 +01:00
Dietmar Maurer	2348e77edf	src/cli/completion.rs - get_simple_completion: remove all_arg_param parameters Seems useless.	2019-12-02 08:31:03 +01:00
Dietmar Maurer	28c855c0a2	src/cli: improve docs	2019-12-01 16:41:49 +01:00
Dietmar Maurer	3bf920527c	src/cli/command.rs: make set_help_context private - no need to expose	2019-12-01 12:54:41 +01:00
Dietmar Maurer	bca6d9120a	src/cli/command.rs: improve help command description	2019-12-01 12:40:37 +01:00
Dietmar Maurer	2f6ecc1ccd	src/cli/completion.rs: moved all completion related code here	2019-12-01 11:44:31 +01:00
Dietmar Maurer	a810e05288	src/cli/readline.rs: moved readline related code here	2019-12-01 11:31:35 +01:00
Dietmar Maurer	5d64a0d083	src/cli/shellword.rs: moved shellword related code into this file	2019-12-01 11:17:57 +01:00
Dietmar Maurer	fb8a00bc00	src/cli/command.rs - get_nested_completion: allow fuzzy match	2019-12-01 11:06:54 +01:00
Dietmar Maurer	9498fcf9c5	src/cli/command.rs - get_nested_completion: return nothing if first argument does not match	2019-12-01 10:24:07 +01:00
Dietmar Maurer	309e449db2	src/cli/command.rs: improve get_help_completion	2019-12-01 10:18:47 +01:00
Dietmar Maurer	b8a82922c7	src/cli/command.rs: handle multilevel sub commands in help	2019-12-01 09:42:51 +01:00
Dietmar Maurer	b7bf71dfa1	src/cli/command.rs: fix help completions	2019-11-30 15:43:44 +01:00
Dietmar Maurer	7afc1af7ab	src/bin/completion.rs: add history support	2019-11-30 15:08:37 +01:00
Dietmar Maurer	4b8573da65	src/cli/command.rs - handle_command: return Result instead of exit()	2019-11-30 15:08:00 +01:00
Dietmar Maurer	2b691daf6f	src/cli.rs: add insert_help() method	2019-11-30 14:56:31 +01:00
Dietmar Maurer	1201abcffa	src/cli/command.rs: implement help	2019-11-30 13:02:00 +01:00
Wolfgang Bumiller	8423c1fe64	cli/completion: use match statements and .contains() `match` is a bit more readable than the if-else chains, also replace space_chars.iter().any(\|s\| c == *s) with space_chars.contains(&c) which is also more readable. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-29 13:37:34 +01:00
Dietmar Maurer	e399398444	remove shellwords crate	2019-11-29 12:49:44 +01:00
Dietmar Maurer	ca60ac13d9	use rustyline to replace readline	2019-11-29 12:49:44 +01:00
Wolfgang Bumiller	41874331ed	whitespace fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-29 09:52:15 +01:00
Wolfgang Bumiller	2905f2b5e6	update api macro example usage Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-29 09:51:27 +01:00
Dietmar Maurer	8b130e71da	src/cli/command.rs: cleanup, make handlers more generic	2019-11-27 14:44:21 +01:00
Wolfgang Bumiller	7b6c41078b	update api macro invocation to new style Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-27 14:19:46 +01:00
Christian Ebner	314bb35868	catalog_shell: fix several clippy warnings Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-27 10:13:50 +01:00
Wolfgang Bumiller	6486cb853f	first api macro usage test/example Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-27 10:05:37 +01:00
Christian Ebner	951cf17ee3	catalog_shell: major refactoring of cli command definition and parsing By changing the way shell commands are defined and parsed, this makes it more straight forward to extend the current functionality. The readline input is parsed based on the provided command definition and the given parameters and options are passed to a command specific callback function. In addition, the provided command definition including its description is used to generate a help string to display. The help command shows a list of all supported commands or the help string for the provided command. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-26 13:01:00 +01:00
Christian Ebner	59bc6ad676	catalog_shell: check and return on empty list in order to avoid division by zero Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-26 12:55:33 +01:00
Christian Ebner	4e56b2f792	catalog_shell: do not restore full archive if no entries are selected for restore-selected Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-26 12:55:18 +01:00
Dietmar Maurer	12b4098660	src/cli/format.rs: move doc generation code into extra file	2019-11-26 12:12:53 +01:00
Dietmar Maurer	2f2b87e659	src/pxar/fuse.rs: add impl Context	2019-11-26 11:16:46 +01:00
Dietmar Maurer	eedd1f95eb	src/pxar/fuse.rs: move init and destroy into Session	2019-11-26 11:08:55 +01:00
Dietmar Maurer	970687c982	src/pxar/fuse.rs: cleanup - remove setup_session (moved into new)	2019-11-26 11:04:46 +01:00
Dietmar Maurer	2a11191072	src/pxar/fuse.rs: rename from_decoder into new, new into from_path	2019-11-26 10:56:41 +01:00
Dietmar Maurer	99b5b6cba9	src/pxar/decoder.rs: use trait object as reader So that the Decoder is no longer genertic.	2019-11-26 10:45:11 +01:00
Dietmar Maurer	f701d0335e	src/pxar/sequential_decoder.rs: remove callback from new() And use an extra functzion set_callback() to configure that. Also rewrite pxar/fuse.rs and implement a generic Session (will get further cleanups with next patches).	2019-11-26 09:56:48 +01:00
Dietmar Maurer	49fddd985c	src/cli/command.rs: use static array for arg_param	2019-11-25 11:16:41 +01:00
Dietmar Maurer	329d40b50b	rename ApiHandler::Async into ApiHandler::AsyncHttp	2019-11-23 09:03:21 +01:00
Dietmar Maurer	70fbac84da	src/server/rest.rs: simplify code	2019-11-22 18:44:14 +01:00
Dietmar Maurer	2bbd835b9b	src/server/rest.rs: rename get_request_parameters_async to get_request_parameters	2019-11-22 17:24:16 +01:00
Dietmar Maurer	75a5a68944	src/server/rest.rs - only pass ObjectSchema to get_request_parameters_async()	2019-11-22 17:22:07 +01:00
Dietmar Maurer	ad51d02aa9	src/server/rest.rs: cleanup async code	2019-11-22 13:02:05 +01:00
Wolfgang Bumiller	be2bb37205	remove api_schema submodule Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-22 09:31:45 +01:00
Dietmar Maurer	e57e1cd82d	move src/api_schema/config.rs -> src/server/config.rs	2019-11-22 09:23:03 +01:00
Dietmar Maurer	8256b0e417	src/api_schema/registry.rs: remove dead code	2019-11-22 09:04:52 +01:00
Christian Ebner	3cf73c4e53	src/bin/proxmox-backup-client.rs: impl shell subcommand for client Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 08:22:50 +01:00
Christian Ebner	f14c96ea38	src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:54:04 +01:00
Christian Ebner	38446a9551	src/backup/readline.rs: impl wrapper for GNU readline In order to provide the context needed for tab completion via the readline callback, the needed mut ref is passed via a thread local storage key. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:46:24 +01:00
Christian Ebner	6dba015043	src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute This is needed in order to explicitly clone the values when needed in the catalog shell implementation. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:15:39 +01:00
Christian Ebner	c9f002213e	src/backup/catalog.rs: impl DirEntry::is_directory() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:14:00 +01:00
Christian Ebner	d3bcc7f261	src/pxar/sequential_decoder.rs: create all directories in target path Create the full target path and not fail if an intermediate directory does not exist. This is needed in order to restore multiple archives via the catalog, where the target should further contain each archive name as subdir. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:11:49 +01:00
Christian Ebner	b6ca690095	src/pxar/decoder.rs: return also payload size in lookup Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:05:14 +01:00
Dietmar Maurer	45cc157f7c	docs: add backup reader protocol	2019-11-22 06:59:37 +01:00
Wolfgang Bumiller	cad540e969	api/compat: remove remaining api_schema references Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:53:15 +01:00
Wolfgang Bumiller	a2479cfa1a	api/compat: drop more compat imports from api_schema.rs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:36:28 +01:00
Wolfgang Bumiller	cc4ecf80b0	api/compat: drop api_handler submodule Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:18:41 +01:00
Wolfgang Bumiller	4f1963d259	api/comapt: drop schema submodule Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:17:36 +01:00
Wolfgang Bumiller	2194824f77	api: BoxFut -> ApiFuture Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:16:37 +01:00
Wolfgang Bumiller	3d482025b3	move api schema into proxmox::api crate And leave some compat imports in api_schema.rs to get it to build with minimal changes. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:14:54 +01:00
Dietmar Maurer	552c225948	sort all property lookup tables Required, because we use binary sreach to find items.	2019-11-21 13:32:09 +01:00
Dietmar Maurer	255f378a1b	use const api definitions	2019-11-21 13:32:09 +01:00
Wolfgang Bumiller	c5946faffd	drop md5 crate dependency we already depend on openssl which also provides md5 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-14 11:19:43 +01:00
Wolfgang Bumiller	f569acc5e2	drop uuid crate dependency proxmox::tools now has a Uuid module using the native libuuid. Adds build dependency: libuuid1 (which is a Pre-Depends of util-linux, so always installed anyway). Drops uuid + 16 more crate dependencies. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-14 10:22:29 +01:00
Christian Ebner	b423958d8e	catalog: fix issue with DirEntry start start has to be the parent start - offset Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-12 17:18:13 +01:00
Dietmar Maurer	780dd2b0a1	src/backup/catalog.rs: add some docs	2019-11-12 13:23:40 +01:00
Dietmar Maurer	6aa906b547	src/backup/catalog.rs - CatalogReader: verify magic number	2019-11-12 12:54:06 +01:00
Dietmar Maurer	2ec208aef5	src/backup/catalog.rs - CatalogReader::dump(): use root()	2019-11-12 12:47:21 +01:00
Dietmar Maurer	8f24a9ea18	src/backup/catalog.rs - Direntry::parse: abort if callback return false	2019-11-12 11:41:44 +01:00
Dietmar Maurer	7d017123fd	src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code	2019-11-12 11:41:44 +01:00
Dietmar Maurer	dc9596de45	src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs	2019-11-12 11:41:44 +01:00
Dietmar Maurer	55c0b3cc7c	src/backup/catalog.rs - DirEntry: factor out name attribute And avoid allocations inside parse().	2019-11-12 11:41:44 +01:00
Christian Ebner	3f1c5b5e65	catalog: impl std::fmt::Display trait for CatalogEntryType Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-12 07:00:08 +01:00
Dietmar Maurer	f89359c2f3	src/backup/catalog.rs: clenup - use single impl block	2019-11-10 11:38:31 +01:00
Dietmar Maurer	e8d8fdc61d	src/backup/catalog.rs: remove unused code	2019-11-10 11:37:06 +01:00
Dietmar Maurer	c74c074bfd	src/backup/catalog.rs: write MAGIC at start of file	2019-11-10 11:36:45 +01:00
Dietmar Maurer	265664517a	src/backup/catalog.rs: new parse() helper	2019-11-09 14:37:33 +01:00
Dietmar Maurer	5d92935e03	src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block	2019-11-09 13:21:20 +01:00
Dietmar Maurer	12a1975af9	src/backup/catalog.rs: store u64 compressed This way catalogs get about 1/3 smaller.	2019-11-09 13:07:47 +01:00
Dietmar Maurer	bb19af7302	src/client/backup_reader.rs: factor out download_blob() helper	2019-11-08 13:16:41 +01:00
Dietmar Maurer	7205050059	src/client/backup_reader.rs: factor out download_fixed_index() helper	2019-11-08 12:51:52 +01:00
Dietmar Maurer	c3d84a2281	src/client/backup_reader.rs: factor out download_dynamic_index() helper	2019-11-08 12:51:52 +01:00
Dietmar Maurer	89245fb530	renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs	2019-11-08 10:41:00 +01:00
Dietmar Maurer	bf6e321744	change catalog format, use dynamic index to store catalog. In order to remove size restriction of a single blob.	2019-11-08 10:35:48 +01:00
Dietmar Maurer	e016f9ff2f	src/client/backup_writer.rs: rename force_close to cancel And do not consume self.	2019-11-08 10:34:16 +01:00
Dietmar Maurer	0997967d65	do no restrict index names too much	2019-11-08 10:28:14 +01:00
Christian Ebner	e66621182b	src/pxar/decoder.rs: impl lookup for `Decoder` Allows to lookup an entry in a directory based on the provided `DirectoryEntry`. This is needed to navigate the filesystem based on `DirectoryEntry`s and similar to the find_goodbye_entry() function in src/pxar/fuse.rs Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-06 10:00:41 +01:00
Christian Ebner	fb2554de29	src/pxar/decoder.rs: fix wrong filename check for Decoder::restore() As it turns out the original implementation was correct and the start in `DirectoryEntry` points to the `PxarEntry` and not as wrongly stated to the filename. This reverts the incorrect code and adds comments to the fields clarifying this. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-06 09:59:35 +01:00
Christian Ebner	33ad183a40	src/pxar/decoder.rs: allow to pass match pattern to restore In order to partially restore the archive starting from the directory provided as parameter. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-05 14:19:09 +01:00
Christian Ebner	5e8d600c71	src/pxar/decoder.rs: fix issue with restore `Decoder::restore()` calls the `SequentialDecoder::restore()` which expects to encounter a `PxarEntry` at first. But the start of `DirectoryEntry` points to the filename (except for the root dir), so skip over it. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-05 14:18:29 +01:00
Christian Ebner	9708b2d98c	src/pxar/fuse.rs: refactor Iterator: use find() instead of position() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-04 13:59:16 +01:00
Christian Ebner	b82a101c86	catalog: store current writer position in CatalogBlobWriter Counts the bytes written by the CatalogBlobWriter in order to obtain the stream position, needed to get offset to reference catalog items. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-31 09:11:51 +01:00
Dietmar Maurer	62ee2eb405	avoid some clippy warnings	2019-10-26 11:42:05 +02:00
Dietmar Maurer	834a2f95a0	avoid some clippy warnings	2019-10-25 18:44:51 +02:00
Dietmar Maurer	f58f426e86	src/pxar/binary_search_tree.rs: allow(clippy::many_single_char_names)	2019-10-25 18:22:19 +02:00
Dietmar Maurer	11377a47bb	avoid some clippy warnings	2019-10-25 18:04:37 +02:00
Dietmar Maurer	9fe2f639b8	allow(clippy::cast_ptr_alignment)	2019-10-25 18:03:55 +02:00
Dietmar Maurer	cd69d36b45	src/backup/dynamic_index.rs: use write_all()	2019-10-25 17:22:41 +02:00
Christian Ebner	8abd12f1f3	pxar: decoder: cleanup un-needed Decoder::get_dir() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 08:02:23 +02:00
Christian Ebner	9c25c0330a	pxar: fuse: optimize fuse readdir callback The expensive call to Decoder::read_directory_entry() can be omitted as Decoder::attributes() returns all the information the fuse response needs. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 08:02:03 +02:00
Dietmar Maurer	ffe171c5db	src/pxar/encoder.rs: avoid duplicate check for MAX_DIRECTORY_ENTRIES Simply by reordering code.	2019-10-23 07:55:56 +02:00
Christian Ebner	62d123e50a	pxar: add functionality to pass exclude MatchPatterns on create This exposes the option to pass a list of exclude MatchPattern via the '--exclude' option. The list is encoded as file '.pxarexclude-cli' in the archives root directory. If such a file is present in the filesystem, it is skipped and not included in the archive in order to avoid conflicting information. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 07:31:49 +02:00
Christian Ebner	ba8165c607	pxar: encoder: add encode_pxar_exclude_cli() function. This provides the functionality needed to encode MatchPatterns passed on the cli in the root directory. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 07:30:30 +02:00
Dietmar Maurer	920243b12d	pxar: simplify MatchPattern::to_bytes	2019-10-23 07:29:02 +02:00
Christian Ebner	e50a90e010	pxar: Implement to_bytes() for MatchPattern in order to write them to file. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 07:21:11 +02:00
Dietmar Maurer	a8f10f849e	src/bin/proxmox-backup-client.rs: fix parameters for get_encryption_key_password	2019-10-17 10:28:21 +02:00
Christian Ebner	40c9ff9d81	src/pxar/encoder.rs: include .pxarexclude files into the catalog Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-17 10:25:20 +02:00
Christian Ebner	20feec9fac	pxar: fuse: include nsec information in stat Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-17 10:24:13 +02:00
Dietmar Maurer	649c5ee7b1	load_and_decrtypt_key: allow function closures to readf passphrase	2019-10-16 13:23:52 +02:00
Dietmar Maurer	60ffa9649c	src/backup/manifest.rs: fix property names (use '-' instead of '_')	2019-10-14 12:28:27 +02:00
Dietmar Maurer	4c9f753c56	src/backup/data_blob.rs: define create_zero_chunk as member function and rename it to build_zero_chunk()	2019-10-14 10:58:26 +02:00
Dietmar Maurer	2745151d93	src/backup/data_blob.rs: add create_zero_chunk() helper	2019-10-14 10:44:46 +02:00
Dietmar Maurer	f06b820ac0	src/backup/manifest.rs: add verify_file	2019-10-13 10:09:12 +02:00
Dietmar Maurer	511a47bd73	cleanup	2019-10-13 09:39:21 +02:00
Dietmar Maurer	e17d5d864d	src/backup/manifest.rs: improve parser (try_from)	2019-10-13 08:39:49 +02:00
Dietmar Maurer	59e9ba01c6	src/backup/manifest.rs: new class to generate/parse index.json	2019-10-12 17:58:08 +02:00
Dietmar Maurer	ad6e5a6f51	rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME	2019-10-12 15:53:27 +02:00
Dietmar Maurer	296c50ba3d	src/client/backup_reader.rs: add download_manifest Also add crypt_config as member variable.	2019-10-12 15:50:26 +02:00
Dietmar Maurer	cf9271e27c	src/client/backup_writer.rs: move BackupClient code into extra file And rename it to BackupWriter.	2019-10-12 13:53:11 +02:00
Dietmar Maurer	913acb41fd	src/client/backup_reader.rs: add documentation	2019-10-12 13:26:47 +02:00
Dietmar Maurer	9e490a7479	src/client/backup_reader.rs: split BackupReader code into separate file	2019-10-12 12:57:08 +02:00
Dietmar Maurer	fd04ca7a5a	src/bin/proxmox-backup-client.rs: pass verbose flag to dump_image And print some useful status to stderr.	2019-10-09 12:04:02 +02:00
Dietmar Maurer	2c2097ee75	fix typo in error message	2019-10-07 12:48:56 +02:00
Dietmar Maurer	f7aa6f15d3	src/api2/reader.rs: use tokio::fs::read() to read file async	2019-10-07 12:24:06 +02:00
Dietmar Maurer	7123ff7d43	src/backup/data_blob.rs: avoid Arc<CryptConfig> use simple reference instead.	2019-10-07 11:41:35 +02:00
Dietmar Maurer	4ee8f53d07	remove DataChunk file format - use DataBlob instead	2019-10-06 10:31:06 +02:00
Dietmar Maurer	c26d4b4573	cleanup: remove dead code	2019-10-05 16:10:41 +02:00
Dietmar Maurer	88892ea801	src/bin/proxmox-backup-client.rs: cleanup restore - write directly to /dev/stdout (instead of using stdout line writer) - avoid BufferedFixedReader	2019-10-05 11:52:12 +02:00
Dietmar Maurer	62e05bb3b3	src/client/remote_chunk_reader.rs: pass reference to download_chunk Make sure we do not move data around.	2019-10-05 11:45:20 +02:00
Dietmar Maurer	c0b1b14c05	src/api2/reader.rs - download_chunk: use blocking io Turns out to be twice as fast as tokio async io...	2019-10-05 11:41:19 +02:00
Dietmar Maurer	ff01c1e393	src/client/http_client.rs: store raw data/blob size in index.json	2019-10-02 10:54:06 +02:00
Dietmar Maurer	f16aea687a	src/client/http_client.rs: new helper send_upload_request()	2019-09-28 18:22:48 +02:00
Dietmar Maurer	9e603e254c	src/backup/fixed_index.rs: do not use offset for checksum	2019-09-23 11:30:20 +02:00
Dietmar Maurer	fb6026b66b	backup writer api: verify checksum inside close	2019-09-23 10:56:53 +02:00
Dietmar Maurer	c8c4051aa7	src/client/http_client.rs - H2Client::upload: fix parameter order	2019-09-20 13:19:55 +02:00
Dietmar Maurer	f011dba0f7	src/client/http_client.rs: add method parameter to H2 upload	2019-09-20 12:40:23 +02:00
Dietmar Maurer	792a70b9e1	src/client/http_client.rs: add content type parameter to H2 upload/requets_builder	2019-09-20 12:26:21 +02:00
Dietmar Maurer	a55b2975db	src/client/http_client.rs: cleanup (rename parameter)	2019-09-20 11:57:03 +02:00
Dietmar Maurer	32efac1cce	remove trailing whitespace	2019-09-20 08:12:31 +02:00
Christian Ebner	70235f72f4	src/bin/proxmox-backup-client.rs: add functionality to mount pxar archive. This adds the basic functionality to mount an archive stored in a repository locally via fuse. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 08:01:01 +02:00
Christian Ebner	2fa91f5221	src/bin/pxar.rs: adapt to new deamonize flag Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 08:00:20 +02:00
Christian Ebner	d388e244f5	src/pxar/fuse.rs: make verbose flag independent from deamonize by adding this as new flag Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 08:00:04 +02:00
Christian Ebner	e717e9a3bf	src/pxar/fuse.rs: refactor Session::new() and add Session::from_decoder() in order to be able to create a fuse session with a `Decoder` given as argument. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 07:59:51 +02:00
Christian Ebner	946b72a6b2	src/pxar/fuse.rs: introduce `Context` in order to add caching. This patch introduces `Context` to hold the decoder, ino_offset and caches for the attributes and the goodbye table. By caching, certain callbacks can be handled without the need to read additional data via the decoder, which improves performance. The searching of the goodbye table is refactored as well, avoiding recursive function calls in case of a hash collision. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 07:56:40 +02:00
Christian Ebner	1c93182371	src/pxar/decoder.rs: fix bug in decoder read `offset` points to the `PXAR_FILENAME`, therefore read the filename before the `PXAR_ENTRY`. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 07:54:16 +02:00
Dietmar Maurer	2aba16bde1	src/backup/data_chunk.rs: add into_raw()	2019-09-19 14:24:17 +02:00
Dietmar Maurer	15bb1bba89	BackupClient: add upload() function	2019-09-19 13:59:24 +02:00
Dietmar Maurer	6d4df36c3c	make download_chunk_list() public	2019-09-19 13:17:23 +02:00
Dietmar Maurer	cc2ce4a92f	src/client/http_client.rs: add password parameter to HttpClient::new()	2019-09-16 12:35:23 +02:00
Christian Ebner	0e20b336e1	pxar: fuse: avoid possible hash collision in lookup by additional checking against filename The hash of the filename in the goodbye table items allows to quickly compare to a hashed filename. Unfortunately, a matching hash is no garantee for matching filenames as hash collisions are possible. This patch fixes such possible collisions by further checking the filenames once a matching hash has been found. This introduces no significant extra cost (except for the filename comparison) for cases with matching hashes, as the lookup call has to seek and read the file attributes (including the filename) anyway. In cases with hash collision, the next matching item is read and treaded analogously (what means we need at least one extra seek). As collisions should be not that frequent, this should be an acceptable penalty. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-16 10:22:22 +02:00
Wolfgang Bumiller	ba5e67475a	clippy: fix access to unaligned buffer in dynamic reader Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:50:45 +02:00
Wolfgang Bumiller	b526bd1408	backup: cleanup unused imports Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:46:46 +02:00
Wolfgang Bumiller	4dc79bb1d9	clippy: fix access to unaligned buffer via the newly introduced ReadExt::read_host_value_boxed() from proxmox::tools::io. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:45:44 +02:00
Wolfgang Bumiller	ccdf3ad1ec	backup: use static assertion in fixed index reader Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:43:25 +02:00
Wolfgang Bumiller	34a3845b01	clippy: don't use clone a double reference Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:02:30 +02:00
Wolfgang Bumiller	b21851cc5c	clippy: collapse identical if branches Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:57:58 +02:00
Wolfgang Bumiller	299a2f18b2	fixup variable naming Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:57:23 +02:00
Wolfgang Bumiller	175eeb870d	clippy: use write_all in file logger Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:56:09 +02:00
Wolfgang Bumiller	5a54935bc9	clippy: fix pointer to temporary value Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:55:56 +02:00
Wolfgang Bumiller	bd430c225b	clippy: remove a loop{} which never actually loops Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 12:31:39 +02:00
Wolfgang Bumiller	44fed91e17	more clippy fixups mostly indentation changes, view with `-w` Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 12:23:12 +02:00
Wolfgang Bumiller	653b1ca10e	some clippy fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 12:14:05 +02:00
Christian Ebner	03310bea12	src/pxar/fuse.rs: s/Buf/ReplyBuf/ and s/BufState/ReplyBufState/ Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-11 11:27:18 +02:00
Christian Ebner	48cc1b8234	src/pxar/fuse.rs: refactor stat and fix i-node mapping The functionality of stat is split into smaller sub-functions, which allows to reuse them more flexible, as the code flow is similar but not always the same. By this, the ugly and incorrect re-setting of the i-node in the lookup callback function is avoided. The correct i-node is now calculated beforehand and stat simply creates a `libc::stat` struct from the provided parameters. Also, this fixes incorrect i-node assignments in the readdir callback function. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-11 11:26:55 +02:00
Christian Ebner	22eaa905a4	src/pxar/fuse.rs: impl readdir callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:38:58 +02:00
Christian Ebner	1652192f4d	src/pxar/fuse.rs: impl readlink callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:38:48 +02:00
Christian Ebner	3710b23db5	src/pxar/decoder.rs: read_link has to read filename before entry. The offset points to the start of the item, which is the filename Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:36:04 +02:00
Christian Ebner	7eb9848ba7	src/pxar/fuse.rs: impl opendir callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:35:13 +02:00
Christian Ebner	2c5fd378d3	src/pxar/fuse.rs: impl read callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:34:55 +02:00
Christian Ebner	97e4db9dd5	src/pxar/fuse.rs: impl open callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:34:18 +02:00
Christian Ebner	b00689254a	src/pxar/fuse.rs: impl lookup callback for fuse The lookup call checks if the given filename is found in the directory referenced by the i-node by calclulating the filenames hash and looking it up in the directories goodbye table. If found, the entries parameters are returned. In order to be able to lookup the parent offset by a given file offset in the readdir callback, this also stores the corresponding values in a HashMap. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:54:23 +02:00
Christian Ebner	1443927c02	src/pxar/decoder.rs: correctly handle hardlinks in `Decoder`s attribute function If the attributes are then returned by seeking to the new offset. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:46:39 +02:00
Christian Ebner	d00097a0e6	src/pxar/decoder.rs: split functionality of list_dir into list_dir and goodbye_table In order to read the contents of the goodbye table while keeping the functionality of list_dir in place as is. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:46:29 +02:00
Christian Ebner	ac12570e99	src/pxar/decoder.rs: make read_directory_entry usable by fuse ... and thereby allow it to read a single directory entry based on the start and end archive offsets. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:32:48 +02:00
Christian Ebner	3f7f4f34be	src/pxar/decoder.rs: remove `DirectoryEntry` start_offset and end_offset functions ... as they are not needed with the latest iteration of the fuse callback function impl (which never made it into the repos). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:30:45 +02:00
Christian Ebner	ac30bc3326	src/pxar/decoder.rs: correctly read, check and skip filename The previous implementation simply skipped over `size` bytes, which is not correct as size includes also the header. By relying on `SequentailDecoder`s read_filename function, this is correctly taken care of plus some more integrity checks. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:30:21 +02:00
Dietmar Maurer	4a7de56e2f	avoid injecting ENV vars from Makefile So that we can run "cargo build" without setting vars manually.	2019-09-09 10:51:08 +02:00
Christian Ebner	d21ae955a6	src/pxar/fuse.rs: impl getattr callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 12:24:14 +02:00
Christian Ebner	ec04ea81f2	src/pxar/fuse.rs: Refactor run_in_context and remove inode_to_offset and offset_to_inode as their functionality is used only once so it makes more sense to keep them inline. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 12:22:53 +02:00
Christian Ebner	8413083fc5	src/pxar/fuse.rs: Improve comments, add rustfmt skip tag and reafctor Also, removes an unused println statement in the decoder callback function and fixes a typo. Further, use ABI compatible Option<&T> for FFI to avoid use of raw pointers. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 12:22:17 +02:00
Dietmar Maurer	54a5a885a3	src/client/http_client.rs: use async for upload_speedtest()	2019-09-06 08:55:47 +02:00

... 10 11 12 13 14 ...

2516 Commits