proxmox-backup

Author	SHA1	Message	Date
Dietmar Maurer	28c855c0a2	src/cli: improve docs	2019-12-01 16:41:49 +01:00
Dietmar Maurer	3bf920527c	src/cli/command.rs: make set_help_context private - no need to expose	2019-12-01 12:54:41 +01:00
Dietmar Maurer	bca6d9120a	src/cli/command.rs: improve help command description	2019-12-01 12:40:37 +01:00
Dietmar Maurer	2f6ecc1ccd	src/cli/completion.rs: moved all completion related code here	2019-12-01 11:44:31 +01:00
Dietmar Maurer	a810e05288	src/cli/readline.rs: moved readline related code here	2019-12-01 11:31:35 +01:00
Dietmar Maurer	5d64a0d083	src/cli/shellword.rs: moved shellword related code into this file	2019-12-01 11:17:57 +01:00
Dietmar Maurer	fb8a00bc00	src/cli/command.rs - get_nested_completion: allow fuzzy match	2019-12-01 11:06:54 +01:00
Dietmar Maurer	9498fcf9c5	src/cli/command.rs - get_nested_completion: return nothing if first argument does not match	2019-12-01 10:24:07 +01:00
Dietmar Maurer	309e449db2	src/cli/command.rs: improve get_help_completion	2019-12-01 10:18:47 +01:00
Dietmar Maurer	b8a82922c7	src/cli/command.rs: handle multilevel sub commands in help	2019-12-01 09:42:51 +01:00
Dietmar Maurer	b7bf71dfa1	src/cli/command.rs: fix help completions	2019-11-30 15:43:44 +01:00
Dietmar Maurer	7afc1af7ab	src/bin/completion.rs: add history support	2019-11-30 15:08:37 +01:00
Dietmar Maurer	4b8573da65	src/cli/command.rs - handle_command: return Result instead of exit()	2019-11-30 15:08:00 +01:00
Dietmar Maurer	2b691daf6f	src/cli.rs: add insert_help() method	2019-11-30 14:56:31 +01:00
Dietmar Maurer	1201abcffa	src/cli/command.rs: implement help	2019-11-30 13:02:00 +01:00
Wolfgang Bumiller	8423c1fe64	cli/completion: use match statements and .contains() `match` is a bit more readable than the if-else chains, also replace space_chars.iter().any(\|s\| c == *s) with space_chars.contains(&c) which is also more readable. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-29 13:37:34 +01:00
Dietmar Maurer	e399398444	remove shellwords crate	2019-11-29 12:49:44 +01:00
Dietmar Maurer	ca60ac13d9	use rustyline to replace readline	2019-11-29 12:49:44 +01:00
Wolfgang Bumiller	41874331ed	whitespace fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-29 09:52:15 +01:00
Wolfgang Bumiller	2905f2b5e6	update api macro example usage Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-29 09:51:27 +01:00
Dietmar Maurer	8b130e71da	src/cli/command.rs: cleanup, make handlers more generic	2019-11-27 14:44:21 +01:00
Wolfgang Bumiller	7b6c41078b	update api macro invocation to new style Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-27 14:19:46 +01:00
Christian Ebner	314bb35868	catalog_shell: fix several clippy warnings Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-27 10:13:50 +01:00
Wolfgang Bumiller	6486cb853f	first api macro usage test/example Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-27 10:05:37 +01:00
Christian Ebner	951cf17ee3	catalog_shell: major refactoring of cli command definition and parsing By changing the way shell commands are defined and parsed, this makes it more straight forward to extend the current functionality. The readline input is parsed based on the provided command definition and the given parameters and options are passed to a command specific callback function. In addition, the provided command definition including its description is used to generate a help string to display. The help command shows a list of all supported commands or the help string for the provided command. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-26 13:01:00 +01:00
Christian Ebner	59bc6ad676	catalog_shell: check and return on empty list in order to avoid division by zero Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-26 12:55:33 +01:00
Christian Ebner	4e56b2f792	catalog_shell: do not restore full archive if no entries are selected for restore-selected Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-26 12:55:18 +01:00
Dietmar Maurer	12b4098660	src/cli/format.rs: move doc generation code into extra file	2019-11-26 12:12:53 +01:00
Dietmar Maurer	2f2b87e659	src/pxar/fuse.rs: add impl Context	2019-11-26 11:16:46 +01:00
Dietmar Maurer	eedd1f95eb	src/pxar/fuse.rs: move init and destroy into Session	2019-11-26 11:08:55 +01:00
Dietmar Maurer	970687c982	src/pxar/fuse.rs: cleanup - remove setup_session (moved into new)	2019-11-26 11:04:46 +01:00
Dietmar Maurer	2a11191072	src/pxar/fuse.rs: rename from_decoder into new, new into from_path	2019-11-26 10:56:41 +01:00
Dietmar Maurer	99b5b6cba9	src/pxar/decoder.rs: use trait object as reader So that the Decoder is no longer genertic.	2019-11-26 10:45:11 +01:00
Dietmar Maurer	f701d0335e	src/pxar/sequential_decoder.rs: remove callback from new() And use an extra functzion set_callback() to configure that. Also rewrite pxar/fuse.rs and implement a generic Session (will get further cleanups with next patches).	2019-11-26 09:56:48 +01:00
Dietmar Maurer	49fddd985c	src/cli/command.rs: use static array for arg_param	2019-11-25 11:16:41 +01:00
Dietmar Maurer	329d40b50b	rename ApiHandler::Async into ApiHandler::AsyncHttp	2019-11-23 09:03:21 +01:00
Dietmar Maurer	70fbac84da	src/server/rest.rs: simplify code	2019-11-22 18:44:14 +01:00
Dietmar Maurer	2bbd835b9b	src/server/rest.rs: rename get_request_parameters_async to get_request_parameters	2019-11-22 17:24:16 +01:00
Dietmar Maurer	75a5a68944	src/server/rest.rs - only pass ObjectSchema to get_request_parameters_async()	2019-11-22 17:22:07 +01:00
Dietmar Maurer	ad51d02aa9	src/server/rest.rs: cleanup async code	2019-11-22 13:02:05 +01:00
Wolfgang Bumiller	be2bb37205	remove api_schema submodule Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-22 09:31:45 +01:00
Dietmar Maurer	e57e1cd82d	move src/api_schema/config.rs -> src/server/config.rs	2019-11-22 09:23:03 +01:00
Dietmar Maurer	8256b0e417	src/api_schema/registry.rs: remove dead code	2019-11-22 09:04:52 +01:00
Christian Ebner	3cf73c4e53	src/bin/proxmox-backup-client.rs: impl shell subcommand for client Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 08:22:50 +01:00
Christian Ebner	f14c96ea38	src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:54:04 +01:00
Christian Ebner	38446a9551	src/backup/readline.rs: impl wrapper for GNU readline In order to provide the context needed for tab completion via the readline callback, the needed mut ref is passed via a thread local storage key. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:46:24 +01:00
Christian Ebner	6dba015043	src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute This is needed in order to explicitly clone the values when needed in the catalog shell implementation. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:15:39 +01:00
Christian Ebner	c9f002213e	src/backup/catalog.rs: impl DirEntry::is_directory() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:14:00 +01:00
Christian Ebner	d3bcc7f261	src/pxar/sequential_decoder.rs: create all directories in target path Create the full target path and not fail if an intermediate directory does not exist. This is needed in order to restore multiple archives via the catalog, where the target should further contain each archive name as subdir. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:11:49 +01:00
Christian Ebner	b6ca690095	src/pxar/decoder.rs: return also payload size in lookup Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-22 07:05:14 +01:00
Dietmar Maurer	45cc157f7c	docs: add backup reader protocol	2019-11-22 06:59:37 +01:00
Wolfgang Bumiller	cad540e969	api/compat: remove remaining api_schema references Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:53:15 +01:00
Wolfgang Bumiller	a2479cfa1a	api/compat: drop more compat imports from api_schema.rs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:36:28 +01:00
Wolfgang Bumiller	cc4ecf80b0	api/compat: drop api_handler submodule Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:18:41 +01:00
Wolfgang Bumiller	4f1963d259	api/comapt: drop schema submodule Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:17:36 +01:00
Wolfgang Bumiller	2194824f77	api: BoxFut -> ApiFuture Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:16:37 +01:00
Wolfgang Bumiller	3d482025b3	move api schema into proxmox::api crate And leave some compat imports in api_schema.rs to get it to build with minimal changes. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-21 14:14:54 +01:00
Dietmar Maurer	552c225948	sort all property lookup tables Required, because we use binary sreach to find items.	2019-11-21 13:32:09 +01:00
Dietmar Maurer	255f378a1b	use const api definitions	2019-11-21 13:32:09 +01:00
Wolfgang Bumiller	c5946faffd	drop md5 crate dependency we already depend on openssl which also provides md5 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-14 11:19:43 +01:00
Wolfgang Bumiller	f569acc5e2	drop uuid crate dependency proxmox::tools now has a Uuid module using the native libuuid. Adds build dependency: libuuid1 (which is a Pre-Depends of util-linux, so always installed anyway). Drops uuid + 16 more crate dependencies. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-11-14 10:22:29 +01:00
Christian Ebner	b423958d8e	catalog: fix issue with DirEntry start start has to be the parent start - offset Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-12 17:18:13 +01:00
Dietmar Maurer	780dd2b0a1	src/backup/catalog.rs: add some docs	2019-11-12 13:23:40 +01:00
Dietmar Maurer	6aa906b547	src/backup/catalog.rs - CatalogReader: verify magic number	2019-11-12 12:54:06 +01:00
Dietmar Maurer	2ec208aef5	src/backup/catalog.rs - CatalogReader::dump(): use root()	2019-11-12 12:47:21 +01:00
Dietmar Maurer	8f24a9ea18	src/backup/catalog.rs - Direntry::parse: abort if callback return false	2019-11-12 11:41:44 +01:00
Dietmar Maurer	7d017123fd	src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code	2019-11-12 11:41:44 +01:00
Dietmar Maurer	dc9596de45	src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs	2019-11-12 11:41:44 +01:00
Dietmar Maurer	55c0b3cc7c	src/backup/catalog.rs - DirEntry: factor out name attribute And avoid allocations inside parse().	2019-11-12 11:41:44 +01:00
Christian Ebner	3f1c5b5e65	catalog: impl std::fmt::Display trait for CatalogEntryType Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-12 07:00:08 +01:00
Dietmar Maurer	f89359c2f3	src/backup/catalog.rs: clenup - use single impl block	2019-11-10 11:38:31 +01:00
Dietmar Maurer	e8d8fdc61d	src/backup/catalog.rs: remove unused code	2019-11-10 11:37:06 +01:00
Dietmar Maurer	c74c074bfd	src/backup/catalog.rs: write MAGIC at start of file	2019-11-10 11:36:45 +01:00
Dietmar Maurer	265664517a	src/backup/catalog.rs: new parse() helper	2019-11-09 14:37:33 +01:00
Dietmar Maurer	5d92935e03	src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block	2019-11-09 13:21:20 +01:00
Dietmar Maurer	12a1975af9	src/backup/catalog.rs: store u64 compressed This way catalogs get about 1/3 smaller.	2019-11-09 13:07:47 +01:00
Dietmar Maurer	bb19af7302	src/client/backup_reader.rs: factor out download_blob() helper	2019-11-08 13:16:41 +01:00
Dietmar Maurer	7205050059	src/client/backup_reader.rs: factor out download_fixed_index() helper	2019-11-08 12:51:52 +01:00
Dietmar Maurer	c3d84a2281	src/client/backup_reader.rs: factor out download_dynamic_index() helper	2019-11-08 12:51:52 +01:00
Dietmar Maurer	89245fb530	renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs	2019-11-08 10:41:00 +01:00
Dietmar Maurer	bf6e321744	change catalog format, use dynamic index to store catalog. In order to remove size restriction of a single blob.	2019-11-08 10:35:48 +01:00
Dietmar Maurer	e016f9ff2f	src/client/backup_writer.rs: rename force_close to cancel And do not consume self.	2019-11-08 10:34:16 +01:00
Dietmar Maurer	0997967d65	do no restrict index names too much	2019-11-08 10:28:14 +01:00
Christian Ebner	e66621182b	src/pxar/decoder.rs: impl lookup for `Decoder` Allows to lookup an entry in a directory based on the provided `DirectoryEntry`. This is needed to navigate the filesystem based on `DirectoryEntry`s and similar to the find_goodbye_entry() function in src/pxar/fuse.rs Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-06 10:00:41 +01:00
Christian Ebner	fb2554de29	src/pxar/decoder.rs: fix wrong filename check for Decoder::restore() As it turns out the original implementation was correct and the start in `DirectoryEntry` points to the `PxarEntry` and not as wrongly stated to the filename. This reverts the incorrect code and adds comments to the fields clarifying this. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-06 09:59:35 +01:00
Christian Ebner	33ad183a40	src/pxar/decoder.rs: allow to pass match pattern to restore In order to partially restore the archive starting from the directory provided as parameter. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-05 14:19:09 +01:00
Christian Ebner	5e8d600c71	src/pxar/decoder.rs: fix issue with restore `Decoder::restore()` calls the `SequentialDecoder::restore()` which expects to encounter a `PxarEntry` at first. But the start of `DirectoryEntry` points to the filename (except for the root dir), so skip over it. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-05 14:18:29 +01:00
Christian Ebner	9708b2d98c	src/pxar/fuse.rs: refactor Iterator: use find() instead of position() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-11-04 13:59:16 +01:00
Christian Ebner	b82a101c86	catalog: store current writer position in CatalogBlobWriter Counts the bytes written by the CatalogBlobWriter in order to obtain the stream position, needed to get offset to reference catalog items. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-31 09:11:51 +01:00
Dietmar Maurer	62ee2eb405	avoid some clippy warnings	2019-10-26 11:42:05 +02:00
Dietmar Maurer	834a2f95a0	avoid some clippy warnings	2019-10-25 18:44:51 +02:00
Dietmar Maurer	f58f426e86	src/pxar/binary_search_tree.rs: allow(clippy::many_single_char_names)	2019-10-25 18:22:19 +02:00
Dietmar Maurer	11377a47bb	avoid some clippy warnings	2019-10-25 18:04:37 +02:00
Dietmar Maurer	9fe2f639b8	allow(clippy::cast_ptr_alignment)	2019-10-25 18:03:55 +02:00
Dietmar Maurer	cd69d36b45	src/backup/dynamic_index.rs: use write_all()	2019-10-25 17:22:41 +02:00
Christian Ebner	8abd12f1f3	pxar: decoder: cleanup un-needed Decoder::get_dir() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 08:02:23 +02:00
Christian Ebner	9c25c0330a	pxar: fuse: optimize fuse readdir callback The expensive call to Decoder::read_directory_entry() can be omitted as Decoder::attributes() returns all the information the fuse response needs. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 08:02:03 +02:00
Dietmar Maurer	ffe171c5db	src/pxar/encoder.rs: avoid duplicate check for MAX_DIRECTORY_ENTRIES Simply by reordering code.	2019-10-23 07:55:56 +02:00
Christian Ebner	62d123e50a	pxar: add functionality to pass exclude MatchPatterns on create This exposes the option to pass a list of exclude MatchPattern via the '--exclude' option. The list is encoded as file '.pxarexclude-cli' in the archives root directory. If such a file is present in the filesystem, it is skipped and not included in the archive in order to avoid conflicting information. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 07:31:49 +02:00
Christian Ebner	ba8165c607	pxar: encoder: add encode_pxar_exclude_cli() function. This provides the functionality needed to encode MatchPatterns passed on the cli in the root directory. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 07:30:30 +02:00
Dietmar Maurer	920243b12d	pxar: simplify MatchPattern::to_bytes	2019-10-23 07:29:02 +02:00
Christian Ebner	e50a90e010	pxar: Implement to_bytes() for MatchPattern in order to write them to file. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-23 07:21:11 +02:00
Dietmar Maurer	a8f10f849e	src/bin/proxmox-backup-client.rs: fix parameters for get_encryption_key_password	2019-10-17 10:28:21 +02:00
Christian Ebner	40c9ff9d81	src/pxar/encoder.rs: include .pxarexclude files into the catalog Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-17 10:25:20 +02:00
Christian Ebner	20feec9fac	pxar: fuse: include nsec information in stat Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-10-17 10:24:13 +02:00
Dietmar Maurer	649c5ee7b1	load_and_decrtypt_key: allow function closures to readf passphrase	2019-10-16 13:23:52 +02:00
Dietmar Maurer	60ffa9649c	src/backup/manifest.rs: fix property names (use '-' instead of '_')	2019-10-14 12:28:27 +02:00
Dietmar Maurer	4c9f753c56	src/backup/data_blob.rs: define create_zero_chunk as member function and rename it to build_zero_chunk()	2019-10-14 10:58:26 +02:00
Dietmar Maurer	2745151d93	src/backup/data_blob.rs: add create_zero_chunk() helper	2019-10-14 10:44:46 +02:00
Dietmar Maurer	f06b820ac0	src/backup/manifest.rs: add verify_file	2019-10-13 10:09:12 +02:00
Dietmar Maurer	511a47bd73	cleanup	2019-10-13 09:39:21 +02:00
Dietmar Maurer	e17d5d864d	src/backup/manifest.rs: improve parser (try_from)	2019-10-13 08:39:49 +02:00
Dietmar Maurer	59e9ba01c6	src/backup/manifest.rs: new class to generate/parse index.json	2019-10-12 17:58:08 +02:00
Dietmar Maurer	ad6e5a6f51	rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME	2019-10-12 15:53:27 +02:00
Dietmar Maurer	296c50ba3d	src/client/backup_reader.rs: add download_manifest Also add crypt_config as member variable.	2019-10-12 15:50:26 +02:00
Dietmar Maurer	cf9271e27c	src/client/backup_writer.rs: move BackupClient code into extra file And rename it to BackupWriter.	2019-10-12 13:53:11 +02:00
Dietmar Maurer	913acb41fd	src/client/backup_reader.rs: add documentation	2019-10-12 13:26:47 +02:00
Dietmar Maurer	9e490a7479	src/client/backup_reader.rs: split BackupReader code into separate file	2019-10-12 12:57:08 +02:00
Dietmar Maurer	fd04ca7a5a	src/bin/proxmox-backup-client.rs: pass verbose flag to dump_image And print some useful status to stderr.	2019-10-09 12:04:02 +02:00
Dietmar Maurer	2c2097ee75	fix typo in error message	2019-10-07 12:48:56 +02:00
Dietmar Maurer	f7aa6f15d3	src/api2/reader.rs: use tokio::fs::read() to read file async	2019-10-07 12:24:06 +02:00
Dietmar Maurer	7123ff7d43	src/backup/data_blob.rs: avoid Arc<CryptConfig> use simple reference instead.	2019-10-07 11:41:35 +02:00
Dietmar Maurer	4ee8f53d07	remove DataChunk file format - use DataBlob instead	2019-10-06 10:31:06 +02:00
Dietmar Maurer	c26d4b4573	cleanup: remove dead code	2019-10-05 16:10:41 +02:00
Dietmar Maurer	88892ea801	src/bin/proxmox-backup-client.rs: cleanup restore - write directly to /dev/stdout (instead of using stdout line writer) - avoid BufferedFixedReader	2019-10-05 11:52:12 +02:00
Dietmar Maurer	62e05bb3b3	src/client/remote_chunk_reader.rs: pass reference to download_chunk Make sure we do not move data around.	2019-10-05 11:45:20 +02:00
Dietmar Maurer	c0b1b14c05	src/api2/reader.rs - download_chunk: use blocking io Turns out to be twice as fast as tokio async io...	2019-10-05 11:41:19 +02:00
Dietmar Maurer	ff01c1e393	src/client/http_client.rs: store raw data/blob size in index.json	2019-10-02 10:54:06 +02:00
Dietmar Maurer	f16aea687a	src/client/http_client.rs: new helper send_upload_request()	2019-09-28 18:22:48 +02:00
Dietmar Maurer	9e603e254c	src/backup/fixed_index.rs: do not use offset for checksum	2019-09-23 11:30:20 +02:00
Dietmar Maurer	fb6026b66b	backup writer api: verify checksum inside close	2019-09-23 10:56:53 +02:00
Dietmar Maurer	c8c4051aa7	src/client/http_client.rs - H2Client::upload: fix parameter order	2019-09-20 13:19:55 +02:00
Dietmar Maurer	f011dba0f7	src/client/http_client.rs: add method parameter to H2 upload	2019-09-20 12:40:23 +02:00
Dietmar Maurer	792a70b9e1	src/client/http_client.rs: add content type parameter to H2 upload/requets_builder	2019-09-20 12:26:21 +02:00
Dietmar Maurer	a55b2975db	src/client/http_client.rs: cleanup (rename parameter)	2019-09-20 11:57:03 +02:00
Dietmar Maurer	32efac1cce	remove trailing whitespace	2019-09-20 08:12:31 +02:00
Christian Ebner	70235f72f4	src/bin/proxmox-backup-client.rs: add functionality to mount pxar archive. This adds the basic functionality to mount an archive stored in a repository locally via fuse. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 08:01:01 +02:00
Christian Ebner	2fa91f5221	src/bin/pxar.rs: adapt to new deamonize flag Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 08:00:20 +02:00
Christian Ebner	d388e244f5	src/pxar/fuse.rs: make verbose flag independent from deamonize by adding this as new flag Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 08:00:04 +02:00
Christian Ebner	e717e9a3bf	src/pxar/fuse.rs: refactor Session::new() and add Session::from_decoder() in order to be able to create a fuse session with a `Decoder` given as argument. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 07:59:51 +02:00
Christian Ebner	946b72a6b2	src/pxar/fuse.rs: introduce `Context` in order to add caching. This patch introduces `Context` to hold the decoder, ino_offset and caches for the attributes and the goodbye table. By caching, certain callbacks can be handled without the need to read additional data via the decoder, which improves performance. The searching of the goodbye table is refactored as well, avoiding recursive function calls in case of a hash collision. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 07:56:40 +02:00
Christian Ebner	1c93182371	src/pxar/decoder.rs: fix bug in decoder read `offset` points to the `PXAR_FILENAME`, therefore read the filename before the `PXAR_ENTRY`. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-20 07:54:16 +02:00
Dietmar Maurer	2aba16bde1	src/backup/data_chunk.rs: add into_raw()	2019-09-19 14:24:17 +02:00
Dietmar Maurer	15bb1bba89	BackupClient: add upload() function	2019-09-19 13:59:24 +02:00
Dietmar Maurer	6d4df36c3c	make download_chunk_list() public	2019-09-19 13:17:23 +02:00
Dietmar Maurer	cc2ce4a92f	src/client/http_client.rs: add password parameter to HttpClient::new()	2019-09-16 12:35:23 +02:00
Christian Ebner	0e20b336e1	pxar: fuse: avoid possible hash collision in lookup by additional checking against filename The hash of the filename in the goodbye table items allows to quickly compare to a hashed filename. Unfortunately, a matching hash is no garantee for matching filenames as hash collisions are possible. This patch fixes such possible collisions by further checking the filenames once a matching hash has been found. This introduces no significant extra cost (except for the filename comparison) for cases with matching hashes, as the lookup call has to seek and read the file attributes (including the filename) anyway. In cases with hash collision, the next matching item is read and treaded analogously (what means we need at least one extra seek). As collisions should be not that frequent, this should be an acceptable penalty. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-16 10:22:22 +02:00
Wolfgang Bumiller	ba5e67475a	clippy: fix access to unaligned buffer in dynamic reader Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:50:45 +02:00
Wolfgang Bumiller	b526bd1408	backup: cleanup unused imports Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:46:46 +02:00
Wolfgang Bumiller	4dc79bb1d9	clippy: fix access to unaligned buffer via the newly introduced ReadExt::read_host_value_boxed() from proxmox::tools::io. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:45:44 +02:00
Wolfgang Bumiller	ccdf3ad1ec	backup: use static assertion in fixed index reader Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:43:25 +02:00
Wolfgang Bumiller	34a3845b01	clippy: don't use clone a double reference Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 14:02:30 +02:00
Wolfgang Bumiller	b21851cc5c	clippy: collapse identical if branches Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:57:58 +02:00
Wolfgang Bumiller	299a2f18b2	fixup variable naming Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:57:23 +02:00
Wolfgang Bumiller	175eeb870d	clippy: use write_all in file logger Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:56:09 +02:00
Wolfgang Bumiller	5a54935bc9	clippy: fix pointer to temporary value Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 13:55:56 +02:00
Wolfgang Bumiller	bd430c225b	clippy: remove a loop{} which never actually loops Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 12:31:39 +02:00
Wolfgang Bumiller	44fed91e17	more clippy fixups mostly indentation changes, view with `-w` Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 12:23:12 +02:00
Wolfgang Bumiller	653b1ca10e	some clippy fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-11 12:14:05 +02:00
Christian Ebner	03310bea12	src/pxar/fuse.rs: s/Buf/ReplyBuf/ and s/BufState/ReplyBufState/ Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-11 11:27:18 +02:00
Christian Ebner	48cc1b8234	src/pxar/fuse.rs: refactor stat and fix i-node mapping The functionality of stat is split into smaller sub-functions, which allows to reuse them more flexible, as the code flow is similar but not always the same. By this, the ugly and incorrect re-setting of the i-node in the lookup callback function is avoided. The correct i-node is now calculated beforehand and stat simply creates a `libc::stat` struct from the provided parameters. Also, this fixes incorrect i-node assignments in the readdir callback function. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-11 11:26:55 +02:00
Christian Ebner	22eaa905a4	src/pxar/fuse.rs: impl readdir callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:38:58 +02:00
Christian Ebner	1652192f4d	src/pxar/fuse.rs: impl readlink callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:38:48 +02:00
Christian Ebner	3710b23db5	src/pxar/decoder.rs: read_link has to read filename before entry. The offset points to the start of the item, which is the filename Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:36:04 +02:00
Christian Ebner	7eb9848ba7	src/pxar/fuse.rs: impl opendir callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:35:13 +02:00
Christian Ebner	2c5fd378d3	src/pxar/fuse.rs: impl read callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:34:55 +02:00
Christian Ebner	97e4db9dd5	src/pxar/fuse.rs: impl open callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 11:34:18 +02:00
Christian Ebner	b00689254a	src/pxar/fuse.rs: impl lookup callback for fuse The lookup call checks if the given filename is found in the directory referenced by the i-node by calclulating the filenames hash and looking it up in the directories goodbye table. If found, the entries parameters are returned. In order to be able to lookup the parent offset by a given file offset in the readdir callback, this also stores the corresponding values in a HashMap. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:54:23 +02:00
Christian Ebner	1443927c02	src/pxar/decoder.rs: correctly handle hardlinks in `Decoder`s attribute function If the attributes are then returned by seeking to the new offset. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:46:39 +02:00
Christian Ebner	d00097a0e6	src/pxar/decoder.rs: split functionality of list_dir into list_dir and goodbye_table In order to read the contents of the goodbye table while keeping the functionality of list_dir in place as is. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:46:29 +02:00
Christian Ebner	ac12570e99	src/pxar/decoder.rs: make read_directory_entry usable by fuse ... and thereby allow it to read a single directory entry based on the start and end archive offsets. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:32:48 +02:00
Christian Ebner	3f7f4f34be	src/pxar/decoder.rs: remove `DirectoryEntry` start_offset and end_offset functions ... as they are not needed with the latest iteration of the fuse callback function impl (which never made it into the repos). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:30:45 +02:00
Christian Ebner	ac30bc3326	src/pxar/decoder.rs: correctly read, check and skip filename The previous implementation simply skipped over `size` bytes, which is not correct as size includes also the header. By relying on `SequentailDecoder`s read_filename function, this is correctly taken care of plus some more integrity checks. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-10 10:30:21 +02:00
Dietmar Maurer	4a7de56e2f	avoid injecting ENV vars from Makefile So that we can run "cargo build" without setting vars manually.	2019-09-09 10:51:08 +02:00
Christian Ebner	d21ae955a6	src/pxar/fuse.rs: impl getattr callback for fuse Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 12:24:14 +02:00
Christian Ebner	ec04ea81f2	src/pxar/fuse.rs: Refactor run_in_context and remove inode_to_offset and offset_to_inode as their functionality is used only once so it makes more sense to keep them inline. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 12:22:53 +02:00
Christian Ebner	8413083fc5	src/pxar/fuse.rs: Improve comments, add rustfmt skip tag and reafctor Also, removes an unused println statement in the decoder callback function and fixes a typo. Further, use ABI compatible Option<&T> for FFI to avoid use of raw pointers. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 12:22:17 +02:00
Dietmar Maurer	54a5a885a3	src/client/http_client.rs: use async for upload_speedtest()	2019-09-06 08:55:47 +02:00
Christian Ebner	bbd055bf45	src/pxar/decoder.rs: impl functionality needed for fuse implementation Implements functions attributes, open, read, read_link and get_dir to be used by the fuse implementation which uses file offsets within the archive as inodes to reference the archives items. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 07:18:14 +02:00
Christian Ebner	132cb0d0db	src/pxar/decoder.rs: Add comment for unsolved issue and refactor if statement. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-06 07:16:31 +02:00
Dietmar Maurer	2a05048b93	src/client/http_client.rs: simplify send_request	2019-09-05 15:07:37 +02:00
Dietmar Maurer	9edd3bf1b8	src/client/http_client.rs: use async for h2api_response()	2019-09-05 14:56:52 +02:00
Dietmar Maurer	c18fddf80f	src/client/http_client.rs - download_chunk_list: simplify async code	2019-09-05 14:16:45 +02:00
Dietmar Maurer	44f59dc731	src/client/http_client.rs - H2Client: fix error message in download()	2019-09-05 13:26:06 +02:00
Dietmar Maurer	c2a5a9f353	src/client/http_client.rs: use async for download_chunk_list	2019-09-05 13:12:03 +02:00
Dietmar Maurer	2f831baec0	src/client/http_client.rs - BackupClient: use async	2019-09-05 12:55:22 +02:00
Dietmar Maurer	2a1e6d7dea	src/client/http_client.rs - BackupClient: use async	2019-09-05 11:42:12 +02:00
Dietmar Maurer	d4a085e564	src/client/http_client.rs - BackupReader: use async	2019-09-05 11:42:05 +02:00
Dietmar Maurer	9d456cf8ec	src/backup/catalog_blob.rs: use write_all() instead of write() To correctly handle interrupted system calls.	2019-09-05 11:40:48 +02:00
Dietmar Maurer	1a7a0e74c0	src/client/http_client.rs: use async for more functions	2019-09-04 13:48:16 +02:00
Dietmar Maurer	96f5e80abb	src/client/http_client.rs: use async for login()	2019-09-04 10:01:46 +02:00
Dietmar Maurer	9d35dbbb8f	src/client/http_client.rs: use async for credentials	2019-09-04 09:57:29 +02:00
Dietmar Maurer	d2267b112d	src/bin/proxmox-backup-client.rs: verify blob/catlog checksums	2019-09-04 08:47:14 +02:00
Christian Ebner	70b03a07c8	src/pxar/sequential_decoder.rs: make functions needed in non-sequential decoder accessible from within crate Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-04 06:15:05 +02:00
Christian Ebner	f92e8266ae	src/pxar/format_definitions.rs: move header type in PxarHeader and hash in PxarGoodbyeItem to top of struct By this it is possible to read and check just the first u64 of the corresponding structs in order to identify the items. This is needed for the fuse implementation in order to get entries based on the archive offset, used as inode. Directories are referenced by the offset to the goodbye tail while other items are referenced by the offset of the filename followed by the entry. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-04 06:14:30 +02:00
Christian Ebner	75933d1e83	src/pxar/fuse.rs: add function to map offset to inode and vice versa Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-04 06:13:59 +02:00
Christian Ebner	c7fee396c6	src/pxar/decoder.rs: add getter functions to obtain start/end offsets for directories Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-04 06:10:56 +02:00
Christian Ebner	3626ac611f	src/pxar/decoder.rs: rustfmt fixups Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-04 06:09:17 +02:00
Christian Ebner	3fa71727ee	src/bin/proxmox-backup-client.rs: refactoring of match statement Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-09-04 06:08:00 +02:00
Dietmar Maurer	a40220c056	src/bin/proxmox-backup-client.rs: code cleanup - factor out some functions	2019-09-03 13:31:13 +02:00
Dietmar Maurer	36493d4d7e	src/backup.rs: define INDEX_BLOB_NAME here	2019-09-03 13:15:44 +02:00
Dietmar Maurer	df65bd3da6	src/bin/proxmox-backup-client.rs - restore: verify checksums	2019-09-03 13:12:16 +02:00
Dietmar Maurer	0a51fe0011	src/backup/fixed_index.rs: new helper to compute checksum and file size	2019-09-03 13:11:45 +02:00
Dietmar Maurer	82c85a21a1	src/backup/dynamic_index.rs: new helper to compute checksum and file size	2019-09-03 13:11:09 +02:00
Dietmar Maurer	a3e032b754	src/client/http_client.rs: fix dynamic index checksum	2019-09-03 13:10:18 +02:00
Wolfgang Bumiller	c77563146b	comment fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-03 11:17:33 +02:00
Wolfgang Bumiller	daef93f481	tools: add tokio::main() replacement to deal with block_on() not allowing blocking() Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-03 11:16:30 +02:00
Wolfgang Bumiller	66fbf5bad0	add WrappedReaderStream test Mostly to just document the rt.block_on() issue with blocking(). Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-03 10:05:45 +02:00
Wolfgang Bumiller	2b92971fba	merge fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:22:36 +02:00
Wolfgang Bumiller	083ff3fd5d	update to tokio 0.2.0-alpha.4 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	1434f4f8df	drop hyper_openssl We can use .await now, which means the whole connection state machine doesn't need to be typed out as "types" anymore, so, at least until hyper_openssl gets updated to proper dependencies, let's drop it. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	556eb70ea3	add generic either-or AsyncRead/Write type The HttpsConnector will use this. Instead of implementing a specialized MaybeTlsStream, this is simply a generic "either this or that kind of Async Read/Write type". Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	b30415d244	src/server/state.rs: update to tokio alpha.2 Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	aadcf7c621	src/server/rest.rs: use tokio::timer::delay Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	0cdb8d9c5b	src/tools/wrapped_reader_stream.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	e668912a99	src/tools/futures.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	0f5856acca	src/tools/daemon.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	56e351c911	src/tools/broadcast_future.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	627bb7d114	src/tools/async_mutex.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	75fef4b463	src/server/worker_task.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	aa4110cc64	src/server/state.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	91e4587343	src/server/rest.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	b9b7f7ec1f	src/server/h2service.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	160fc8147f	src/server/command_socket.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	3c0facc787	src/client/remote_chunk_reader.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	369a87e3a2	src/client/pxar_backup_stream.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	2107bb40c1	src/client/pipe_to_stream.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	5b3911995b	src/client/merge_known_chunks.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	a6782ca10b	src/client/http_client.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	8630557918	src/bin/upload-speed.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	cab6816969	src/bin/test_chunk_speed2.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	fda5797b8a	src/bin/proxmox-backup-proxy.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	e9722f8bde	src/bin/proxmox-backup-client.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:21:26 +02:00
Wolfgang Bumiller	e76ac3a499	src/bin/proxmox-backup-api.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	e235c8f719	src/bin/h2s-server.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	74be6dc9b7	src/bin/h2server.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	55d8a631fc	src/bin/h2s-client.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	15d0e4a3bd	src/bin/h2client.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	b9203d87f4	src/bin/download-speed.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	745e652a7f	src/backup/index.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	32bef1e2d1	src/backup/chunk_stream.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	f2d9b73c91	src/api_schema/router.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	ffb6434485	src/api2/reader.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	7622005574	src/api2/backup/upload_chunk.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	59b2baa0f6	src/api2/backup.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	6be147b78c	src/api2/admin/datastore.rs: switch to async Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	5c1130df9f	implement Sync for DynamicIndexReader and FixedIndexReader hyper's wrap_stream now needs this Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Wolfgang Bumiller	236761a3e6	drop src/storage/futures.rs it's unused and uses a tokio::Task I don't want to lookup the std-future conversion for... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-09-02 15:17:58 +02:00
Dietmar Maurer	02fcf372e4	rc/bin/proxmox-backup-client.rs - restore: always download index.json.blob	2019-09-02 14:14:32 +02:00
Dietmar Maurer	0d9862803b	src/bin/proxmox-backup-client.rs - reatore: avoid loading (large) blobs into memory	2019-09-02 14:13:31 +02:00
Christian Ebner	66c8eb9383	src/client/http_client.rs: Refactor handling Option and Result types Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-29 09:15:16 +02:00
Wolfgang Bumiller	40f8680e37	tools: remove SignalFd We don't use it anymore and it'll just be more work when switching to async-await. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 13:44:53 +02:00
Wolfgang Bumiller	d21f8a5b5b	backup/dynamic_index: explicitly zero-initialize the header The writer.write_all() call accessed data marked as undefined to valgrind. Note that we shouldn't write out uninitialized memory for security reasons anyway. (note that vec::undefined already did zero-initialize the data, but also marked it as undefined for valgrind when compiling with the valgrind feature) Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 11:42:35 +02:00
Wolfgang Bumiller	990b930f22	backup/dynamic_index: use static assertion, fix size The type was sized properly but the number was still wrong, fixed this. TODO! Once unions with non-Copy values are stable make this a `union { full: [u8; 4096], data: TheActualHeader }`; Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 11:42:35 +02:00
Christian Ebner	bcb664cb69	pxar/fuse: add pxar cli flag to set single- or multi-threaded session loop Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:43:14 +02:00
Christian Ebner	de2791868a	src/pxar/sequentail_decoder.rs: followup: cleanup S_IFMT masking Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:42:22 +02:00
Christian Ebner	9cb9e1b7db	src/pxar/sequential_decoder.rs: style fixups Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:42:19 +02:00
Christian Ebner	dac88033ba	src/pxar/dir_stack.rs: style fixups Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:39:14 +02:00
Christian Ebner	b69169e7be	src/pxar/fuse.rs: style fixups Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:37:58 +02:00
Christian Ebner	d792dc3cd2	src/pxar/match_pattern.rs: style fixups Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:37:56 +02:00
Christian Ebner	4ea22b68d9	src/pxar/format_definition.rs: style fixup Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-23 09:37:53 +02:00
Wolfgang Bumiller	7d83440c60	remove proxmox-protocol subcrate AFAICT we have no use for it anymore, its api entry points are gone. If we do end up needing something from it, it's still in the git history anyway. (And about two thirds of it can be made much less awkward by utilizing async-await anyway, so no love lost there...) Moved the chunker back into src/backup/chunker.rs Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 14:08:25 +02:00
Wolfgang Bumiller	7a57cb77e1	more formatting & use statement fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 13:50:27 +02:00
Christian Ebner	35dfd3ceb7	src/pxar/encoder.rs: style fixups by rustfmt Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 13:29:11 +02:00
Christian Ebner	1af30bc2fe	(pxar: replace deprecated std::mem::uninitialized() ... and use std::mem::MaybeUninit or proxmox::tools::vec::uninitialized() instead. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 13:27:25 +02:00
Wolfgang Bumiller	5e58e1bb7d	{dynamic,fixed}_index: replace mem::uninitialized Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:35:36 +02:00
Wolfgang Bumiller	f93b55b057	src/backup/index: style fixup & unsafe copy removal We can use the safe .copy_from_slice alternative in this case. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:31:45 +02:00
Wolfgang Bumiller	fa17b1ce2a	src/backup/index: replace mem::uninitialized Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:27:35 +02:00
Wolfgang Bumiller	f3a8d1d7e0	api2/node/time: replace mem::uninitialized and fixup use statements Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:14:41 +02:00
Wolfgang Bumiller	48b85e8e3b	tools: tty: replace mem::uninitialized and style fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:14:19 +02:00
Wolfgang Bumiller	b528fddfc6	tools: timer: style fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:07:56 +02:00
Wolfgang Bumiller	b7f44ace92	timer: replace mem::uninitialized Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 11:07:10 +02:00
Wolfgang Bumiller	6100071f4e	tools: style & use statement fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-22 10:57:56 +02:00
Wolfgang Bumiller	5c20e2da6b	src/config: more style fixups Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 14:14:26 +02:00
Wolfgang Bumiller	91640ab567	src/config: style fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 14:11:07 +02:00
Wolfgang Bumiller	928650c4eb	src/config: use statement cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 14:10:50 +02:00
Wolfgang Bumiller	2ec979e4ed	/var/run -> /run /var/run is considered deprecated and for instance in systemd unit files lintian complains... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 12:28:24 +02:00
Wolfgang Bumiller	35238e234d	switch from create_dir_chown to create_path Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 12:26:10 +02:00
Christian Ebner	e5471b4836	pxar/decoder: avoid failing on hardlinks and return corresponding target file entry Signed-off-by: Christian Ebner <c.ebner@proxmox.com> [Fixup: remove unnecessary 'mut'] Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 09:32:52 +02:00
Christian Ebner	58262f40f7	src/pxar/decoder.rs: get correct stats for root directory Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 09:31:41 +02:00
Christian Ebner	7d26720e79	src/pxar/decoder.rs: move GOODBYE_ITEM_SIZE from function to module scope. Signed-off-by: Christian Ebner <c.ebner@proxmox.com> Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-21 09:31:39 +02:00
Wolfgang Bumiller	367f002ee2	use statement cleanup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-08-16 12:28:40 +02:00
Dietmar Maurer	9d135fe617	src/backup/catalog_blob.rs: moved catalog impl. from pxar And avoid loading catalog into memory.	2019-08-16 12:27:17 +02:00
Wolfgang Bumiller	9025312aa6	avoid lifetimes in blob reader/writer	2019-08-16 10:07:24 +02:00
Dietmar Maurer	71d08e00b7	src/backup/data_blob_reader.rs: fix compiler warning	2019-08-16 08:10:39 +02:00
Christian Ebner	ac0b435344	src/pxar/fuse.rs: fix compiler warnings Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-14 15:14:57 +02:00
Dietmar Maurer	a84ef4c205	src/bin/proxmox-backup-client.rs: avoid loading catalog into memory We can use the new DataBlobReader instead.	2019-08-14 15:07:28 +02:00
Dietmar Maurer	b791804f4b	src/backup/data_blob_reader.rs: impl for Read instead of BufRead	2019-08-14 14:56:01 +02:00
Dietmar Maurer	39a4df61d6	improve docs	2019-08-14 14:08:27 +02:00
Dietmar Maurer	018d11bb84	src/backup/data_blob.rs: move parts into single files	2019-08-14 13:24:41 +02:00
Dietmar Maurer	e27c5a559f	src/backup/data_blob.rs: avoid excessive stack usage	2019-08-14 12:44:08 +02:00
Dietmar Maurer	c638542b4b	src/backup/file_formats.rs: moved header_size	2019-08-14 12:35:53 +02:00
Christian Ebner	41d08aa6de	src/pxar/fuse.rs: enable libfuse debug output in verbose mode Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-14 08:59:15 +02:00
Christian Ebner	742e64ea0b	src/pxar/fuse.rs: cleanup callback interface and store decoder within session context Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-14 08:58:44 +02:00
Christian Ebner	7750b7f2b7	pxar: decoder: take ownership of underlying reader By taking ownership it is easier to move the decoder into another struct, e.g. into a session context in fuse. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-14 08:55:08 +02:00
Christian Ebner	f50b4fd6a0	src/pxar/decoder.rs: cleanup s/CaDirectoryEntry/DirectoryEntry Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-14 08:54:08 +02:00
Dietmar Maurer	6082216343	src/backup/data_blob.rs - CryptReader/CryptWriter: fix buffer size handling	2019-08-13 15:23:15 +02:00
Dietmar Maurer	548c9489d7	src/backup/data_blob.rs - DataBlobReader: impl compressed, encrypted blobs	2019-08-13 13:17:08 +02:00
Dietmar Maurer	2aa0bfff59	src/backup/data_blob.rs - DataBlobReader: implement reader for encrtypted blobs	2019-08-13 13:17:08 +02:00
Dietmar Maurer	e9a385a78e	src/backup/data_blob.rs - DataBlobReader: impl compressed, signed blobs	2019-08-12 17:47:25 +02:00
Dietmar Maurer	4bfa147eaf	src/backup/data_blob.rs - DataBlobReader: impl. reader for signed blobs	2019-08-12 17:41:25 +02:00
Dietmar Maurer	09785b2795	src/backup/data_blob.rs - DataBlobReader: start serious impl.	2019-08-12 17:20:41 +02:00
Dietmar Maurer	a32bd8a516	src/backup/data_blob.rs - DataBlobWriter: fix decryption	2019-08-12 11:57:29 +02:00
Dietmar Maurer	5d15cb49b4	src/backup/data_blob.rs - compute_crc: start after blob header	2019-08-12 11:32:26 +02:00
Dietmar Maurer	1cbdfd19a2	src/backup/file_formats.rs; use const instzead of static	2019-08-12 11:20:21 +02:00
Dietmar Maurer	18be4ec24a	src/backup/data_blob.rs - DataBlobWriter: fix magic number for uncompressed blobs	2019-08-12 10:46:05 +02:00
Dietmar Maurer	5622a3fcdb	src/backup/data_blob.rs - DataBlobWriter: impl. compressed encrypted blobs	2019-08-12 10:16:42 +02:00
Dietmar Maurer	f4942e9ffd	src/backup/data_blob.rs - DataBlobWriter: impl. encrypted blobs	2019-08-12 10:08:00 +02:00
Dietmar Maurer	c57ec43a53	src/backup/crypt_config.rs: new helper data_crypter to get openssl Crypter	2019-08-12 10:06:51 +02:00
Dietmar Maurer	7776becf5d	src/backup/data_blob.rs - DataBlobWriter: impl. compressed signed blobs	2019-08-12 09:01:08 +02:00
Dietmar Maurer	706638f803	src/backup/data_blob.rs: impl. ChecksumWriter To correctly compute crc,hmac of final data.	2019-08-12 08:53:53 +02:00
Dietmar Maurer	f796351c54	src/backup/data_blob.rs: impl. DataBlobWriter for authenticated blobs	2019-08-12 07:34:41 +02:00
Dietmar Maurer	cb0eea29d9	src/backup/crypt_config.rs: new method to get hmac signer	2019-08-12 07:33:15 +02:00
Dietmar Maurer	a762ce54fd	src/backup/data_blob.rs: make DataBlobWriter more generic Allow to write compressed and uncompressed blobs.	2019-08-11 12:14:04 +02:00
Dietmar Maurer	1f26fdef19	src/backup/data_blob.rs: impl. simple reader/writer To avoid loading blob into memory.	2019-08-11 11:32:36 +02:00
Dietmar Maurer	e240d8be0b	src/bin/proxmox-backup-client.rs: implement login/logout	2019-08-10 09:12:17 +02:00
Christian Ebner	f71e8cc96f	pxar: add command 'mount' to cli of pxar Allows to mount an archive to a specified mountpoint via the cli. Once the archive is mounted, the process is send to the background. By passing the --verbose flag, the process is kept in foreground and debug output is provided. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-09 16:46:16 +02:00
Dietmar Maurer	02ba8a5d11	pxar: add fuse module and expose its pub functionality.	2019-08-09 16:45:13 +02:00
Christian Ebner	c50f87442c	pxar: add basic code for FUSE implementation. This adds the basic code in order to create a fuse session and mount an archive. It adds libfuse3-3 as runtime dependency and libfuse3-dev as build dependency. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-09 16:42:31 +02:00
Dietmar Maurer	781ac11c6a	src/backup/data_blob.rs: define const MAX_BLOB_SIZE	2019-08-09 11:49:06 +02:00
Dietmar Maurer	9049a8cfe6	src/bin/proxmox-backup-client.rs: implement catalog command	2019-08-09 11:36:33 +02:00
Dietmar Maurer	7926a3a1cf	src/bin/proxmox-backup-client.rs: add archive filename to catalog	2019-08-09 11:05:56 +02:00
Dietmar Maurer	863be2e6e2	src/backup/data_blob.rs: allow largert blob sizes (128MB) Need this for catalog files.	2019-08-09 10:22:56 +02:00
Dietmar Maurer	3fb53e07b4	src/bin/proxmox-backup-client.rs: avoid compiler warning	2019-08-09 10:22:32 +02:00
Dietmar Maurer	543a260ff9	src/bin/proxmox-backup-client.rs: new completion helper for snapshots	2019-08-09 10:08:45 +02:00
Dietmar Maurer	2761d6a4f3	src/pxar/encoder.rs: use BackupCatalogWriter	2019-08-09 09:46:49 +02:00
Dietmar Maurer	aea0815d32	src/pxar/catalog.rs: new catalog helper	2019-08-09 09:41:43 +02:00
Dietmar Maurer	977eeb24f6	src/bin/proxmox-backup-client.rs: avoid doubled file extensions (.blob.blob) in index.json	2019-08-07 10:23:08 +02:00
Dietmar Maurer	4af0ee055a	api: always use complete file names (including add exctensions)	2019-08-07 10:10:14 +02:00
Dietmar Maurer	d7c243977d	src/api2/admin/datastore.rs - list_snapshot_files: list all files Add files that are not mentioned in index.json.	2019-08-07 09:16:14 +02:00
Dietmar Maurer	34066e2eeb	src/api2/backup/environment.rs - add_blob: always verify blob crc	2019-08-07 09:15:13 +02:00
Dietmar Maurer	eecb23560b	src/backup/data_blob.rs - encode: always compute crc	2019-08-07 08:30:27 +02:00
Dietmar Maurer	38a6cddad7	src/backup/backup_info.rs: add BackupInfo::new constructor	2019-08-07 08:27:52 +02:00
Dietmar Maurer	c4f025ebbf	src/bin/proxmox-backup-client.rs: fix remote filename completion	2019-08-07 06:48:18 +02:00
Dietmar Maurer	dbed4c8cd7	src/client/http_client.rs: compute checksums for chunk streams	2019-08-06 12:23:06 +02:00
Dietmar Maurer	bd15e96dd9	src/client/http_client.rs: minor code cleanup	2019-08-06 11:46:15 +02:00
Dietmar Maurer	f889b15867	src/backup/data_blob.rs: always compute crc	2019-08-06 11:42:14 +02:00
Dietmar Maurer	c807d231eb	src/client/http_client.rs: compute file checksums	2019-08-06 11:26:06 +02:00
Dietmar Maurer	a17a0e7a9f	src/api2/admin/datastore.rs: extract backup size from index.json	2019-08-06 10:56:21 +02:00
Christian Ebner	eecb182845	pxar: add initial docs for MatchPattern Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-06 09:03:02 +02:00
Christian Ebner	43e892d293	pxar: add error handling for MatchPattern::matches_filename() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-06 09:02:44 +02:00
Dietmar Maurer	8c70e3eb18	src/api2/admin/datastore.rs - files: return data from index.json	2019-08-05 13:22:19 +02:00
Dietmar Maurer	f69adc81a6	use new proxmox::tools::nodename	2019-08-03 17:06:23 +02:00
Dietmar Maurer	7f66c29e5c	remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead	2019-08-03 16:44:31 +02:00
Dietmar Maurer	fd329bdcd7	remove src/tools/procfs.rs, use proxmox::sys::linux::procfs instead	2019-08-03 16:26:44 +02:00
Dietmar Maurer	779e4c90b1	src/tools/vec.rs: remove unused file We already moved that to proxmox::tools	2019-08-03 16:11:47 +02:00
Dietmar Maurer	e18a6c9ee5	update to nix 0.14, use code from proxmox:tools	2019-08-03 13:05:38 +02:00
Dietmar Maurer	b86f263ced	use new proxmox::sys::linux::magic	2019-08-03 09:28:57 +02:00
Christian Ebner	4ea0f7d95b	pxar: remove flag to change digest algorithm The flag CA_FORMAT_SHA512_256 is used to switch between sha512 and sha256 to calculate digest in casync. As we use sha256, we can get rid of this flag for now. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:57:53 +02:00
Christian Ebner	4d142ea79e	pxar: cleanup: refactor and rename exclude pattern The original name PxarExcludePattern makes no sense anymore as the patterns are also used to match filenames during restore of the archive. Therefore, exclude_pattern.rs is moved to match_pattern.rs and PxarExcludePattern rename to MatchPattern. Further, since it makes more sense the MatchTypes are now declared as None, Positive, Negative, PartialPositive or PartialNegative, as this makes more sense and seems more readable. Positive matches are those without '!' prefix, Negatives with '!' prefix. This makes also the filename matching in the encoder/decoder more intuitive and the logic was adapted accordingly. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:52:32 +02:00
Christian Ebner	fe076c8259	pxar: cleanup: s/PxarDirBuf/PxarDirStack/g and move dir_buffer.rs to dir_stack.rs Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:52:10 +02:00
Christian Ebner	ead7546a96	pxar: change uid and gid to u32 instead of u64 Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:51:51 +02:00
Christian Ebner	5e50c606b0	pxar: cleanup: s/CA_FORMAT/PXAR/g and s/CaFormat/Pxar/g Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:51:33 +02:00
Christian Ebner	47651f9530	pxar: cleanup: move feature flags to src/pxar/flags.rs and omit CA_FORMAT prefix on all of them Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:51:06 +02:00
Christian Ebner	91451c4355	pxar: cleanup: move filesystem related magic numbers to tools/fs/magic.rs Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-03 08:48:12 +02:00
Dietmar Maurer	b335f5b713	src/bin/proxmox-backup-client.rs: sign index.json	2019-08-02 10:00:15 +02:00
Dietmar Maurer	69ecd8d5b0	src/backup/data_blob.rs: implement signed blobs	2019-08-02 09:56:01 +02:00
Dietmar Maurer	93205f942a	src/backup/crypt_config.rs: new compute_auth_tag helper	2019-08-02 08:55:37 +02:00
Dietmar Maurer	c68d2170d5	src/backup/crypt_config.rs: fix typo	2019-08-02 08:29:40 +02:00
Christian Ebner	81a9905e0a	pxar: implement feature flag support for device nodes, fifos and sockets Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:55:17 +02:00
Christian Ebner	8abc95a145	pxar: remove some unused feature flags from format definition Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:54:46 +02:00
Christian Ebner	3a905cac2b	src/pxar/sequential_decoder.rs: remove of obsolete function dir_mkdirat Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:52:49 +02:00
Christian Ebner	beffac999f	src/pxar/sequentail_decoder.rs: fix issue when restoring with glob pattern. Partial extraction of an archive with a glob pattern, e.g. '*/.conf' lead to the unexpected behaviour of restoring all partially matched directories (in this example all of them). This patch fixes this unexpected behaviour by only restoring those directories were the directory or one of its sub-items fully matched the pattern and should therefore be restored. To achive this behavoiur, directory metadata is pushed onto a stack and restored on demand. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:49:47 +02:00
Christian Ebner	51ac99c314	src/pxar/exclude_pattern.rs: add Clone and Copy trait to MatchType Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:49:47 +02:00
Christian Ebner	6a584cfd76	pxar: add PxarDir and PxarDirBuf to buffer directory metadata In order to restore only directories when some of their content fully matched a match pattern on partial restores, these directories and their metadata are pushed onto this buffer and only restored successivley on demand. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:45:12 +02:00
Dietmar Maurer	27e6e180b6	src/pxar/format_definition.rs - PxarAttributes: derive Default And remove unnecessary impl new().	2019-08-02 07:16:10 +02:00
Christian Ebner	a92e6c9011	src/pxar/sequential_decoder.rs: split read and restore of attributes into separate functions Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:11:38 +02:00
Christian Ebner	e60cfbcdd0	src/pxar/format_definition.rs: add struct PxarAttributes This structure contains all the attributes allowing to easily store those within a e.g. dir buffer. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 07:10:22 +02:00
Christian Ebner	2be3eff5c3	pxar: avoid taking ownership of xattrs, fcaps and quota_projid on corresponding restore functions. By borrowing these objects we preserve the functionality but make sure that ownership doesn't change, avoiding problems when contained within other structs such as e.g. a buffer storing these attributes. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-08-02 06:44:00 +02:00
Dietmar Maurer	2c3891d1c3	src/bin/proxmox-backup-client.rs: upload backup index.json The plan is to use this file to verify the backup content.	2019-08-01 12:39:02 +02:00
Dietmar Maurer	953d5e1531	src/tools.rs - file_set_contents_full: only call fchmod when we pass permissions So that we can use it to write into /etc/pve/ (which does not support chmod).	2019-07-31 12:44:27 +02:00
Dietmar Maurer	cec17a3ec7	src/bin/proxmox-backup-client.rs - files: allow --repository parameter	2019-07-31 11:45:55 +02:00
Dietmar Maurer	52c171e49d	src/bin/proxmox-backup-client.rs: implement files command	2019-07-31 11:23:53 +02:00
Dietmar Maurer	43a406fd3f	src/bin/proxmox-backup-client.rs: supress useless output from prune command	2019-07-31 10:15:16 +02:00
Christian Ebner	9731c8b862	pxar: add support for storing/restoring the quota project id on ZFS Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-30 09:40:48 +02:00
Christian Ebner	6a87910949	pxar: implement allow_existing_dirs for pxar decoder By default, restoring an archive will fail if files with the same filename already exist in the target directory. By setting the allow_existing_dirs flag, the restore will not fail if an existing directory is encountered. The metadata (permissions, acls, ...) of the existing directory will be set to the ones from the archive. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-30 09:40:30 +02:00
Dietmar Maurer	46d5aa0a09	src/bin/proxmox-backup-client.rs: add restore flag --allow-existing-dirs Not jet implemented ...	2019-07-29 12:49:15 +02:00
Dietmar Maurer	ea7a7ef299	src/bin/proxmox-backup-client.rs: fixup - correctly pass prune parameters	2019-07-27 09:24:23 +02:00
Dietmar Maurer	9fdc3ef46f	src/api2/admin/datastore.rs: pass backup type/id (group) to prune IMHO, prune over all backup groups is a bit dangerous, considering that more than one user might use a datastore.	2019-07-27 08:49:14 +02:00
Dietmar Maurer	7a6cfbd98d	src/bin/proxmox-backup-client.rs: correctly compute duration And format time with SecondsFormat::Secs	2019-07-26 11:13:49 +02:00
Dietmar Maurer	bbf9e7e951	src/api2/backup.rs: cleanup schema definitions	2019-07-26 09:07:29 +02:00
Dietmar Maurer	ca5d0b61ca	src/api2/backup.rs: new required backup-time parameter The client should pass the time as parameter.	2019-07-25 13:44:01 +02:00
Dietmar Maurer	e128d4e84f	src/api2/admin/datastore.rs: backup logs may not be written twice	2019-07-25 12:29:04 +02:00
Dietmar Maurer	ef3254c086	src/api2/admin/datastore.rs: use correct .blob extension for log	2019-07-25 12:22:50 +02:00
Dietmar Maurer	ec34f7eba0	src/bin/proxmox-backup-client.rs: implement upload-log	2019-07-25 12:18:15 +02:00
Dietmar Maurer	04512d3068	src/client/http_client.rs: allow upload with parameters	2019-07-25 12:17:35 +02:00
Dietmar Maurer	07ee223590	src/api2/admin/datastore.rs: add api to upload backup client log file	2019-07-25 12:15:30 +02:00
Dietmar Maurer	79679c2d6e	src/bin/proxmox-backup-client.rs: allow to uploag .log files (as binary blobs)	2019-07-24 13:07:02 +02:00
Dietmar Maurer	5b72c9b4f7	src/pxar/encoder.rs: add new skip_lost_and_found parameter	2019-07-24 12:21:25 +02:00
Dietmar Maurer	6c3c9bceb5	pxar_backup_stream.rs: limit lock scope to avoid blocking forever	2019-07-24 11:30:43 +02:00
Dietmar Maurer	684233aa3b	remove debug output	2019-07-24 09:33:52 +02:00
Dietmar Maurer	5be106eeae	src/client/pxar_backup_stream.rs: correctly pass errors to stream	2019-07-24 09:24:35 +02:00
Dietmar Maurer	2eeaacb974	src/pxar/encoder.rs: allow to pass list of devices For better mount point include control...	2019-07-24 08:11:59 +02:00
Dietmar Maurer	fa5d6977dd	Revert previous, commit, use UTC RFC3339 without timezone (Z) We now have human readable data/time, and names are still sortable.	2019-07-22 11:49:30 +02:00
Dietmar Maurer	10c2a21cfe	src/backup/backup_info.rs: use unix epoch timestamps instead of RFC3339 RFC3339 contains additional timezone information, but IMHO this just adds more confusion. The API also uses timestamps.	2019-07-22 07:57:03 +02:00
Christian Ebner	1fdef50f52	src/pxar/sequential_decoder.rs: Call callback only on restored paths. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-19 11:58:51 +02:00
Dietmar Maurer	40a13369a3	src/bin/pxar.rs - extract: print archive name with --verbose flag	2019-07-19 07:01:57 +02:00
Dietmar Maurer	bbd4365c61	src/bin/pxar.rs: remove wrong debug message	2019-07-19 07:00:17 +02:00
Christian Ebner	a0ec687cd7	src/bin/pxar.rs: allow to pass paths and match patterns as args to pxar extract To improve usability it is now possible to directly pass paths or match patterns as arguments to pxar extract to partially restore an archive. The patterns provided via CLI are appended to the ones read from file by the --files-from option in order to have priority over those. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-19 06:50:39 +02:00
Dietmar Maurer	9aa3f68278	src/bin/proxmox-backup-client.rs: code cleanup	2019-07-18 13:16:32 +02:00
Dietmar Maurer	f6ede796ca	src/bin/proxmox-backup-client.rs - status: only print result[data]	2019-07-18 09:52:11 +02:00
Dietmar Maurer	15c847f136	src/api2/admin/datastore.rs: allow to list all snapshots	2019-07-18 09:11:43 +02:00
Dietmar Maurer	0f73ee73bb	src/cli/command.rs: set exit status on error	2019-07-17 13:40:10 +02:00
Christian Ebner	129dda4729	src/bin/pxar.rs: add --files-from parameter to pxar extract Allows to pass a list of match pattern to the pxar extract call by reading them from file. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 12:20:29 +02:00
Christian Ebner	fa7e957ca3	pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 12:19:31 +02:00
Christian Ebner	d32c2e4d27	src/pxar/sequential_decoder.rs: adapt code for partial restore by match pattern Allows to partially restore an archive by passing match patterns to the restore function. The whole restore is performed in sequential, therefore the whole archive has to be read. By wrapping the RawFd into an Option it can be controlled if the corresponding part is restored (in case of Some(fd)) or if the Reader reads over it without restore (in case of None). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 12:11:37 +02:00
Christian Ebner	4902291673	src/pxar/sequential_decoder.rs: implement match_filename to match include pattern for partial restores Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 12:07:54 +02:00
Christian Ebner	894cd49ac4	src/pxar/sequential_decoder.rs: Also check for nul bytes when reading the filename Check if the filename does not contain invalid nul byes when reading it from the archive. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 12:04:41 +02:00
Christian Ebner	0b39674895	src/pxar/sequential_decoder.rs: remove old comments and refactoring Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 09:46:02 +02:00
Christian Ebner	9f8fcdd050	src/pxar/encoder.rs: refactoring of match_exclude_pattern() match_exclude_pattern() does not need a '&mut self' reference to the encoder, move it therefore out of the impl. Further, this patch contains some naming and formatting cosmetics. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 09:45:19 +02:00
Christian Ebner	3ff4ef28e6	src/pxar/encoder.rs: remove unneeded dir_count dir_count was used to track the number of directory entries to store in the archive and bail if the maximum is exceeded. As the number of entries is equally obtained from the list of the filenames to include, use that one instead. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 09:44:59 +02:00
Christian Ebner	a771f90745	src/pxar/exclude_pattern.rs: do not return partial matches for files Partial matches make only sense for directories, files are always leafs of the tree. Take this into account in order to avoid restoring of files which only matched the front of a match pattern. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 09:44:06 +02:00
Christian Ebner	bdf0d82ced	src/bin/pxar.rs: Make pxar extract target optional In order to improve usablity, the target on archive extraction will be the current working directory by default. A different target can be provided via the optional --target <PATH> parameter. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-17 09:40:02 +02:00
Dietmar Maurer	34a816cc7b	src/bin/proxmox-backup-client.rs: add output-fromat options, implement status api	2019-07-16 13:35:25 +02:00
Dietmar Maurer	0eecf38fbf	src/api2/admin/datastore.rs: add status api call	2019-07-16 13:34:38 +02:00
Dietmar Maurer	25500cfa26	src/cli/command.rs: add helpers/schema to format output	2019-07-16 11:46:58 +02:00
Dietmar Maurer	aad2ee4947	src/bin/pxar.rs: only print archive name in verbose mode	2019-07-16 08:38:02 +02:00
Dietmar Maurer	6b9a071028	src/bin/pxar.rs: add --verbose parameter to list (replace dump)	2019-07-16 08:31:40 +02:00
Dietmar Maurer	2665cef703	src/bin/proxmox-backup-client.rs: make repository parameter optional And allow to read it from $ENV{PBS_REPOSITORY}	2019-07-16 07:30:04 +02:00
Christian Ebner	3f5192730e	src/pxar/format_definition.rs: fix typo Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-15 10:49:34 +02:00
Dietmar Maurer	9eae781ab8	src/bin/pxar.rs - extract: allow to read from stdin	2019-07-05 12:39:25 +02:00
Dietmar Maurer	c6c9e093d0	src/bin/pxar.rs - dump: allow to read from stdin	2019-07-05 12:32:15 +02:00
Dietmar Maurer	bf1252619a	src/bin/proxmox-backup-client.rs - restore: allow to pipe output to stdout	2019-07-05 12:14:50 +02:00
Dietmar Maurer	f8100e9623	src/bin/proxmox-backup-client.rs: remove download cli, do everything in restore	2019-07-05 11:36:45 +02:00
Dietmar Maurer	f4bf7dfcc7	src/client/remote_chunk_reader.rs: implement simple caching	2019-07-05 10:42:46 +02:00
Dietmar Maurer	afb4cd28be	src/backup/fixed_index.rs: implement BufferedFixedReader	2019-07-05 09:19:56 +02:00
Christian Ebner	be9b39e17a	src/pxar/sequential_decoder.rs: major refactoring of restore_sequential() This splits the functionality of restore_sequential() into several smaller functions in order to allow to reuse them when restoring by seeking based on the goodbye table offsets. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-05 06:45:50 +02:00
Christian Ebner	48a4509c67	src/pxar/encoder.rs: fix bug with incorrect offset stored in goodbye table Commit `cd7dc87903` introduced the special treatment for .pxarexclude files when stored in the archive. The incorrect placement of a code snipplet from this path leads to an incorrect offset and size stored in the goodbye table. This fix places the start to the correct position, restoring the previously correct behaviour. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-05 06:39:39 +02:00
Christian Ebner	7dcbe051e9	pxar: Refactor SequentialDecoder to store the callback function within the struct Reduces the number of arguments for the function calls within the decoder. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-05 06:38:22 +02:00
Dietmar Maurer	b46c3fad03	src/backup/fixed_index.rs: use correct size We need to consider that the last chunk may have smaller size.	2019-07-04 15:13:22 +02:00
Dietmar Maurer	9cc88a7c00	src/client/http_client.rs - finish: hold reference to avoid early connection close	2019-07-04 15:11:23 +02:00
Dietmar Maurer	5e04ec707d	src/api2/backup/environment.rs: improve chunk alignment/size checks	2019-07-04 13:40:43 +02:00
Dietmar Maurer	a24e3993e0	src/backup/chunk_store.rs: coding style fixes	2019-07-04 11:39:10 +02:00
Dietmar Maurer	e4c2fbf170	src/backup/chunk_store.rs: additionally log chunk count	2019-07-04 11:27:11 +02:00
Dietmar Maurer	9850bcdf19	src/backup/chunk_store.rs: improve error reporting	2019-07-04 11:21:54 +02:00
Wolfgang Bumiller	a3f3e91da2	backup/chunk_store: rework chunk iterator We can now use iter::from_fn() which makes for a much nicer logic. The only thing better is going to be when we can use generators with `yield`. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-04 10:55:17 +02:00
Dietmar Maurer	99168f43e6	src/client/http_client.rs: s/set_recv_buf_size/set_recv_buffer_size/	2019-07-04 10:51:18 +02:00
Dietmar Maurer	c698636a5d	src/tools/fs.rs - read_dir: return nix::Result	2019-07-04 10:47:16 +02:00
Dietmar Maurer	a57360983b	src/backup/chunk_store.rs - get_chunk_iterator: return percentage inside iterator item	2019-07-04 09:26:44 +02:00
Dietmar Maurer	0f0a35b390	src/backup/dynamic_index.rs: lock file inside new, code cleanup	2019-07-04 08:17:30 +02:00
Dietmar Maurer	a7c72ad9eb	src/backup/fixed_index.rs - FixedIndexReader: remove reference to chunk store	2019-07-04 08:13:16 +02:00
Dietmar Maurer	a660978c9a	src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats	2019-07-04 07:57:43 +02:00
Dietmar Maurer	86eda3eb0d	src/bin/proxmox-backup-client.rs: implement restore using BackupReader	2019-07-03 15:45:12 +02:00
Dietmar Maurer	b2a6897982	src/client/remote_chunk_reader.rs: remove dead code	2019-07-03 14:56:24 +02:00
Dietmar Maurer	7f99bf691a	src/client/remote_chunk_reader.rs: implement remote chunk reader	2019-07-03 14:39:13 +02:00
Dietmar Maurer	4f6aaf542c	src/client/http_client.rs: avoid compiler warning	2019-07-03 14:36:02 +02:00
Dietmar Maurer	3467cd91e9	src/client/http_client.rs: return Arc<Self> Avoid to call canceller for each clone on drop ...	2019-07-03 14:26:07 +02:00
Dietmar Maurer	1e0784841c	src/backup/read_chunk.rs: use &mut self	2019-07-03 14:25:18 +02:00
Christian Ebner	0d70535a36	src/pxar/encoder.rs: refactor simple match statements to map_err() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-03 12:18:36 +02:00
Christian Ebner	6e1c31163b	src/pxar/sequential_decoder.rs: refactoring of simple match statements to map_err() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-03 12:18:24 +02:00
Christian Ebner	9d227350a3	src/pxar/sequential_decoder.rs: Cleanup and fix incorrect error message Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-07-03 12:16:59 +02:00
Dietmar Maurer	80af046794	src/server/rest.rs: avoid unwrap	2019-07-03 12:00:43 +02:00
Dietmar Maurer	7fb4f5642a	src/server/rest.rs: log peer address, use hyper MakeService	2019-07-03 11:54:35 +02:00
Dietmar Maurer	6d1f61b208	use hyper/tokio-openssl instead of hyper/tokio-tls This exposes the complete SSL setup. And download is much faster now (600MB/s instead of 130MB/s)!	2019-07-02 13:36:28 +02:00
Dietmar Maurer	74d0a6bc23	src/bin/proxmox-backup-proxy.rs: set socket options for max. performance	2019-07-02 10:48:58 +02:00
Dietmar Maurer	93f8bb0445	src/bin/h2s-client.rs: avoid compiler warnings	2019-07-02 08:59:33 +02:00
Dietmar Maurer	877dd11133	src/backup/{fixed, dynamic}_index.rs: improve error messages	2019-07-02 08:56:56 +02:00
Dietmar Maurer	5c593a4d56	src/backup/read_chunk.rs: use Arc for DataStore	2019-07-02 08:49:16 +02:00
Dietmar Maurer	b850673634	src/backup/read_chunk.rs: move read chunk trait into extra file And implement LocalChunkReader.	2019-07-02 08:22:29 +02:00
Dietmar Maurer	8fad30a4b1	use latest hyper master branch to include socket buffer size fix	2019-07-02 07:09:46 +02:00
Wolfgang Bumiller	f35197f449	replace tools::vec with proxmox::tools::vec Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-01 11:03:25 +02:00
Wolfgang Bumiller	d0162d53d3	tools: remove io module now completely replaced by proxmox::tools::io. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-01 10:44:12 +02:00
Wolfgang Bumiller	9b2b627fe0	update remaining users of tools::io::ops Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-01 10:44:03 +02:00
Wolfgang Bumiller	9110a69bd8	tools: remove read/write now completely replaced by proxmox::tools::io::{ReadExt, WriteExt} Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-01 10:39:42 +02:00
Wolfgang Bumiller	ca3c3ce997	daemon: remove last use of tools::read/write Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-01 10:39:13 +02:00
Wolfgang Bumiller	5485b579a1	backup: use new proxmox::tools::io traits This replaces all `write_value` calls with `write_le_value` calls! Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-07-01 10:37:02 +02:00
Dietmar Maurer	dc9775d1b0	src/bin/h2s-client.rs: increase tokio tcp stream read buffer size This improves download speed by factor 10.	2019-06-29 18:09:50 +02:00
Dietmar Maurer	34f7131160	src/bin/h2s-client.rs: avoid hyper, use h2 directly But performance is still bad.	2019-06-29 15:58:18 +02:00
Dietmar Maurer	5ed2277f0e	src/bin/h2s-server.rs: test to reproduce slow h2 download Simplified code, use with h2s-client.rs.	2019-06-29 14:57:54 +02:00
Dietmar Maurer	17243003b3	src/bin/download-speed.rs: h2 download speedtest for our server I get about 30MB/s (much too slow)	2019-06-29 13:43:10 +02:00
Dietmar Maurer	a2b29b68d4	src/client/http_client.rs: use maximal h2 window sizes	2019-06-29 11:05:36 +02:00
Dietmar Maurer	872debdefa	src/bin/h2client.rs: tune h2 client parameters to get reasonable speed We now get about 2.5GB/s (not really good, but better)	2019-06-29 10:41:24 +02:00
Dietmar Maurer	38294bcc56	src/client.rs: make module put_to_stream public	2019-06-29 10:40:43 +02:00
Dietmar Maurer	fded1f3154	add code to test H2 speed I currently get about 1MB/s, which is ways too slow (must be a bug)?	2019-06-29 10:08:12 +02:00
Dietmar Maurer	d48a9955a5	src/backup/dynamic_index.rs: introduce ReadChunk trait	2019-06-28 16:35:00 +02:00
Dietmar Maurer	09d7dc5024	src/api2/reader.rs: add download chunk api	2019-06-28 16:27:01 +02:00
Dietmar Maurer	fcf5dea56d	src/client/http_client.rs: increase h2 window size, set tcp nodelay option To increase h2 download speed.	2019-06-28 16:00:58 +02:00
Dietmar Maurer	81a6ce6fde	src/backup/chunk_store.rs: new method chunk_path() Returns the absolute path.	2019-06-28 15:48:09 +02:00
Dietmar Maurer	fcfb84fedf	file download: avoid unnecessary copy	2019-06-28 07:07:52 +02:00
Dietmar Maurer	984a7c3502	src/client/http_client.rs - h2 download: implement flow control	2019-06-28 07:02:43 +02:00
Dietmar Maurer	dd066d28e2	src/api2/reader.rs: implement backup reader protocol	2019-06-27 09:01:41 +02:00
Dietmar Maurer	42a87f7b96	src/server/h2service.rs: implement generic h2 service	2019-06-26 17:38:33 +02:00
Dietmar Maurer	fb0470837b	src/client/http_client.rs: split out code to start new h2 connections	2019-06-26 12:09:18 +02:00
Dietmar Maurer	b208da8393	src/backup/data_{chunk,blob}.rs: add verify_crc method	2019-06-26 09:54:25 +02:00
Dietmar Maurer	fef44d4f78	src/bin/proxmox-backup-client.rs: decode blobs on download No more need for that clumsy blob cli (removed).	2019-06-26 09:18:59 +02:00
Dietmar Maurer	c2b945341c	src/client/http_client.rs - download: use generic Write type, return writer. Make it possible to write int Vec<u8>.	2019-06-26 09:17:13 +02:00
Dietmar Maurer	bb8231409e	src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json Use the KeyConfig serialization with kdf = None.	2019-06-26 07:32:34 +02:00
Dietmar Maurer	3031e44c58	src/bin/proxmox-backup-client.rs: remove stale download file, start blob cli	2019-06-25 13:00:37 +02:00
Dietmar Maurer	f14a8c9a85	src/api2/admin/datastore.rs: use correct path for download	2019-06-25 12:59:36 +02:00
Dietmar Maurer	81820b0d4d	src/client/http_client.rs - download: fix error handling	2019-06-25 12:43:55 +02:00
Dietmar Maurer	45db6f89e1	src/bin/proxmox-backup-client.rs: implement download command	2019-06-25 11:17:24 +02:00
Dietmar Maurer	9e47c0a56e	src/api2/admin/datastore.rs: implement generic download method We simply allow to download raw binary data.	2019-06-25 10:16:59 +02:00
Dietmar Maurer	da7d67213e	src/api2/admin/datastore/pxar.rs: remove pxar upload api Upload needs to be done using the backup protocol.	2019-06-25 09:45:35 +02:00
Dietmar Maurer	dd8e744f9a	src/api2/admin/datastore.rs: use a worker log log prune actions	2019-06-25 08:12:25 +02:00
Dietmar Maurer	6899dbfb47	src/bin/proxmox-backup-client.rs: strip .blob extension	2019-06-25 07:11:02 +02:00
Dietmar Maurer	c6d203bbd3	src/backup/backup_info.rs: also list .blob files	2019-06-25 07:07:45 +02:00
Dietmar Maurer	045e8a568d	src/bin/proxmox-backup-client.rs: change key name to rsa-encrypted.key	2019-06-25 07:00:07 +02:00
Dietmar Maurer	79bc7345e8	src/cli/command.rs: fix ReST output format	2019-06-25 06:19:51 +02:00
Dietmar Maurer	9f46c7de4b	src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup	2019-06-24 13:56:37 +02:00
Christian Ebner	cd7dc87903	pxar: impl .pxarexclude parsing and exclude matching .pxarexclude files allow to exclude or include parts of a subtree by matching with a glob pattern. The globs are used according to the matches of fnmatch. In addition '**' can be used to match multiple directories within the path. Order of the entries matter, as later ones win over previous ones. As the .pxarexclude files can be placed at any node of the directory hirarchy, this implies that matching child entries win over parent entries. The only exception to this behaviour is, when a parent entry already fully matched the path, thereby excluding the child entries which would match otherwise. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-24 10:26:20 +02:00
Dietmar Maurer	46bd880041	src/api2/backup/environment.rs: new helper add_blob()	2019-06-24 09:35:37 +02:00
Dietmar Maurer	cb08ac3efe	src/api2/backup.rs: replace upload_config with upload_blob	2019-06-23 11:44:30 +02:00
Dietmar Maurer	a38c5d4d12	src/backup/data_blob.rs: implement from_raw()	2019-06-23 09:35:44 +02:00
Dietmar Maurer	6f083b7a92	src/api2/backup/upload_chunk.rs: compute crc on upload	2019-06-23 09:04:42 +02:00
Dietmar Maurer	ba01828d38	try to use use proxmox::tools::io::ops::ReadExtOps	2019-06-22 16:29:10 +02:00
Dietmar Maurer	27042ce637	src/backup/file_formats.rs: improve docs	2019-06-22 15:45:36 +02:00
Dietmar Maurer	9f83e0f7ee	src/backup/crypt_config.rs: simplify decode interface	2019-06-22 13:24:29 +02:00
Dietmar Maurer	0066c6d972	src/backup/crypt_config.rs: remove encode_chunk, use encrypt_to instead	2019-06-22 13:02:53 +02:00
Dietmar Maurer	ee8a7e8080	src/backup/crypt_config.rs: implement encrypt_to (output to writer)	2019-06-22 12:25:04 +02:00
Dietmar Maurer	991abfa8b4	src/backup/file_formats.rs: split out file format data	2019-06-22 09:12:25 +02:00
Dietmar Maurer	b7f4f27d6c	add crc field for binary blobs formats	2019-06-21 18:49:04 +02:00
Dietmar Maurer	a7f67a9a9c	depend on crc32fast We can use this to compute fast checksums to test file integrity (not crypto safe).	2019-06-21 15:58:00 +02:00
Dietmar Maurer	3025b3a53c	src/backup/data_blob.rs: new file format for binary blobs	2019-06-21 11:32:07 +02:00
Dietmar Maurer	077a8cae63	src/backup/crypt_config.rs: make code more flexible Allow to pass magic numbers as parameters.	2019-06-21 10:50:48 +02:00
Dietmar Maurer	c38266c18d	renamed: src/backup/crypt_setup.rs -> src/backup/crypt_config.rs	2019-06-21 09:51:18 +02:00
Dietmar Maurer	3ea8bfc9c0	src/bin/proxmox-backup-client.rs: add import-master-pubkey command	2019-06-21 07:34:17 +02:00
Dietmar Maurer	37c5a17505	src/bin/proxmox-backup-client.rs: helper to generate RSA master key	2019-06-21 07:12:19 +02:00
Dietmar Maurer	ab44acff57	src/backup/key_derivation.rs: add modified field to key file	2019-06-20 07:58:33 +02:00
Dietmar Maurer	6d0983dbe1	src/bin/proxmox-backup-client.rs: add keyfile parameter for backup	2019-06-19 17:16:41 +02:00
Dietmar Maurer	181f097af9	src/bin/proxmox-backup-client.rs - key API: pass kdf parameter Allow to store keys without password.	2019-06-18 18:21:01 +02:00
Dietmar Maurer	9b06db4563	src/bin/proxmox-backup-client.rs: use file system path to reference keys	2019-06-18 17:38:50 +02:00
Dietmar Maurer	091540298e	src/bin/proxmox-backup-client.rs - create_key: always read password from tty	2019-06-18 17:27:52 +02:00
Dietmar Maurer	826f309bf5	src/backup/key_derivation.rs: move kdf code into separate file	2019-06-18 11:17:22 +02:00
Dietmar Maurer	11515438cc	Cargo.toml: use serde feature derive	2019-06-18 06:23:25 +02:00
Dietmar Maurer	ac71623483	src/bin/proxmox-backup-client.rs: implement "key change-passphrase"	2019-06-17 13:10:00 +02:00
Dietmar Maurer	f2401311b0	src/bin/proxmox-backup-client.rs: start key management	2019-06-17 10:33:24 +02:00
Dietmar Maurer	a7dd483097	src/backup/*_index.rs: used generated magic numbers	2019-06-14 14:58:37 +02:00
Dietmar Maurer	bffd40d6b7	src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools	2019-06-14 11:40:04 +02:00
Dietmar Maurer	00388226e6	src/api2/backup/environment.rs: log index checksums	2019-06-14 10:36:20 +02:00
Dietmar Maurer	9335d74eba	src/backup/fixed_index.rs: compute checksum over the index	2019-06-14 10:34:29 +02:00
Dietmar Maurer	16ff6b7cd8	src/backup/dynamic_index.rs: compute checksum over the index	2019-06-14 10:33:59 +02:00
Dietmar Maurer	36075475a6	src/api2/backup/environment.rs: log duplicate chunks	2019-06-14 07:12:30 +02:00
Dietmar Maurer	fa148dbd11	src/api2/backup/upload_chunk.rs: verify chunk data if possible	2019-06-14 06:23:46 +02:00
Dietmar Maurer	0f37577092	src/bin/cipherbench.rs: add test for zstd compression	2019-06-13 17:24:57 +02:00
Dietmar Maurer	9399c98f82	src/bin/cipherbench.rs: cipher speed test with large blocks	2019-06-13 17:16:43 +02:00
Dietmar Maurer	f640f23ac5	src/backup/data_chunk.rs: fix typo	2019-06-13 16:42:55 +02:00
Dietmar Maurer	dba72d5016	src/backup/data_chunk.rs: only use compressed data if it is shorter than uncompressed	2019-06-13 12:56:50 +02:00
Dietmar Maurer	68ab376ae9	src/backup/crypt_setup.rs: only use compressed data if it is shorter than uncompressed	2019-06-13 12:47:54 +02:00
Dietmar Maurer	f98ac774ee	backup: Add support for client side encryption first try ...	2019-06-13 11:47:23 +02:00
Dietmar Maurer	51929e4532	src/backup/data_chunk.rs - decode: make crypt_config optional	2019-06-13 11:44:38 +02:00
Dietmar Maurer	046bcb9773	src/backup/crypt_setup.rs: add comment for possible improvements	2019-06-13 07:35:59 +02:00
Dietmar Maurer	bd0e3c7cfa	src/backup/data_chunk.rs: Moved ChunkInfo from merge_known_chunks.rs	2019-06-13 07:35:02 +02:00
Dietmar Maurer	6762db70d6	src/api2/types.rs: define CHUNK_DIGEST_SCHEMA	2019-06-13 07:30:42 +02:00
Dietmar Maurer	b595cb9d2c	src/backup/data_chunk.rs: Data Chunk builder Make handling encoded chunks easier.	2019-06-12 13:37:01 +02:00
Dietmar Maurer	ef27200cec	src/backup/crypt_setup.rs: fix encoder for uncompressed chunks	2019-06-10 09:56:06 +02:00
Dietmar Maurer	74792b95b2	src/backup/crypt_setup.rs: allow compressed and uncompressed chunks	2019-06-10 08:27:35 +02:00
Dietmar Maurer	9e0187a203	src/backup/crypt_setup.rs: remove unused CryptData struct	2019-06-10 07:55:36 +02:00
Dietmar Maurer	bec8498ae3	src/backup/crypt_setup.rs: avoid useless allocations	2019-06-09 11:44:17 +02:00
Dietmar Maurer	c20b17b460	src/backup.rs: define magic numbers for chunk store file format	2019-06-08 10:34:20 +02:00
Dietmar Maurer	48b4b40b57	src/backup/crypt_setup.rs: crypto helpers	2019-06-08 09:56:51 +02:00
Wolfgang Bumiller	dd5495d6dc	tree-wide: use 'dyn' for all trait objects Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-06-07 13:13:48 +02:00
Christian Ebner	e993db91cd	pxar: introduce fs_feature_flags obtained from filesystem magic in Encoder Not all filesystems support features such as xattrs,acl,... and trying to get them is rather expensive. By getting the supported features based on the filesystem magic and masking the user set feature flags, unsupported features are excluded rather inexpensively while encoding the archive. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-07 11:08:20 +02:00
Christian Ebner	1f319e766e	src/pxar/encoder.rs: refactoring: use ioctl calls from src/tools/fs.rs instead of local Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-07 11:08:06 +02:00
Christian Ebner	9b77388f2a	src/pxar/encoder.rs: minor refactoring Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-07 11:07:55 +02:00
Christian Ebner	a4cc8eb7c5	src/pxar/encoder.rs: check if ioctl supported by filesystem Reading the quota project id relies on a ioctl call to get fsxattr. On FUSE filesystems, ioctl calls might not be supported and will fail with an errno indicating no support. For these cases, the error is ignored and the default project id is used (indicated by returning Ok(None)). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-07 11:07:31 +02:00
Dietmar Maurer	4800df7ba6	src/api2/admin/datastore.rs: remove test-upload api	2019-06-07 07:52:11 +02:00
Dietmar Maurer	113898d7e9	src/api2/admin/datastore/upload.rs: remove unused upload api	2019-06-07 07:39:41 +02:00
Dietmar Maurer	4247fccb0f	src/client/http_client.rs: automatically close connection in finish	2019-06-05 11:41:50 +02:00
Wolfgang Bumiller	cb4426b348	make h2 client connection cancellable Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-06-05 09:45:59 +02:00
Dietmar Maurer	986bef16be	src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1 So that we can include it in static doc strings.	2019-06-05 08:41:20 +02:00
Dietmar Maurer	c9ec0956cf	src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1	2019-06-05 08:12:13 +02:00
Dietmar Maurer	44c54845b3	src/tools/xattr.rs: fix test when run as root	2019-06-05 07:57:42 +02:00
Dietmar Maurer	7773ccc11f	src/api2.rs: move backup api to /backup	2019-06-05 07:23:21 +02:00
Christian Ebner	97eeea3b4a	src/bin/proxmox-backup-proxy.rs: fix typo in pkcs12 error message Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-05 06:29:02 +02:00
Dietmar Maurer	1ce2f7cfb9	src/pxar/encoder.rs: style fix	2019-06-05 06:27:39 +02:00
Christian Ebner	e7b60a16c8	pxar: impl storing/dumping/restoring of quota project ids Allows to store/dump/restore the quota project id associated with an inode in order to correctly restore project quotas. The project id is obtained/set via ioctl calls getting/setting the fsxattr associated with the given file descriptor. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-05 06:26:02 +02:00
Christian Ebner	7312ab9eef	src/pxar/format_definition.rs: add CA_FORMAT_QUOTA_PROJID and CaFormatQuotaProjID Defines the types needed to store/restore quota project ids in pxar Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-05 06:23:06 +02:00
Christian Ebner	042babe4e7	src/tools/fs.rs: impl ioctl calls to get/set fsxattr This implements fs_ioc_fsgetxattr/fs_ioc_fssetxattr calls in order to read or write fsxattr for a given file descriptor. This is needed in order to read or write the quota project id for filesystems which support project quotas (EXT4/XFS/FUSE). Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-06-05 06:22:49 +02:00
Dietmar Maurer	22e5aa1d3e	src/bin/dump-backup-api.rs: helper to generate backup API docs	2019-06-04 13:12:42 +02:00
Dietmar Maurer	5e0f305142	src/api_schema/format.rs: depend on textwrap, new wrap_text helper	2019-06-04 12:42:02 +02:00
Dietmar Maurer	339ddfcbfa	src/cli/command.rs: move doc generator code to src/api_schema/format.rs	2019-06-04 12:32:22 +02:00
Dietmar Maurer	ef39bf95b3	src/client/http_client.rs - upload_config: improve error message	2019-06-03 10:53:09 +02:00
Dietmar Maurer	ec8a9bb953	src/bin/proxmox-backup-client.rs: implement upload_config	2019-06-03 10:39:44 +02:00
Dietmar Maurer	39d6846e1e	src/api2/admin/datastore/backup.rs: implement config file upload	2019-06-03 09:45:14 +02:00
Dietmar Maurer	1c0472e88f	avoid compiler warnings	2019-06-03 06:43:47 +02:00
Dietmar Maurer	36898ffce6	src/backup/chunk_stream.rs: add optional chunk_size parameter	2019-05-30 13:28:24 +02:00
Dietmar Maurer	49ef316bcd	src/bin/proxmox-backup-client.rs: backup_image() - pass correct chunk size	2019-05-30 12:52:12 +02:00
Dietmar Maurer	202cadc29e	src/bin/proxmox-backup-client.rs: remove unused code	2019-05-30 12:47:35 +02:00
Dietmar Maurer	6af905c18f	src/bin/proxmox-backup-client.rs: implement image/block-device upload	2019-05-30 12:46:01 +02:00
Dietmar Maurer	eb1804c567	src/bin/proxmox-backup-client.rs: use std lib instead of nix	2019-05-30 12:19:23 +02:00
Dietmar Maurer	6e907505aa	src/bin/proxmox-backup-client.rs: fix file type test	2019-05-30 12:08:34 +02:00
Dietmar Maurer	96482891ae	src/api2/admin/datastore/backup/environment.rs: log upload statistics	2019-05-30 09:21:19 +02:00
Dietmar Maurer	642322b433	src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload So that we can provide better statistics (patches follows).	2019-05-30 09:21:19 +02:00
Wolfgang Bumiller	390e83c9b9	typo fix and doc improvement Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-29 11:57:28 +02:00
Wolfgang Bumiller	7fb49397d5	doc fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-29 11:56:14 +02:00
Wolfgang Bumiller	382609b04e	tools: add Cancellable, start a futures submodule To make a future cancellable, use: let (future, canceller) = crate::tools::futures::cancellable(future); Proceed with using `future` as usual, `canceller` is clonable and can cancel the future via the `.cancel()` method. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-29 11:18:29 +02:00
Wolfgang Bumiller	0d32d71fb7	tools: add AsyncMutex::new_locked Allows creating a pre-locked mutex, returning the mutex and a guard. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-29 11:18:29 +02:00
Dietmar Maurer	e6389f4e75	src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files)	2019-05-29 10:38:57 +02:00
Dietmar Maurer	39e60bd6fa	src/api2/admin/datastore/backup.rs: improve debug logging	2019-05-29 10:17:38 +02:00
Dietmar Maurer	4ae6551c78	src/api_schema/router.rs: disable debug println	2019-05-29 09:42:16 +02:00
Dietmar Maurer	a42d1f55c3	src/api2/admin/datastore/backup.rs: add debug flag	2019-05-29 09:35:21 +02:00
Dietmar Maurer	976595e1a9	src/backup/dynamic_index.rs: split class DynamicIndexWriter	2019-05-29 08:49:57 +02:00
Dietmar Maurer	cb0708dd46	src/backup/fixed_index.rs: remove ChunkStat from struct	2019-05-29 07:08:34 +02:00
Dietmar Maurer	01af11f340	src/backup/fixed_index.rs: improve consistency checks	2019-05-29 07:02:45 +02:00
Christian Ebner	9b38443314	pxar: change logic and impl all command line flags for xattrs/fcaps/acls Allows to individually set the flags for storing/dumping/restoring of xattrs/fcaps/acls in the cli of pxar. Changes logic so that each of them can be threated individually. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-29 06:51:20 +02:00
Dietmar Maurer	60e589a111	src/api2/admin/datastore/backup.rs: log sucessful finish call	2019-05-28 12:55:22 +02:00
Dietmar Maurer	c4ff3dcefd	src/bin/proxmox-backup-client.rs: use new BackupClient to upload data	2019-05-28 10:12:44 +02:00
Dietmar Maurer	10241c20ea	src/tools.rs: implement image_size helper	2019-05-28 09:35:08 +02:00
Dietmar Maurer	006f3ff407	src/api2/admin/datastore/backup/environment.rs: add more consistency checks	2019-05-28 09:21:31 +02:00
Dietmar Maurer	3dc5b2a203	src/client/http_client.rs: use start of chunk as offset	2019-05-28 09:21:31 +02:00
Dietmar Maurer	8de20e5c1f	src/client/http_client.rs: code cleanup	2019-05-28 08:53:27 +02:00
Dietmar Maurer	fc14b84937	src/backup/fixed_index.rs: improve bound checks inside add_digest	2019-05-28 08:47:03 +02:00
Dietmar Maurer	e1225de467	src/backup/fixed_index.rs: store index_length to avoid code duplication	2019-05-28 08:27:57 +02:00
Dietmar Maurer	e3d525feb7	src/api2/admin/datastore/backup.rs: fix typo	2019-05-28 07:13:19 +02:00
Christian Ebner	af309d4d5c	src/bin/pxar.rs: set archive permissions to 0o640 As the archive can contain potentially sensitive data such as key files, it makes sense to restrict the permissions. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-28 06:24:47 +02:00
Christian Ebner	ab87f167f1	src/pxar/encoder.rs: Refactor file stat Introduce helper functions to check file stats Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-28 06:23:49 +02:00
Christian Ebner	578b601159	src/pxar/encoder.rs: Don't bail if endpoint does not support xattrs. The encoder bailed if a endpoint which did not support xattrs was encountered. Instead of bailing, we ignore these errors and simply do not store xattrs for such endpoints. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-28 06:22:16 +02:00
Dietmar Maurer	a42fa400ee	src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api	2019-05-28 06:18:55 +02:00
Dietmar Maurer	29ae5c86a2	src/backup/fixed_index.rs: make chunk_size public	2019-05-28 06:18:24 +02:00
Dietmar Maurer	7dd1bcac58	src/client/http_client.rs: correctly parse error response in download_chunk_list	2019-05-27 09:35:30 +02:00
Dietmar Maurer	a1e7cff3ea	src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk	2019-05-27 07:57:43 +02:00
Dietmar Maurer	771953f9eb	src/api2/admin/datastore/backup.rs: use smaller window sizes	2019-05-27 07:53:40 +02:00
Dietmar Maurer	174ad378d8	src/client/http_client.rs: fix request pipelining	2019-05-27 07:24:32 +02:00
Dietmar Maurer	624362226e	src/client/merge_known_chunks.rs: use MergedChunkInfo as input	2019-05-26 10:52:56 +02:00
Dietmar Maurer	05cba08c9c	src/client/http_client.rs: append chunks in correct order We need to wait for response from upload_chunk. After that we can add the chunk to the index file.	2019-05-26 09:57:34 +02:00
Dietmar Maurer	417cb0731e	src/api2/admin/datastore/backup.rs: verify chunk offset This clearly shows that the current approach does not work - seems we get chunks out of order.	2019-05-26 08:49:29 +02:00
Dietmar Maurer	34114e2606	src/server/rest.rs: correctly verify json parameters	2019-05-24 09:10:59 +02:00
Dietmar Maurer	eb2bdd1bcc	src/client/http_client.rs: encode parameters inside body To avoid problems with long uris.	2019-05-24 08:32:55 +02:00
Dietmar Maurer	9bb675ec23	src/client/merge_known_chunks.rs: reduce RAM usage There is no need to store chunk data for known chunks.	2019-05-24 07:42:58 +02:00
Dietmar Maurer	ecb21b4794	src/client/merge_known_chunks.rs: avoid VecDequeu - a single buffer is enough	2019-05-24 07:36:09 +02:00
Christian Ebner	2dbba78b98	src/pxar/sequential_decoder.rs: impl support to dump/restore ACLs from pxar archives Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-24 06:27:24 +02:00
Christian Ebner	6a19448004	src/pxar/encoder.rs: impl support to store ACLs in pxar achives Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-24 06:21:33 +02:00
Christian Ebner	7cfaade7c8	src/pxar/format_definition.rs: define types and structs needed to impl ACL support in pxar Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-24 06:16:55 +02:00
Christian Ebner	b344461b33	pxar: pass feature_flags to encoder/decoder instead of individual flags Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-24 06:11:36 +02:00
Christian Ebner	687407741b	src/tools/xattr.rs: add test for is_valid_xattr_name() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-24 06:11:36 +02:00
Christian Ebner	357e4614e2	Minor refactoring of pxars xattr encoder/decoder, mostly reformatting and renaming. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-24 06:11:36 +02:00
Wolfgang Bumiller	c05a8c8d18	doc-test fixup cargo test by default compiles and runs all code snippets found in the documentation... oops... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-23 15:22:33 +02:00
Wolfgang Bumiller	cc84a830c5	RFC: schema: make enums static I don't see a reason to allow these to be dynamically modifiable. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-23 12:38:51 +02:00
Dietmar Maurer	aa1b2e04fe	src/client/merge_known_chunks.rs: merge known chunks To decrease the number of api calls required...	2019-05-23 12:31:09 +02:00
Dietmar Maurer	91320f0879	src/client/http_client.rs: use ChunkInfo streams This will make out of order uploads possible...	2019-05-23 12:31:09 +02:00
Wolfgang Bumiller	8ea3b1d188	tree-wide: use the new vec/io tools modules Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-23 09:49:10 +02:00
Wolfgang Bumiller	9cdda3f7c7	tools: add helpful vector and read operations After importing the I/O ops trait via: use crate::tools::io::ops::; Instead of: let mut buffer = vec![0u8; 65536]; file.read_exact(&mut buffer)?; use: let buffer = file.read_exact_allocated(65536)?; After importing the vector helpers via: use crate::tools::vec::{self, ops::}; For a buffer which could be uninitialized but you prefer zero-initialization anyway for security reasons, instead of: let mut buffer = vec![0u8; len]; use: let mut buffer = vec::undefined(len); which zero-initializes, but, if the `valgrind` feature flag is enabled, marks the vector as having undefined contents, so reading from it will cause valgrind errors. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-05-23 09:49:10 +02:00
Dietmar Maurer	8bea85b42e	src/api2/admin/datastore/backup.rs: verify file size and chunk count on close	2019-05-23 08:50:36 +02:00
Dietmar Maurer	1179e15842	src/server/rest.rs: improve error handling	2019-05-23 08:15:32 +02:00
Dietmar Maurer	907f5bb0d8	src/api2/admin/datastore/backup/service.rs: improve error logging	2019-05-23 08:05:39 +02:00
Dietmar Maurer	32e1bf4260	src/bin/upload-speed.rs: code cleanup	2019-05-23 07:20:05 +02:00
Christian Ebner	0d9bab05ab	src/bin/pxar.rs: impl CLI flags to disable archiving/restoring of xattrs and fcaps. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-23 06:40:50 +02:00
Christian Ebner	4add706092	src/client/: pass dummy flags for xattrs and fcaps to pxar encoder/decoder. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-23 06:40:32 +02:00
Christian Ebner	54b14ad741	src/pxar/decoder.rs: pass flags for xattrs and fcaps to decoder. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-23 06:40:09 +02:00
Christian Ebner	1201360def	src/pxar/sequential_decoder.rs: Conditional restore xattrs and fcaps if feature_flags is set Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-23 06:39:16 +02:00
Christian Ebner	8bc8791338	src/pxar/format_definitions.rs: do not store the feature_flags in pxar archive. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-23 06:37:37 +02:00
Christian Ebner	a45a123950	src/pxar/encoder: Use encoder feature_flags, do not store them within the archive. Instead of testing and storing the feature_flags in the pxar archive, set and use the feature flags within the encoder. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-23 06:34:36 +02:00
Dietmar Maurer	d6f204eda6	src/client/http_client.rs: re-order code - public functions first	2019-05-23 06:28:10 +02:00
Dietmar Maurer	9af37c8f0e	src/client/http_client.rs: move low level H2 code into separate class	2019-05-22 17:28:25 +02:00
Dietmar Maurer	d2c48afc6e	src/client/http_client.rs: code cleanup	2019-05-22 13:24:33 +02:00
Dietmar Maurer	6ab34afa88	src/client/http_client.rs: cleanup h2 backup client	2019-05-22 13:05:51 +02:00
Dietmar Maurer	a55fcd740f	src/api2/admin/datastore/backup.rs: suse POST for finish() api	2019-05-22 13:04:58 +02:00
Dietmar Maurer	de6742c04f	src/bin/h2test.rs: removed This test does no work anymore.	2019-05-22 11:14:56 +02:00
Dietmar Maurer	5a0b521809	src/api2/admin/datastore/backup.rs: remove test callbacks	2019-05-22 10:55:27 +02:00
Christian Ebner	de61bc9264	src/tools/xattr.rs: impl unit tests for fsetattr and fgetattr As git does not preserve xattrs they have to be set before they can be read. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-22 10:01:10 +02:00
Dietmar Maurer	553610b43e	src/client/http_client.rs: implement download_chunk_list	2019-05-22 09:46:02 +02:00
Dietmar Maurer	0cc0fffd1e	src/backup/chunk_stream.rs: use BytesMut in ChunkStream	2019-05-22 09:39:02 +02:00
Dietmar Maurer	c052be5c86	src/backup/chunk_stream.rs: use more generics	2019-05-22 09:18:05 +02:00
Dietmar Maurer	169c0e060f	src/backup/chunk_stream.rs: optimize FixedChunkStream (use BytesMut)	2019-05-22 09:05:35 +02:00
Dietmar Maurer	3be3f3dcc4	src/backup/chunk_stream.rs: fix behaviour for large input buffers	2019-05-22 08:05:27 +02:00
Dietmar Maurer	02fa54ff3c	src/backup/chunk_stream.rs: assemble data with correct order	2019-05-22 07:44:51 +02:00
Dietmar Maurer	d0afeaa831	src/api2/admin/datastore/backup.rs: improve error message	2019-05-22 07:43:54 +02:00
Dietmar Maurer	11867a2b54	src/tools.rs: fix hex_to_digest	2019-05-21 13:42:13 +02:00
Dietmar Maurer	7f3d2ffab9	src/backup/index.rs: rename ChunkListReader into DigestListEncoder	2019-05-21 12:28:44 +02:00
Dietmar Maurer	ee53955f52	src/backup/index.rs: implement DigestListDecoder	2019-05-21 12:21:22 +02:00
Dietmar Maurer	a95849321c	src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks	2019-05-21 09:54:11 +02:00
Dietmar Maurer	40f4e198a8	src/backup/dynamic_index.rs: add chunk_info method	2019-05-21 09:52:19 +02:00
Christian Ebner	596182056b	src/pxar/sequential_decoder.rs: impl functionality to restore xattrs/fcaps from pxar achives Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-21 06:15:30 +02:00
Christian Ebner	b303057a19	src/pxar/encoder.rs: impl functionality to encode xattrs/fcaps in pxar Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-21 06:06:26 +02:00
Christian Ebner	bee8d8ea78	src/tools/xattr.rs: add functions name_store and security_capability These functions allow to check if the name for xattrs/fcaps are valid. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-21 06:05:31 +02:00
Dietmar Maurer	a09c0e38d8	src/api2/admin/datastore/backup/environment.rs: register/lookup known chunks	2019-05-20 18:05:10 +02:00
Dietmar Maurer	97f03eff13	src/client/pxar_backup_stream.rs: fix docu	2019-05-20 17:29:00 +02:00
Dietmar Maurer	82ab72304e	src/client/http_client.rs: implement upload_stream	2019-05-20 14:19:24 +02:00
Dietmar Maurer	2698e8a514	src/client/pxar_backup_stream.rs: close pipe inside drop	2019-05-20 11:20:33 +02:00
Dietmar Maurer	f2269d8f99	src/tools.rs: implement hex_to_digest	2019-05-19 12:15:46 +02:00
Dietmar Maurer	751030aa7c	src/bin/test_chunk_speed2.rs: test max chunk size	2019-05-19 11:09:02 +02:00
Dietmar Maurer	ff77dbbea3	Revert "src/backup/chunk_stream.rs: simplify code" This reverts commit `e058744d8f`. The optimization was wrong, and produces wrong chunk boundaries.	2019-05-19 11:05:56 +02:00
Dietmar Maurer	178ac042c4	src/api_schema/schema.rs: improve error messages	2019-05-19 10:18:51 +02:00
Dietmar Maurer	0c516b123d	src/client/pxar_backup_stream.rs: use WrappedStreamReader to make it fully async ...	2019-05-18 15:13:31 +02:00
Dietmar Maurer	c6e28b66c6	src/client/pxar_backup_stream.rs: use 1M pipe buffer size	2019-05-18 12:57:43 +02:00
Dietmar Maurer	6276448519	src/bin/test_chunk_speed2.rs: test chunker speed with real data	2019-05-18 11:59:17 +02:00
Dietmar Maurer	e058744d8f	src/backup/chunk_stream.rs: simplify code	2019-05-18 10:55:46 +02:00
Dietmar Maurer	8a7cc7565f	src/backup/chunk_stream.rs: implement fixed size chunk stream	2019-05-18 10:55:46 +02:00
Dietmar Maurer	9c9ad941c0	src/client/http_client.rs: improve speed test pipeline responses to a separate async channel ...	2019-05-18 10:55:46 +02:00
Christian Ebner	2dcdd3b427	src/tools/xattrs.rs: impl libc wrapper functions to get/set xattrs and fcaps Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-16 11:14:21 +02:00
Christian Ebner	de2016d58a	src/pxar/format_definitions.rs: add format definitions for xattrs and fcaps In addition to the format definition, the traits needed for sorting of xattr entries by name are derived. Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-16 11:14:13 +02:00
Christian Ebner	dec1ff18b8	src/tools/acl.rs: impl add_entry_full for ACL and set_permissions for ACLEntry Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-16 11:10:06 +02:00
Dietmar Maurer	adec8ea263	src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2	2019-05-16 10:24:23 +02:00
Dietmar Maurer	372724afea	src/api2/admin/datastore/backup.rs: implement finish_backup	2019-05-15 12:58:55 +02:00
Dietmar Maurer	bb105f9dae	src/api2/admin/datastore/backup.rs: improve logging	2019-05-15 10:09:46 +02:00
Dietmar Maurer	25ddc15fe0	src/client/pipe_to_stream.rs: add missing file	2019-05-15 09:07:19 +02:00
Dietmar Maurer	a20772525f	src/api2/admin/datastore/backup.rs: implement close for dynamic writers	2019-05-15 07:58:05 +02:00
Dietmar Maurer	4e93f8c164	src/api2/types.rs: add schema/format for file names	2019-05-15 07:14:08 +02:00
Dietmar Maurer	e3dbd41b20	src/client/http_client.rs: add flow control for h2 upload	2019-05-14 14:54:21 +02:00
Dietmar Maurer	97f22ce57d	src/client/http_client.rs: implement h2 upload without flow control so far ...	2019-05-14 12:58:59 +02:00
Dietmar Maurer	dafc27aeff	src/backup/chunk_stream.rs: async chunk stream	2019-05-14 10:05:29 +02:00
Dietmar Maurer	c0e22141ed	src/client/http_client.rs: derive Clone for H2Client	2019-05-14 09:00:24 +02:00
Christian Ebner	25e205a1d3	src/tools.rs: add mod acl to tools Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-14 06:34:24 +02:00
Christian Ebner	d22096ef4b	src/tools/acl.rs: impl POSIX access control list (ACL) manipulation tools Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-05-14 06:28:42 +02:00
Dietmar Maurer	1133fe9a63	src/server/rest.rs: do not log 1xx status codes as errors	2019-05-14 06:23:22 +02:00
Dietmar Maurer	850ac6d0f0	src/client/http_client.rs: directly return H2Client on upgrade	2019-05-13 16:44:59 +02:00
Dietmar Maurer	10130cf48c	src/client/http_client.rs: cleanup, remove debug println	2019-05-13 12:11:18 +02:00
Dietmar Maurer	b57cb26406	src/client/http_client.rs: implement http2 client wrapper	2019-05-13 10:27:22 +02:00
Dietmar Maurer	9e391bb7f0	src/client/http_client.rs: allow to pass parameters (encode them as query string)	2019-05-13 09:12:03 +02:00
Dietmar Maurer	fbb798f6d2	src/api2/admin/datastore/backup.rs: ignore errors from last_backup	2019-05-13 07:06:56 +02:00
Dietmar Maurer	d36113665b	src/api2/admin/datastore/backup.rs: implement chunk index download from previous backup	2019-05-11 16:05:50 +02:00
Dietmar Maurer	457531e7fe	src/backup/index.rs: new ChunkListReader	2019-05-11 16:01:42 +02:00
Dietmar Maurer	af1d68185c	src/api2/admin/datastore/backup.rs: add missing 'archive-name' parameter to schema	2019-05-11 12:25:15 +02:00
Dietmar Maurer	6b95c7df07	src/api2/admin/datastore/backup/environment.rs: do not store path	2019-05-11 12:13:44 +02:00
Dietmar Maurer	51a4f63ff7	src/backup/datastore.rs: move last_backup to BackupInfo	2019-05-11 12:07:09 +02:00
Dietmar Maurer	b02a52e3a5	src/backup/datastore.rs: impl last_backup helper	2019-05-11 11:36:46 +02:00
Dietmar Maurer	c09775011a	src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups	2019-05-11 10:19:34 +02:00
Dietmar Maurer	f9578f3c79	backup-api: allow to create DynamicIndexWriter and add chunks to to	2019-05-10 10:25:40 +02:00
Dietmar Maurer	35a2d8a6a6	src/backup/backup_info.rs: Derive Clone	2019-05-10 06:59:23 +02:00
Dietmar Maurer	0506d1ed04	upload_chunk.rs: add comment about possible unnecessary code	2019-05-10 06:15:25 +02:00
Dietmar Maurer	5f42ef009a	cleanup: remove unused code	2019-05-09 18:11:39 +02:00
Dietmar Maurer	b4b63e520d	handle_async_api_request: put rpcenv into a Box So that we can pass rpcenv into futures.	2019-05-09 18:01:24 +02:00
Dietmar Maurer	2c41096a4b	src/api2/admin/datastore/backup/upload_chunk.rs: fix maximal chunk size	2019-05-09 13:29:50 +02:00
Dietmar Maurer	78b5191550	src/server/formatter.rs: further cleanups and renaming ...	2019-05-09 13:28:26 +02:00
Dietmar Maurer	d55037e471	src/server/formatter.rs: rename format_result to format_data To avoid confusions with Rust Result type.	2019-05-09 13:15:15 +02:00
Dietmar Maurer	21ee7912fa	src/api2/admin/datastore/backup.rs: implement upload chunk	2019-05-09 13:06:09 +02:00
Dietmar Maurer	4ebf0eabb0	src/api2/types.rs: use separate file for schema definitions	2019-05-09 07:44:09 +02:00
Dietmar Maurer	bd1507c4fa	src/api2/admin/datastore/backup/service.rs: move service code into extra file	2019-05-08 17:36:19 +02:00
Dietmar Maurer	0aadd40b4f	src/api2/admin/datastore/backup.rs: add required parameters, cleanups	2019-05-08 13:17:23 +02:00
Dietmar Maurer	d95ced64ad	src/api2/admin/datastore/backup/environment.rs: use separate file	2019-05-08 12:41:58 +02:00
Dietmar Maurer	ca60c371e2	api: rename h2upload to backup	2019-05-08 12:35:15 +02:00
Dietmar Maurer	d906b1124f	src/api2/admin/datastore/h2upload.rs: code cleanups	2019-05-08 11:36:43 +02:00
Dietmar Maurer	58c8d7d91f	rc/api2/admin/datastore/h2upload.rs: implement BackupEnvironment To pass arbitrary information/state to api methods.	2019-05-08 11:26:54 +02:00
Dietmar Maurer	f757b30efc	src/server/rest.rs: use generics to pass RpcEnvironment	2019-05-08 11:09:01 +02:00
Dietmar Maurer	2edc341b29	src/tools.rs: Add AsAyn Trait	2019-05-08 11:07:36 +02:00
Dietmar Maurer	090ac9f79a	src/api2/admin/datastore/h2upload.rs: avoid unwrap	2019-05-08 08:21:18 +02:00
Dietmar Maurer	4e3da4b366	src/api2/admin/datastore/h2upload.rs: do not log client address	2019-05-08 08:15:55 +02:00
Dietmar Maurer	e3a44552ee	src/api2/admin/datastore/h2upload.rs: code cleanups	2019-05-08 08:12:49 +02:00
Dietmar Maurer	a66ab8ae5e	src/api2/admin/datastore/h2upload.rs: make worker abortable	2019-05-08 07:14:40 +02:00
Dietmar Maurer	72375ce6c6	src/api2/admin/datastore/h2upload.rs: log to worker task	2019-05-07 14:07:00 +02:00
Dietmar Maurer	d9bd06eae8	src/api2/admin/datastore/h2upload.rs: use WorkerTask	2019-05-07 13:42:00 +02:00
Dietmar Maurer	52cf506e48	src/api2/admin/datastore/h2upload.rs: implement async test api call	2019-05-07 12:26:55 +02:00
Dietmar Maurer	92ac375aff	src/api2/admin/datastore/h2upload.rs: improve http2 service	2019-05-07 11:24:44 +02:00
Dietmar Maurer	78f7e225e4	src/server/rest.rs: make handle_(a)sync_api_request public	2019-05-07 11:23:52 +02:00
Dietmar Maurer	75097495ad	RestEnvironment: derive Clone	2019-05-07 11:09:18 +02:00
Dietmar Maurer	01bf3b7b5f	move find_method to Router class	2019-05-07 11:08:30 +02:00
Dietmar Maurer	3578d99f3e	move normalize_path to tools::normalize_uri_path	2019-05-07 09:44:34 +02:00
Dietmar Maurer	96e95fc179	src/api2/admin/datastore/h2upload.rs: test if version < 2 We can only upgrade http 1.x connections.	2019-05-06 10:29:34 +02:00
Dietmar Maurer	45cdce069e	src/client/http_client.rs: request password at creation time	2019-04-30 11:44:35 +02:00
Dietmar Maurer	e45afdff9c	src/server/state.rs: use new BroadcastData helper	2019-04-30 10:21:48 +02:00
Dietmar Maurer	824b5ee4ee	src/tools/broadcast_future.rs: cleanup, decompose into two classes In order to make it more usable.	2019-04-30 08:57:20 +02:00
Wolfgang Bumiller	d82ed9b0f5	tools: add AsyncMutex Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-29 14:19:03 +02:00
Dietmar Maurer	3a9bea3918	src/bin/h2test.rs: test h2 upgrade	2019-04-29 11:59:18 +02:00
Dietmar Maurer	cf639a47de	rc/client/http_client.rs: add experimental h2 upgrade code	2019-04-29 11:57:58 +02:00
Dietmar Maurer	5a2df00004	src/client/http_client.rs: cleanup, make login fully async	2019-04-28 10:55:03 +02:00
Dietmar Maurer	3dceb9b304	src/tools/broadcast_future.rs: add new constructor new_oneshot() To simplify usage. Also added a test case.	2019-04-27 10:56:49 +02:00
Dietmar Maurer	152764ec15	src/api2/admin/datastore/h2upload.rs: add test code for http2	2019-04-27 08:57:35 +02:00
Dietmar Maurer	490be29ed6	src/tools/broadcast_future.rs: new helper class	2019-04-26 17:57:57 +02:00
Wolfgang Bumiller	e888155712	proxy: allow .key/.pem files instead of a pkcs12/.pfx file Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-26 10:47:33 +02:00
Christian Ebner	8b7c0def84	src/tools/procfs.rs: clean up tab indentation and use whitespaces instead Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-25 13:14:52 +02:00
Wolfgang Bumiller	5e5eed5c3b	use double-fork for reload To ensure the new process' parent is pid 1, so systemd won't complain about supervising a process it does not own. Fixes the following log spam on reloads: Apr 25 10:50:54 deb-dev systemd[1]: proxmox-backup.service: Supervising process 1625 which is not our child. We'll most likely not notice when it exits. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-25 11:02:12 +00:00
Wolfgang Bumiller	efd1536eb7	add a wrapper around nix::unistd::pipe2 Using O_CLOEXEC by default, and returning Fd handles to ensure they get dropped on bail!() or panic!() if the RawFds aren't used yet. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-25 11:02:12 +00:00
Wolfgang Bumiller	897982e237	tools: add read/write convenience helpers Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-25 11:02:12 +00:00
Wolfgang Bumiller	d98c9a7a38	use service Type=notify Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-25 11:01:33 +00:00
Wolfgang Bumiller	9c351a36c4	tools/daemon: add sd_notify wrapper Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-25 11:01:28 +00:00
Dietmar Maurer	d8871f2f18	src/tools/procfs.rs: avoid useless allocations	2019-04-24 09:35:15 +02:00
Christian Ebner	ae694f44ef	src/tools/procfs.rs: add tests for read_proc_net_route() and read_proc_net_ipv6_route() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-24 06:43:52 +02:00
Christian Ebner	ec8db837fb	src/tools/procfs.rs: refactor read_proc_net_route() and read_proc_net_ipv6_route() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-24 06:43:32 +02:00
Christian Ebner	eb46c3d1ec	src/tools/procfs.rs: implement read_proc_net_ipv6_route() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-23 14:31:23 +02:00
Dietmar Maurer	6358b68236	cleanup: avoid compiler warnings	2019-04-23 12:17:24 +02:00
Dietmar Maurer	e06ffb10d1	src/api_schema/router.rs: fix typo (subdira -> subdir)	2019-04-16 12:20:17 +02:00
Dietmar Maurer	13f1cc17ea	src/api_schema/router.rs: implement list_subdirs() helper	2019-04-16 12:07:02 +02:00
Wolfgang Bumiller	062d4916ff	api_schema: allow generic api handler functions Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-16 11:22:23 +02:00
Christian Ebner	286f0d4099	src/tools/procfs.rs: implement read_proc_net_route() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-15 11:13:16 +02:00
Dietmar Maurer	1a6c9415ae	src/api2/node/services.rs: add proxmox-backup-proxy service	2019-04-15 09:48:02 +02:00
Dietmar Maurer	75bc49bed7	src/server/worker_task.rs: implement abort channel	2019-04-15 09:38:05 +02:00
Dietmar Maurer	912524752e	src/api2/admin/datastore/pxar.rs: create worker task for file upload	2019-04-15 08:38:14 +02:00
Dietmar Maurer	1be71fb02b	src/server/worker_task.rs_ do not write ERROR: prefix into task index	2019-04-15 08:37:13 +02:00
Dietmar Maurer	418def7a78	src/server/worker_task.rs: move common code into log_result Make new() pub	2019-04-15 07:49:03 +02:00
Dietmar Maurer	37b87869b5	src/server/upid.rs: check for illegal characters	2019-04-15 07:47:51 +02:00
Christian Ebner	d2981e2738	src/tools/procfs.rs: implement read_proc_net_dev() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-13 11:52:59 +02:00
Christian Ebner	13abfdc57a	src/tools/procfs.rs: implement read_memory_usage() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-13 11:52:01 +02:00
Christian Ebner	3005ef48b6	src/tools/procfs.rs: Rename variable and struct to be more consistent Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-12 12:13:34 +02:00
Christian Ebner	cc93bacb72	src/tools/procfs.rs: simplify read_cpuinfo() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-12 12:13:14 +02:00
Dietmar Maurer	f2b99c34f7	src/api2/admin/datastore.rs: implement API to return last GC status	2019-04-11 12:04:25 +02:00
Dietmar Maurer	59961b8954	src/server/command_socket.rs: check control socket permissions	2019-04-11 10:51:59 +02:00
Dietmar Maurer	ba70040dbf	src/server/worker_task.rs: change task list sort order - latest on top	2019-04-11 09:43:33 +02:00
Dietmar Maurer	d3f4c08f33	src/server/worker_task.rs: catch panics in worker threads	2019-04-11 07:56:06 +02:00
Wolfgang Bumiller	620dccf1a1	tools/daemon: dup the TcpListener file descriptor Now that we let hyper shutdown gracefully we need an owned version of the listening socket to prevent it from closing before running the reload preparations. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-10 15:20:10 +02:00
Wolfgang Bumiller	d96bb7f163	tools: add Fd helper stores a raw file descriptor with a drop handler for safekeeping in closures Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-04-10 15:20:10 +02:00
Christian Ebner	1cd33633aa	/src/tools: Cleanup tempfiles in folder Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-10 13:56:22 +02:00
Dietmar Maurer	a665dea128	src/api2/node/tasks.rs: implement stop_task api	2019-04-10 13:55:05 +02:00
Dietmar Maurer	98a181f055	src/server/worker_task.rs: log message when we call request_abort()	2019-04-10 13:54:28 +02:00
Christian Ebner	2b6a3e1396	src/tools/procfs.rs: implement read_cpuinfo() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-10 13:11:11 +02:00
Dietmar Maurer	321070b4fa	src/server/worker_task.rs: implement abort_worker (via command_socket)	2019-04-10 12:42:24 +02:00
Dietmar Maurer	cfb2d3c17c	src/server/command_socket.rs: correctly handle/spawn handle parallel connections	2019-04-10 11:05:00 +02:00
Dietmar Maurer	9b002cbc5f	src/server/worker_task.rs: use abstract socket	2019-04-10 09:03:17 +02:00
Dietmar Maurer	5f550fd99f	start hyper server using with_graceful_shutdown() Without, hyper keeps some futures running, and the server does not correctly shutdown.	2019-04-10 08:24:32 +02:00
Dietmar Maurer	7f2078de59	src/tools/procfs.rs: avoid compiler warning	2019-04-10 06:40:29 +02:00
Christian Ebner	1f6fbf1c57	src/tools/procfs.rs: implement read_meminfo() Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-09 15:21:49 +02:00
Dietmar Maurer	c360bd73d3	src/api2/node/tasks.rs: fix task status api	2019-04-09 15:12:20 +02:00
Dietmar Maurer	0bfd87bc5e	src/server/worker_task.rs: speedup upid_read_status()	2019-04-09 15:11:48 +02:00
Dietmar Maurer	d8d40dd0c2	src/api2/node.rs: add node parameter	2019-04-09 14:43:30 +02:00
Dietmar Maurer	7ae0bf52dd	src/server/command_socket.rs: code cleanup - fix error message	2019-04-09 12:47:42 +02:00
Dietmar Maurer	d607b8861b	src/server/worker_task.rs: implement task control socket	2019-04-09 12:15:06 +02:00
Dietmar Maurer	e201753629	src/server/command_socket.rs: implement auto_remove flag Remove the socket file on close.	2019-04-09 11:47:23 +02:00
Dietmar Maurer	634132fe10	src/server/upid.rs: moved code into separate file	2019-04-08 18:43:26 +02:00
Dietmar Maurer	78a39e0517	src/server/command_socket.rs: simple command socket	2019-04-08 17:59:39 +02:00
Dietmar Maurer	e3f41f2165	src/tools/daemon.rs: use new ServerState handler	2019-04-08 14:00:23 +02:00
Dietmar Maurer	7a630df709	implement server state/signal handling, depend on tokio-signal	2019-04-08 13:59:07 +02:00
Dietmar Maurer	5a12c0e2fb	src/api2/node/tasks.rs: start upid API	2019-04-07 14:36:57 +02:00
Dietmar Maurer	b75b968147	src/server/worker_task.rs: implement is_active()	2019-04-07 14:36:25 +02:00
Dietmar Maurer	4494d07819	src/server/worker_task.rs: make log_path a method on UPID	2019-04-07 13:51:46 +02:00
Dietmar Maurer	d2a2e02bb7	src/api2/node/tasks.rs: implement userfilter	2019-04-07 13:17:19 +02:00
Dietmar Maurer	3c3bee2e32	src/api2/node/tasks.rs: correctly handle start parameter	2019-04-07 12:41:24 +02:00
Dietmar Maurer	063ca5be77	src/api2/node/tasks.rs: implement list_tasks	2019-04-07 12:18:58 +02:00
Dietmar Maurer	d4b59ae0b8	src/backup/datastore.rs: pass WorkerTask to GC So that we can log to the task log file.	2019-04-06 17:57:38 +02:00
Dietmar Maurer	35950380a9	src/server/worker_task.rs: carefully handle file permissions	2019-04-06 17:53:12 +02:00
Dietmar Maurer	1619a72063	src/tools.rs: implement create_dir_chown Combinded mkdir with chown.	2019-04-06 16:50:23 +02:00
Dietmar Maurer	eea8131952	src/tools.rs: implement file_set_contents_full()	2019-04-06 13:53:43 +02:00
Dietmar Maurer	0f778e0652	src/api2/admin/datastore.rs: use new WorkerTask::new_thread()	2019-04-06 11:27:23 +02:00
Dietmar Maurer	244abab769	src/tools/file_logger.rs: fix test	2019-04-06 11:24:37 +02:00
Dietmar Maurer	660c684640	src/server/worker_task.rs: return upid_str	2019-04-06 11:23:53 +02:00
Dietmar Maurer	93aebb38bc	src/server/worker_task.rs: implement read_task_list()	2019-04-06 10:17:11 +02:00
Dietmar Maurer	4a36a2c999	src/server/worker_task.rs: improve UPID format description	2019-04-06 09:36:50 +02:00
Dietmar Maurer	882594c5e9	src/server.rs: improve crate layout	2019-04-06 09:17:25 +02:00
Dietmar Maurer	4b01c983f0	src/server/worker_task.rs: implement persistent task list	2019-04-05 12:37:35 +02:00
Christian Ebner	a2474d328f	src/tools/procfs.rs: implement read_proc_uptime and read_proc_uptime_ticks Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-05 06:22:21 +02:00
Christian Ebner	08daa81af6	procfs: implement check_process_running and check_process_running_pstart Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-04-04 13:54:17 +02:00
Dietmar Maurer	9a0d0ff774	server/worker_task.rs: implement UPID parser	2019-04-04 13:49:35 +02:00
Dietmar Maurer	49cf9f3d0b	src/tools.rs: file_get_json() - add new default parameter	2019-04-04 12:24:18 +02:00
Dietmar Maurer	79ad7b7ec2	src/tools/procfs.rs: split regex into several lines	2019-04-04 09:25:19 +02:00
Dietmar Maurer	479f6e40ec	src/server/worker_task.rs: implement task registry	2019-04-04 08:58:39 +02:00
Dietmar Maurer	52ddc2522d	src/tools/procfs.rs: use libc::pid_t	2019-04-04 08:05:43 +02:00
Dietmar Maurer	d3d7ab1273	src/tools/file_logger.rs: new - accept AsRef<Path>	2019-04-03 14:13:33 +02:00
Dietmar Maurer	3c2012f97a	src/tools/procfs.rs: start helper tools to read procfs	2019-04-03 13:39:20 +02:00
Dietmar Maurer	3489936ebb	src/tools/file_logger.rs: change timestamp format to rfc3339	2019-04-03 08:58:43 +02:00
Dietmar Maurer	92da93b245	abort GC on server shutdown	2019-04-01 12:13:02 +02:00
Dietmar Maurer	9136f857cc	add global var to indicate server shutdown requests	2019-04-01 12:05:11 +02:00
Dietmar Maurer	3e6a7deeaa	src/api2/admin/datastore.rs: start GC in a separate thread	2019-04-01 12:02:46 +02:00
Dietmar Maurer	ad281d1bd9	www/DataStoreConfig.js: add button to start GC	2019-04-01 08:08:34 +02:00
Dietmar Maurer	49d123ee3a	src/server/rest.rs: use formatter to encode errors	2019-04-01 08:04:12 +02:00
Dietmar Maurer	7d4ef127cb	src/server/rest.rs: generate csrf token if we have a valid ticket This is important if the user reloads the browser page.	2019-04-01 07:52:30 +02:00
Dietmar Maurer	11861a482d	src/backup/chunk_store.rs: fix GC Added option to get oldest_writer timestamp from ProcessLocker.	2019-03-31 17:21:36 +02:00
Dietmar Maurer	d85987aeeb	fix last commit: the filename var was not ment to be removed, sorry	2019-03-31 16:16:14 +02:00
Dietmar Maurer	15a77c4c2e	src/backup/chunk_store.rs: avoid create/unlink race	2019-03-31 10:03:01 +02:00
Dietmar Maurer	c374f05499	src/backup.rs: improve GC problem description	2019-03-31 09:44:35 +02:00
Dietmar Maurer	c8ec450e37	src/backup.rs: start explaining different GC algorithm	2019-03-30 17:21:40 +01:00
Dietmar Maurer	8a4757342c	src/backup.rs: describe the garbage collection problem	2019-03-30 16:26:52 +01:00
Dietmar Maurer	10eea49d81	src/backup/fixed_index.rs: store reference to file Keep it open and locked while index is in use.	2019-03-27 10:17:03 +01:00
Dietmar Maurer	c597a92c81	index file readers: aquire shared lock	2019-03-27 10:11:14 +01:00
Dietmar Maurer	0465218953	src/backup.rs: add documentation about ChunkStore locking	2019-03-22 10:14:50 +01:00
Dietmar Maurer	43b1303398	datastore: use new ProcessLocker To make sure only one process runs garbage collection while having active writers.	2019-03-22 09:42:15 +01:00
Dietmar Maurer	abfc001f25	src/tools/process_locker.rs: new() - use generic path parameter	2019-03-22 09:41:32 +01:00
Dietmar Maurer	a650f50301	src/tools/process_locker.rs: implement inter-process reader-writer locks	2019-03-22 08:05:51 +01:00
Wolfgang Bumiller	eb90c9e372	tools: style fixup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-20 09:57:13 +01:00
Christian Ebner	e04d1bfadc	Cleanup of if statement with unwraps by a more compact match statement Signed-off-by: Christian Ebner <c.ebner@proxmox.com>	2019-03-20 09:49:18 +01:00
Dietmar Maurer	8346f0d59b	src/server/rest.rs: correctly extract content type	2019-03-19 12:50:15 +01:00
Wolfgang Bumiller	a690ecacf0	daemon: simplify daemon creation Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-19 12:12:54 +01:00
Wolfgang Bumiller	e4311382d9	tools: daemon: rename some structs Reloadable resources are now 'Reloadable' instead of 'ReexecContinue'. The struct handling the reload is a 'Reloader', not a 'ReexecStore'. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-19 11:09:46 +01:00
Wolfgang Bumiller	5e7bc50af1	add reload support to api daemon Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Wolfgang Bumiller	a2ca713701	add reload support to proxy Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Wolfgang Bumiller	af70c1811b	tools: implement ReexecContinue for tokio's TcpListener This is the only thing we currently need to keep alive for reloads. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Wolfgang Bumiller	4422ba2c7f	tools: daemon: add a default signalfd helper Proxy and daemon for now just want to handle reload via `SIGHUP`, so provide a helper creating the signalfd stream doing that - this is simply a filtered stream which passes the remaining signals through, so it can be used exactly like the signalfd stream could before to add more signals. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Wolfgang Bumiller	dce94d0e09	tools: add daemon helpers Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Wolfgang Bumiller	ff7049d481	tools: add fd_change_cloexec helper Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Wolfgang Bumiller	1c042cdc6c	proxy: listener error handling fixup If the listening socket goes into some error state we'll get std::io::Errors rather than higher level errors from the native_tls::TlsAcceptor, those are usually fatal. (Ran into this after performing a shutdown() on the file descriptor, after which the future just endlessly loops in accept().) Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-18 12:29:18 +01:00
Dietmar Maurer	60864b1b56	src/pxar.rs: improve docu	2019-03-18 12:27:30 +01:00
Dietmar Maurer	515688d1f1	avoid compiler warnings	2019-03-18 10:00:58 +01:00
Dietmar Maurer	ba10f2b0d4	src/bin/dynamic-files.rs: test case for file shrink during backup	2019-03-16 14:22:14 +01:00
Dietmar Maurer	af572aaa4c	pxar: implement hardlinks So we are no longer compatible with catar ...	2019-03-16 11:02:12 +01:00
Dietmar Maurer	812dd935ce	src/pxar/encoder.rs: detect hardlinks	2019-03-15 12:19:51 +01:00
Dietmar Maurer	896e803feb	src/pxar/sequential_decoder.rs: dump_entry - fix FIFO/SOCKET handling	2019-03-15 12:15:38 +01:00
Dietmar Maurer	7dfa17c7cc	src/pxar/binary_search_tree.rs: fix test	2019-03-15 11:34:31 +01:00
Dietmar Maurer	9307279faf	cleanup: remove dead code	2019-03-15 10:18:28 +01:00
Dietmar Maurer	46cfe7ab7e	src/pxar/decoder.rs: implement seekable decoder	2019-03-15 09:36:05 +01:00
Dietmar Maurer	0e21b87be3	renamed: src/pxar/decoder.rs -> src/pxar/sequential_decoder.rs	2019-03-15 08:36:02 +01:00
Dietmar Maurer	3dbfe5b142	simplify pxar module structure	2019-03-15 08:24:32 +01:00
Dietmar Maurer	1ef46b81d3	src/bin/pxar.rs: implement extract	2019-03-15 08:03:44 +01:00
Dietmar Maurer	02c93163cf	src/pxar/decoder.rs: correctly call callback()	2019-03-15 08:02:04 +01:00
Dietmar Maurer	5defa71bf9	rename PxarBackupWriter into PxarDecodeWriter	2019-03-15 07:20:22 +01:00
Dietmar Maurer	b005ed12d5	src/client/http_client.rs: avoid compiler warning	2019-03-15 07:16:26 +01:00
Dietmar Maurer	9e26abf16b	src/cli/command.rs: fix iterator usage	2019-03-15 07:12:40 +01:00
Dietmar Maurer	f98746ba6c	src/pxar/decoder.rs: improve docu	2019-03-15 07:12:11 +01:00
Dietmar Maurer	0b2d7fdc81	src/pxar/decoder.rs: avoid compiler warnings	2019-03-14 17:45:47 +01:00
Dietmar Maurer	37940aa1f9	src/bin/pxar.rs: implement dump/list	2019-03-14 17:43:11 +01:00
Dietmar Maurer	40360fde6e	src/pxar/decoder.rs: implement dump_archive use this to debug archive problems ...	2019-03-14 13:10:27 +01:00
Dietmar Maurer	8968258b66	rename catar into pxar To avoid confusion with the casync implementation.	2019-03-14 10:54:09 +01:00
Dietmar Maurer	7c4dd94670	rename catar into pxar Seems impossible to stay compatible with the original catar specification, so we change the name to avoid confusions.	2019-03-14 10:14:32 +01:00
Dietmar Maurer	806500cd09	src/tools.rs: improve file name completion Use current working directory by default.	2019-03-14 08:17:35 +01:00
Dietmar Maurer	08dc340a12	src/bin/proxmox-backup-client.rs: complete archive names	2019-03-14 08:09:35 +01:00
Wolfgang Bumiller	6f90a6a764	protocol: cleanup finish_backup Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-13 14:16:17 +01:00
Dietmar Maurer	b238851832	src/bin/proxmox-backup-client.rs: implement completion for snapshots	2019-03-13 13:31:29 +01:00
Dietmar Maurer	4981134708	src/bin/proxmox-backup-client.rs: implement chunk-size completion	2019-03-13 12:26:01 +01:00
Dietmar Maurer	98f0b97293	src/bin/proxmox-backup-client.rs: cleanup	2019-03-13 12:17:39 +01:00
Dietmar Maurer	024f11bb55	src/bin/proxmox-backup-client.rs: implement completion for backup groups	2019-03-13 11:56:37 +01:00
Dietmar Maurer	edd3c8c605	src/client/backup_repo.rs: implement FromStr trait	2019-03-13 10:09:39 +01:00
Dietmar Maurer	874acb7039	src/client/backup_repo.rs: implement Display trait	2019-03-13 09:57:36 +01:00
Dietmar Maurer	d0a03d40ce	src/bin/proxmox-backup-client.rs: implement repository completions	2019-03-13 09:47:12 +01:00
Dietmar Maurer	bff110308b	src/bin/proxmox-backup-client.rs: fix BACKUPSPEC_REGEX - correctly specify non-capture group And improve bash completion.	2019-03-13 07:23:59 +01:00
Dietmar Maurer	1a71509aad	src/cli/command.rs: fix completion - never display argument parameters as option	2019-03-12 14:53:41 +01:00
Dietmar Maurer	496a67846f	src/cli/command.rs: pass parsed parameters to completion function	2019-03-12 14:39:51 +01:00
Dietmar Maurer	b5fa28251c	src/cli/getopts.rs: factor out reusable code	2019-03-12 14:38:47 +01:00
Dietmar Maurer	e39efdbd33	src/cli/command.rs: use a HashMap to store fixed param as key/value pairs	2019-03-12 12:54:16 +01:00
Dietmar Maurer	f4e4d58327	src/cli/command.rs: improve completions	2019-03-12 12:07:45 +01:00
Wolfgang Bumiller	4ba7b6099c	signalfd cleanup - add comment about the boxed file descriptor (and others) - reexport SigSet and Signal for convenience - remove Result wrapper (only used once) - enforce drop order of the PollEvented2 struct with respect to the boxed fd Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-12 11:07:09 +01:00
Wolfgang Bumiller	f54c19989c	tools: add async signalfd handler This provides a Stream<Item = siginfo> via nix' signalfd, by wrapping it in tokio's PollEvented2 struct to allow polling via tokio's event loop. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-12 08:04:28 +01:00
Dietmar Maurer	d5c34d98c0	src/catar/decoder.rs: simplify public restore API	2019-03-11 14:31:01 +01:00
Dietmar Maurer	8e39232acc	src/bin/proxmox-backup-client.rs: strip .didx file extensions	2019-03-11 10:54:02 +01:00
Dietmar Maurer	74cdb52107	src/bin/proxmox-backup-client.rs: include file extension in backupspec	2019-03-11 10:27:28 +01:00
Dietmar Maurer	fc2bf37ead	src/catar/decoder.rs: cleanup style	2019-03-11 08:14:28 +01:00
Dietmar Maurer	656b23e107	src/catar/decoder.rs: dir_mkdirat - add create_new flag	2019-03-11 07:27:45 +01:00
Dietmar Maurer	e9c9409a99	src/catar/decoder.rs: implement real sequential decoder (no Seek) I copied the old code to src/catar/inspector.rs. Will modify that later to implement a random access decoder ...	2019-03-08 16:55:54 +01:00
Dietmar Maurer	eed6db3923	src/bin/proxmox-backup-client.rs: impl option all-file-systems	2019-03-08 09:36:40 +01:00
Wolfgang Bumiller	06178f13fa	move chunker into proxmox_protocol Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-08 09:29:47 +01:00
Dietmar Maurer	e3c30c50a6	src/bin/catar.rs: implement all-file-systems option	2019-03-08 09:28:12 +01:00
Dietmar Maurer	65092b1e27	src/catar/encoder.rs: log skipped mount points	2019-03-08 09:25:00 +01:00
Dietmar Maurer	6b64d25d64	src/catar/encoder.rs: remove debug output	2019-03-08 08:23:07 +01:00
Dietmar Maurer	2689810c92	src/bin/catar.rs: implement verbose flags	2019-03-08 08:20:56 +01:00
Dietmar Maurer	79c281fdac	src/catar/encoder.rs: use simply flag "all_file_systems"	2019-03-08 08:14:26 +01:00
Dietmar Maurer	25e77d3877	src/cli/command.rs: print_bash_completion - avoid crash	2019-03-07 12:14:26 +01:00
Dietmar Maurer	141f062e08	src/backup/chunk_store.rs: use zstd compression insteadf of lz4 Provides better compressionm rate, and is still fast.	2019-03-07 11:42:59 +01:00
Dietmar Maurer	d2690f74bb	src/backup/chunk_store.rs: read_chunk - clear buffer before calling read_to_end	2019-03-07 10:40:01 +01:00
Dietmar Maurer	2ae7d196b9	src/bin/proxmox-backup-client.rs: correctly strip file extension	2019-03-06 11:18:46 +01:00
Dietmar Maurer	9f91249351	src/bin/proxmox-backup-client.rs: impl restore	2019-03-06 11:02:08 +01:00
Dietmar Maurer	01a1342364	src/api2/admin/datastore.rs: impl list_snapshot_files	2019-03-06 11:02:08 +01:00
Dietmar Maurer	58e99e138e	src/backup/backup_info.rs: new helper to list files	2019-03-06 11:02:08 +01:00
Dietmar Maurer	dd79265a8a	src/api2/admin/datastore/catar.rs: use archive_name directly	2019-03-06 11:02:08 +01:00
Dietmar Maurer	a6b7551311	src/client/http_client.rs: impl download	2019-03-06 11:02:08 +01:00
Wolfgang Bumiller	6716f30bbc	add protocol test client This uses futures for everything which is mostly useful as a test to see if the protocol crate's non-blocking I/O support can handle it... Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-06 10:26:12 +01:00
Wolfgang Bumiller	e2d007f76e	api2/admin/datastore: add a backup protocol test api path Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-06 10:26:12 +01:00
Wolfgang Bumiller	e3062f87b1	fixed_index: split add_chunk/add_digest Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-06 10:25:20 +01:00
Wolfgang Bumiller	c3bb97e59c	fixed_index: impl Send for FixedIndexWriter Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-06 10:25:20 +01:00
Wolfgang Bumiller	3b7ade9e78	backup/datastore: expose insert_chunk method Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-06 10:25:20 +01:00
Wolfgang Bumiller	91ebea8262	router: add upgrade method This is just an Async GET method, but semantically different, so it gets its own router method. Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-06 10:25:20 +01:00
Dietmar Maurer	80822b95b1	src/bin/proxmox-backup-client.rs: sort list results	2019-03-06 07:20:51 +01:00
Dietmar Maurer	fba30411fb	src/bin/proxmox-backup-client.rs: allow to set backup-id	2019-03-06 07:02:52 +01:00
Dietmar Maurer	21ea0158b4	src/client/http_client.rs: remove debug statement	2019-03-06 06:45:51 +01:00
Dietmar Maurer	ba3a60b2a4	src/client/http_client.rs: store/load ticket in xdg runtime dir, depend on crate xdg	2019-03-05 12:56:21 +01:00
Dietmar Maurer	f8f94534d4	src/api2/access.rs: authenticate_user() - add ticket login	2019-03-05 12:56:21 +01:00
Dietmar Maurer	e5662b04e9	src/tools/ticket.rs: define const TICKET_LIFETIME	2019-03-05 12:56:21 +01:00
Dietmar Maurer	53cafb5925	src/tools.rs: impl file_get_json()	2019-03-05 12:56:21 +01:00
Wolfgang Bumiller	391d310741	backup_info: BackupDir shortcut Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-05 10:15:06 +01:00
Wolfgang Bumiller	93b49ce38a	backup_info: use two independent Into<String> types there's no reason to force them to be the same input type Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>	2019-03-05 09:13:08 +01:00
Dietmar Maurer	3ec3ec3f9a	src/bin/proxmox-backup-client.rs: print backup duration	2019-03-05 08:11:40 +01:00
Dietmar Maurer	d57474e055	src/backup/backup_info.rs: improve docs	2019-03-05 07:28:13 +01:00
Dietmar Maurer	b3483782d1	src/backup/backup_info.rs: move code into separate file Also changed create_backup_dir() parameters - uses &BackupDir now.	2019-03-05 07:18:12 +01:00
Dietmar Maurer	2b01a22507	src/backup/datastore.rs: add helper to sort backup lists	2019-03-04 18:20:57 +01:00
Dietmar Maurer	875fb1c01a	src/backup/datastore.rs: use unix epoch to create DateTime To make sure that we have a timestamp without nanosecond.	2019-03-04 17:58:22 +01:00
Dietmar Maurer	38f8815925	src/backup/datastore.rs: use new() function to create structs	2019-03-04 17:44:27 +01:00
Dietmar Maurer	9b492eb256	src/backup/datastore.rs: protect BackupDir fields, impl new()	2019-03-04 13:51:36 +01:00
Dietmar Maurer	1e9a94e579	src/backup/datastore.rs: protect BackupGroup fields, impl new()	2019-03-04 13:38:23 +01:00
Dietmar Maurer	cdebd467e6	src/bin/proxmox-backup-client.rs: simply log to stdout	2019-03-04 09:38:34 +01:00
Dietmar Maurer	f72dfd2d10	src/backup/datastore.rs: make sure timestamp does not contains nanoseconds Because we want simple, readable file names.	2019-03-04 09:37:10 +01:00
Dietmar Maurer	219ef0e6ca	src/catar/encoder.rs: add verbose flags	2019-03-04 08:01:09 +01:00
Dietmar Maurer	e0b8ce6424	src/catar/encoder.rs: skip RAMFS unless explicitly selected.	2019-03-04 06:48:47 +01:00
Dietmar Maurer	597a92039e	src/bin/proxmox-backup-client.rs: rename create into backup	2019-03-03 11:46:53 +01:00
Dietmar Maurer	6f62c9240a	src/api2/admin/datastore.rs: imp delete_snapshot	2019-03-03 11:29:00 +01:00
Dietmar Maurer	af926291e0	avoid compiler warning	2019-03-03 09:52:06 +01:00
Dietmar Maurer	e77a02ed87	catar api: cleanup parameter names	2019-03-03 09:48:32 +01:00

... 17 18 19 20 21 ...

2444 Commits