cca67dc0c3
src/backup/readline.rs: removed in favor of new readline impl
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:03:04 +01:00
6934c6fe77
src/backup/catalog_shell.rs: adapt to use API Schema definition and rustyline
...
This major refactoring of the catalog based shell utilizes the new API macro and
the API Schema as well as rustyline instead of the old GNU readline C API.
The code now has these 3 main components:
* The `Shell` which handles the readline loop via rustyline.
* The shell functions defined via the API macro.
* The `Context` which holds catalog and decoder instances.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:02:44 +01:00
9b78352188
src/backup/prune.rs: define new struct PruneOptions
2019-12-06 08:56:27 +01:00
92acbd69f7
src/backup/backup_info.rs: remove old prune code
2019-12-06 08:29:27 +01:00
408434360b
src/backup/prune.rs: add PruneMark::KeepPartial
...
Use separate mark for partial backup - dont mess them up with regular ones.
2019-12-06 08:15:32 +01:00
a8c8366cfa
src/backup/prune.rs: factor out code into remove_incomplete_snapshots()
2019-12-06 08:12:08 +01:00
dc18849156
src/backup/prune.rs: moved prune related code into extra file
2019-12-06 08:06:21 +01:00
6f47dd8a0f
src/backup/backup_info.rs - prune: improve algorythm
2019-12-05 18:41:28 +01:00
8f0b4c1f90
src/api2/admin/datastore.rs - prune: log info about all snapshots
2019-12-05 15:35:11 +01:00
9ce42759ec
src/backup/backup_info.rs - compute_prune_list: remove unfinished backups
2019-12-05 11:18:10 +01:00
2c034f8d0a
src/backup/backup_info.rs: improve prune algorithm
2019-12-05 08:55:19 +01:00
aeeac29bb1
src/backup/backup_info.rs: new compute_prune_list helper
...
So that we can write regression tests for this.
2019-12-04 15:49:11 +01:00
3d229a4a01
Revert "test another api macro use"
...
This reverts commit a9aa52e6a8
2019-12-04 14:18:11 +01:00
4f1e40a29e
src/api2/admin/datastore.rs - read_backup_index: add manifest with size
...
So that the client can display the size of the index file.
2019-12-04 10:03:52 +01:00
a9aa52e6a8
test another api macro use
...
The api macro now supports hyphens in parameter names and
referencing externally defined `Schema`s, so here's an
example.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-03 13:41:40 +01:00
de0aa53174
pxar: decoder: fix incorrect read for hardlinks.
...
Decoders read must check if the file is a hardlink and read data from the
corresponding offset if so.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-03 12:08:23 +01:00
b7e18004a2
pxar: decoder: return original name in attributes()
...
The returned filename should be the one of the file given at the offset, not of
the one the hardlink points to.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-03 12:08:07 +01:00
7eea56ca00
use cli from proxmox::api
2019-12-02 11:56:29 +01:00
524b22df35
src/cli/completion.rs - get_help_completion: fix for "help -" expansion
2019-12-02 11:11:31 +01:00
e8e9bae4d1
src/bin/completion.rs: add echo command
2019-12-02 11:04:40 +01:00
19d7a59236
src/bin/completion.rs: use api macros
2019-12-02 10:57:19 +01:00
f9a51322cb
src/cli/command.rs: use insert_help()
2019-12-02 08:52:08 +01:00
e3e72546dd
src/cli/command.rs: cleanup, remove unused top_def parameter
2019-12-02 08:47:03 +01:00
2348e77edf
src/cli/completion.rs - get_simple_completion: remove all_arg_param parameters
...
Seems useless.
2019-12-02 08:31:03 +01:00
28c855c0a2
src/cli: improve docs
2019-12-01 16:41:49 +01:00
3bf920527c
src/cli/command.rs: make set_help_context private - no need to expose
2019-12-01 12:54:41 +01:00
bca6d9120a
src/cli/command.rs: improve help command description
2019-12-01 12:40:37 +01:00
2f6ecc1ccd
src/cli/completion.rs: moved all completion related code here
2019-12-01 11:44:31 +01:00
a810e05288
src/cli/readline.rs: moved readline related code here
2019-12-01 11:31:35 +01:00
5d64a0d083
src/cli/shellword.rs: moved shellword related code into this file
2019-12-01 11:17:57 +01:00
fb8a00bc00
src/cli/command.rs - get_nested_completion: allow fuzzy match
2019-12-01 11:06:54 +01:00
9498fcf9c5
src/cli/command.rs - get_nested_completion: return nothing if first argument does not match
2019-12-01 10:24:07 +01:00
309e449db2
src/cli/command.rs: improve get_help_completion
2019-12-01 10:18:47 +01:00
b8a82922c7
src/cli/command.rs: handle multilevel sub commands in help
2019-12-01 09:42:51 +01:00
b7bf71dfa1
src/cli/command.rs: fix help completions
2019-11-30 15:43:44 +01:00
7afc1af7ab
src/bin/completion.rs: add history support
2019-11-30 15:08:37 +01:00
4b8573da65
src/cli/command.rs - handle_command: return Result instead of exit()
2019-11-30 15:08:00 +01:00
2b691daf6f
src/cli.rs: add insert_help() method
2019-11-30 14:56:31 +01:00
1201abcffa
src/cli/command.rs: implement help
2019-11-30 13:02:00 +01:00
8423c1fe64
cli/completion: use match statements and .contains()
...
`match` is a bit more readable than the if-else chains,
also replace
space_chars.iter().any(|s| c == *s)
with
space_chars.contains(&c)
which is also more readable.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 13:37:34 +01:00
e399398444
remove shellwords crate
2019-11-29 12:49:44 +01:00
ca60ac13d9
use rustyline to replace readline
2019-11-29 12:49:44 +01:00
41874331ed
whitespace fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:52:15 +01:00
2905f2b5e6
update api macro example usage
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:51:27 +01:00
8b130e71da
src/cli/command.rs: cleanup, make handlers more generic
2019-11-27 14:44:21 +01:00
7b6c41078b
update api macro invocation to new style
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 14:19:46 +01:00
314bb35868
catalog_shell: fix several clippy warnings
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-27 10:13:50 +01:00
6486cb853f
first api macro usage test/example
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 10:05:37 +01:00
951cf17ee3
catalog_shell: major refactoring of cli command definition and parsing
...
By changing the way shell commands are defined and parsed, this makes it more
straight forward to extend the current functionality.
The readline input is parsed based on the provided command definition and the
given parameters and options are passed to a command specific callback function.
In addition, the provided command definition including its description is used
to generate a help string to display.
The help command shows a list of all supported commands or the help string for
the provided command.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 13:01:00 +01:00
59bc6ad676
catalog_shell: check and return on empty list in order to avoid division by zero
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:33 +01:00
4e56b2f792
catalog_shell: do not restore full archive if no entries are selected for restore-selected
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:18 +01:00
12b4098660
src/cli/format.rs: move doc generation code into extra file
2019-11-26 12:12:53 +01:00
2f2b87e659
src/pxar/fuse.rs: add impl Context
2019-11-26 11:16:46 +01:00
eedd1f95eb
src/pxar/fuse.rs: move init and destroy into Session
2019-11-26 11:08:55 +01:00
970687c982
src/pxar/fuse.rs: cleanup - remove setup_session (moved into new)
2019-11-26 11:04:46 +01:00
2a11191072
src/pxar/fuse.rs: rename from_decoder into new, new into from_path
2019-11-26 10:56:41 +01:00
99b5b6cba9
src/pxar/decoder.rs: use trait object as reader
...
So that the Decoder is no longer genertic.
2019-11-26 10:45:11 +01:00
f701d0335e
src/pxar/sequential_decoder.rs: remove callback from new()
...
And use an extra functzion set_callback() to configure that.
Also rewrite pxar/fuse.rs and implement a generic Session (will get
further cleanups with next patches).
2019-11-26 09:56:48 +01:00
49fddd985c
src/cli/command.rs: use static array for arg_param
2019-11-25 11:16:41 +01:00
329d40b50b
rename ApiHandler::Async into ApiHandler::AsyncHttp
2019-11-23 09:03:21 +01:00
70fbac84da
src/server/rest.rs: simplify code
2019-11-22 18:44:14 +01:00
2bbd835b9b
src/server/rest.rs: rename get_request_parameters_async to get_request_parameters
2019-11-22 17:24:16 +01:00
75a5a68944
src/server/rest.rs - only pass ObjectSchema to get_request_parameters_async()
2019-11-22 17:22:07 +01:00
ad51d02aa9
src/server/rest.rs: cleanup async code
2019-11-22 13:02:05 +01:00
be2bb37205
remove api_schema submodule
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-22 09:31:45 +01:00
e57e1cd82d
move src/api_schema/config.rs -> src/server/config.rs
2019-11-22 09:23:03 +01:00
8256b0e417
src/api_schema/registry.rs: remove dead code
2019-11-22 09:04:52 +01:00
3cf73c4e53
src/bin/proxmox-backup-client.rs: impl shell subcommand for client
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 08:22:50 +01:00
f14c96ea38
src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:54:04 +01:00
38446a9551
src/backup/readline.rs: impl wrapper for GNU readline
...
In order to provide the context needed for tab completion via the readline
callback, the needed mut ref is passed via a thread local storage key.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:46:24 +01:00
6dba015043
src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute
...
This is needed in order to explicitly clone the values when needed in the
catalog shell implementation.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:15:39 +01:00
c9f002213e
src/backup/catalog.rs: impl DirEntry::is_directory()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:14:00 +01:00
d3bcc7f261
src/pxar/sequential_decoder.rs: create all directories in target path
...
Create the full target path and not fail if an intermediate directory does not
exist.
This is needed in order to restore multiple archives via the catalog, where the
target should further contain each archive name as subdir.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:11:49 +01:00
b6ca690095
src/pxar/decoder.rs: return also payload size in lookup
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:05:14 +01:00
45cc157f7c
docs: add backup reader protocol
2019-11-22 06:59:37 +01:00
cad540e969
api/compat: remove remaining api_schema references
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:53:15 +01:00
a2479cfa1a
api/compat: drop more compat imports from api_schema.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:36:28 +01:00
cc4ecf80b0
api/compat: drop api_handler submodule
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:18:41 +01:00
4f1963d259
api/comapt: drop schema submodule
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:17:36 +01:00
2194824f77
api: BoxFut -> ApiFuture
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:16:37 +01:00
3d482025b3
move api schema into proxmox::api crate
...
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:14:54 +01:00
552c225948
sort all property lookup tables
...
Required, because we use binary sreach to find items.
2019-11-21 13:32:09 +01:00
255f378a1b
use const api definitions
2019-11-21 13:32:09 +01:00
c5946faffd
drop md5 crate dependency
...
we already depend on openssl which also provides md5
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-14 11:19:43 +01:00
f569acc5e2
drop uuid crate dependency
...
proxmox::tools now has a Uuid module using the native
libuuid.
Adds build dependency: libuuid1 (which is a Pre-Depends of
util-linux, so always installed anyway).
Drops uuid + 16 more crate dependencies.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-14 10:22:29 +01:00
b423958d8e
catalog: fix issue with DirEntry start
...
start has to be the parent start - offset
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-12 17:18:13 +01:00
780dd2b0a1
src/backup/catalog.rs: add some docs
2019-11-12 13:23:40 +01:00
6aa906b547
src/backup/catalog.rs - CatalogReader: verify magic number
2019-11-12 12:54:06 +01:00
2ec208aef5
src/backup/catalog.rs - CatalogReader::dump(): use root()
2019-11-12 12:47:21 +01:00
8f24a9ea18
src/backup/catalog.rs - Direntry::parse: abort if callback return false
2019-11-12 11:41:44 +01:00
7d017123fd
src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code
2019-11-12 11:41:44 +01:00
dc9596de45
src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs
2019-11-12 11:41:44 +01:00
55c0b3cc7c
src/backup/catalog.rs - DirEntry: factor out name attribute
...
And avoid allocations inside parse().
2019-11-12 11:41:44 +01:00
3f1c5b5e65
catalog: impl std::fmt::Display trait for CatalogEntryType
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-12 07:00:08 +01:00
f89359c2f3
src/backup/catalog.rs: clenup - use single impl block
2019-11-10 11:38:31 +01:00
e8d8fdc61d
src/backup/catalog.rs: remove unused code
2019-11-10 11:37:06 +01:00
c74c074bfd
src/backup/catalog.rs: write MAGIC at start of file
2019-11-10 11:36:45 +01:00
265664517a
src/backup/catalog.rs: new parse() helper
2019-11-09 14:37:33 +01:00
5d92935e03
src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block
2019-11-09 13:21:20 +01:00
12a1975af9
src/backup/catalog.rs: store u64 compressed
...
This way catalogs get about 1/3 smaller.
2019-11-09 13:07:47 +01:00
bb19af7302
src/client/backup_reader.rs: factor out download_blob() helper
2019-11-08 13:16:41 +01:00
7205050059
src/client/backup_reader.rs: factor out download_fixed_index() helper
2019-11-08 12:51:52 +01:00
c3d84a2281
src/client/backup_reader.rs: factor out download_dynamic_index() helper
2019-11-08 12:51:52 +01:00
89245fb530
renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs
2019-11-08 10:41:00 +01:00
bf6e321744
change catalog format, use dynamic index to store catalog.
...
In order to remove size restriction of a single blob.
2019-11-08 10:35:48 +01:00
e016f9ff2f
src/client/backup_writer.rs: rename force_close to cancel
...
And do not consume self.
2019-11-08 10:34:16 +01:00
0997967d65
do no restrict index names too much
2019-11-08 10:28:14 +01:00
e66621182b
src/pxar/decoder.rs: impl lookup for `Decoder`
...
Allows to lookup an entry in a directory based on the provided `DirectoryEntry`.
This is needed to navigate the filesystem based on `DirectoryEntry`s and similar
to the find_goodbye_entry() function in src/pxar/fuse.rs
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-06 10:00:41 +01:00
fb2554de29
src/pxar/decoder.rs: fix wrong filename check for Decoder::restore()
...
As it turns out the original implementation was correct and the start in
`DirectoryEntry` points to the `PxarEntry` and not as wrongly stated to the
filename.
This reverts the incorrect code and adds comments to the fields clarifying this.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-06 09:59:35 +01:00
33ad183a40
src/pxar/decoder.rs: allow to pass match pattern to restore
...
In order to partially restore the archive starting from the directory provided as
parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-05 14:19:09 +01:00
5e8d600c71
src/pxar/decoder.rs: fix issue with restore
...
`Decoder::restore()` calls the `SequentialDecoder::restore()` which expects to
encounter a `PxarEntry` at first. But the start of `DirectoryEntry` points to the
filename (except for the root dir), so skip over it.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-05 14:18:29 +01:00
9708b2d98c
src/pxar/fuse.rs: refactor Iterator: use find() instead of position()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-04 13:59:16 +01:00
b82a101c86
catalog: store current writer position in CatalogBlobWriter
...
Counts the bytes written by the CatalogBlobWriter in order to obtain the
stream position, needed to get offset to reference catalog items.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-31 09:11:51 +01:00
62ee2eb405
avoid some clippy warnings
2019-10-26 11:42:05 +02:00
834a2f95a0
avoid some clippy warnings
2019-10-25 18:44:51 +02:00
f58f426e86
src/pxar/binary_search_tree.rs: allow(clippy::many_single_char_names)
2019-10-25 18:22:19 +02:00
11377a47bb
avoid some clippy warnings
2019-10-25 18:04:37 +02:00
9fe2f639b8
allow(clippy::cast_ptr_alignment)
2019-10-25 18:03:55 +02:00
cd69d36b45
src/backup/dynamic_index.rs: use write_all()
2019-10-25 17:22:41 +02:00
8abd12f1f3
pxar: decoder: cleanup un-needed Decoder::get_dir()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 08:02:23 +02:00
9c25c0330a
pxar: fuse: optimize fuse readdir callback
...
The expensive call to Decoder::read_directory_entry() can be omitted as
Decoder::attributes() returns all the information the fuse response needs.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 08:02:03 +02:00
ffe171c5db
src/pxar/encoder.rs: avoid duplicate check for MAX_DIRECTORY_ENTRIES
...
Simply by reordering code.
2019-10-23 07:55:56 +02:00
62d123e50a
pxar: add functionality to pass exclude MatchPatterns on create
...
This exposes the option to pass a list of exclude MatchPattern via the
'--exclude' option.
The list is encoded as file '.pxarexclude-cli' in the archives root directory.
If such a file is present in the filesystem, it is skipped and not included in
the archive in order to avoid conflicting information.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:31:49 +02:00
ba8165c607
pxar: encoder: add encode_pxar_exclude_cli() function.
...
This provides the functionality needed to encode MatchPatterns passed on the cli
in the root directory.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:30:30 +02:00
920243b12d
pxar: simplify MatchPattern::to_bytes
2019-10-23 07:29:02 +02:00
e50a90e010
pxar: Implement to_bytes() for MatchPattern in order to write them to file.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:21:11 +02:00
a8f10f849e
src/bin/proxmox-backup-client.rs: fix parameters for get_encryption_key_password
2019-10-17 10:28:21 +02:00
40c9ff9d81
src/pxar/encoder.rs: include .pxarexclude files into the catalog
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-17 10:25:20 +02:00
20feec9fac
pxar: fuse: include nsec information in stat
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-17 10:24:13 +02:00
649c5ee7b1
load_and_decrtypt_key: allow function closures to readf passphrase
2019-10-16 13:23:52 +02:00
60ffa9649c
src/backup/manifest.rs: fix property names (use '-' instead of '_')
2019-10-14 12:28:27 +02:00
4c9f753c56
src/backup/data_blob.rs: define create_zero_chunk as member function
...
and rename it to build_zero_chunk()
2019-10-14 10:58:26 +02:00
2745151d93
src/backup/data_blob.rs: add create_zero_chunk() helper
2019-10-14 10:44:46 +02:00
f06b820ac0
src/backup/manifest.rs: add verify_file
2019-10-13 10:09:12 +02:00
511a47bd73
cleanup
2019-10-13 09:39:21 +02:00
e17d5d864d
src/backup/manifest.rs: improve parser (try_from)
2019-10-13 08:39:49 +02:00
59e9ba01c6
src/backup/manifest.rs: new class to generate/parse index.json
2019-10-12 17:58:08 +02:00
ad6e5a6f51
rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME
2019-10-12 15:53:27 +02:00
296c50ba3d
src/client/backup_reader.rs: add download_manifest
...
Also add crypt_config as member variable.
2019-10-12 15:50:26 +02:00
cf9271e27c
src/client/backup_writer.rs: move BackupClient code into extra file
...
And rename it to BackupWriter.
2019-10-12 13:53:11 +02:00
913acb41fd
src/client/backup_reader.rs: add documentation
2019-10-12 13:26:47 +02:00
9e490a7479
src/client/backup_reader.rs: split BackupReader code into separate file
2019-10-12 12:57:08 +02:00
fd04ca7a5a
src/bin/proxmox-backup-client.rs: pass verbose flag to dump_image
...
And print some useful status to stderr.
2019-10-09 12:04:02 +02:00
2c2097ee75
fix typo in error message
2019-10-07 12:48:56 +02:00
f7aa6f15d3
src/api2/reader.rs: use tokio::fs::read() to read file async
2019-10-07 12:24:06 +02:00
7123ff7d43
src/backup/data_blob.rs: avoid Arc<CryptConfig>
...
use simple reference instead.
2019-10-07 11:41:35 +02:00
4ee8f53d07
remove DataChunk file format - use DataBlob instead
2019-10-06 10:31:06 +02:00
c26d4b4573
cleanup: remove dead code
2019-10-05 16:10:41 +02:00
88892ea801
src/bin/proxmox-backup-client.rs: cleanup restore
...
- write directly to /dev/stdout (instead of using stdout line writer)
- avoid BufferedFixedReader
2019-10-05 11:52:12 +02:00
62e05bb3b3
src/client/remote_chunk_reader.rs: pass reference to download_chunk
...
Make sure we do not move data around.
2019-10-05 11:45:20 +02:00
Christian Ebner
Dietmar Maurer
Wolfgang Bumiller