7b6c41078b
update api macro invocation to new style
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 14:19:46 +01:00
314bb35868
catalog_shell: fix several clippy warnings
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-27 10:13:50 +01:00
6486cb853f
first api macro usage test/example
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 10:05:37 +01:00
951cf17ee3
catalog_shell: major refactoring of cli command definition and parsing
...
By changing the way shell commands are defined and parsed, this makes it more
straight forward to extend the current functionality.
The readline input is parsed based on the provided command definition and the
given parameters and options are passed to a command specific callback function.
In addition, the provided command definition including its description is used
to generate a help string to display.
The help command shows a list of all supported commands or the help string for
the provided command.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 13:01:00 +01:00
59bc6ad676
catalog_shell: check and return on empty list in order to avoid division by zero
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:33 +01:00
4e56b2f792
catalog_shell: do not restore full archive if no entries are selected for restore-selected
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:18 +01:00
12b4098660
src/cli/format.rs: move doc generation code into extra file
2019-11-26 12:12:53 +01:00
2f2b87e659
src/pxar/fuse.rs: add impl Context
2019-11-26 11:16:46 +01:00
eedd1f95eb
src/pxar/fuse.rs: move init and destroy into Session
2019-11-26 11:08:55 +01:00
970687c982
src/pxar/fuse.rs: cleanup - remove setup_session (moved into new)
2019-11-26 11:04:46 +01:00
2a11191072
src/pxar/fuse.rs: rename from_decoder into new, new into from_path
2019-11-26 10:56:41 +01:00
99b5b6cba9
src/pxar/decoder.rs: use trait object as reader
...
So that the Decoder is no longer genertic.
2019-11-26 10:45:11 +01:00
f701d0335e
src/pxar/sequential_decoder.rs: remove callback from new()
...
And use an extra functzion set_callback() to configure that.
Also rewrite pxar/fuse.rs and implement a generic Session (will get
further cleanups with next patches).
2019-11-26 09:56:48 +01:00
49fddd985c
src/cli/command.rs: use static array for arg_param
2019-11-25 11:16:41 +01:00
329d40b50b
rename ApiHandler::Async into ApiHandler::AsyncHttp
2019-11-23 09:03:21 +01:00
70fbac84da
src/server/rest.rs: simplify code
2019-11-22 18:44:14 +01:00
2bbd835b9b
src/server/rest.rs: rename get_request_parameters_async to get_request_parameters
2019-11-22 17:24:16 +01:00
75a5a68944
src/server/rest.rs - only pass ObjectSchema to get_request_parameters_async()
2019-11-22 17:22:07 +01:00
ad51d02aa9
src/server/rest.rs: cleanup async code
2019-11-22 13:02:05 +01:00
be2bb37205
remove api_schema submodule
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-22 09:31:45 +01:00
e57e1cd82d
move src/api_schema/config.rs -> src/server/config.rs
2019-11-22 09:23:03 +01:00
8256b0e417
src/api_schema/registry.rs: remove dead code
2019-11-22 09:04:52 +01:00
3cf73c4e53
src/bin/proxmox-backup-client.rs: impl shell subcommand for client
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 08:22:50 +01:00
f14c96ea38
src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:54:04 +01:00
38446a9551
src/backup/readline.rs: impl wrapper for GNU readline
...
In order to provide the context needed for tab completion via the readline
callback, the needed mut ref is passed via a thread local storage key.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:46:24 +01:00
6dba015043
src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute
...
This is needed in order to explicitly clone the values when needed in the
catalog shell implementation.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:15:39 +01:00
c9f002213e
src/backup/catalog.rs: impl DirEntry::is_directory()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:14:00 +01:00
d3bcc7f261
src/pxar/sequential_decoder.rs: create all directories in target path
...
Create the full target path and not fail if an intermediate directory does not
exist.
This is needed in order to restore multiple archives via the catalog, where the
target should further contain each archive name as subdir.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:11:49 +01:00
b6ca690095
src/pxar/decoder.rs: return also payload size in lookup
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:05:14 +01:00
45cc157f7c
docs: add backup reader protocol
2019-11-22 06:59:37 +01:00
cad540e969
api/compat: remove remaining api_schema references
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:53:15 +01:00
a2479cfa1a
api/compat: drop more compat imports from api_schema.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:36:28 +01:00
cc4ecf80b0
api/compat: drop api_handler submodule
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:18:41 +01:00
4f1963d259
api/comapt: drop schema submodule
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:17:36 +01:00
2194824f77
api: BoxFut -> ApiFuture
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:16:37 +01:00
3d482025b3
move api schema into proxmox::api crate
...
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:14:54 +01:00
552c225948
sort all property lookup tables
...
Required, because we use binary sreach to find items.
2019-11-21 13:32:09 +01:00
255f378a1b
use const api definitions
2019-11-21 13:32:09 +01:00
c5946faffd
drop md5 crate dependency
...
we already depend on openssl which also provides md5
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-14 11:19:43 +01:00
f569acc5e2
drop uuid crate dependency
...
proxmox::tools now has a Uuid module using the native
libuuid.
Adds build dependency: libuuid1 (which is a Pre-Depends of
util-linux, so always installed anyway).
Drops uuid + 16 more crate dependencies.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-14 10:22:29 +01:00
b423958d8e
catalog: fix issue with DirEntry start
...
start has to be the parent start - offset
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-12 17:18:13 +01:00
780dd2b0a1
src/backup/catalog.rs: add some docs
2019-11-12 13:23:40 +01:00
6aa906b547
src/backup/catalog.rs - CatalogReader: verify magic number
2019-11-12 12:54:06 +01:00
2ec208aef5
src/backup/catalog.rs - CatalogReader::dump(): use root()
2019-11-12 12:47:21 +01:00
8f24a9ea18
src/backup/catalog.rs - Direntry::parse: abort if callback return false
2019-11-12 11:41:44 +01:00
7d017123fd
src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code
2019-11-12 11:41:44 +01:00
dc9596de45
src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs
2019-11-12 11:41:44 +01:00
55c0b3cc7c
src/backup/catalog.rs - DirEntry: factor out name attribute
...
And avoid allocations inside parse().
2019-11-12 11:41:44 +01:00
3f1c5b5e65
catalog: impl std::fmt::Display trait for CatalogEntryType
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-12 07:00:08 +01:00
f89359c2f3
src/backup/catalog.rs: clenup - use single impl block
2019-11-10 11:38:31 +01:00
e8d8fdc61d
src/backup/catalog.rs: remove unused code
2019-11-10 11:37:06 +01:00
c74c074bfd
src/backup/catalog.rs: write MAGIC at start of file
2019-11-10 11:36:45 +01:00
265664517a
src/backup/catalog.rs: new parse() helper
2019-11-09 14:37:33 +01:00
5d92935e03
src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block
2019-11-09 13:21:20 +01:00
12a1975af9
src/backup/catalog.rs: store u64 compressed
...
This way catalogs get about 1/3 smaller.
2019-11-09 13:07:47 +01:00
bb19af7302
src/client/backup_reader.rs: factor out download_blob() helper
2019-11-08 13:16:41 +01:00
7205050059
src/client/backup_reader.rs: factor out download_fixed_index() helper
2019-11-08 12:51:52 +01:00
c3d84a2281
src/client/backup_reader.rs: factor out download_dynamic_index() helper
2019-11-08 12:51:52 +01:00
89245fb530
renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs
2019-11-08 10:41:00 +01:00
bf6e321744
change catalog format, use dynamic index to store catalog.
...
In order to remove size restriction of a single blob.
2019-11-08 10:35:48 +01:00
e016f9ff2f
src/client/backup_writer.rs: rename force_close to cancel
...
And do not consume self.
2019-11-08 10:34:16 +01:00
0997967d65
do no restrict index names too much
2019-11-08 10:28:14 +01:00
e66621182b
src/pxar/decoder.rs: impl lookup for `Decoder`
...
Allows to lookup an entry in a directory based on the provided `DirectoryEntry`.
This is needed to navigate the filesystem based on `DirectoryEntry`s and similar
to the find_goodbye_entry() function in src/pxar/fuse.rs
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-06 10:00:41 +01:00
fb2554de29
src/pxar/decoder.rs: fix wrong filename check for Decoder::restore()
...
As it turns out the original implementation was correct and the start in
`DirectoryEntry` points to the `PxarEntry` and not as wrongly stated to the
filename.
This reverts the incorrect code and adds comments to the fields clarifying this.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-06 09:59:35 +01:00
33ad183a40
src/pxar/decoder.rs: allow to pass match pattern to restore
...
In order to partially restore the archive starting from the directory provided as
parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-05 14:19:09 +01:00
5e8d600c71
src/pxar/decoder.rs: fix issue with restore
...
`Decoder::restore()` calls the `SequentialDecoder::restore()` which expects to
encounter a `PxarEntry` at first. But the start of `DirectoryEntry` points to the
filename (except for the root dir), so skip over it.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-05 14:18:29 +01:00
9708b2d98c
src/pxar/fuse.rs: refactor Iterator: use find() instead of position()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-04 13:59:16 +01:00
b82a101c86
catalog: store current writer position in CatalogBlobWriter
...
Counts the bytes written by the CatalogBlobWriter in order to obtain the
stream position, needed to get offset to reference catalog items.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-31 09:11:51 +01:00
62ee2eb405
avoid some clippy warnings
2019-10-26 11:42:05 +02:00
834a2f95a0
avoid some clippy warnings
2019-10-25 18:44:51 +02:00
f58f426e86
src/pxar/binary_search_tree.rs: allow(clippy::many_single_char_names)
2019-10-25 18:22:19 +02:00
11377a47bb
avoid some clippy warnings
2019-10-25 18:04:37 +02:00
9fe2f639b8
allow(clippy::cast_ptr_alignment)
2019-10-25 18:03:55 +02:00
cd69d36b45
src/backup/dynamic_index.rs: use write_all()
2019-10-25 17:22:41 +02:00
8abd12f1f3
pxar: decoder: cleanup un-needed Decoder::get_dir()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 08:02:23 +02:00
9c25c0330a
pxar: fuse: optimize fuse readdir callback
...
The expensive call to Decoder::read_directory_entry() can be omitted as
Decoder::attributes() returns all the information the fuse response needs.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 08:02:03 +02:00
ffe171c5db
src/pxar/encoder.rs: avoid duplicate check for MAX_DIRECTORY_ENTRIES
...
Simply by reordering code.
2019-10-23 07:55:56 +02:00
62d123e50a
pxar: add functionality to pass exclude MatchPatterns on create
...
This exposes the option to pass a list of exclude MatchPattern via the
'--exclude' option.
The list is encoded as file '.pxarexclude-cli' in the archives root directory.
If such a file is present in the filesystem, it is skipped and not included in
the archive in order to avoid conflicting information.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:31:49 +02:00
ba8165c607
pxar: encoder: add encode_pxar_exclude_cli() function.
...
This provides the functionality needed to encode MatchPatterns passed on the cli
in the root directory.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:30:30 +02:00
920243b12d
pxar: simplify MatchPattern::to_bytes
2019-10-23 07:29:02 +02:00
e50a90e010
pxar: Implement to_bytes() for MatchPattern in order to write them to file.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:21:11 +02:00
a8f10f849e
src/bin/proxmox-backup-client.rs: fix parameters for get_encryption_key_password
2019-10-17 10:28:21 +02:00
40c9ff9d81
src/pxar/encoder.rs: include .pxarexclude files into the catalog
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-17 10:25:20 +02:00
20feec9fac
pxar: fuse: include nsec information in stat
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-17 10:24:13 +02:00
649c5ee7b1
load_and_decrtypt_key: allow function closures to readf passphrase
2019-10-16 13:23:52 +02:00
60ffa9649c
src/backup/manifest.rs: fix property names (use '-' instead of '_')
2019-10-14 12:28:27 +02:00
4c9f753c56
src/backup/data_blob.rs: define create_zero_chunk as member function
...
and rename it to build_zero_chunk()
2019-10-14 10:58:26 +02:00
2745151d93
src/backup/data_blob.rs: add create_zero_chunk() helper
2019-10-14 10:44:46 +02:00
f06b820ac0
src/backup/manifest.rs: add verify_file
2019-10-13 10:09:12 +02:00
511a47bd73
cleanup
2019-10-13 09:39:21 +02:00
e17d5d864d
src/backup/manifest.rs: improve parser (try_from)
2019-10-13 08:39:49 +02:00
59e9ba01c6
src/backup/manifest.rs: new class to generate/parse index.json
2019-10-12 17:58:08 +02:00
ad6e5a6f51
rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME
2019-10-12 15:53:27 +02:00
296c50ba3d
src/client/backup_reader.rs: add download_manifest
...
Also add crypt_config as member variable.
2019-10-12 15:50:26 +02:00
cf9271e27c
src/client/backup_writer.rs: move BackupClient code into extra file
...
And rename it to BackupWriter.
2019-10-12 13:53:11 +02:00
913acb41fd
src/client/backup_reader.rs: add documentation
2019-10-12 13:26:47 +02:00
9e490a7479
src/client/backup_reader.rs: split BackupReader code into separate file
2019-10-12 12:57:08 +02:00
fd04ca7a5a
src/bin/proxmox-backup-client.rs: pass verbose flag to dump_image
...
And print some useful status to stderr.
2019-10-09 12:04:02 +02:00
2c2097ee75
fix typo in error message
2019-10-07 12:48:56 +02:00
f7aa6f15d3
src/api2/reader.rs: use tokio::fs::read() to read file async
2019-10-07 12:24:06 +02:00
7123ff7d43
src/backup/data_blob.rs: avoid Arc<CryptConfig>
...
use simple reference instead.
2019-10-07 11:41:35 +02:00
4ee8f53d07
remove DataChunk file format - use DataBlob instead
2019-10-06 10:31:06 +02:00
c26d4b4573
cleanup: remove dead code
2019-10-05 16:10:41 +02:00
88892ea801
src/bin/proxmox-backup-client.rs: cleanup restore
...
- write directly to /dev/stdout (instead of using stdout line writer)
- avoid BufferedFixedReader
2019-10-05 11:52:12 +02:00
62e05bb3b3
src/client/remote_chunk_reader.rs: pass reference to download_chunk
...
Make sure we do not move data around.
2019-10-05 11:45:20 +02:00
c0b1b14c05
src/api2/reader.rs - download_chunk: use blocking io
...
Turns out to be twice as fast as tokio async io...
2019-10-05 11:41:19 +02:00
ff01c1e393
src/client/http_client.rs: store raw data/blob size in index.json
2019-10-02 10:54:06 +02:00
f16aea687a
src/client/http_client.rs: new helper send_upload_request()
2019-09-28 18:22:48 +02:00
9e603e254c
src/backup/fixed_index.rs: do not use offset for checksum
2019-09-23 11:30:20 +02:00
fb6026b66b
backup writer api: verify checksum inside close
2019-09-23 10:56:53 +02:00
c8c4051aa7
src/client/http_client.rs - H2Client::upload: fix parameter order
2019-09-20 13:19:55 +02:00
f011dba0f7
src/client/http_client.rs: add method parameter to H2 upload
2019-09-20 12:40:23 +02:00
792a70b9e1
src/client/http_client.rs: add content type parameter to H2 upload/requets_builder
2019-09-20 12:26:21 +02:00
a55b2975db
src/client/http_client.rs: cleanup (rename parameter)
2019-09-20 11:57:03 +02:00
32efac1cce
remove trailing whitespace
2019-09-20 08:12:31 +02:00
70235f72f4
src/bin/proxmox-backup-client.rs: add functionality to mount pxar archive.
...
This adds the basic functionality to mount an archive stored in a repository
locally via fuse.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-20 08:01:01 +02:00
2fa91f5221
src/bin/pxar.rs: adapt to new deamonize flag
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-20 08:00:20 +02:00
d388e244f5
src/pxar/fuse.rs: make verbose flag independent from deamonize by adding this as new flag
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-20 08:00:04 +02:00
e717e9a3bf
src/pxar/fuse.rs: refactor Session::new()
...
and add Session::from_decoder() in order to be able to create a fuse session
with a `Decoder` given as argument.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-20 07:59:51 +02:00
946b72a6b2
src/pxar/fuse.rs: introduce `Context` in order to add caching.
...
This patch introduces `Context` to hold the decoder, ino_offset and caches for
the attributes and the goodbye table.
By caching, certain callbacks can be handled without the need to read additional
data via the decoder, which improves performance.
The searching of the goodbye table is refactored as well, avoiding recursive
function calls in case of a hash collision.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-20 07:56:40 +02:00
1c93182371
src/pxar/decoder.rs: fix bug in decoder read
...
`offset` points to the `PXAR_FILENAME`, therefore read the filename before
the `PXAR_ENTRY`.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-20 07:54:16 +02:00
2aba16bde1
src/backup/data_chunk.rs: add into_raw()
2019-09-19 14:24:17 +02:00
15bb1bba89
BackupClient: add upload() function
2019-09-19 13:59:24 +02:00
6d4df36c3c
make download_chunk_list() public
2019-09-19 13:17:23 +02:00
cc2ce4a92f
src/client/http_client.rs: add password parameter to HttpClient::new()
2019-09-16 12:35:23 +02:00
0e20b336e1
pxar: fuse: avoid possible hash collision in lookup by additional checking against filename
...
The hash of the filename in the goodbye table items allows to quickly compare to
a hashed filename.
Unfortunately, a matching hash is no garantee for matching filenames as hash
collisions are possible.
This patch fixes such possible collisions by further checking the filenames once
a matching hash has been found.
This introduces no significant extra cost (except for the filename comparison)
for cases with matching hashes, as the lookup call has to seek and read the file
attributes (including the filename) anyway.
In cases with hash collision, the next matching item is read and treaded
analogously (what means we need at least one extra seek).
As collisions should be not that frequent, this should be an acceptable penalty.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-16 10:22:22 +02:00
ba5e67475a
clippy: fix access to unaligned buffer in dynamic reader
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:50:45 +02:00
b526bd1408
backup: cleanup unused imports
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:46:46 +02:00
4dc79bb1d9
clippy: fix access to unaligned buffer
...
via the newly introduced ReadExt::read_host_value_boxed()
from proxmox::tools::io.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:45:44 +02:00
ccdf3ad1ec
backup: use static assertion in fixed index reader
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:43:25 +02:00
34a3845b01
clippy: don't use clone a double reference
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:02:30 +02:00
b21851cc5c
clippy: collapse identical if branches
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 13:57:58 +02:00
299a2f18b2
fixup variable naming
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 13:57:23 +02:00
175eeb870d
clippy: use write_all in file logger
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 13:56:09 +02:00
5a54935bc9
clippy: fix pointer to temporary value
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 13:55:56 +02:00
bd430c225b
clippy: remove a loop{} which never actually loops
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:31:39 +02:00
44fed91e17
more clippy fixups
...
mostly indentation changes, view with `-w`
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:23:12 +02:00
653b1ca10e
some clippy fixups
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:14:05 +02:00
03310bea12
src/pxar/fuse.rs: s/Buf/ReplyBuf/ and s/BufState/ReplyBufState/
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-11 11:27:18 +02:00
48cc1b8234
src/pxar/fuse.rs: refactor stat and fix i-node mapping
...
The functionality of stat is split into smaller sub-functions, which allows
to reuse them more flexible, as the code flow is similar but not always the same.
By this, the ugly and incorrect re-setting of the i-node in the lookup callback
function is avoided.
The correct i-node is now calculated beforehand and stat simply creates a
`libc::stat` struct from the provided parameters.
Also, this fixes incorrect i-node assignments in the readdir callback function.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-11 11:26:55 +02:00
22eaa905a4
src/pxar/fuse.rs: impl readdir callback for fuse
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 11:38:58 +02:00
1652192f4d
src/pxar/fuse.rs: impl readlink callback for fuse
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 11:38:48 +02:00
3710b23db5
src/pxar/decoder.rs: read_link has to read filename before entry.
...
The offset points to the start of the item, which is the filename
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 11:36:04 +02:00
7eb9848ba7
src/pxar/fuse.rs: impl opendir callback for fuse
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 11:35:13 +02:00
2c5fd378d3
src/pxar/fuse.rs: impl read callback for fuse
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 11:34:55 +02:00
97e4db9dd5
src/pxar/fuse.rs: impl open callback for fuse
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 11:34:18 +02:00
b00689254a
src/pxar/fuse.rs: impl lookup callback for fuse
...
The lookup call checks if the given filename is found in the directory referenced
by the i-node by calclulating the filenames hash and looking it up in the
directories goodbye table.
If found, the entries parameters are returned.
In order to be able to lookup the parent offset by a given file offset in the
readdir callback, this also stores the corresponding values in a HashMap.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 10:54:23 +02:00
1443927c02
src/pxar/decoder.rs: correctly handle hardlinks in `Decoder`s attribute function
...
If the attributes are then returned by seeking to the new offset.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 10:46:39 +02:00
d00097a0e6
src/pxar/decoder.rs: split functionality of list_dir into list_dir and goodbye_table
...
In order to read the contents of the goodbye table while keeping the
functionality of list_dir in place as is.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 10:46:29 +02:00
ac12570e99
src/pxar/decoder.rs: make read_directory_entry usable by fuse
...
... and thereby allow it to read a single directory entry based on the
start and end archive offsets.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 10:32:48 +02:00
3f7f4f34be
src/pxar/decoder.rs: remove `DirectoryEntry` start_offset and end_offset functions
...
... as they are not needed with the latest iteration of the fuse callback
function impl (which never made it into the repos).
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 10:30:45 +02:00
ac30bc3326
src/pxar/decoder.rs: correctly read, check and skip filename
...
The previous implementation simply skipped over `size` bytes, which is not
correct as size includes also the header.
By relying on `SequentailDecoder`s read_filename function, this is correctly
taken care of plus some more integrity checks.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-10 10:30:21 +02:00
4a7de56e2f
avoid injecting ENV vars from Makefile
...
So that we can run "cargo build" without setting vars manually.
2019-09-09 10:51:08 +02:00
d21ae955a6
src/pxar/fuse.rs: impl getattr callback for fuse
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-06 12:24:14 +02:00
ec04ea81f2
src/pxar/fuse.rs: Refactor run_in_context and remove inode_to_offset and offset_to_inode
...
as their functionality is used only once so it makes more sense to keep them
inline.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-06 12:22:53 +02:00
8413083fc5
src/pxar/fuse.rs: Improve comments, add rustfmt skip tag and reafctor
...
Also, removes an unused println statement in the decoder callback function and
fixes a typo.
Further, use ABI compatible Option<&T> for FFI to avoid use of raw pointers.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-06 12:22:17 +02:00
54a5a885a3
src/client/http_client.rs: use async for upload_speedtest()
2019-09-06 08:55:47 +02:00
bbd055bf45
src/pxar/decoder.rs: impl functionality needed for fuse implementation
...
Implements functions attributes, open, read, read_link and get_dir
to be used by the fuse implementation which uses file offsets within the archive
as inodes to reference the archives items.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-06 07:18:14 +02:00
132cb0d0db
src/pxar/decoder.rs: Add comment for unsolved issue and refactor if statement.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-06 07:16:31 +02:00
2a05048b93
src/client/http_client.rs: simplify send_request
2019-09-05 15:07:37 +02:00
9edd3bf1b8
src/client/http_client.rs: use async for h2api_response()
2019-09-05 14:56:52 +02:00
c18fddf80f
src/client/http_client.rs - download_chunk_list: simplify async code
2019-09-05 14:16:45 +02:00
44f59dc731
src/client/http_client.rs - H2Client: fix error message in download()
2019-09-05 13:26:06 +02:00
c2a5a9f353
src/client/http_client.rs: use async for download_chunk_list
2019-09-05 13:12:03 +02:00
2f831baec0
src/client/http_client.rs - BackupClient: use async
2019-09-05 12:55:22 +02:00
2a1e6d7dea
src/client/http_client.rs - BackupClient: use async
2019-09-05 11:42:12 +02:00
d4a085e564
src/client/http_client.rs - BackupReader: use async
2019-09-05 11:42:05 +02:00
9d456cf8ec
src/backup/catalog_blob.rs: use write_all() instead of write()
...
To correctly handle interrupted system calls.
2019-09-05 11:40:48 +02:00
1a7a0e74c0
src/client/http_client.rs: use async for more functions
2019-09-04 13:48:16 +02:00
96f5e80abb
src/client/http_client.rs: use async for login()
2019-09-04 10:01:46 +02:00
9d35dbbb8f
src/client/http_client.rs: use async for credentials
2019-09-04 09:57:29 +02:00
d2267b112d
src/bin/proxmox-backup-client.rs: verify blob/catlog checksums
2019-09-04 08:47:14 +02:00
70b03a07c8
src/pxar/sequential_decoder.rs: make functions needed in non-sequential decoder accessible from within crate
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-04 06:15:05 +02:00
f92e8266ae
src/pxar/format_definitions.rs: move header type in PxarHeader and hash in PxarGoodbyeItem to top of struct
...
By this it is possible to read and check just the first u64 of the corresponding
structs in order to identify the items.
This is needed for the fuse implementation in order to get entries based on the
archive offset, used as inode.
Directories are referenced by the offset to the goodbye tail while other items
are referenced by the offset of the filename followed by the entry.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-04 06:14:30 +02:00
75933d1e83
src/pxar/fuse.rs: add function to map offset to inode and vice versa
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-04 06:13:59 +02:00
c7fee396c6
src/pxar/decoder.rs: add getter functions to obtain start/end offsets for directories
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-04 06:10:56 +02:00
3626ac611f
src/pxar/decoder.rs: rustfmt fixups
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-04 06:09:17 +02:00
3fa71727ee
src/bin/proxmox-backup-client.rs: refactoring of match statement
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-09-04 06:08:00 +02:00
a40220c056
src/bin/proxmox-backup-client.rs: code cleanup - factor out some functions
2019-09-03 13:31:13 +02:00
36493d4d7e
src/backup.rs: define INDEX_BLOB_NAME here
2019-09-03 13:15:44 +02:00
df65bd3da6
src/bin/proxmox-backup-client.rs - restore: verify checksums
2019-09-03 13:12:16 +02:00
0a51fe0011
src/backup/fixed_index.rs: new helper to compute checksum and file size
2019-09-03 13:11:45 +02:00
82c85a21a1
src/backup/dynamic_index.rs: new helper to compute checksum and file size
2019-09-03 13:11:09 +02:00
a3e032b754
src/client/http_client.rs: fix dynamic index checksum
2019-09-03 13:10:18 +02:00
c77563146b
comment fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-03 11:17:33 +02:00
daef93f481
tools: add tokio::main() replacement
...
to deal with block_on() not allowing blocking()
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-03 11:16:30 +02:00
66fbf5bad0
add WrappedReaderStream test
...
Mostly to just document the rt.block_on() issue with
blocking().
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-03 10:05:45 +02:00
2b92971fba
merge fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:22:36 +02:00
083ff3fd5d
update to tokio 0.2.0-alpha.4
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
1434f4f8df
drop hyper_openssl
...
We can use .await now, which means the whole connection
state machine doesn't need to be typed out as "types"
anymore, so, at least until hyper_openssl gets updated to
proper dependencies, let's drop it.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
556eb70ea3
add generic either-or AsyncRead/Write type
...
The HttpsConnector will use this. Instead of implementing a
specialized MaybeTlsStream, this is simply a generic "either
this or that kind of Async Read/Write type".
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
b30415d244
src/server/state.rs: update to tokio alpha.2
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
aadcf7c621
src/server/rest.rs: use tokio::timer::delay
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
0cdb8d9c5b
src/tools/wrapped_reader_stream.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
e668912a99
src/tools/futures.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
0f5856acca
src/tools/daemon.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
56e351c911
src/tools/broadcast_future.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
627bb7d114
src/tools/async_mutex.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
75fef4b463
src/server/worker_task.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
aa4110cc64
src/server/state.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
91e4587343
src/server/rest.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
b9b7f7ec1f
src/server/h2service.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
160fc8147f
src/server/command_socket.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
3c0facc787
src/client/remote_chunk_reader.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
369a87e3a2
src/client/pxar_backup_stream.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
2107bb40c1
src/client/pipe_to_stream.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
5b3911995b
src/client/merge_known_chunks.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
a6782ca10b
src/client/http_client.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
8630557918
src/bin/upload-speed.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
cab6816969
src/bin/test_chunk_speed2.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
fda5797b8a
src/bin/proxmox-backup-proxy.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
e9722f8bde
src/bin/proxmox-backup-client.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
e76ac3a499
src/bin/proxmox-backup-api.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
e235c8f719
src/bin/h2s-server.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
74be6dc9b7
src/bin/h2server.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
55d8a631fc
src/bin/h2s-client.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
15d0e4a3bd
src/bin/h2client.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
b9203d87f4
src/bin/download-speed.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
745e652a7f
src/backup/index.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
32bef1e2d1
src/backup/chunk_stream.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
f2d9b73c91
src/api_schema/router.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
ffb6434485
src/api2/reader.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
7622005574
src/api2/backup/upload_chunk.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
59b2baa0f6
src/api2/backup.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
6be147b78c
src/api2/admin/datastore.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
5c1130df9f
implement Sync for DynamicIndexReader and FixedIndexReader
...
hyper's wrap_stream now needs this
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
236761a3e6
drop src/storage/futures.rs
...
it's unused and uses a tokio::Task I don't want to lookup
the std-future conversion for...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
02fcf372e4
rc/bin/proxmox-backup-client.rs - restore: always download index.json.blob
2019-09-02 14:14:32 +02:00
0d9862803b
src/bin/proxmox-backup-client.rs - reatore: avoid loading (large) blobs into memory
2019-09-02 14:13:31 +02:00
66c8eb9383
src/client/http_client.rs: Refactor handling Option and Result types
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-29 09:15:16 +02:00
40f8680e37
tools: remove SignalFd
...
We don't use it anymore and it'll just be more work when
switching to async-await.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 13:44:53 +02:00
d21f8a5b5b
backup/dynamic_index: explicitly zero-initialize the header
...
The writer.write_all() call accessed data marked as
undefined to valgrind. Note that we shouldn't write out
uninitialized memory for security reasons anyway.
(note that vec::undefined already did zero-initialize the
data, but also marked it as undefined for valgrind when
compiling with the valgrind feature)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 11:42:35 +02:00
990b930f22
backup/dynamic_index: use static assertion, fix size
...
The type was sized properly but the number was still wrong,
fixed this.
TODO! Once unions with non-Copy values are stable make this
a `union { full: [u8; 4096], data: TheActualHeader }`;
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 11:42:35 +02:00
bcb664cb69
pxar/fuse: add pxar cli flag to set single- or multi-threaded session loop
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:43:14 +02:00
de2791868a
src/pxar/sequentail_decoder.rs: followup: cleanup S_IFMT masking
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:42:22 +02:00
9cb9e1b7db
src/pxar/sequential_decoder.rs: style fixups
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:42:19 +02:00
dac88033ba
src/pxar/dir_stack.rs: style fixups
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:39:14 +02:00
b69169e7be
src/pxar/fuse.rs: style fixups
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:37:58 +02:00
d792dc3cd2
src/pxar/match_pattern.rs: style fixups
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:37:56 +02:00
4ea22b68d9
src/pxar/format_definition.rs: style fixup
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 09:37:53 +02:00
7d83440c60
remove proxmox-protocol subcrate
...
AFAICT we have no use for it anymore, its api entry points
are gone. If we do end up needing something from it, it's
still in the git history anyway. (And about two thirds of it
can be made much less awkward by utilizing async-await
anyway, so no love lost there...)
Moved the chunker back into src/backup/chunker.rs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 14:08:25 +02:00
7a57cb77e1
more formatting & use statement fixups
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 13:50:27 +02:00
35dfd3ceb7
src/pxar/encoder.rs: style fixups by rustfmt
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 13:29:11 +02:00
1af30bc2fe
(pxar: replace deprecated std::mem::uninitialized()
...
... and use std::mem::MaybeUninit or proxmox::tools::vec::uninitialized() instead.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 13:27:25 +02:00
5e58e1bb7d
{dynamic,fixed}_index: replace mem::uninitialized
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:35:36 +02:00
f93b55b057
src/backup/index: style fixup & unsafe copy removal
...
We can use the safe .copy_from_slice alternative in this
case.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:31:45 +02:00
fa17b1ce2a
src/backup/index: replace mem::uninitialized
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:27:35 +02:00
f3a8d1d7e0
api2/node/time: replace mem::uninitialized
...
and fixup use statements
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:14:41 +02:00
48b85e8e3b
tools: tty: replace mem::uninitialized and style fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:14:19 +02:00
b528fddfc6
tools: timer: style fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:07:56 +02:00
Wolfgang Bumiller
Christian Ebner
Dietmar Maurer