tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,100 Commits 2 Branches 54 Tags 14 MiB
c2f9149461
Commit Graph

416 Commits

Author SHA1 Message Date
Dietmar Maurer
55c3cb69cd rename catalog_shell_api() into catalog_shell_cli() 2019-12-06 09:42:05 +01:00
Dietmar Maurer
ecbaa38fa4 src/backup/catalog_shell.rs: factor out catalog_shell_api() 
In order to extract documentation ...
 2019-12-06 09:35:47 +01:00
Christian Ebner
cca67dc0c3 src/backup/readline.rs: removed in favor of new readline impl 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 09:03:04 +01:00
Christian Ebner
6934c6fe77 src/backup/catalog_shell.rs: adapt to use API Schema definition and rustyline 
This major refactoring of the catalog based shell utilizes the new API macro and
the API Schema as well as rustyline instead of the old GNU readline C API.

The code now has these 3 main components:
 * The `Shell` which handles the readline loop via rustyline.
 * The shell functions defined via the API macro.
 * The `Context` which holds catalog and decoder instances.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 09:02:44 +01:00
Dietmar Maurer
9b78352188 src/backup/prune.rs: define new struct PruneOptions 2019-12-06 08:56:27 +01:00
Dietmar Maurer
92acbd69f7 src/backup/backup_info.rs: remove old prune code 2019-12-06 08:29:27 +01:00
Dietmar Maurer
408434360b src/backup/prune.rs: add PruneMark::KeepPartial 
Use separate mark for partial backup - dont mess them up with regular ones.
 2019-12-06 08:15:32 +01:00
Dietmar Maurer
a8c8366cfa src/backup/prune.rs: factor out code into remove_incomplete_snapshots() 2019-12-06 08:12:08 +01:00
Dietmar Maurer
dc18849156 src/backup/prune.rs: moved prune related code into extra file 2019-12-06 08:06:21 +01:00
Dietmar Maurer
6f47dd8a0f src/backup/backup_info.rs - prune: improve algorythm 2019-12-05 18:41:28 +01:00
Dietmar Maurer
8f0b4c1f90 src/api2/admin/datastore.rs - prune: log info about all snapshots 2019-12-05 15:35:11 +01:00
Dietmar Maurer
9ce42759ec src/backup/backup_info.rs - compute_prune_list: remove unfinished backups 2019-12-05 11:18:10 +01:00
Dietmar Maurer
2c034f8d0a src/backup/backup_info.rs: improve prune algorithm 2019-12-05 08:55:19 +01:00
Dietmar Maurer
aeeac29bb1 src/backup/backup_info.rs: new compute_prune_list helper 
So that we can write regression tests for this.
 2019-12-04 15:49:11 +01:00
Christian Ebner
314bb35868 catalog_shell: fix several clippy warnings 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-27 10:13:50 +01:00
Christian Ebner
951cf17ee3 catalog_shell: major refactoring of cli command definition and parsing 
By changing the way shell commands are defined and parsed, this makes it more
straight forward to extend the current functionality.
The readline input is parsed based on the provided command definition and the
given parameters and options are passed to a command specific callback function.
In addition, the provided command definition including its description is used
to generate a help string to display.
The help command shows a list of all supported commands or the help string for
the provided command.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-26 13:01:00 +01:00
Christian Ebner
59bc6ad676 catalog_shell: check and return on empty list in order to avoid division by zero 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-26 12:55:33 +01:00
Christian Ebner
4e56b2f792 catalog_shell: do not restore full archive if no entries are selected for restore-selected 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-26 12:55:18 +01:00
Dietmar Maurer
99b5b6cba9 src/pxar/decoder.rs: use trait object as reader 
So that the Decoder is no longer genertic.
 2019-11-26 10:45:11 +01:00
Dietmar Maurer
f701d0335e src/pxar/sequential_decoder.rs: remove callback from new() 
And use an extra functzion  set_callback() to configure that.

Also rewrite pxar/fuse.rs and implement a generic Session (will get
further cleanups with next patches).
 2019-11-26 09:56:48 +01:00
Christian Ebner
f14c96ea38 src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:54:04 +01:00
Christian Ebner
38446a9551 src/backup/readline.rs: impl wrapper for GNU readline 
In order to provide the context needed for tab completion via the readline
callback, the needed mut ref is passed via a thread local storage key.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:46:24 +01:00
Christian Ebner
6dba015043 src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute 
This is needed in order to explicitly clone the values when needed in the
catalog shell implementation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:15:39 +01:00
Christian Ebner
c9f002213e src/backup/catalog.rs: impl DirEntry::is_directory() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:14:00 +01:00
Wolfgang Bumiller
f569acc5e2 drop uuid crate dependency 
proxmox::tools now has a Uuid module using the native
libuuid.

Adds build dependency: libuuid1 (which is a Pre-Depends of
util-linux, so always installed anyway).

Drops uuid + 16 more crate dependencies.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-14 10:22:29 +01:00
Christian Ebner
b423958d8e catalog: fix issue with DirEntry start 
start has to be the parent start - offset

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-12 17:18:13 +01:00
Dietmar Maurer
780dd2b0a1 src/backup/catalog.rs: add some docs 2019-11-12 13:23:40 +01:00
Dietmar Maurer
6aa906b547 src/backup/catalog.rs - CatalogReader: verify magic number 2019-11-12 12:54:06 +01:00
Dietmar Maurer
2ec208aef5 src/backup/catalog.rs - CatalogReader::dump(): use root() 2019-11-12 12:47:21 +01:00
Dietmar Maurer
8f24a9ea18 src/backup/catalog.rs - Direntry::parse: abort if callback return false 2019-11-12 11:41:44 +01:00
Dietmar Maurer
7d017123fd src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code 2019-11-12 11:41:44 +01:00
Dietmar Maurer
dc9596de45 src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs 2019-11-12 11:41:44 +01:00
Dietmar Maurer
55c0b3cc7c src/backup/catalog.rs - DirEntry: factor out name attribute 
And avoid allocations inside parse().
 2019-11-12 11:41:44 +01:00
Christian Ebner
3f1c5b5e65 catalog: impl std::fmt::Display trait for CatalogEntryType 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-12 07:00:08 +01:00
Dietmar Maurer
f89359c2f3 src/backup/catalog.rs: clenup - use single impl block 2019-11-10 11:38:31 +01:00
Dietmar Maurer
e8d8fdc61d src/backup/catalog.rs: remove unused code 2019-11-10 11:37:06 +01:00
Dietmar Maurer
c74c074bfd src/backup/catalog.rs: write MAGIC at start of file 2019-11-10 11:36:45 +01:00
Dietmar Maurer
265664517a src/backup/catalog.rs: new parse() helper 2019-11-09 14:37:33 +01:00
Dietmar Maurer
5d92935e03 src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block 2019-11-09 13:21:20 +01:00
Dietmar Maurer
12a1975af9 src/backup/catalog.rs: store u64 compressed 
This way catalogs get about 1/3 smaller.
 2019-11-09 13:07:47 +01:00
Dietmar Maurer
89245fb530 renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs 2019-11-08 10:41:00 +01:00
Dietmar Maurer
bf6e321744 change catalog format, use dynamic index to store catalog. 
In order to remove size restriction of a single blob.
 2019-11-08 10:35:48 +01:00
Christian Ebner
b82a101c86 catalog: store current writer position in CatalogBlobWriter 
Counts the bytes written by the CatalogBlobWriter in order to obtain the
stream position, needed to get offset to reference catalog items.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-31 09:11:51 +01:00
Dietmar Maurer
62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer
834a2f95a0 avoid some clippy warnings 2019-10-25 18:44:51 +02:00
Dietmar Maurer
9fe2f639b8 allow(clippy::cast_ptr_alignment) 2019-10-25 18:03:55 +02:00
Dietmar Maurer
cd69d36b45 src/backup/dynamic_index.rs: use write_all() 2019-10-25 17:22:41 +02:00
Dietmar Maurer
649c5ee7b1 load_and_decrtypt_key: allow function closures to readf passphrase 2019-10-16 13:23:52 +02:00
Dietmar Maurer
60ffa9649c src/backup/manifest.rs: fix property names (use '-' instead of '_') 2019-10-14 12:28:27 +02:00
Dietmar Maurer
4c9f753c56 src/backup/data_blob.rs: define create_zero_chunk as member function 
and rename it to build_zero_chunk()
 2019-10-14 10:58:26 +02:00
Dietmar Maurer
2745151d93 src/backup/data_blob.rs: add create_zero_chunk() helper 2019-10-14 10:44:46 +02:00
Dietmar Maurer
f06b820ac0 src/backup/manifest.rs: add verify_file 2019-10-13 10:09:12 +02:00
Dietmar Maurer
511a47bd73 cleanup 2019-10-13 09:39:21 +02:00
Dietmar Maurer
e17d5d864d src/backup/manifest.rs: improve parser (try_from) 2019-10-13 08:39:49 +02:00
Dietmar Maurer
59e9ba01c6 src/backup/manifest.rs: new class to generate/parse index.json 2019-10-12 17:58:08 +02:00
Dietmar Maurer
7123ff7d43 src/backup/data_blob.rs: avoid Arc<CryptConfig> 
use simple reference instead.
 2019-10-07 11:41:35 +02:00
Dietmar Maurer
4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
Dietmar Maurer
9e603e254c src/backup/fixed_index.rs: do not use offset for checksum 2019-09-23 11:30:20 +02:00
Dietmar Maurer
2aba16bde1 src/backup/data_chunk.rs: add into_raw() 2019-09-19 14:24:17 +02:00
Wolfgang Bumiller
ba5e67475a clippy: fix access to unaligned buffer in dynamic reader 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:50:45 +02:00
Wolfgang Bumiller
b526bd1408 backup: cleanup unused imports 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:46:46 +02:00
Wolfgang Bumiller
4dc79bb1d9 clippy: fix access to unaligned buffer 
via the newly introduced ReadExt::read_host_value_boxed()
from proxmox::tools::io.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:45:44 +02:00
Wolfgang Bumiller
ccdf3ad1ec backup: use static assertion in fixed index reader 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:43:25 +02:00
Wolfgang Bumiller
b21851cc5c clippy: collapse identical if branches 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 13:57:58 +02:00
Wolfgang Bumiller
653b1ca10e some clippy fixups 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 12:14:05 +02:00
Dietmar Maurer
9d456cf8ec src/backup/catalog_blob.rs: use write_all() instead of write() 
To correctly handle interrupted system calls.
 2019-09-05 11:40:48 +02:00
Dietmar Maurer
0a51fe0011 src/backup/fixed_index.rs: new helper to compute checksum and file size 2019-09-03 13:11:45 +02:00
Dietmar Maurer
82c85a21a1 src/backup/dynamic_index.rs: new helper to compute checksum and file size 2019-09-03 13:11:09 +02:00
Wolfgang Bumiller
745e652a7f src/backup/index.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
32bef1e2d1 src/backup/chunk_stream.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
5c1130df9f implement Sync for DynamicIndexReader and FixedIndexReader 
hyper's wrap_stream now needs this

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
d21f8a5b5b backup/dynamic_index: explicitly zero-initialize the header 
The writer.write_all() call accessed data marked as
undefined to valgrind. Note that we shouldn't write out
uninitialized memory for security reasons anyway.
(note that vec::undefined already did zero-initialize the
data, but also marked it as undefined for valgrind when
compiling with the valgrind feature)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 11:42:35 +02:00
Wolfgang Bumiller
990b930f22 backup/dynamic_index: use static assertion, fix size 
The type was sized properly but the number was still wrong,
fixed this.
TODO! Once unions with non-Copy values are stable make this
a `union { full: [u8; 4096], data: TheActualHeader }`;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 11:42:35 +02:00
Wolfgang Bumiller
7d83440c60 remove proxmox-protocol subcrate 
AFAICT we have no use for it anymore, its api entry points
are gone. If we do end up needing something from it, it's
still in the git history anyway. (And about two thirds of it
can be made much less awkward by utilizing async-await
anyway, so no love lost there...)

Moved the chunker back into src/backup/chunker.rs

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 14:08:25 +02:00
Wolfgang Bumiller
5e58e1bb7d {dynamic,fixed}_index: replace mem::uninitialized 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:35:36 +02:00
Wolfgang Bumiller
f93b55b057 src/backup/index: style fixup & unsafe copy removal 
We can use the safe .copy_from_slice alternative in this
case.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:31:45 +02:00
Wolfgang Bumiller
fa17b1ce2a src/backup/index: replace mem::uninitialized 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:27:35 +02:00
Wolfgang Bumiller
367f002ee2 use statement cleanup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-16 12:28:40 +02:00
Dietmar Maurer
9d135fe617 src/backup/catalog_blob.rs: moved catalog impl. from pxar 
And avoid loading catalog into memory.
 2019-08-16 12:27:17 +02:00
Wolfgang Bumiller
9025312aa6 avoid lifetimes in blob reader/writer 2019-08-16 10:07:24 +02:00
Dietmar Maurer
71d08e00b7 src/backup/data_blob_reader.rs: fix compiler warning 2019-08-16 08:10:39 +02:00
Dietmar Maurer
b791804f4b src/backup/data_blob_reader.rs: impl for Read instead of BufRead 2019-08-14 14:56:01 +02:00
Dietmar Maurer
39a4df61d6 improve docs 2019-08-14 14:08:27 +02:00
Dietmar Maurer
018d11bb84 src/backup/data_blob.rs: move parts into single files 2019-08-14 13:24:41 +02:00
Dietmar Maurer
e27c5a559f src/backup/data_blob.rs: avoid excessive stack usage 2019-08-14 12:44:08 +02:00
Dietmar Maurer
c638542b4b src/backup/file_formats.rs: moved header_size 2019-08-14 12:35:53 +02:00
Dietmar Maurer
6082216343 src/backup/data_blob.rs - CryptReader/CryptWriter: fix buffer size handling 2019-08-13 15:23:15 +02:00
Dietmar Maurer
548c9489d7 src/backup/data_blob.rs - DataBlobReader: impl compressed, encrypted blobs 2019-08-13 13:17:08 +02:00
Dietmar Maurer
2aa0bfff59 src/backup/data_blob.rs - DataBlobReader: implement reader for encrtypted blobs 2019-08-13 13:17:08 +02:00
Dietmar Maurer
e9a385a78e src/backup/data_blob.rs - DataBlobReader: impl compressed, signed blobs 2019-08-12 17:47:25 +02:00
Dietmar Maurer
4bfa147eaf src/backup/data_blob.rs - DataBlobReader: impl. reader for signed blobs 2019-08-12 17:41:25 +02:00
Dietmar Maurer
09785b2795 src/backup/data_blob.rs - DataBlobReader: start serious impl. 2019-08-12 17:20:41 +02:00
Dietmar Maurer
a32bd8a516 src/backup/data_blob.rs - DataBlobWriter: fix decryption 2019-08-12 11:57:29 +02:00
Dietmar Maurer
5d15cb49b4 src/backup/data_blob.rs - compute_crc: start after blob header 2019-08-12 11:32:26 +02:00
Dietmar Maurer
1cbdfd19a2 src/backup/file_formats.rs; use const instzead of static 2019-08-12 11:20:21 +02:00
Dietmar Maurer
18be4ec24a src/backup/data_blob.rs - DataBlobWriter: fix magic number for uncompressed blobs 2019-08-12 10:46:05 +02:00
Dietmar Maurer
5622a3fcdb src/backup/data_blob.rs - DataBlobWriter: impl. compressed encrypted blobs 2019-08-12 10:16:42 +02:00
Dietmar Maurer
f4942e9ffd src/backup/data_blob.rs - DataBlobWriter: impl. encrypted blobs 2019-08-12 10:08:00 +02:00
Dietmar Maurer
c57ec43a53 src/backup/crypt_config.rs: new helper data_crypter to get openssl Crypter 2019-08-12 10:06:51 +02:00
Dietmar Maurer
7776becf5d src/backup/data_blob.rs - DataBlobWriter: impl. compressed signed blobs 2019-08-12 09:01:08 +02:00