tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,950 Commits 2 Branches 54 Tags 14 MiB
cef03f4149
Commit Graph

2950 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Dietmar Maurer
fc14b84937 src/backup/fixed_index.rs: improve bound checks inside add_digest 2019-05-28 08:47:03 +02:00
Dietmar Maurer
e1225de467 src/backup/fixed_index.rs: store index_length to avoid code duplication 2019-05-28 08:27:57 +02:00
Dietmar Maurer
e3d525feb7 src/api2/admin/datastore/backup.rs: fix typo 2019-05-28 07:13:19 +02:00
Christian Ebner
af309d4d5c src/bin/pxar.rs: set archive permissions to 0o640 
As the archive can contain potentially sensitive data such as key files, it makes
sense to restrict the permissions.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-28 06:24:47 +02:00
Christian Ebner
ab87f167f1 src/pxar/encoder.rs: Refactor file stat 
Introduce helper functions to check file stats

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-28 06:23:49 +02:00
Christian Ebner
578b601159 src/pxar/encoder.rs: Don't bail if endpoint does not support xattrs. 
The encoder bailed if a endpoint which did not support xattrs was encountered.
Instead of bailing, we ignore these errors and simply do not store xattrs for
such endpoints.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-28 06:22:16 +02:00
Dietmar Maurer
a42fa400ee src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api 2019-05-28 06:18:55 +02:00
Dietmar Maurer
29ae5c86a2 src/backup/fixed_index.rs: make chunk_size public 2019-05-28 06:18:24 +02:00
Dietmar Maurer
7dd1bcac58 src/client/http_client.rs: correctly parse error response in download_chunk_list 2019-05-27 09:35:30 +02:00
Dietmar Maurer
a1e7cff3ea src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk 2019-05-27 07:57:43 +02:00
Dietmar Maurer
771953f9eb src/api2/admin/datastore/backup.rs: use smaller window sizes 2019-05-27 07:53:40 +02:00
Dietmar Maurer
174ad378d8 src/client/http_client.rs: fix request pipelining 2019-05-27 07:24:32 +02:00
Dietmar Maurer
624362226e src/client/merge_known_chunks.rs: use MergedChunkInfo as input 2019-05-26 10:52:56 +02:00
Dietmar Maurer
05cba08c9c src/client/http_client.rs: append chunks in correct order 
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
 2019-05-26 09:57:34 +02:00
Dietmar Maurer
417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset 
This clearly shows that the current approach does not work - seems we get
chunks out of order.
 2019-05-26 08:49:29 +02:00
Dietmar Maurer
34114e2606 src/server/rest.rs: correctly verify json parameters 2019-05-24 09:10:59 +02:00
Dietmar Maurer
eb2bdd1bcc src/client/http_client.rs: encode parameters inside body 
To avoid problems with long uris.
 2019-05-24 08:32:55 +02:00
Dietmar Maurer
9bb675ec23 src/client/merge_known_chunks.rs: reduce RAM usage 
There is no need to store chunk data for known chunks.
 2019-05-24 07:42:58 +02:00
Dietmar Maurer
ecb21b4794 src/client/merge_known_chunks.rs: avoid VecDequeu - a single buffer is enough 2019-05-24 07:36:09 +02:00
Christian Ebner
2dbba78b98 src/pxar/sequential_decoder.rs: impl support to dump/restore ACLs from pxar archives 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:27:24 +02:00
Christian Ebner
6a19448004 src/pxar/encoder.rs: impl support to store ACLs in pxar achives 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:21:33 +02:00
Christian Ebner
7cfaade7c8 src/pxar/format_definition.rs: define types and structs needed to impl ACL support in pxar 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:16:55 +02:00
Christian Ebner
b344461b33 pxar: pass feature_flags to encoder/decoder instead of individual flags 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00
Christian Ebner
687407741b src/tools/xattr.rs: add test for is_valid_xattr_name() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00
Christian Ebner
357e4614e2 Minor refactoring of pxars xattr encoder/decoder, mostly reformatting and renaming. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00
Wolfgang Bumiller
c05a8c8d18 doc-test fixup 
cargo test by default compiles and runs all code snippets
found in the documentation...

oops...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 15:22:33 +02:00
Wolfgang Bumiller
cc84a830c5 RFC: schema: make enums static 
I don't see a reason to allow these to be dynamically
modifiable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 12:38:51 +02:00
Dietmar Maurer
aa1b2e04fe src/client/merge_known_chunks.rs: merge known chunks 
To decrease the number of api calls required...
 2019-05-23 12:31:09 +02:00
Dietmar Maurer
91320f0879 src/client/http_client.rs: use ChunkInfo streams 
This will make out of order uploads possible...
 2019-05-23 12:31:09 +02:00
Wolfgang Bumiller
8ea3b1d188 tree-wide: use the new vec/io tools modules 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 09:49:10 +02:00
Wolfgang Bumiller
6f0069ae1e buildsys: make valgrind=1 to enable valgrind support 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 09:49:10 +02:00
Wolfgang Bumiller
9cdda3f7c7 tools: add helpful vector and read operations 
After importing the I/O ops trait via:
    use crate::tools::io::ops::*;

Instead of:
    let mut buffer = vec![0u8; 65536];
    file.read_exact(&mut buffer)?;
use:
    let buffer = file.read_exact_allocated(65536)?;

After importing the vector helpers via:
    use crate::tools::vec::{self, ops::*};

For a buffer which *could* be uninitialized but you prefer
zero-initialization anyway for security reasons, instead of:
    let mut buffer = vec![0u8; len];
use:
    let mut buffer = vec::undefined(len);
which zero-initializes, but, if the `valgrind` feature flag
is enabled, marks the vector as having undefined contents,
so reading from it will cause valgrind errors.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 09:49:10 +02:00
Dietmar Maurer
8bea85b42e src/api2/admin/datastore/backup.rs: verify file size and chunk count on close 2019-05-23 08:50:36 +02:00
Dietmar Maurer
1179e15842 src/server/rest.rs: improve error handling 2019-05-23 08:15:32 +02:00
Dietmar Maurer
907f5bb0d8 src/api2/admin/datastore/backup/service.rs: improve error logging 2019-05-23 08:05:39 +02:00
Dietmar Maurer
32e1bf4260 src/bin/upload-speed.rs: code cleanup 2019-05-23 07:20:05 +02:00
Christian Ebner
238dd1b7ab tests/catar.rs: Pass dummy xattr/fcaps feature_flags to encoder. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:41:03 +02:00
Christian Ebner
0d9bab05ab src/bin/pxar.rs: impl CLI flags to disable archiving/restoring of xattrs and fcaps. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:40:50 +02:00
Christian Ebner
4add706092 src/client/: pass dummy flags for xattrs and fcaps to pxar encoder/decoder. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:40:32 +02:00
Christian Ebner
54b14ad741 src/pxar/decoder.rs: pass flags for xattrs and fcaps to decoder. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:40:09 +02:00
Christian Ebner
1201360def src/pxar/sequential_decoder.rs: Conditional restore xattrs and fcaps if feature_flags is set 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:39:16 +02:00
Christian Ebner
8bc8791338 src/pxar/format_definitions.rs: do not store the feature_flags in pxar archive. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:37:37 +02:00
Christian Ebner
a45a123950 src/pxar/encoder: Use encoder feature_flags, do not store them within the archive. 
Instead of testing and storing the feature_flags in the pxar archive, set and use
the feature flags within the encoder.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:34:36 +02:00
Christian Ebner
fe61cc1099 tests/pxar.rs: Make sure the correct pxar binary is run in the tests. 
If compiled in debug mode, the debug binary should be tested, in release mode
the release binary has to be invoked.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:31:37 +02:00
Christian Ebner
ec56f07bec Makefile: make the tests folder available in the build folder on 'make deb' 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:31:06 +02:00
Dietmar Maurer
d6f204eda6 src/client/http_client.rs: re-order code - public functions first 2019-05-23 06:28:10 +02:00
Dietmar Maurer
9af37c8f0e src/client/http_client.rs: move low level H2 code into separate class 2019-05-22 17:28:25 +02:00
Dietmar Maurer
d2c48afc6e src/client/http_client.rs: code cleanup 2019-05-22 13:24:33 +02:00
Dietmar Maurer
6ab34afa88 src/client/http_client.rs: cleanup h2 backup client 2019-05-22 13:05:51 +02:00
Dietmar Maurer
a55fcd740f src/api2/admin/datastore/backup.rs: suse POST for finish() api 2019-05-22 13:04:58 +02:00