tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,038 Commits 2 Branches 54 Tags 14 MiB
624362226e
Commit Graph

931 Commits

Author SHA1 Message Date
Dietmar Maurer
624362226e src/client/merge_known_chunks.rs: use MergedChunkInfo as input 2019-05-26 10:52:56 +02:00
Dietmar Maurer
05cba08c9c src/client/http_client.rs: append chunks in correct order 
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
 2019-05-26 09:57:34 +02:00
Dietmar Maurer
417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset 
This clearly shows that the current approach does not work - seems we get
chunks out of order.
 2019-05-26 08:49:29 +02:00
Dietmar Maurer
34114e2606 src/server/rest.rs: correctly verify json parameters 2019-05-24 09:10:59 +02:00
Dietmar Maurer
eb2bdd1bcc src/client/http_client.rs: encode parameters inside body 
To avoid problems with long uris.
 2019-05-24 08:32:55 +02:00
Dietmar Maurer
9bb675ec23 src/client/merge_known_chunks.rs: reduce RAM usage 
There is no need to store chunk data for known chunks.
 2019-05-24 07:42:58 +02:00
Dietmar Maurer
ecb21b4794 src/client/merge_known_chunks.rs: avoid VecDequeu - a single buffer is enough 2019-05-24 07:36:09 +02:00
Christian Ebner
2dbba78b98 src/pxar/sequential_decoder.rs: impl support to dump/restore ACLs from pxar archives 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:27:24 +02:00
Christian Ebner
6a19448004 src/pxar/encoder.rs: impl support to store ACLs in pxar achives 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:21:33 +02:00
Christian Ebner
7cfaade7c8 src/pxar/format_definition.rs: define types and structs needed to impl ACL support in pxar 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:16:55 +02:00
Christian Ebner
b344461b33 pxar: pass feature_flags to encoder/decoder instead of individual flags 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00
Christian Ebner
687407741b src/tools/xattr.rs: add test for is_valid_xattr_name() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00
Christian Ebner
357e4614e2 Minor refactoring of pxars xattr encoder/decoder, mostly reformatting and renaming. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00
Wolfgang Bumiller
c05a8c8d18 doc-test fixup 
cargo test by default compiles and runs all code snippets
found in the documentation...

oops...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 15:22:33 +02:00
Wolfgang Bumiller
cc84a830c5 RFC: schema: make enums static 
I don't see a reason to allow these to be dynamically
modifiable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 12:38:51 +02:00
Dietmar Maurer
aa1b2e04fe src/client/merge_known_chunks.rs: merge known chunks 
To decrease the number of api calls required...
 2019-05-23 12:31:09 +02:00
Dietmar Maurer
91320f0879 src/client/http_client.rs: use ChunkInfo streams 
This will make out of order uploads possible...
 2019-05-23 12:31:09 +02:00
Wolfgang Bumiller
8ea3b1d188 tree-wide: use the new vec/io tools modules 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 09:49:10 +02:00
Wolfgang Bumiller
9cdda3f7c7 tools: add helpful vector and read operations 
After importing the I/O ops trait via:
    use crate::tools::io::ops::*;

Instead of:
    let mut buffer = vec![0u8; 65536];
    file.read_exact(&mut buffer)?;
use:
    let buffer = file.read_exact_allocated(65536)?;

After importing the vector helpers via:
    use crate::tools::vec::{self, ops::*};

For a buffer which *could* be uninitialized but you prefer
zero-initialization anyway for security reasons, instead of:
    let mut buffer = vec![0u8; len];
use:
    let mut buffer = vec::undefined(len);
which zero-initializes, but, if the `valgrind` feature flag
is enabled, marks the vector as having undefined contents,
so reading from it will cause valgrind errors.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 09:49:10 +02:00
Dietmar Maurer
8bea85b42e src/api2/admin/datastore/backup.rs: verify file size and chunk count on close 2019-05-23 08:50:36 +02:00
Dietmar Maurer
1179e15842 src/server/rest.rs: improve error handling 2019-05-23 08:15:32 +02:00
Dietmar Maurer
907f5bb0d8 src/api2/admin/datastore/backup/service.rs: improve error logging 2019-05-23 08:05:39 +02:00
Dietmar Maurer
32e1bf4260 src/bin/upload-speed.rs: code cleanup 2019-05-23 07:20:05 +02:00
Christian Ebner
0d9bab05ab src/bin/pxar.rs: impl CLI flags to disable archiving/restoring of xattrs and fcaps. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:40:50 +02:00
Christian Ebner
4add706092 src/client/: pass dummy flags for xattrs and fcaps to pxar encoder/decoder. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:40:32 +02:00
Christian Ebner
54b14ad741 src/pxar/decoder.rs: pass flags for xattrs and fcaps to decoder. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:40:09 +02:00
Christian Ebner
1201360def src/pxar/sequential_decoder.rs: Conditional restore xattrs and fcaps if feature_flags is set 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:39:16 +02:00
Christian Ebner
8bc8791338 src/pxar/format_definitions.rs: do not store the feature_flags in pxar archive. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:37:37 +02:00
Christian Ebner
a45a123950 src/pxar/encoder: Use encoder feature_flags, do not store them within the archive. 
Instead of testing and storing the feature_flags in the pxar archive, set and use
the feature flags within the encoder.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-23 06:34:36 +02:00
Dietmar Maurer
d6f204eda6 src/client/http_client.rs: re-order code - public functions first 2019-05-23 06:28:10 +02:00
Dietmar Maurer
9af37c8f0e src/client/http_client.rs: move low level H2 code into separate class 2019-05-22 17:28:25 +02:00
Dietmar Maurer
d2c48afc6e src/client/http_client.rs: code cleanup 2019-05-22 13:24:33 +02:00
Dietmar Maurer
6ab34afa88 src/client/http_client.rs: cleanup h2 backup client 2019-05-22 13:05:51 +02:00
Dietmar Maurer
a55fcd740f src/api2/admin/datastore/backup.rs: suse POST for finish() api 2019-05-22 13:04:58 +02:00
Dietmar Maurer
de6742c04f src/bin/h2test.rs: removed 
This test does no work anymore.
 2019-05-22 11:14:56 +02:00
Dietmar Maurer
5a0b521809 src/api2/admin/datastore/backup.rs: remove test callbacks 2019-05-22 10:55:27 +02:00
Christian Ebner
de61bc9264 src/tools/xattr.rs: impl unit tests for fsetattr and fgetattr 
As git does not preserve xattrs they have to be set before they can be read.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-22 10:01:10 +02:00
Dietmar Maurer
553610b43e src/client/http_client.rs: implement download_chunk_list 2019-05-22 09:46:02 +02:00
Dietmar Maurer
0cc0fffd1e src/backup/chunk_stream.rs: use BytesMut in ChunkStream 2019-05-22 09:39:02 +02:00
Dietmar Maurer
c052be5c86 src/backup/chunk_stream.rs: use more generics 2019-05-22 09:18:05 +02:00
Dietmar Maurer
169c0e060f src/backup/chunk_stream.rs: optimize FixedChunkStream (use BytesMut) 2019-05-22 09:05:35 +02:00
Dietmar Maurer
3be3f3dcc4 src/backup/chunk_stream.rs: fix behaviour for large input buffers 2019-05-22 08:05:27 +02:00
Dietmar Maurer
02fa54ff3c src/backup/chunk_stream.rs: assemble data with correct order 2019-05-22 07:44:51 +02:00
Dietmar Maurer
d0afeaa831 src/api2/admin/datastore/backup.rs: improve error message 2019-05-22 07:43:54 +02:00
Dietmar Maurer
11867a2b54 src/tools.rs: fix hex_to_digest 2019-05-21 13:42:13 +02:00
Dietmar Maurer
7f3d2ffab9 src/backup/index.rs: rename ChunkListReader into DigestListEncoder 2019-05-21 12:28:44 +02:00
Dietmar Maurer
ee53955f52 src/backup/index.rs: implement DigestListDecoder 2019-05-21 12:21:22 +02:00
Dietmar Maurer
a95849321c src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks 2019-05-21 09:54:11 +02:00
Dietmar Maurer
40f4e198a8 src/backup/dynamic_index.rs: add chunk_info method 2019-05-21 09:52:19 +02:00
Christian Ebner
596182056b src/pxar/sequential_decoder.rs: impl functionality to restore xattrs/fcaps from pxar achives 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-21 06:15:30 +02:00