Commit Graph

43 Commits

Author SHA1 Message Date
f569acc5e2 drop uuid crate dependency
proxmox::tools now has a Uuid module using the native
libuuid.

Adds build dependency: libuuid1 (which is a Pre-Depends of
util-linux, so always installed anyway).

Drops uuid + 16 more crate dependencies.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-14 10:22:29 +01:00
62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
9fe2f639b8 allow(clippy::cast_ptr_alignment) 2019-10-25 18:03:55 +02:00
cd69d36b45 src/backup/dynamic_index.rs: use write_all() 2019-10-25 17:22:41 +02:00
4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
ba5e67475a clippy: fix access to unaligned buffer in dynamic reader
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:50:45 +02:00
653b1ca10e some clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:14:05 +02:00
82c85a21a1 src/backup/dynamic_index.rs: new helper to compute checksum and file size 2019-09-03 13:11:09 +02:00
5c1130df9f implement Sync for DynamicIndexReader and FixedIndexReader
hyper's wrap_stream now needs this

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
d21f8a5b5b backup/dynamic_index: explicitly zero-initialize the header
The writer.write_all() call accessed data marked as
undefined to valgrind. Note that we shouldn't write out
uninitialized memory for security reasons anyway.
(note that vec::undefined already did zero-initialize the
data, but also marked it as undefined for valgrind when
compiling with the valgrind feature)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 11:42:35 +02:00
990b930f22 backup/dynamic_index: use static assertion, fix size
The type was sized properly but the number was still wrong,
fixed this.
TODO! Once unions with non-Copy values are stable make this
a `union { full: [u8; 4096], data: TheActualHeader }`;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 11:42:35 +02:00
7d83440c60 remove proxmox-protocol subcrate
AFAICT we have no use for it anymore, its api entry points
are gone. If we do end up needing something from it, it's
still in the git history anyway. (And about two thirds of it
can be made much less awkward by utilizing async-await
anyway, so no love lost there...)

Moved the chunker back into src/backup/chunker.rs

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 14:08:25 +02:00
5e58e1bb7d {dynamic,fixed}_index: replace mem::uninitialized
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:35:36 +02:00
367f002ee2 use statement cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-16 12:28:40 +02:00
0f0a35b390 src/backup/dynamic_index.rs: lock file inside new, code cleanup 2019-07-04 08:17:30 +02:00
a660978c9a src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats 2019-07-04 07:57:43 +02:00
877dd11133 src/backup/{fixed, dynamic}_index.rs: improve error messages 2019-07-02 08:56:56 +02:00
b850673634 src/backup/read_chunk.rs: move read chunk trait into extra file
And implement LocalChunkReader.
2019-07-02 08:22:29 +02:00
f35197f449 replace tools::vec with proxmox::tools::vec
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 11:03:25 +02:00
9b2b627fe0 update remaining users of tools::io::ops
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:44:03 +02:00
d48a9955a5 src/backup/dynamic_index.rs: introduce ReadChunk trait 2019-06-28 16:35:00 +02:00
a7dd483097 src/backup/*_index.rs: used generated magic numbers 2019-06-14 14:58:37 +02:00
bffd40d6b7 src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools 2019-06-14 11:40:04 +02:00
16ff6b7cd8 src/backup/dynamic_index.rs: compute checksum over the index 2019-06-14 10:33:59 +02:00
f98ac774ee backup: Add support for client side encryption
first try ...
2019-06-13 11:47:23 +02:00
dd5495d6dc tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
976595e1a9 src/backup/dynamic_index.rs: split class DynamicIndexWriter 2019-05-29 08:49:57 +02:00
8ea3b1d188 tree-wide: use the new vec/io tools modules
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 09:49:10 +02:00
40f4e198a8 src/backup/dynamic_index.rs: add chunk_info method 2019-05-21 09:52:19 +02:00
92da93b245 abort GC on server shutdown 2019-04-01 12:13:02 +02:00
c597a92c81 index file readers: aquire shared lock 2019-03-27 10:11:14 +01:00
43b1303398 datastore: use new ProcessLocker
To make sure only one process runs garbage collection while having active writers.
2019-03-22 09:42:15 +01:00
8968258b66 rename catar into pxar
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
06178f13fa move chunker into proxmox_protocol
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-08 09:29:47 +01:00
7bc1d72778 add IndexFile trait
We want to be able to iterate through digests of index
files, but without always having to distinguish between
dynamic and fixed types, so add a trait we can use as a
trait object.

Unfortunately the iterator needs to yield copies as
iterators cannot yield values with life times when
represented as trait objects (Box<dyn Iterator<Item = ?>>)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
5be4065baf make FixedIndexReader Send and update comments
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
fded74d0c7 style: BufferedReader -> BufferedRead
Single-method traits usually use the same name as their
method and aren't usually the 'noun' (which is usually an
implementation of them instead).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
f661f37427 backup/dynamic_index: split out add_chunk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:50:45 +01:00
7e3365554e src/backup/chunk_stat.rs: new struct to track chunk statistics 2019-02-25 12:52:10 +01:00
798f7fa065 src/backup/chunk_store.rs: return info about compressed chunk size
So that we can generate better statistics ..
2019-02-25 11:36:05 +01:00
247cdbce72 src/api2/admin/datastore/catar.rs: allow to configure chunk-size 2019-02-19 15:19:12 +01:00
e5064ba607 simplify backup lib structure (pub use xxx:*), improve doc 2019-02-12 14:13:31 +01:00
93d5d77952 rename ArchiveIndex to DynamicIndex
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00