62f2422f6a
backup/chunk_store: verify chunk file names
...
get_chunk_iterator() should skip over files which aren't an
actual chunk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
3dacd9a660
backup/datastore: fn open is a private implementation detail
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
9739aca41f
backup/chunk_store: open base dir in get_chunk_iterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
c7f481b642
backup: replace ChunkIterator
...
Instead we just build the iterator using our
tools::fs::read_subdir iterator as follows:
Use the index range (0..0x10000) as an iterator, map() it
to yield a subdirectory iterator for each index, then
flatten it.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
e5064ba607
simplify backup lib structure (pub use xxx:*), improve doc
2019-02-12 14:13:31 +01:00
93d5d77952
rename ArchiveIndex to DynamicIndex
...
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00
91a905b6dd
rename ImageIndex to FixedIndex
...
also changed the file extension fron .iidx to .fidx
2019-02-12 11:50:45 +01:00
82bc0ad40c
backup: fixup warnings and readd lost file type check
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:43:43 +01:00
9349d2a1d2
backup: cleanup long lines
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
fdd71f5286
backup: use ChunkIterator for sweep_unused_chunks
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
c9e7f4dc8b
backup: add optional progress callback to ChunkIterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
4070096c6a
backup: add ChunkIterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
e0a5d1ca2f
tests: chunk_store: cleanup .testdir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
9f49fe1d5d
avoid compiler warnings
2019-01-30 18:25:37 +01:00
2296860027
tools.rs: move digest_to_hex() into tools
2019-01-25 10:58:28 +01:00
332dcc2277
backup/chunk_store.rs: fix test case - use absolute path
2019-01-25 10:14:25 +01:00
784252dbc5
backup/datastore.rs: use single lazy_static block
2019-01-24 14:58:47 +01:00
7ca80246ea
backup/datastore.rs: use DateTime<Utc> to store backup time
2019-01-21 10:51:52 +01:00
be0084b001
white space fix
2019-01-21 10:32:07 +01:00
e25736b4c3
backup/datastore.rs: implement list_backups
2019-01-20 18:02:27 +01:00
68469eebb4
backup/chunk_store.rs: verify base path is absolute
2019-01-20 16:49:22 +01:00
0b05fd5830
api3/admin/datastore/catar.rs: implement download
2019-01-20 09:39:32 +01:00
6a4c091616
api3/admin/datastore/catar.rs: start impl. download
2019-01-19 17:27:02 +01:00
728797d0c1
reduce compiler warnings
2019-01-18 16:50:15 +01:00
95cea65b04
backup/datastore.rs: list all index files using walkdir crate
2019-01-18 12:24:58 +01:00
ff3d3100d4
backup/datastore.rs: try to create useful directory layout
...
store/type/ID/TIME/xxx.yyy
2019-01-18 12:01:37 +01:00
150f1bd8f6
backup/datastore.rs: use Arc for chunk_store references
2019-01-15 12:36:16 +01:00
1629d2ad7b
api3/admin/datastore/upload_catar.rs: implement upload future
2019-01-15 11:38:26 +01:00
ddbdf80d51
avoid compiler warnings
2019-01-11 08:41:33 +01:00
373ef4a504
backup/archive_index.rs: correctly check for errors
2019-01-10 11:19:54 +01:00
4624fe29cc
impl Read for BufferedArchiveReader
2019-01-06 10:04:45 +01:00
343370504b
impl Seek for BufferedArchiveReader
2019-01-06 09:35:39 +01:00
318564ac03
buffered_read: return empty buffer on eof
2019-01-06 09:17:28 +01:00
0a72e26704
define and use BufferedReader trait
2019-01-05 17:28:20 +01:00
39c6bd86cc
backup/archive_index.rs: implement BufferedArchiveReader
...
Implement relativly fast random read using binary search.
2019-01-05 14:47:56 +01:00
060c4811a0
backup/chunk_store.rs: resize buffer by powers of two
2019-01-05 11:34:49 +01:00
df9973e8f5
backup/chunk_store.rs: avoid allocation in read_chunk
2019-01-04 17:16:56 +01:00
7394ca3e95
white space cleanups
2019-01-04 12:51:43 +01:00
96df2fb44b
backup/chunk_store.rs: implement read_chunk()
2019-01-04 12:50:54 +01:00
bc616633d5
backup/chunk_store.rs: only use one directory level
2019-01-04 10:35:22 +01:00
b663789bdb
backup/archive_index.rs: pass correct size to unmap
2019-01-04 09:28:41 +01:00
48d0d35688
backup/image_index.rs: use little endian
2019-01-04 08:45:45 +01:00
050f8a6355
chunker.rs: make sure chunk_size_avg is a power of two
2019-01-03 15:10:54 +01:00
ea4ea34baf
improve chunker speed (avoid modulo)
2019-01-03 14:35:19 +01:00
0b8e75ed16
image_index.rs: verify file size
2019-01-02 18:14:02 +01:00
77703d95aa
implement garbage collection for .aidx files
2019-01-02 14:27:04 +01:00
a360f6fa2d
image_index.rs: verify header magic/version
2019-01-02 13:13:13 +01:00
44b3f62b42
fix typo
2019-01-02 12:56:04 +01:00
5032b57b44
archive_index.rs: correctly write .aidx file
2019-01-02 12:54:40 +01:00
580dc84bac
image_index.rs: print stats
2019-01-02 12:53:49 +01:00
Wolfgang Bumiller
Dietmar Maurer