Dominik Csapak
abdb976340
add Display trait to BackupDir
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:28:35 +02:00
Dominik Csapak
3b62116ce6
implement AsyncReadChunk for LocalChunkReader
...
same as the sync ReadChunk but uses tokio::fs::read instead
of file_get_contents
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:54:23 +02:00
Dominik Csapak
e181d2f6da
add encrypted info to Manifest
...
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:35:39 +02:00
Dominik Csapak
bde8e243cf
remove unsafe copy code
...
copy_nonoverlapping is basically a memcpy which can also be done
via copy_from_slice which is not unsafe
(copy_from_slice uses copy_nonoverlapping internally)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:56:15 +02:00
Wolfgang Bumiller
bb59df9134
catalog: don't panic on invalid file mtimes
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:25:54 +02:00
Wolfgang Bumiller
4264e52220
reuse some extractor code in catalog shell
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:54:54 +02:00
Wolfgang Bumiller
6988b29bdc
use O_EXCL when creating files during extraction
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:33:27 +02:00
Wolfgang Bumiller
d30c192589
AsyncReadChunk: require Send
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 09:50:29 +02:00
Wolfgang Bumiller
7a6b549270
dynamic index: make it hard to mess up endianess
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-15 09:19:35 +02:00
Wolfgang Bumiller
57e50fb906
use new Mmap helper for dynamic index
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 13:57:56 +02:00
Wolfgang Bumiller
4d16badf6f
add an AsyncReadChunk trait
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:38:21 +02:00
Wolfgang Bumiller
1498659b4e
cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:59:34 +02:00
Dominik Csapak
e693818afc
refactor time functions to tools
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:10 +02:00
Wolfgang Bumiller
26e78a2efb
downgrade some FIXMEs to TODOs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:09:23 +02:00
Wolfgang Bumiller
5444fa940b
turn pxar::flags into bitflags, pxar::Flags
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:05:53 +02:00
Wolfgang Bumiller
fab2413741
catalog: remove unused SenderWriter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 10:42:42 +02:00
Wolfgang Bumiller
c443f58b09
switch to external pxar and fuse crates
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Dominik Csapak
a95a3fb893
fix csum calculation of not 'chunk_size' aligned images
...
the last chunk does not have to be as big as the chunk_size,
just use the already available 'chunk_end' function which does the
correct thing
this fixes restoration of images whose sizes are not a multiple of
'chunk_size' as well
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 10:18:30 +02:00
Thomas Lamprecht
55919bf141
verify_file: add missing closing parenthesis in error message
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 19:10:01 +02:00
Dietmar Maurer
4e14781aec
fix typo
2020-06-03 06:59:43 +02:00
Thomas Lamprecht
add5861e8d
typo fixes all over the place
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 16:39:08 +02:00
Dietmar Maurer
1610c45a86
src/client/pull.rs: also download client.log.blob
2020-05-30 14:51:33 +02:00
Dietmar Maurer
96d65fbcd0
cleanup: define/use const for predefined blob file names.
2020-05-30 14:04:15 +02:00
Dietmar Maurer
8545480a31
src/bin/proxmox-backup-proxy.rs: add simple task scheduler for garbage collection
2020-05-20 08:59:45 +02:00
Dietmar Maurer
07ce44a633
avoid compiler warnings
2020-05-19 07:03:41 +02:00
Dietmar Maurer
7b22acd0c2
src/config/network.rs: make it compatible with pve
...
and depend on proxmox 0.1.26
2020-05-07 09:28:25 +02:00
Dietmar Maurer
99641a6bbb
garbage_collect: call fail_on_abort to abort GV when requested.
2020-05-05 09:06:34 +02:00
Dietmar Maurer
54552dda59
implemnt backup ownership, improve datastore access permissions
2020-04-28 10:22:25 +02:00
Wolfgang Bumiller
f7d4e4b506
switch from failure to anyhow
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-17 18:43:30 +02:00
Dietmar Maurer
cf459b1982
gc: log pending removals
2020-04-06 09:50:40 +02:00
Christian Ebner
32d192a952
catalog: shell: Use the new logic including resolving symlinks for catalog
...
and remove the old unused code.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:51:01 +01:00
Christian Ebner
fee5528e59
catalog: shell: introduce new CatalogPathStack to navigate in catalog shell.
...
This is basically a rewrite of the current logic for navigating the catalog,
but in addition allows to follow symlinks.
Following symlinks introduces the issue that generation of canonical paths
(needed in the actual pxar archive) is more complex, as symlinks have to be
resolved and loops avoided.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:50:43 +01:00
Christian Ebner
4145c36749
catalog: add missing function documentation.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:50:08 +01:00
Christian Ebner
c2f9149461
catalog: introduce is_symlink() to check if DirEntry is a symlink.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:49:55 +01:00
Christian Ebner
11ee5c0563
catalog: derive PartEq for equality checks of two DirEntry's.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:49:39 +01:00
Christian Ebner
536683e73b
src/backup/dynamic_index.rs: Add LruCache for chunks.
...
In order to improve non-sequential reads of chunks as e.g. in FUSE.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:47:16 +01:00
Christian Ebner
35ddf0b419
catalog: shell: Introduce clear-selected command.
...
'clear-selected' allows to clear all the match patterns from the list of
patterns for a subsequent restore.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:40:55 +01:00
Christian Ebner
8e464141cf
catalog: shell: Improve list-selected command.
...
'list-selected' now shows the filenames matching the patterns for a restore
instead of the patterns themselfs.
The patterns can be displayed by passing the '--pattern' flag.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:40:42 +01:00
Christian Ebner
03f779c6f5
catalog: shell: Improve output of stat command.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-27 06:53:17 +01:00
Dietmar Maurer
8ce49a76da
src/backup/backup_info.rs: fix SNAPSHOT_PATH_REGEX and GROUP_PATH_REGEX
2020-02-18 13:16:35 +01:00
Christian Ebner
ba050e3788
catalog: shell: Allow two or more successive slashes in path.
...
Two or more successive slashes should be allowed and treated as a single slash.
We also do not treat two successive slashes at the beginning of a path any
different.
Details are found here:
https://pubs.opengroup.org/onlinepubs/000095399/basedefs/xbd_chap04.html#tag_04_11
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-02-05 09:32:29 +01:00
Dietmar Maurer
501f4fa220
depend on proxmox 0.1.13, use new tty helpers from there
2020-01-31 08:16:00 +01:00
Dietmar Maurer
d08bc483db
use proxmox 0.1.9 with new cli command helpers
2020-01-27 18:08:43 +01:00
Dietmar Maurer
a92830dc39
src/api2/types.rs: define and use api type GarbageCollectionStatus
2020-01-23 13:40:12 +01:00
Dietmar Maurer
51534c8de9
src/backup/manifest.rs: check if manifest contains files
2020-01-23 11:16:12 +01:00
Dietmar Maurer
6abce6c2bb
src/backup/datastore.rs: remove_backup_(group/dir) - return Error instead of io::Error
2020-01-23 10:14:46 +01:00
Dietmar Maurer
8a1d68c8b9
src/backup/datastore.rs: improve error messages
2020-01-23 09:58:14 +01:00
Dietmar Maurer
41b373eced
src/backup/datastore.rs: new helpers
...
last_successful_backup: Returns the time of the last successful backup
group_path: Returns the absolute path for a backup_group
snapshot_path: Returns the absolute path for a backup_dir
2020-01-22 15:05:47 +01:00
Wolfgang Bumiller
9ea4bce444
bump proxmox crate to 0.1.7
...
The -sys, -tools and -api crate have now been merged into
the proxmx crate directly. Only macro crates are separate
(but still reexported by the proxmox crate in their
designated locations).
When we need to depend on "parts" of the crate later on
we'll just have to use features.
The reason is mostly that these modules had
inter-dependencies which really make them not independent
enough to be their own crates.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 13:48:37 +01:00
Wolfgang Bumiller
d973aa827c
introduce new runtime tokio helpers
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-20 13:12:40 +01:00
Dietmar Maurer
4b4eba0b9e
src/api2/pull.rs: implement delete flag for vanished groups
2020-01-17 11:24:55 +01:00
Dietmar Maurer
11d89239c3
src/backup/backup_info.rs: new list_groups helper
2020-01-17 10:42:03 +01:00
Dietmar Maurer
d2dd827877
src/backup/catalog.rs - SenderWriter: use tokio::task::block_in_place
...
Make sure we do not block the executor.
2020-01-16 14:24:15 +01:00
Dietmar Maurer
d0187a51a9
src/section_config.rs - convert_to_array: optionally add digest
...
datastore::config() -> also return digest
remotes::config() -> also return digest
2020-01-14 12:57:03 +01:00
Oguz Bektas
6d20a29d73
fix typo for function name load_and_decrypt_key
...
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-01-13 11:14:47 +01:00
Christian Ebner
90fc97af6a
pxar::decoder::Decoder: include xattrs and payload size in `DirectoryEntry`.
...
By reading and including xattrs and payload size in struct `DirectoryEntry`,
the tuple of return types is avoided and the code is simpler.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-10 13:45:40 +01:00
Dietmar Maurer
8f14e8fe4c
src/backup/backup_info.rs: implement last_successful_backup()
...
A backup is considered successful if there is a manifest file.
2020-01-06 11:35:22 +01:00
Dietmar Maurer
247a8ca5bb
src/backup/manifest.rs: impl TryFrom DataBlob
2020-01-05 16:28:18 +01:00
Dietmar Maurer
7759eef552
src/backup/datastore.rs: implement cleanup_backup_dir()
2020-01-05 15:15:12 +01:00
Dietmar Maurer
3758b398cd
src/backup/read_chunk.rs: add read_raw_chunk()
...
Allow to read chunk data without decoding.
2020-01-02 13:29:10 +01:00
Dietmar Maurer
2585a8a4e2
src/backup/chunk_store.rs: implement cond_touch_chunk()
...
This will be used by backup sync to test if a chunk already exists.
2020-01-02 13:26:28 +01:00
Dietmar Maurer
1cf5178ac5
src/backup/datastore.rs: new helper try_shared_chunk_store_lock()
2020-01-02 11:00:33 +01:00
Dietmar Maurer
1e8da0a789
src/backup/manifest.rs: new helper archive_type()
2019-12-31 15:23:41 +01:00
Dietmar Maurer
645995634a
src/api2/config/datastore.rs - create: pass uid and gid instead of User
2019-12-20 09:23:58 +01:00
Dietmar Maurer
e67770d496
src/backup/chunk_store.rs - create: pass User instead of CreateOptions
2019-12-20 09:11:40 +01:00
Wolfgang Bumiller
afdcfb5bc9
let ChunkStore::create take CreateOptions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-19 13:14:49 +01:00
Dietmar Maurer
f74a03da1f
remove tools::getpwnam_ugid, impl. crate::backup::backup_user()
...
And use new nix::unistd::User struct.
2019-12-19 10:20:13 +01:00
Dietmar Maurer
868c585219
src/backup/datastore.rs: avoid divide by zero
2019-12-19 07:14:23 +01:00
Dietmar Maurer
7e210bd0b4
src/backup/chunk_store.rs: create lock file with correct owner
2019-12-19 06:55:53 +01:00
Dietmar Maurer
0b97bc6158
src/backup/chunk_store.rs: use proxmox::tools::fs::create_path
2019-12-18 12:26:43 +01:00
Wolfgang Bumiller
feaa1ad35f
replace file_set_contents with replace_file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-18 11:16:04 +01:00
Oguz Bektas
14f1e63067
chunk_store: create parent directories
...
'datastore create storename /path/to/dir/that/may/not/exist' should
work.
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-12-17 15:39:42 +01:00
Christian Ebner
38d9a69875
catalog: decoder and shell: fix incorrect logic in find matching
...
The find matching was incorrectly performed starting from the parent directroy
and not as intended from the entries of the parent directory.
Further, the match pattern passed from the catalog shell contains the absolute
path of the search entry point as prefix, so find() must always start from the
archive root. This is because the match pattern has to be stored in the selected
list for a subsequent restore-selected command in the shell.
All matching paths are shown as absolute paths with all contents in the subdir,
equal to what would be restored by the given pattern.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 14:57:34 +01:00
Christian Ebner
25cdd0e0a1
catalog_shell: impl find for catalog shell.
...
Implements the find command which allows to find and select files for subsequent
restore.
Files selected for restore are now stored in a Vec instead of a HashSet.
This is needed, since instead of the full paths for each file, selected files are
now identified by a list of match pattern, where ordering matters.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner
90dfd0a71d
catalog: impl find() for file search by match pattern.
...
find() iterates over the file tree and matches each node against a list of match
patterns provided at function call.
For each matching node, a callback function with the current directroy stack is
called.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Dietmar Maurer
236a396aa1
src/api2/admin/datastore.rs - prune: log retention options
2019-12-14 16:32:16 +01:00
Wolfgang Bumiller
db0cb9ce0b
update a chunk of stuff to the hyper release
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
Christian Ebner
f084505ec5
src/backup/catalog_shell.rs: sort output of list-selected
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-11 10:54:20 +01:00
Dietmar Maurer
73e57f244e
src/backup/datastore.rs - garbage-collection: fix average chunk size
2019-12-10 11:25:29 +01:00
Dietmar Maurer
48ef3c3327
cli: avoid useless .into()
...
This needs latest proxmox-api updates.
2019-12-09 17:40:34 +01:00
Dietmar Maurer
102d8d4136
src/backup/prune.rs: implement --keep-hourly
2019-12-07 11:23:33 +01:00
Dietmar Maurer
052c4aa160
src/backup/prune.rs: prune - correctly use iso_week year
2019-12-06 19:41:38 +01:00
Dietmar Maurer
9e3f008804
src/backup/prune.rs: add new helper keeps_something()
2019-12-06 12:28:31 +01:00
Dietmar Maurer
55c3cb69cd
rename catalog_shell_api() into catalog_shell_cli()
2019-12-06 09:42:05 +01:00
Dietmar Maurer
ecbaa38fa4
src/backup/catalog_shell.rs: factor out catalog_shell_api()
...
In order to extract documentation ...
2019-12-06 09:35:47 +01:00
Christian Ebner
cca67dc0c3
src/backup/readline.rs: removed in favor of new readline impl
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:03:04 +01:00
Christian Ebner
6934c6fe77
src/backup/catalog_shell.rs: adapt to use API Schema definition and rustyline
...
This major refactoring of the catalog based shell utilizes the new API macro and
the API Schema as well as rustyline instead of the old GNU readline C API.
The code now has these 3 main components:
* The `Shell` which handles the readline loop via rustyline.
* The shell functions defined via the API macro.
* The `Context` which holds catalog and decoder instances.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:02:44 +01:00
Dietmar Maurer
9b78352188
src/backup/prune.rs: define new struct PruneOptions
2019-12-06 08:56:27 +01:00
Dietmar Maurer
92acbd69f7
src/backup/backup_info.rs: remove old prune code
2019-12-06 08:29:27 +01:00
Dietmar Maurer
408434360b
src/backup/prune.rs: add PruneMark::KeepPartial
...
Use separate mark for partial backup - dont mess them up with regular ones.
2019-12-06 08:15:32 +01:00
Dietmar Maurer
a8c8366cfa
src/backup/prune.rs: factor out code into remove_incomplete_snapshots()
2019-12-06 08:12:08 +01:00
Dietmar Maurer
dc18849156
src/backup/prune.rs: moved prune related code into extra file
2019-12-06 08:06:21 +01:00
Dietmar Maurer
6f47dd8a0f
src/backup/backup_info.rs - prune: improve algorythm
2019-12-05 18:41:28 +01:00
Dietmar Maurer
8f0b4c1f90
src/api2/admin/datastore.rs - prune: log info about all snapshots
2019-12-05 15:35:11 +01:00
Dietmar Maurer
9ce42759ec
src/backup/backup_info.rs - compute_prune_list: remove unfinished backups
2019-12-05 11:18:10 +01:00
Dietmar Maurer
2c034f8d0a
src/backup/backup_info.rs: improve prune algorithm
2019-12-05 08:55:19 +01:00
Dietmar Maurer
aeeac29bb1
src/backup/backup_info.rs: new compute_prune_list helper
...
So that we can write regression tests for this.
2019-12-04 15:49:11 +01:00
Christian Ebner
314bb35868
catalog_shell: fix several clippy warnings
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-27 10:13:50 +01:00
Christian Ebner
951cf17ee3
catalog_shell: major refactoring of cli command definition and parsing
...
By changing the way shell commands are defined and parsed, this makes it more
straight forward to extend the current functionality.
The readline input is parsed based on the provided command definition and the
given parameters and options are passed to a command specific callback function.
In addition, the provided command definition including its description is used
to generate a help string to display.
The help command shows a list of all supported commands or the help string for
the provided command.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 13:01:00 +01:00
Christian Ebner
59bc6ad676
catalog_shell: check and return on empty list in order to avoid division by zero
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:33 +01:00
Christian Ebner
4e56b2f792
catalog_shell: do not restore full archive if no entries are selected for restore-selected
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:18 +01:00
Dietmar Maurer
99b5b6cba9
src/pxar/decoder.rs: use trait object as reader
...
So that the Decoder is no longer genertic.
2019-11-26 10:45:11 +01:00
Dietmar Maurer
f701d0335e
src/pxar/sequential_decoder.rs: remove callback from new()
...
And use an extra functzion set_callback() to configure that.
Also rewrite pxar/fuse.rs and implement a generic Session (will get
further cleanups with next patches).
2019-11-26 09:56:48 +01:00
Christian Ebner
f14c96ea38
src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:54:04 +01:00
Christian Ebner
38446a9551
src/backup/readline.rs: impl wrapper for GNU readline
...
In order to provide the context needed for tab completion via the readline
callback, the needed mut ref is passed via a thread local storage key.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:46:24 +01:00
Christian Ebner
6dba015043
src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute
...
This is needed in order to explicitly clone the values when needed in the
catalog shell implementation.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:15:39 +01:00
Christian Ebner
c9f002213e
src/backup/catalog.rs: impl DirEntry::is_directory()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:14:00 +01:00
Wolfgang Bumiller
f569acc5e2
drop uuid crate dependency
...
proxmox::tools now has a Uuid module using the native
libuuid.
Adds build dependency: libuuid1 (which is a Pre-Depends of
util-linux, so always installed anyway).
Drops uuid + 16 more crate dependencies.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-14 10:22:29 +01:00
Christian Ebner
b423958d8e
catalog: fix issue with DirEntry start
...
start has to be the parent start - offset
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-12 17:18:13 +01:00
Dietmar Maurer
780dd2b0a1
src/backup/catalog.rs: add some docs
2019-11-12 13:23:40 +01:00
Dietmar Maurer
6aa906b547
src/backup/catalog.rs - CatalogReader: verify magic number
2019-11-12 12:54:06 +01:00
Dietmar Maurer
2ec208aef5
src/backup/catalog.rs - CatalogReader::dump(): use root()
2019-11-12 12:47:21 +01:00
Dietmar Maurer
8f24a9ea18
src/backup/catalog.rs - Direntry::parse: abort if callback return false
2019-11-12 11:41:44 +01:00
Dietmar Maurer
7d017123fd
src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code
2019-11-12 11:41:44 +01:00
Dietmar Maurer
dc9596de45
src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs
2019-11-12 11:41:44 +01:00
Dietmar Maurer
55c0b3cc7c
src/backup/catalog.rs - DirEntry: factor out name attribute
...
And avoid allocations inside parse().
2019-11-12 11:41:44 +01:00
Christian Ebner
3f1c5b5e65
catalog: impl std::fmt::Display trait for CatalogEntryType
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-12 07:00:08 +01:00
Dietmar Maurer
f89359c2f3
src/backup/catalog.rs: clenup - use single impl block
2019-11-10 11:38:31 +01:00
Dietmar Maurer
e8d8fdc61d
src/backup/catalog.rs: remove unused code
2019-11-10 11:37:06 +01:00
Dietmar Maurer
c74c074bfd
src/backup/catalog.rs: write MAGIC at start of file
2019-11-10 11:36:45 +01:00
Dietmar Maurer
265664517a
src/backup/catalog.rs: new parse() helper
2019-11-09 14:37:33 +01:00
Dietmar Maurer
5d92935e03
src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block
2019-11-09 13:21:20 +01:00
Dietmar Maurer
12a1975af9
src/backup/catalog.rs: store u64 compressed
...
This way catalogs get about 1/3 smaller.
2019-11-09 13:07:47 +01:00
Dietmar Maurer
89245fb530
renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs
2019-11-08 10:41:00 +01:00
Dietmar Maurer
bf6e321744
change catalog format, use dynamic index to store catalog.
...
In order to remove size restriction of a single blob.
2019-11-08 10:35:48 +01:00
Christian Ebner
b82a101c86
catalog: store current writer position in CatalogBlobWriter
...
Counts the bytes written by the CatalogBlobWriter in order to obtain the
stream position, needed to get offset to reference catalog items.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-31 09:11:51 +01:00
Dietmar Maurer
62ee2eb405
avoid some clippy warnings
2019-10-26 11:42:05 +02:00
Dietmar Maurer
834a2f95a0
avoid some clippy warnings
2019-10-25 18:44:51 +02:00
Dietmar Maurer
9fe2f639b8
allow(clippy::cast_ptr_alignment)
2019-10-25 18:03:55 +02:00
Dietmar Maurer
cd69d36b45
src/backup/dynamic_index.rs: use write_all()
2019-10-25 17:22:41 +02:00
Dietmar Maurer
649c5ee7b1
load_and_decrtypt_key: allow function closures to readf passphrase
2019-10-16 13:23:52 +02:00
Dietmar Maurer
60ffa9649c
src/backup/manifest.rs: fix property names (use '-' instead of '_')
2019-10-14 12:28:27 +02:00
Dietmar Maurer
4c9f753c56
src/backup/data_blob.rs: define create_zero_chunk as member function
...
and rename it to build_zero_chunk()
2019-10-14 10:58:26 +02:00
Dietmar Maurer
2745151d93
src/backup/data_blob.rs: add create_zero_chunk() helper
2019-10-14 10:44:46 +02:00
Dietmar Maurer
f06b820ac0
src/backup/manifest.rs: add verify_file
2019-10-13 10:09:12 +02:00
Dietmar Maurer
511a47bd73
cleanup
2019-10-13 09:39:21 +02:00
Dietmar Maurer
e17d5d864d
src/backup/manifest.rs: improve parser (try_from)
2019-10-13 08:39:49 +02:00
Dietmar Maurer
59e9ba01c6
src/backup/manifest.rs: new class to generate/parse index.json
2019-10-12 17:58:08 +02:00
Dietmar Maurer
7123ff7d43
src/backup/data_blob.rs: avoid Arc<CryptConfig>
...
use simple reference instead.
2019-10-07 11:41:35 +02:00
Dietmar Maurer
4ee8f53d07
remove DataChunk file format - use DataBlob instead
2019-10-06 10:31:06 +02:00
Dietmar Maurer
9e603e254c
src/backup/fixed_index.rs: do not use offset for checksum
2019-09-23 11:30:20 +02:00
Dietmar Maurer
2aba16bde1
src/backup/data_chunk.rs: add into_raw()
2019-09-19 14:24:17 +02:00
Wolfgang Bumiller
ba5e67475a
clippy: fix access to unaligned buffer in dynamic reader
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:50:45 +02:00
Wolfgang Bumiller
b526bd1408
backup: cleanup unused imports
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:46:46 +02:00
Wolfgang Bumiller
4dc79bb1d9
clippy: fix access to unaligned buffer
...
via the newly introduced ReadExt::read_host_value_boxed()
from proxmox::tools::io.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:45:44 +02:00
Wolfgang Bumiller
ccdf3ad1ec
backup: use static assertion in fixed index reader
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:43:25 +02:00
Wolfgang Bumiller
b21851cc5c
clippy: collapse identical if branches
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 13:57:58 +02:00
Wolfgang Bumiller
653b1ca10e
some clippy fixups
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:14:05 +02:00
Dietmar Maurer
9d456cf8ec
src/backup/catalog_blob.rs: use write_all() instead of write()
...
To correctly handle interrupted system calls.
2019-09-05 11:40:48 +02:00
Dietmar Maurer
0a51fe0011
src/backup/fixed_index.rs: new helper to compute checksum and file size
2019-09-03 13:11:45 +02:00
Dietmar Maurer
82c85a21a1
src/backup/dynamic_index.rs: new helper to compute checksum and file size
2019-09-03 13:11:09 +02:00
Wolfgang Bumiller
745e652a7f
src/backup/index.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
32bef1e2d1
src/backup/chunk_stream.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
5c1130df9f
implement Sync for DynamicIndexReader and FixedIndexReader
...
hyper's wrap_stream now needs this
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
d21f8a5b5b
backup/dynamic_index: explicitly zero-initialize the header
...
The writer.write_all() call accessed data marked as
undefined to valgrind. Note that we shouldn't write out
uninitialized memory for security reasons anyway.
(note that vec::undefined already did zero-initialize the
data, but also marked it as undefined for valgrind when
compiling with the valgrind feature)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 11:42:35 +02:00
Wolfgang Bumiller
990b930f22
backup/dynamic_index: use static assertion, fix size
...
The type was sized properly but the number was still wrong,
fixed this.
TODO! Once unions with non-Copy values are stable make this
a `union { full: [u8; 4096], data: TheActualHeader }`;
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 11:42:35 +02:00
Wolfgang Bumiller
7d83440c60
remove proxmox-protocol subcrate
...
AFAICT we have no use for it anymore, its api entry points
are gone. If we do end up needing something from it, it's
still in the git history anyway. (And about two thirds of it
can be made much less awkward by utilizing async-await
anyway, so no love lost there...)
Moved the chunker back into src/backup/chunker.rs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 14:08:25 +02:00
Wolfgang Bumiller
5e58e1bb7d
{dynamic,fixed}_index: replace mem::uninitialized
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:35:36 +02:00
Wolfgang Bumiller
f93b55b057
src/backup/index: style fixup & unsafe copy removal
...
We can use the safe .copy_from_slice alternative in this
case.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:31:45 +02:00
Wolfgang Bumiller
fa17b1ce2a
src/backup/index: replace mem::uninitialized
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:27:35 +02:00
Wolfgang Bumiller
367f002ee2
use statement cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-16 12:28:40 +02:00
Dietmar Maurer
9d135fe617
src/backup/catalog_blob.rs: moved catalog impl. from pxar
...
And avoid loading catalog into memory.
2019-08-16 12:27:17 +02:00
Wolfgang Bumiller
9025312aa6
avoid lifetimes in blob reader/writer
2019-08-16 10:07:24 +02:00
Dietmar Maurer
71d08e00b7
src/backup/data_blob_reader.rs: fix compiler warning
2019-08-16 08:10:39 +02:00
Dietmar Maurer
b791804f4b
src/backup/data_blob_reader.rs: impl for Read instead of BufRead
2019-08-14 14:56:01 +02:00
Dietmar Maurer
39a4df61d6
improve docs
2019-08-14 14:08:27 +02:00
Dietmar Maurer
018d11bb84
src/backup/data_blob.rs: move parts into single files
2019-08-14 13:24:41 +02:00
Dietmar Maurer
e27c5a559f
src/backup/data_blob.rs: avoid excessive stack usage
2019-08-14 12:44:08 +02:00
Dietmar Maurer
c638542b4b
src/backup/file_formats.rs: moved header_size
2019-08-14 12:35:53 +02:00
Dietmar Maurer
6082216343
src/backup/data_blob.rs - CryptReader/CryptWriter: fix buffer size handling
2019-08-13 15:23:15 +02:00
Dietmar Maurer
548c9489d7
src/backup/data_blob.rs - DataBlobReader: impl compressed, encrypted blobs
2019-08-13 13:17:08 +02:00
Dietmar Maurer
2aa0bfff59
src/backup/data_blob.rs - DataBlobReader: implement reader for encrtypted blobs
2019-08-13 13:17:08 +02:00
Dietmar Maurer
e9a385a78e
src/backup/data_blob.rs - DataBlobReader: impl compressed, signed blobs
2019-08-12 17:47:25 +02:00
Dietmar Maurer
4bfa147eaf
src/backup/data_blob.rs - DataBlobReader: impl. reader for signed blobs
2019-08-12 17:41:25 +02:00
Dietmar Maurer
09785b2795
src/backup/data_blob.rs - DataBlobReader: start serious impl.
2019-08-12 17:20:41 +02:00
Dietmar Maurer
a32bd8a516
src/backup/data_blob.rs - DataBlobWriter: fix decryption
2019-08-12 11:57:29 +02:00
Dietmar Maurer
5d15cb49b4
src/backup/data_blob.rs - compute_crc: start after blob header
2019-08-12 11:32:26 +02:00
Dietmar Maurer
1cbdfd19a2
src/backup/file_formats.rs; use const instzead of static
2019-08-12 11:20:21 +02:00
Dietmar Maurer
18be4ec24a
src/backup/data_blob.rs - DataBlobWriter: fix magic number for uncompressed blobs
2019-08-12 10:46:05 +02:00
Dietmar Maurer
5622a3fcdb
src/backup/data_blob.rs - DataBlobWriter: impl. compressed encrypted blobs
2019-08-12 10:16:42 +02:00
Dietmar Maurer
f4942e9ffd
src/backup/data_blob.rs - DataBlobWriter: impl. encrypted blobs
2019-08-12 10:08:00 +02:00
Dietmar Maurer
c57ec43a53
src/backup/crypt_config.rs: new helper data_crypter to get openssl Crypter
2019-08-12 10:06:51 +02:00
Dietmar Maurer
7776becf5d
src/backup/data_blob.rs - DataBlobWriter: impl. compressed signed blobs
2019-08-12 09:01:08 +02:00
Dietmar Maurer
706638f803
src/backup/data_blob.rs: impl. ChecksumWriter
...
To correctly compute crc,hmac of final data.
2019-08-12 08:53:53 +02:00
Dietmar Maurer
f796351c54
src/backup/data_blob.rs: impl. DataBlobWriter for authenticated blobs
2019-08-12 07:34:41 +02:00
Dietmar Maurer
cb0eea29d9
src/backup/crypt_config.rs: new method to get hmac signer
2019-08-12 07:33:15 +02:00
Dietmar Maurer
a762ce54fd
src/backup/data_blob.rs: make DataBlobWriter more generic
...
Allow to write compressed and uncompressed blobs.
2019-08-11 12:14:04 +02:00
Dietmar Maurer
1f26fdef19
src/backup/data_blob.rs: impl. simple reader/writer
...
To avoid loading blob into memory.
2019-08-11 11:32:36 +02:00
Dietmar Maurer
781ac11c6a
src/backup/data_blob.rs: define const MAX_BLOB_SIZE
2019-08-09 11:49:06 +02:00
Dietmar Maurer
863be2e6e2
src/backup/data_blob.rs: allow largert blob sizes (128MB)
...
Need this for catalog files.
2019-08-09 10:22:56 +02:00
Dietmar Maurer
eecb23560b
src/backup/data_blob.rs - encode: always compute crc
2019-08-07 08:30:27 +02:00
Dietmar Maurer
38a6cddad7
src/backup/backup_info.rs: add BackupInfo::new constructor
2019-08-07 08:27:52 +02:00
Dietmar Maurer
f889b15867
src/backup/data_blob.rs: always compute crc
2019-08-06 11:42:14 +02:00
Dietmar Maurer
e18a6c9ee5
update to nix 0.14, use code from proxmox:tools
2019-08-03 13:05:38 +02:00
Dietmar Maurer
69ecd8d5b0
src/backup/data_blob.rs: implement signed blobs
2019-08-02 09:56:01 +02:00
Dietmar Maurer
93205f942a
src/backup/crypt_config.rs: new compute_auth_tag helper
2019-08-02 08:55:37 +02:00
Dietmar Maurer
c68d2170d5
src/backup/crypt_config.rs: fix typo
2019-08-02 08:29:40 +02:00
Dietmar Maurer
fa5d6977dd
Revert previous, commit, use UTC RFC3339 without timezone (Z)
...
We now have human readable data/time, and names are still sortable.
2019-07-22 11:49:30 +02:00
Dietmar Maurer
10c2a21cfe
src/backup/backup_info.rs: use unix epoch timestamps instead of RFC3339
...
RFC3339 contains additional timezone information, but IMHO this just adds
more confusion. The API also uses timestamps.
2019-07-22 07:57:03 +02:00
Dietmar Maurer
f4bf7dfcc7
src/client/remote_chunk_reader.rs: implement simple caching
2019-07-05 10:42:46 +02:00
Dietmar Maurer
afb4cd28be
src/backup/fixed_index.rs: implement BufferedFixedReader
2019-07-05 09:19:56 +02:00
Dietmar Maurer
b46c3fad03
src/backup/fixed_index.rs: use correct size
...
We need to consider that the last chunk may have smaller size.
2019-07-04 15:13:22 +02:00
Dietmar Maurer
5e04ec707d
src/api2/backup/environment.rs: improve chunk alignment/size checks
2019-07-04 13:40:43 +02:00
Dietmar Maurer
a24e3993e0
src/backup/chunk_store.rs: coding style fixes
2019-07-04 11:39:10 +02:00
Dietmar Maurer
e4c2fbf170
src/backup/chunk_store.rs: additionally log chunk count
2019-07-04 11:27:11 +02:00
Dietmar Maurer
9850bcdf19
src/backup/chunk_store.rs: improve error reporting
2019-07-04 11:21:54 +02:00
Wolfgang Bumiller
a3f3e91da2
backup/chunk_store: rework chunk iterator
...
We can now use iter::from_fn() which makes for a much nicer
logic. The only thing better is going to be when we can use
generators with `yield`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-04 10:55:17 +02:00
Dietmar Maurer
a57360983b
src/backup/chunk_store.rs - get_chunk_iterator: return percentage inside iterator item
2019-07-04 09:26:44 +02:00
Dietmar Maurer
0f0a35b390
src/backup/dynamic_index.rs: lock file inside new, code cleanup
2019-07-04 08:17:30 +02:00
Dietmar Maurer
a7c72ad9eb
src/backup/fixed_index.rs - FixedIndexReader: remove reference to chunk store
2019-07-04 08:13:16 +02:00
Dietmar Maurer
a660978c9a
src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats
2019-07-04 07:57:43 +02:00
Dietmar Maurer
1e0784841c
src/backup/read_chunk.rs: use &mut self
2019-07-03 14:25:18 +02:00
Dietmar Maurer
877dd11133
src/backup/{fixed, dynamic}_index.rs: improve error messages
2019-07-02 08:56:56 +02:00
Dietmar Maurer
5c593a4d56
src/backup/read_chunk.rs: use Arc for DataStore
2019-07-02 08:49:16 +02:00
Dietmar Maurer
b850673634
src/backup/read_chunk.rs: move read chunk trait into extra file
...
And implement LocalChunkReader.
2019-07-02 08:22:29 +02:00
Wolfgang Bumiller
f35197f449
replace tools::vec with proxmox::tools::vec
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 11:03:25 +02:00
Wolfgang Bumiller
9b2b627fe0
update remaining users of tools::io::ops
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:44:03 +02:00
Wolfgang Bumiller
5485b579a1
backup: use new proxmox::tools::io traits
...
This replaces all `write_value` calls with `write_le_value`
calls!
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:37:02 +02:00
Dietmar Maurer
d48a9955a5
src/backup/dynamic_index.rs: introduce ReadChunk trait
2019-06-28 16:35:00 +02:00
Dietmar Maurer
81a6ce6fde
src/backup/chunk_store.rs: new method chunk_path()
...
Returns the absolute path.
2019-06-28 15:48:09 +02:00
Dietmar Maurer
b208da8393
src/backup/data_{chunk,blob}.rs: add verify_crc method
2019-06-26 09:54:25 +02:00
Dietmar Maurer
bb8231409e
src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json
...
Use the KeyConfig serialization with kdf = None.
2019-06-26 07:32:34 +02:00
Dietmar Maurer
c6d203bbd3
src/backup/backup_info.rs: also list .blob files
2019-06-25 07:07:45 +02:00
Dietmar Maurer
9f46c7de4b
src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup
2019-06-24 13:56:37 +02:00
Dietmar Maurer
cb08ac3efe
src/api2/backup.rs: replace upload_config with upload_blob
2019-06-23 11:44:30 +02:00
Dietmar Maurer
a38c5d4d12
src/backup/data_blob.rs: implement from_raw()
2019-06-23 09:35:44 +02:00
Dietmar Maurer
6f083b7a92
src/api2/backup/upload_chunk.rs: compute crc on upload
2019-06-23 09:04:42 +02:00
Dietmar Maurer
ba01828d38
try to use use proxmox::tools::io::ops::ReadExtOps
2019-06-22 16:29:10 +02:00
Dietmar Maurer
27042ce637
src/backup/file_formats.rs: improve docs
2019-06-22 15:45:36 +02:00
Dietmar Maurer
9f83e0f7ee
src/backup/crypt_config.rs: simplify decode interface
2019-06-22 13:24:29 +02:00
Dietmar Maurer
0066c6d972
src/backup/crypt_config.rs: remove encode_chunk, use encrypt_to instead
2019-06-22 13:02:53 +02:00
Dietmar Maurer
ee8a7e8080
src/backup/crypt_config.rs: implement encrypt_to (output to writer)
2019-06-22 12:25:04 +02:00
Dietmar Maurer
991abfa8b4
src/backup/file_formats.rs: split out file format data
2019-06-22 09:12:25 +02:00
Dietmar Maurer
b7f4f27d6c
add crc field for binary blobs formats
2019-06-21 18:49:04 +02:00
Dietmar Maurer
3025b3a53c
src/backup/data_blob.rs: new file format for binary blobs
2019-06-21 11:32:07 +02:00
Dietmar Maurer
077a8cae63
src/backup/crypt_config.rs: make code more flexible
...
Allow to pass magic numbers as parameters.
2019-06-21 10:50:48 +02:00
Dietmar Maurer
c38266c18d
renamed: src/backup/crypt_setup.rs -> src/backup/crypt_config.rs
2019-06-21 09:51:18 +02:00
Dietmar Maurer
ab44acff57
src/backup/key_derivation.rs: add modified field to key file
2019-06-20 07:58:33 +02:00
Dietmar Maurer
6d0983dbe1
src/bin/proxmox-backup-client.rs: add keyfile parameter for backup
2019-06-19 17:16:41 +02:00
Dietmar Maurer
181f097af9
src/bin/proxmox-backup-client.rs - key API: pass kdf parameter
...
Allow to store keys without password.
2019-06-18 18:21:01 +02:00
Dietmar Maurer
826f309bf5
src/backup/key_derivation.rs: move kdf code into separate file
2019-06-18 11:17:22 +02:00
Dietmar Maurer
11515438cc
Cargo.toml: use serde feature derive
2019-06-18 06:23:25 +02:00
Dietmar Maurer
f2401311b0
src/bin/proxmox-backup-client.rs: start key management
2019-06-17 10:33:24 +02:00
Dietmar Maurer
a7dd483097
src/backup/*_index.rs: used generated magic numbers
2019-06-14 14:58:37 +02:00
Dietmar Maurer
bffd40d6b7
src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools
2019-06-14 11:40:04 +02:00
Dietmar Maurer
9335d74eba
src/backup/fixed_index.rs: compute checksum over the index
2019-06-14 10:34:29 +02:00
Dietmar Maurer
16ff6b7cd8
src/backup/dynamic_index.rs: compute checksum over the index
2019-06-14 10:33:59 +02:00
Dietmar Maurer
fa148dbd11
src/api2/backup/upload_chunk.rs: verify chunk data if possible
2019-06-14 06:23:46 +02:00
Dietmar Maurer
f640f23ac5
src/backup/data_chunk.rs: fix typo
2019-06-13 16:42:55 +02:00
Dietmar Maurer
dba72d5016
src/backup/data_chunk.rs: only use compressed data if it is shorter than uncompressed
2019-06-13 12:56:50 +02:00
Dietmar Maurer
68ab376ae9
src/backup/crypt_setup.rs: only use compressed data if it is shorter than uncompressed
2019-06-13 12:47:54 +02:00
Dietmar Maurer
f98ac774ee
backup: Add support for client side encryption
...
first try ...
2019-06-13 11:47:23 +02:00
Dietmar Maurer
51929e4532
src/backup/data_chunk.rs - decode: make crypt_config optional
2019-06-13 11:44:38 +02:00
Dietmar Maurer
046bcb9773
src/backup/crypt_setup.rs: add comment for possible improvements
2019-06-13 07:35:59 +02:00
Dietmar Maurer
bd0e3c7cfa
src/backup/data_chunk.rs: Moved ChunkInfo from merge_known_chunks.rs
2019-06-13 07:35:02 +02:00
Dietmar Maurer
b595cb9d2c
src/backup/data_chunk.rs: Data Chunk builder
...
Make handling encoded chunks easier.
2019-06-12 13:37:01 +02:00
Dietmar Maurer
ef27200cec
src/backup/crypt_setup.rs: fix encoder for uncompressed chunks
2019-06-10 09:56:06 +02:00
Dietmar Maurer
74792b95b2
src/backup/crypt_setup.rs: allow compressed and uncompressed chunks
2019-06-10 08:27:35 +02:00
Dietmar Maurer
9e0187a203
src/backup/crypt_setup.rs: remove unused CryptData struct
2019-06-10 07:55:36 +02:00
Dietmar Maurer
bec8498ae3
src/backup/crypt_setup.rs: avoid useless allocations
2019-06-09 11:44:17 +02:00
Dietmar Maurer
48b4b40b57
src/backup/crypt_setup.rs: crypto helpers
2019-06-08 09:56:51 +02:00
Wolfgang Bumiller
dd5495d6dc
tree-wide: use 'dyn' for all trait objects
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
Dietmar Maurer
36898ffce6
src/backup/chunk_stream.rs: add optional chunk_size parameter
2019-05-30 13:28:24 +02:00
Dietmar Maurer
976595e1a9
src/backup/dynamic_index.rs: split class DynamicIndexWriter
2019-05-29 08:49:57 +02:00
Dietmar Maurer
cb0708dd46
src/backup/fixed_index.rs: remove ChunkStat from struct
2019-05-29 07:08:34 +02:00
Dietmar Maurer
01af11f340
src/backup/fixed_index.rs: improve consistency checks
2019-05-29 07:02:45 +02:00
Dietmar Maurer
006f3ff407
src/api2/admin/datastore/backup/environment.rs: add more consistency checks
2019-05-28 09:21:31 +02:00
Dietmar Maurer
fc14b84937
src/backup/fixed_index.rs: improve bound checks inside add_digest
2019-05-28 08:47:03 +02:00
Dietmar Maurer
e1225de467
src/backup/fixed_index.rs: store index_length to avoid code duplication
2019-05-28 08:27:57 +02:00
Dietmar Maurer
29ae5c86a2
src/backup/fixed_index.rs: make chunk_size public
2019-05-28 06:18:24 +02:00
Wolfgang Bumiller
8ea3b1d188
tree-wide: use the new vec/io tools modules
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 09:49:10 +02:00
Dietmar Maurer
0cc0fffd1e
src/backup/chunk_stream.rs: use BytesMut in ChunkStream
2019-05-22 09:39:02 +02:00
Dietmar Maurer
c052be5c86
src/backup/chunk_stream.rs: use more generics
2019-05-22 09:18:05 +02:00
Dietmar Maurer
169c0e060f
src/backup/chunk_stream.rs: optimize FixedChunkStream (use BytesMut)
2019-05-22 09:05:35 +02:00
Dietmar Maurer
3be3f3dcc4
src/backup/chunk_stream.rs: fix behaviour for large input buffers
2019-05-22 08:05:27 +02:00
Dietmar Maurer
02fa54ff3c
src/backup/chunk_stream.rs: assemble data with correct order
2019-05-22 07:44:51 +02:00
Dietmar Maurer
7f3d2ffab9
src/backup/index.rs: rename ChunkListReader into DigestListEncoder
2019-05-21 12:28:44 +02:00
Dietmar Maurer
ee53955f52
src/backup/index.rs: implement DigestListDecoder
2019-05-21 12:21:22 +02:00
Dietmar Maurer
40f4e198a8
src/backup/dynamic_index.rs: add chunk_info method
2019-05-21 09:52:19 +02:00
Dietmar Maurer
ff77dbbea3
Revert "src/backup/chunk_stream.rs: simplify code"
...
This reverts commit e058744d8f
.
The optimization was wrong, and produces wrong chunk boundaries.
2019-05-19 11:05:56 +02:00
Dietmar Maurer
e058744d8f
src/backup/chunk_stream.rs: simplify code
2019-05-18 10:55:46 +02:00
Dietmar Maurer
8a7cc7565f
src/backup/chunk_stream.rs: implement fixed size chunk stream
2019-05-18 10:55:46 +02:00
Dietmar Maurer
dafc27aeff
src/backup/chunk_stream.rs: async chunk stream
2019-05-14 10:05:29 +02:00
Dietmar Maurer
457531e7fe
src/backup/index.rs: new ChunkListReader
2019-05-11 16:01:42 +02:00
Dietmar Maurer
51a4f63ff7
src/backup/datastore.rs: move last_backup to BackupInfo
2019-05-11 12:07:09 +02:00
Dietmar Maurer
b02a52e3a5
src/backup/datastore.rs: impl last_backup helper
2019-05-11 11:36:46 +02:00
Dietmar Maurer
c09775011a
src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups
2019-05-11 10:19:34 +02:00
Dietmar Maurer
35a2d8a6a6
src/backup/backup_info.rs: Derive Clone
2019-05-10 06:59:23 +02:00
Dietmar Maurer
f2b99c34f7
src/api2/admin/datastore.rs: implement API to return last GC status
2019-04-11 12:04:25 +02:00
Dietmar Maurer
d4b59ae0b8
src/backup/datastore.rs: pass WorkerTask to GC
...
So that we can log to the task log file.
2019-04-06 17:57:38 +02:00
Dietmar Maurer
92da93b245
abort GC on server shutdown
2019-04-01 12:13:02 +02:00
Dietmar Maurer
11861a482d
src/backup/chunk_store.rs: fix GC
...
Added option to get oldest_writer timestamp from ProcessLocker.
2019-03-31 17:21:36 +02:00
Dietmar Maurer
d85987aeeb
fix last commit: the filename var was not ment to be removed, sorry
2019-03-31 16:16:14 +02:00
Dietmar Maurer
15a77c4c2e
src/backup/chunk_store.rs: avoid create/unlink race
2019-03-31 10:03:01 +02:00
Dietmar Maurer
10eea49d81
src/backup/fixed_index.rs: store reference to file
...
Keep it open and locked while index is in use.
2019-03-27 10:17:03 +01:00
Dietmar Maurer
c597a92c81
index file readers: aquire shared lock
2019-03-27 10:11:14 +01:00
Dietmar Maurer
43b1303398
datastore: use new ProcessLocker
...
To make sure only one process runs garbage collection while having active writers.
2019-03-22 09:42:15 +01:00
Dietmar Maurer
515688d1f1
avoid compiler warnings
2019-03-18 10:00:58 +01:00
Dietmar Maurer
8968258b66
rename catar into pxar
...
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
Dietmar Maurer
8e39232acc
src/bin/proxmox-backup-client.rs: strip .didx file extensions
2019-03-11 10:54:02 +01:00