da78b90f9c
backup: remove AsyncIndexReader
...
superseded by CachedChunkReader, with less code and more speed
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:46 +02:00
70a152deb7
backup: add CachedChunkReader utilizing AsyncLruCache
...
Provides a fast arbitrary read implementation with full async and
concurrency support.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-06-08 09:42:37 +02:00
5e42d38598
api2/types: add TAPE_RESTORE_SNAPSHOT_SCHEMA
...
which is 'store:type/id/time'
needed to refactor SNAPSHOT_PATH_REGEX_STR from backup_info
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-05-07 12:27:50 +02:00
d1d74c4367
typo fixes all over the place
...
found and semi-manually replaced by using:
codespell -L mut -L crate -i 3 -w
Mostly in comments, but also email notification and two occurrences
of misspelled 'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 16:39:57 +01:00
2260f065d4
cleanup: use extra file for StoreProgress
2020-12-01 06:34:33 +01:00
b64e9a97f3
rustdoc: overhaul backup rustdoc and add locking table
...
Rewrite most of the documentation to be more readable and correct
(according to the current implementations).
Add a table visualizing all different locks used to synchronize
concurrent operations.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-16 09:38:01 +02:00
a6ed5e1273
backup: add BACKUP_GROUP_NAME const and backup_group helper
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-26 10:27:47 +02:00
faa8e6948a
backup: Fix typos and grammar
...
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-07-21 12:51:41 +02:00
c2009e5309
src/api2/admin/datastore.rs: add verify api
2020-06-24 13:35:21 +02:00
4a3adc3de8
add AsyncIndexReader
...
implements AsyncRead as well as Stream for an IndexFile and a store
that implements AsyncReadChunk
we can use this to asyncread or stream the content of a FixedIndex or
DynamicIndex
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:32:33 +02:00
f7d4e4b506
switch from failure to anyhow
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-17 18:43:30 +02:00
f74a03da1f
remove tools::getpwnam_ugid, impl. crate::backup::backup_user()
...
And use new nix::unistd::User struct.
2019-12-19 10:20:13 +01:00
cca67dc0c3
src/backup/readline.rs: removed in favor of new readline impl
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:03:04 +01:00
dc18849156
src/backup/prune.rs: moved prune related code into extra file
2019-12-06 08:06:21 +01:00
f14c96ea38
src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-22 07:54:04 +01:00
89245fb530
renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs
2019-11-08 10:41:00 +01:00
bf6e321744
change catalog format, use dynamic index to store catalog.
...
In order to remove size restriction of a single blob.
2019-11-08 10:35:48 +01:00
59e9ba01c6
src/backup/manifest.rs: new class to generate/parse index.json
2019-10-12 17:58:08 +02:00
ad6e5a6f51
rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME
2019-10-12 15:53:27 +02:00
4ee8f53d07
remove DataChunk file format - use DataBlob instead
2019-10-06 10:31:06 +02:00
d2267b112d
src/bin/proxmox-backup-client.rs: verify blob/catlog checksums
2019-09-04 08:47:14 +02:00
36493d4d7e
src/backup.rs: define INDEX_BLOB_NAME here
2019-09-03 13:15:44 +02:00
7d83440c60
remove proxmox-protocol subcrate
...
AFAICT we have no use for it anymore, its api entry points
are gone. If we do end up needing something from it, it's
still in the git history anyway. (And about two thirds of it
can be made much less awkward by utilizing async-await
anyway, so no love lost there...)
Moved the chunker back into src/backup/chunker.rs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 14:08:25 +02:00
9d135fe617
src/backup/catalog_blob.rs: moved catalog impl. from pxar
...
And avoid loading catalog into memory.
2019-08-16 12:27:17 +02:00
39a4df61d6
improve docs
2019-08-14 14:08:27 +02:00
018d11bb84
src/backup/data_blob.rs: move parts into single files
2019-08-14 13:24:41 +02:00
b850673634
src/backup/read_chunk.rs: move read chunk trait into extra file
...
And implement LocalChunkReader.
2019-07-02 08:22:29 +02:00
dd066d28e2
src/api2/reader.rs: implement backup reader protocol
2019-06-27 09:01:41 +02:00
991abfa8b4
src/backup/file_formats.rs: split out file format data
2019-06-22 09:12:25 +02:00
3025b3a53c
src/backup/data_blob.rs: new file format for binary blobs
2019-06-21 11:32:07 +02:00
c38266c18d
renamed: src/backup/crypt_setup.rs -> src/backup/crypt_config.rs
2019-06-21 09:51:18 +02:00
826f309bf5
src/backup/key_derivation.rs: move kdf code into separate file
2019-06-18 11:17:22 +02:00
a7dd483097
src/backup/*_index.rs: used generated magic numbers
2019-06-14 14:58:37 +02:00
b595cb9d2c
src/backup/data_chunk.rs: Data Chunk builder
...
Make handling encoded chunks easier.
2019-06-12 13:37:01 +02:00
74792b95b2
src/backup/crypt_setup.rs: allow compressed and uncompressed chunks
2019-06-10 08:27:35 +02:00
c20b17b460
src/backup.rs: define magic numbers for chunk store file format
2019-06-08 10:34:20 +02:00
48b4b40b57
src/backup/crypt_setup.rs: crypto helpers
2019-06-08 09:56:51 +02:00
986bef16be
src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
...
So that we can include it in static doc strings.
2019-06-05 08:41:20 +02:00
c9ec0956cf
src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1
2019-06-05 08:12:13 +02:00
dafc27aeff
src/backup/chunk_stream.rs: async chunk stream
2019-05-14 10:05:29 +02:00
c374f05499
src/backup.rs: improve GC problem description
2019-03-31 09:44:35 +02:00
c8ec450e37
src/backup.rs: start explaining different GC algorithm
2019-03-30 17:21:40 +01:00
8a4757342c
src/backup.rs: describe the garbage collection problem
2019-03-30 16:26:52 +01:00
0465218953
src/backup.rs: add documentation about ChunkStore locking
2019-03-22 10:14:50 +01:00
06178f13fa
move chunker into proxmox_protocol
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-08 09:29:47 +01:00
b3483782d1
src/backup/backup_info.rs: move code into separate file
...
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
7bc1d72778
add IndexFile trait
...
We want to be able to iterate through digests of index
files, but without always having to distinguish between
dynamic and fixed types, so add a trait we can use as a
trait object.
Unfortunately the iterator needs to yield copies as
iterators cannot yield values with life times when
represented as trait objects (Box<dyn Iterator<Item = ?>>)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
7e3365554e
src/backup/chunk_stat.rs: new struct to track chunk statistics
2019-02-25 12:52:10 +01:00
e5064ba607
simplify backup lib structure (pub use xxx:*), improve doc
2019-02-12 14:13:31 +01:00
d78345bcfc
src/backup.rs - improve doc
2019-02-12 13:27:11 +01:00
Stefan Reiter
Dominik Csapak
Thomas Lamprecht
Dietmar Maurer
Aaron Lauterer
Wolfgang Bumiller
Christian Ebner