Commit Graph

4677 Commits

Author SHA1 Message Date
Dietmar Maurer 1c0472e88f avoid compiler warnings 2019-06-03 06:43:47 +02:00
Dietmar Maurer 36898ffce6 src/backup/chunk_stream.rs: add optional chunk_size parameter 2019-05-30 13:28:24 +02:00
Dietmar Maurer 49ef316bcd src/bin/proxmox-backup-client.rs: backup_image() - pass correct chunk size 2019-05-30 12:52:12 +02:00
Dietmar Maurer 202cadc29e src/bin/proxmox-backup-client.rs: remove unused code 2019-05-30 12:47:35 +02:00
Dietmar Maurer 6af905c18f src/bin/proxmox-backup-client.rs: implement image/block-device upload 2019-05-30 12:46:01 +02:00
Dietmar Maurer eb1804c567 src/bin/proxmox-backup-client.rs: use std lib instead of nix 2019-05-30 12:19:23 +02:00
Dietmar Maurer 6e907505aa src/bin/proxmox-backup-client.rs: fix file type test 2019-05-30 12:08:34 +02:00
Dietmar Maurer 96482891ae src/api2/admin/datastore/backup/environment.rs: log upload statistics 2019-05-30 09:21:19 +02:00
Dietmar Maurer 642322b433 src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload
So that we can provide better statistics (patches follows).
2019-05-30 09:21:19 +02:00
Wolfgang Bumiller 390e83c9b9 typo fix and doc improvement
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:57:28 +02:00
Wolfgang Bumiller 7fb49397d5 doc fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:56:14 +02:00
Wolfgang Bumiller 382609b04e tools: add Cancellable, start a futures submodule
To make a future cancellable, use:
let (future, canceller) =
    crate::tools::futures::cancellable(future);

Proceed with using `future` as usual, `canceller` is
clonable and can cancel the future via the `.cancel()`
method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:18:29 +02:00
Wolfgang Bumiller 0d32d71fb7 tools: add AsyncMutex::new_locked
Allows creating a pre-locked mutex, returning the mutex and
a guard.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:18:29 +02:00
Dietmar Maurer e6389f4e75 src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files) 2019-05-29 10:38:57 +02:00
Dietmar Maurer 39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Dietmar Maurer 4ae6551c78 src/api_schema/router.rs: disable debug println 2019-05-29 09:42:16 +02:00
Dietmar Maurer a42d1f55c3 src/api2/admin/datastore/backup.rs: add debug flag 2019-05-29 09:35:21 +02:00
Dietmar Maurer 976595e1a9 src/backup/dynamic_index.rs: split class DynamicIndexWriter 2019-05-29 08:49:57 +02:00
Dietmar Maurer cb0708dd46 src/backup/fixed_index.rs: remove ChunkStat from struct 2019-05-29 07:08:34 +02:00
Dietmar Maurer 01af11f340 src/backup/fixed_index.rs: improve consistency checks 2019-05-29 07:02:45 +02:00
Christian Ebner 9b38443314 pxar: change logic and impl all command line flags for xattrs/fcaps/acls
Allows to individually set the flags for storing/dumping/restoring of
xattrs/fcaps/acls in the cli of pxar.
Changes logic so that each of them can be threated individually.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-29 06:51:20 +02:00
Dietmar Maurer 60e589a111 src/api2/admin/datastore/backup.rs: log sucessful finish call 2019-05-28 12:55:22 +02:00
Dietmar Maurer c4ff3dcefd src/bin/proxmox-backup-client.rs: use new BackupClient to upload data 2019-05-28 10:12:44 +02:00
Dietmar Maurer 10241c20ea src/tools.rs: implement image_size helper 2019-05-28 09:35:08 +02:00
Dietmar Maurer 006f3ff407 src/api2/admin/datastore/backup/environment.rs: add more consistency checks 2019-05-28 09:21:31 +02:00
Dietmar Maurer 3dc5b2a203 src/client/http_client.rs: use start of chunk as offset 2019-05-28 09:21:31 +02:00
Dietmar Maurer 8de20e5c1f src/client/http_client.rs: code cleanup 2019-05-28 08:53:27 +02:00
Dietmar Maurer fc14b84937 src/backup/fixed_index.rs: improve bound checks inside add_digest 2019-05-28 08:47:03 +02:00
Dietmar Maurer e1225de467 src/backup/fixed_index.rs: store index_length to avoid code duplication 2019-05-28 08:27:57 +02:00
Dietmar Maurer e3d525feb7 src/api2/admin/datastore/backup.rs: fix typo 2019-05-28 07:13:19 +02:00
Christian Ebner af309d4d5c src/bin/pxar.rs: set archive permissions to 0o640
As the archive can contain potentially sensitive data such as key files, it makes
sense to restrict the permissions.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-28 06:24:47 +02:00
Christian Ebner ab87f167f1 src/pxar/encoder.rs: Refactor file stat
Introduce helper functions to check file stats

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-28 06:23:49 +02:00
Christian Ebner 578b601159 src/pxar/encoder.rs: Don't bail if endpoint does not support xattrs.
The encoder bailed if a endpoint which did not support xattrs was encountered.
Instead of bailing, we ignore these errors and simply do not store xattrs for
such endpoints.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-28 06:22:16 +02:00
Dietmar Maurer a42fa400ee src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api 2019-05-28 06:18:55 +02:00
Dietmar Maurer 29ae5c86a2 src/backup/fixed_index.rs: make chunk_size public 2019-05-28 06:18:24 +02:00
Dietmar Maurer 7dd1bcac58 src/client/http_client.rs: correctly parse error response in download_chunk_list 2019-05-27 09:35:30 +02:00
Dietmar Maurer a1e7cff3ea src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk 2019-05-27 07:57:43 +02:00
Dietmar Maurer 771953f9eb src/api2/admin/datastore/backup.rs: use smaller window sizes 2019-05-27 07:53:40 +02:00
Dietmar Maurer 174ad378d8 src/client/http_client.rs: fix request pipelining 2019-05-27 07:24:32 +02:00
Dietmar Maurer 624362226e src/client/merge_known_chunks.rs: use MergedChunkInfo as input 2019-05-26 10:52:56 +02:00
Dietmar Maurer 05cba08c9c src/client/http_client.rs: append chunks in correct order
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
2019-05-26 09:57:34 +02:00
Dietmar Maurer 417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset
This clearly shows that the current approach does not work - seems we get
chunks out of order.
2019-05-26 08:49:29 +02:00
Dietmar Maurer 34114e2606 src/server/rest.rs: correctly verify json parameters 2019-05-24 09:10:59 +02:00
Dietmar Maurer eb2bdd1bcc src/client/http_client.rs: encode parameters inside body
To avoid problems with long uris.
2019-05-24 08:32:55 +02:00
Dietmar Maurer 9bb675ec23 src/client/merge_known_chunks.rs: reduce RAM usage
There is no need to store chunk data for known chunks.
2019-05-24 07:42:58 +02:00
Dietmar Maurer ecb21b4794 src/client/merge_known_chunks.rs: avoid VecDequeu - a single buffer is enough 2019-05-24 07:36:09 +02:00
Christian Ebner 2dbba78b98 src/pxar/sequential_decoder.rs: impl support to dump/restore ACLs from pxar archives
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:27:24 +02:00
Christian Ebner 6a19448004 src/pxar/encoder.rs: impl support to store ACLs in pxar achives
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:21:33 +02:00
Christian Ebner 7cfaade7c8 src/pxar/format_definition.rs: define types and structs needed to impl ACL support in pxar
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:16:55 +02:00
Christian Ebner b344461b33 pxar: pass feature_flags to encoder/decoder instead of individual flags
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:11:36 +02:00