Commit Graph

239 Commits

Author SHA1 Message Date
Dietmar Maurer e016f9ff2f src/client/backup_writer.rs: rename force_close to cancel
And do not consume self.
2019-11-08 10:34:16 +01:00
Dietmar Maurer 62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer 11377a47bb avoid some clippy warnings 2019-10-25 18:04:37 +02:00
Christian Ebner 62d123e50a pxar: add functionality to pass exclude MatchPatterns on create
This exposes the option to pass a list of exclude MatchPattern via the
'--exclude' option.
The list is encoded as file '.pxarexclude-cli' in the archives root directory.
If such a file is present in the filesystem, it is skipped and not included in
the archive in order to avoid conflicting information.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-10-23 07:31:49 +02:00
Dietmar Maurer f06b820ac0 src/backup/manifest.rs: add verify_file 2019-10-13 10:09:12 +02:00
Dietmar Maurer ad6e5a6f51 rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME 2019-10-12 15:53:27 +02:00
Dietmar Maurer 296c50ba3d src/client/backup_reader.rs: add download_manifest
Also add crypt_config as member variable.
2019-10-12 15:50:26 +02:00
Dietmar Maurer cf9271e27c src/client/backup_writer.rs: move BackupClient code into extra file
And rename it to BackupWriter.
2019-10-12 13:53:11 +02:00
Dietmar Maurer 913acb41fd src/client/backup_reader.rs: add documentation 2019-10-12 13:26:47 +02:00
Dietmar Maurer 9e490a7479 src/client/backup_reader.rs: split BackupReader code into separate file 2019-10-12 12:57:08 +02:00
Dietmar Maurer 7123ff7d43 src/backup/data_blob.rs: avoid Arc<CryptConfig>
use simple reference instead.
2019-10-07 11:41:35 +02:00
Dietmar Maurer 4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
Dietmar Maurer 62e05bb3b3 src/client/remote_chunk_reader.rs: pass reference to download_chunk
Make sure we do not move data around.
2019-10-05 11:45:20 +02:00
Dietmar Maurer ff01c1e393 src/client/http_client.rs: store raw data/blob size in index.json 2019-10-02 10:54:06 +02:00
Dietmar Maurer f16aea687a src/client/http_client.rs: new helper send_upload_request() 2019-09-28 18:22:48 +02:00
Dietmar Maurer 9e603e254c src/backup/fixed_index.rs: do not use offset for checksum 2019-09-23 11:30:20 +02:00
Dietmar Maurer fb6026b66b backup writer api: verify checksum inside close 2019-09-23 10:56:53 +02:00
Dietmar Maurer c8c4051aa7 src/client/http_client.rs - H2Client::upload: fix parameter order 2019-09-20 13:19:55 +02:00
Dietmar Maurer f011dba0f7 src/client/http_client.rs: add method parameter to H2 upload 2019-09-20 12:40:23 +02:00
Dietmar Maurer 792a70b9e1 src/client/http_client.rs: add content type parameter to H2 upload/requets_builder 2019-09-20 12:26:21 +02:00
Dietmar Maurer a55b2975db src/client/http_client.rs: cleanup (rename parameter) 2019-09-20 11:57:03 +02:00
Dietmar Maurer 15bb1bba89 BackupClient: add upload() function 2019-09-19 13:59:24 +02:00
Dietmar Maurer 6d4df36c3c make download_chunk_list() public 2019-09-19 13:17:23 +02:00
Dietmar Maurer cc2ce4a92f src/client/http_client.rs: add password parameter to HttpClient::new() 2019-09-16 12:35:23 +02:00
Wolfgang Bumiller 34a3845b01 clippy: don't use clone a double reference
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 14:02:30 +02:00
Wolfgang Bumiller bd430c225b clippy: remove a loop{} which never actually loops
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:31:39 +02:00
Wolfgang Bumiller 44fed91e17 more clippy fixups
mostly indentation changes, view with `-w`

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:23:12 +02:00
Dietmar Maurer 54a5a885a3 src/client/http_client.rs: use async for upload_speedtest() 2019-09-06 08:55:47 +02:00
Dietmar Maurer 2a05048b93 src/client/http_client.rs: simplify send_request 2019-09-05 15:07:37 +02:00
Dietmar Maurer 9edd3bf1b8 src/client/http_client.rs: use async for h2api_response() 2019-09-05 14:56:52 +02:00
Dietmar Maurer c18fddf80f src/client/http_client.rs - download_chunk_list: simplify async code 2019-09-05 14:16:45 +02:00
Dietmar Maurer 44f59dc731 src/client/http_client.rs - H2Client: fix error message in download() 2019-09-05 13:26:06 +02:00
Dietmar Maurer c2a5a9f353 src/client/http_client.rs: use async for download_chunk_list 2019-09-05 13:12:03 +02:00
Dietmar Maurer 2f831baec0 src/client/http_client.rs - BackupClient: use async 2019-09-05 12:55:22 +02:00
Dietmar Maurer 2a1e6d7dea src/client/http_client.rs - BackupClient: use async 2019-09-05 11:42:12 +02:00
Dietmar Maurer d4a085e564 src/client/http_client.rs - BackupReader: use async 2019-09-05 11:42:05 +02:00
Dietmar Maurer 1a7a0e74c0 src/client/http_client.rs: use async for more functions 2019-09-04 13:48:16 +02:00
Dietmar Maurer 96f5e80abb src/client/http_client.rs: use async for login() 2019-09-04 10:01:46 +02:00
Dietmar Maurer 9d35dbbb8f src/client/http_client.rs: use async for credentials 2019-09-04 09:57:29 +02:00
Dietmar Maurer a3e032b754 src/client/http_client.rs: fix dynamic index checksum 2019-09-03 13:10:18 +02:00
Wolfgang Bumiller 1434f4f8df drop hyper_openssl
We can use .await now, which means the whole connection
state machine doesn't need to be typed out as "types"
anymore, so, at least until hyper_openssl gets updated to
proper dependencies, let's drop it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 3c0facc787 src/client/remote_chunk_reader.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 369a87e3a2 src/client/pxar_backup_stream.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 2107bb40c1 src/client/pipe_to_stream.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 5b3911995b src/client/merge_known_chunks.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller a6782ca10b src/client/http_client.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Christian Ebner 66c8eb9383 src/client/http_client.rs: Refactor handling Option and Result types
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-29 09:15:16 +02:00
Wolfgang Bumiller 7a57cb77e1 more formatting & use statement fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 13:50:27 +02:00
Dietmar Maurer 9d135fe617 src/backup/catalog_blob.rs: moved catalog impl. from pxar
And avoid loading catalog into memory.
2019-08-16 12:27:17 +02:00
Wolfgang Bumiller 9025312aa6 avoid lifetimes in blob reader/writer 2019-08-16 10:07:24 +02:00
Dietmar Maurer e240d8be0b src/bin/proxmox-backup-client.rs: implement login/logout 2019-08-10 09:12:17 +02:00
Dietmar Maurer 2761d6a4f3 src/pxar/encoder.rs: use BackupCatalogWriter 2019-08-09 09:46:49 +02:00
Dietmar Maurer dbed4c8cd7 src/client/http_client.rs: compute checksums for chunk streams 2019-08-06 12:23:06 +02:00
Dietmar Maurer bd15e96dd9 src/client/http_client.rs: minor code cleanup 2019-08-06 11:46:15 +02:00
Dietmar Maurer c807d231eb src/client/http_client.rs: compute file checksums 2019-08-06 11:26:06 +02:00
Dietmar Maurer e18a6c9ee5 update to nix 0.14, use code from proxmox:tools 2019-08-03 13:05:38 +02:00
Christian Ebner 47651f9530 pxar: cleanup: move feature flags to src/pxar/flags.rs and omit CA_FORMAT prefix on all of them
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-03 08:51:06 +02:00
Dietmar Maurer b335f5b713 src/bin/proxmox-backup-client.rs: sign index.json 2019-08-02 10:00:15 +02:00
Dietmar Maurer 2c3891d1c3 src/bin/proxmox-backup-client.rs: upload backup index.json
The plan is to use this file to verify the backup content.
2019-08-01 12:39:02 +02:00
Dietmar Maurer ca5d0b61ca src/api2/backup.rs: new required backup-time parameter
The client should pass the time as parameter.
2019-07-25 13:44:01 +02:00
Dietmar Maurer 04512d3068 src/client/http_client.rs: allow upload with parameters 2019-07-25 12:17:35 +02:00
Dietmar Maurer 5b72c9b4f7 src/pxar/encoder.rs: add new skip_lost_and_found parameter 2019-07-24 12:21:25 +02:00
Dietmar Maurer 6c3c9bceb5 pxar_backup_stream.rs: limit lock scope to avoid blocking forever 2019-07-24 11:30:43 +02:00
Dietmar Maurer 684233aa3b remove debug output 2019-07-24 09:33:52 +02:00
Dietmar Maurer 5be106eeae src/client/pxar_backup_stream.rs: correctly pass errors to stream 2019-07-24 09:24:35 +02:00
Dietmar Maurer 2eeaacb974 src/pxar/encoder.rs: allow to pass list of devices
For better mount point include control...
2019-07-24 08:11:59 +02:00
Dietmar Maurer fa5d6977dd Revert previous, commit, use UTC RFC3339 without timezone (Z)
We now have human readable data/time, and names are still sortable.
2019-07-22 11:49:30 +02:00
Christian Ebner fa7e957ca3 pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-17 12:19:31 +02:00
Dietmar Maurer f4bf7dfcc7 src/client/remote_chunk_reader.rs: implement simple caching 2019-07-05 10:42:46 +02:00
Christian Ebner 7dcbe051e9 pxar: Refactor SequentialDecoder to store the callback function within the struct
Reduces the number of arguments for the function calls within the decoder.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-05 06:38:22 +02:00
Dietmar Maurer 9cc88a7c00 src/client/http_client.rs - finish: hold reference to avoid early connection close 2019-07-04 15:11:23 +02:00
Dietmar Maurer 99168f43e6 src/client/http_client.rs: s/set_recv_buf_size/set_recv_buffer_size/ 2019-07-04 10:51:18 +02:00
Dietmar Maurer b2a6897982 src/client/remote_chunk_reader.rs: remove dead code 2019-07-03 14:56:24 +02:00
Dietmar Maurer 7f99bf691a src/client/remote_chunk_reader.rs: implement remote chunk reader 2019-07-03 14:39:13 +02:00
Dietmar Maurer 4f6aaf542c src/client/http_client.rs: avoid compiler warning 2019-07-03 14:36:02 +02:00
Dietmar Maurer 3467cd91e9 src/client/http_client.rs: return Arc<Self>
Avoid to call canceller for each clone on drop ...
2019-07-03 14:26:07 +02:00
Dietmar Maurer 6d1f61b208 use hyper/tokio-openssl instead of hyper/tokio-tls
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
2019-07-02 13:36:28 +02:00
Dietmar Maurer 8fad30a4b1 use latest hyper master branch to include socket buffer size fix 2019-07-02 07:09:46 +02:00
Dietmar Maurer 17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
Dietmar Maurer a2b29b68d4 src/client/http_client.rs: use maximal h2 window sizes 2019-06-29 11:05:36 +02:00
Dietmar Maurer fcf5dea56d src/client/http_client.rs: increase h2 window size, set tcp nodelay option
To increase h2 download speed.
2019-06-28 16:00:58 +02:00
Dietmar Maurer 984a7c3502 src/client/http_client.rs - h2 download: implement flow control 2019-06-28 07:02:43 +02:00
Dietmar Maurer dd066d28e2 src/api2/reader.rs: implement backup reader protocol 2019-06-27 09:01:41 +02:00
Dietmar Maurer fb0470837b src/client/http_client.rs: split out code to start new h2 connections 2019-06-26 12:09:18 +02:00
Dietmar Maurer c2b945341c src/client/http_client.rs - download: use generic Write type, return writer.
Make it possible to write int Vec<u8>.
2019-06-26 09:17:13 +02:00
Dietmar Maurer 81820b0d4d src/client/http_client.rs - download: fix error handling 2019-06-25 12:43:55 +02:00
Dietmar Maurer 9f46c7de4b src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup 2019-06-24 13:56:37 +02:00
Dietmar Maurer cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer bffd40d6b7 src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools 2019-06-14 11:40:04 +02:00
Dietmar Maurer f98ac774ee backup: Add support for client side encryption
first try ...
2019-06-13 11:47:23 +02:00
Dietmar Maurer bd0e3c7cfa src/backup/data_chunk.rs: Moved ChunkInfo from merge_known_chunks.rs 2019-06-13 07:35:02 +02:00
Wolfgang Bumiller dd5495d6dc tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
Dietmar Maurer 4247fccb0f src/client/http_client.rs: automatically close connection in finish 2019-06-05 11:41:50 +02:00
Wolfgang Bumiller cb4426b348 make h2 client connection cancellable
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-05 09:45:59 +02:00
Dietmar Maurer 986bef16be src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
So that we can include it in static doc strings.
2019-06-05 08:41:20 +02:00
Dietmar Maurer c9ec0956cf src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1 2019-06-05 08:12:13 +02:00
Dietmar Maurer 7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Dietmar Maurer ef39bf95b3 src/client/http_client.rs - upload_config: improve error message 2019-06-03 10:53:09 +02:00
Dietmar Maurer ec8a9bb953 src/bin/proxmox-backup-client.rs: implement upload_config 2019-06-03 10:39:44 +02:00
Dietmar Maurer 39d6846e1e src/api2/admin/datastore/backup.rs: implement config file upload 2019-06-03 09:45:14 +02:00