Commit Graph

225 Commits

Author SHA1 Message Date
Dietmar Maurer 5be106eeae src/client/pxar_backup_stream.rs: correctly pass errors to stream 2019-07-24 09:24:35 +02:00
Dietmar Maurer 2eeaacb974 src/pxar/encoder.rs: allow to pass list of devices
For better mount point include control...
2019-07-24 08:11:59 +02:00
Dietmar Maurer fa5d6977dd Revert previous, commit, use UTC RFC3339 without timezone (Z)
We now have human readable data/time, and names are still sortable.
2019-07-22 11:49:30 +02:00
Christian Ebner fa7e957ca3 pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-17 12:19:31 +02:00
Dietmar Maurer f4bf7dfcc7 src/client/remote_chunk_reader.rs: implement simple caching 2019-07-05 10:42:46 +02:00
Christian Ebner 7dcbe051e9 pxar: Refactor SequentialDecoder to store the callback function within the struct
Reduces the number of arguments for the function calls within the decoder.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-05 06:38:22 +02:00
Dietmar Maurer 9cc88a7c00 src/client/http_client.rs - finish: hold reference to avoid early connection close 2019-07-04 15:11:23 +02:00
Dietmar Maurer 99168f43e6 src/client/http_client.rs: s/set_recv_buf_size/set_recv_buffer_size/ 2019-07-04 10:51:18 +02:00
Dietmar Maurer b2a6897982 src/client/remote_chunk_reader.rs: remove dead code 2019-07-03 14:56:24 +02:00
Dietmar Maurer 7f99bf691a src/client/remote_chunk_reader.rs: implement remote chunk reader 2019-07-03 14:39:13 +02:00
Dietmar Maurer 4f6aaf542c src/client/http_client.rs: avoid compiler warning 2019-07-03 14:36:02 +02:00
Dietmar Maurer 3467cd91e9 src/client/http_client.rs: return Arc<Self>
Avoid to call canceller for each clone on drop ...
2019-07-03 14:26:07 +02:00
Dietmar Maurer 6d1f61b208 use hyper/tokio-openssl instead of hyper/tokio-tls
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
2019-07-02 13:36:28 +02:00
Dietmar Maurer 8fad30a4b1 use latest hyper master branch to include socket buffer size fix 2019-07-02 07:09:46 +02:00
Dietmar Maurer 17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
Dietmar Maurer a2b29b68d4 src/client/http_client.rs: use maximal h2 window sizes 2019-06-29 11:05:36 +02:00
Dietmar Maurer fcf5dea56d src/client/http_client.rs: increase h2 window size, set tcp nodelay option
To increase h2 download speed.
2019-06-28 16:00:58 +02:00
Dietmar Maurer 984a7c3502 src/client/http_client.rs - h2 download: implement flow control 2019-06-28 07:02:43 +02:00
Dietmar Maurer dd066d28e2 src/api2/reader.rs: implement backup reader protocol 2019-06-27 09:01:41 +02:00
Dietmar Maurer fb0470837b src/client/http_client.rs: split out code to start new h2 connections 2019-06-26 12:09:18 +02:00
Dietmar Maurer c2b945341c src/client/http_client.rs - download: use generic Write type, return writer.
Make it possible to write int Vec<u8>.
2019-06-26 09:17:13 +02:00
Dietmar Maurer 81820b0d4d src/client/http_client.rs - download: fix error handling 2019-06-25 12:43:55 +02:00
Dietmar Maurer 9f46c7de4b src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup 2019-06-24 13:56:37 +02:00
Dietmar Maurer cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer bffd40d6b7 src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools 2019-06-14 11:40:04 +02:00
Dietmar Maurer f98ac774ee backup: Add support for client side encryption
first try ...
2019-06-13 11:47:23 +02:00
Dietmar Maurer bd0e3c7cfa src/backup/data_chunk.rs: Moved ChunkInfo from merge_known_chunks.rs 2019-06-13 07:35:02 +02:00
Wolfgang Bumiller dd5495d6dc tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
Dietmar Maurer 4247fccb0f src/client/http_client.rs: automatically close connection in finish 2019-06-05 11:41:50 +02:00
Wolfgang Bumiller cb4426b348 make h2 client connection cancellable
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-05 09:45:59 +02:00
Dietmar Maurer 986bef16be src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
So that we can include it in static doc strings.
2019-06-05 08:41:20 +02:00
Dietmar Maurer c9ec0956cf src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1 2019-06-05 08:12:13 +02:00
Dietmar Maurer 7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Dietmar Maurer ef39bf95b3 src/client/http_client.rs - upload_config: improve error message 2019-06-03 10:53:09 +02:00
Dietmar Maurer ec8a9bb953 src/bin/proxmox-backup-client.rs: implement upload_config 2019-06-03 10:39:44 +02:00
Dietmar Maurer 39d6846e1e src/api2/admin/datastore/backup.rs: implement config file upload 2019-06-03 09:45:14 +02:00
Dietmar Maurer 642322b433 src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload
So that we can provide better statistics (patches follows).
2019-05-30 09:21:19 +02:00
Dietmar Maurer 39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Dietmar Maurer 3dc5b2a203 src/client/http_client.rs: use start of chunk as offset 2019-05-28 09:21:31 +02:00
Dietmar Maurer 8de20e5c1f src/client/http_client.rs: code cleanup 2019-05-28 08:53:27 +02:00
Dietmar Maurer a42fa400ee src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api 2019-05-28 06:18:55 +02:00
Dietmar Maurer 7dd1bcac58 src/client/http_client.rs: correctly parse error response in download_chunk_list 2019-05-27 09:35:30 +02:00
Dietmar Maurer a1e7cff3ea src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk 2019-05-27 07:57:43 +02:00
Dietmar Maurer 771953f9eb src/api2/admin/datastore/backup.rs: use smaller window sizes 2019-05-27 07:53:40 +02:00
Dietmar Maurer 174ad378d8 src/client/http_client.rs: fix request pipelining 2019-05-27 07:24:32 +02:00
Dietmar Maurer 624362226e src/client/merge_known_chunks.rs: use MergedChunkInfo as input 2019-05-26 10:52:56 +02:00
Dietmar Maurer 05cba08c9c src/client/http_client.rs: append chunks in correct order
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
2019-05-26 09:57:34 +02:00
Dietmar Maurer 417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset
This clearly shows that the current approach does not work - seems we get
chunks out of order.
2019-05-26 08:49:29 +02:00
Dietmar Maurer eb2bdd1bcc src/client/http_client.rs: encode parameters inside body
To avoid problems with long uris.
2019-05-24 08:32:55 +02:00
Dietmar Maurer 9bb675ec23 src/client/merge_known_chunks.rs: reduce RAM usage
There is no need to store chunk data for known chunks.
2019-05-24 07:42:58 +02:00
Dietmar Maurer ecb21b4794 src/client/merge_known_chunks.rs: avoid VecDequeu - a single buffer is enough 2019-05-24 07:36:09 +02:00
Christian Ebner b344461b33 pxar: pass feature_flags to encoder/decoder instead of individual flags
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:11:36 +02:00
Dietmar Maurer aa1b2e04fe src/client/merge_known_chunks.rs: merge known chunks
To decrease the number of api calls required...
2019-05-23 12:31:09 +02:00
Dietmar Maurer 91320f0879 src/client/http_client.rs: use ChunkInfo streams
This will make out of order uploads possible...
2019-05-23 12:31:09 +02:00
Dietmar Maurer 8bea85b42e src/api2/admin/datastore/backup.rs: verify file size and chunk count on close 2019-05-23 08:50:36 +02:00
Christian Ebner 4add706092 src/client/: pass dummy flags for xattrs and fcaps to pxar encoder/decoder.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-23 06:40:32 +02:00
Dietmar Maurer d6f204eda6 src/client/http_client.rs: re-order code - public functions first 2019-05-23 06:28:10 +02:00
Dietmar Maurer 9af37c8f0e src/client/http_client.rs: move low level H2 code into separate class 2019-05-22 17:28:25 +02:00
Dietmar Maurer d2c48afc6e src/client/http_client.rs: code cleanup 2019-05-22 13:24:33 +02:00
Dietmar Maurer 6ab34afa88 src/client/http_client.rs: cleanup h2 backup client 2019-05-22 13:05:51 +02:00
Dietmar Maurer 553610b43e src/client/http_client.rs: implement download_chunk_list 2019-05-22 09:46:02 +02:00
Dietmar Maurer 97f03eff13 src/client/pxar_backup_stream.rs: fix docu 2019-05-20 17:29:00 +02:00
Dietmar Maurer 82ab72304e src/client/http_client.rs: implement upload_stream 2019-05-20 14:19:24 +02:00
Dietmar Maurer 2698e8a514 src/client/pxar_backup_stream.rs: close pipe inside drop 2019-05-20 11:20:33 +02:00
Dietmar Maurer 0c516b123d src/client/pxar_backup_stream.rs: use WrappedStreamReader
to make it fully async ...
2019-05-18 15:13:31 +02:00
Dietmar Maurer c6e28b66c6 src/client/pxar_backup_stream.rs: use 1M pipe buffer size 2019-05-18 12:57:43 +02:00
Dietmar Maurer 9c9ad941c0 src/client/http_client.rs: improve speed test
pipeline responses to a separate async channel ...
2019-05-18 10:55:46 +02:00
Dietmar Maurer adec8ea263 src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
2019-05-16 10:24:23 +02:00
Dietmar Maurer 25ddc15fe0 src/client/pipe_to_stream.rs: add missing file 2019-05-15 09:07:19 +02:00
Dietmar Maurer e3dbd41b20 src/client/http_client.rs: add flow control for h2 upload 2019-05-14 14:54:21 +02:00
Dietmar Maurer 97f22ce57d src/client/http_client.rs: implement h2 upload
without flow control so far ...
2019-05-14 12:58:59 +02:00
Dietmar Maurer c0e22141ed src/client/http_client.rs: derive Clone for H2Client 2019-05-14 09:00:24 +02:00
Dietmar Maurer 850ac6d0f0 src/client/http_client.rs: directly return H2Client on upgrade 2019-05-13 16:44:59 +02:00
Dietmar Maurer 10130cf48c src/client/http_client.rs: cleanup, remove debug println 2019-05-13 12:11:18 +02:00
Dietmar Maurer b57cb26406 src/client/http_client.rs: implement http2 client wrapper 2019-05-13 10:27:22 +02:00
Dietmar Maurer 9e391bb7f0 src/client/http_client.rs: allow to pass parameters (encode them as query string) 2019-05-13 09:12:03 +02:00
Dietmar Maurer 45cdce069e src/client/http_client.rs: request password at creation time 2019-04-30 11:44:35 +02:00
Dietmar Maurer cf639a47de rc/client/http_client.rs: add experimental h2 upgrade code 2019-04-29 11:57:58 +02:00
Dietmar Maurer 5a2df00004 src/client/http_client.rs: cleanup, make login fully async 2019-04-28 10:55:03 +02:00
Dietmar Maurer 49cf9f3d0b src/tools.rs: file_get_json() - add new default parameter 2019-04-04 12:24:18 +02:00
Dietmar Maurer 3dbfe5b142 simplify pxar module structure 2019-03-15 08:24:32 +01:00
Dietmar Maurer 02c93163cf src/pxar/decoder.rs: correctly call callback() 2019-03-15 08:02:04 +01:00
Dietmar Maurer 5defa71bf9 rename PxarBackupWriter into PxarDecodeWriter 2019-03-15 07:20:22 +01:00
Dietmar Maurer b005ed12d5 src/client/http_client.rs: avoid compiler warning 2019-03-15 07:16:26 +01:00
Dietmar Maurer 8968258b66 rename catar into pxar
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
Dietmar Maurer 024f11bb55 src/bin/proxmox-backup-client.rs: implement completion for backup groups 2019-03-13 11:56:37 +01:00
Dietmar Maurer edd3c8c605 src/client/backup_repo.rs: implement FromStr trait 2019-03-13 10:09:39 +01:00
Dietmar Maurer 874acb7039 src/client/backup_repo.rs: implement Display trait 2019-03-13 09:57:36 +01:00
Dietmar Maurer d0a03d40ce src/bin/proxmox-backup-client.rs: implement repository completions 2019-03-13 09:47:12 +01:00
Dietmar Maurer d5c34d98c0 src/catar/decoder.rs: simplify public restore API 2019-03-11 14:31:01 +01:00
Dietmar Maurer 656b23e107 src/catar/decoder.rs: dir_mkdirat - add create_new flag 2019-03-11 07:27:45 +01:00
Dietmar Maurer e9c9409a99 src/catar/decoder.rs: implement real sequential decoder (no Seek)
I copied the old code to src/catar/inspector.rs. Will modify
that later to implement a random access decoder ...
2019-03-08 16:55:54 +01:00
Dietmar Maurer eed6db3923 src/bin/proxmox-backup-client.rs: impl option all-file-systems 2019-03-08 09:36:40 +01:00
Dietmar Maurer 79c281fdac src/catar/encoder.rs: use simply flag "all_file_systems" 2019-03-08 08:14:26 +01:00
Dietmar Maurer a6b7551311 src/client/http_client.rs: impl download 2019-03-06 11:02:08 +01:00
Dietmar Maurer 21ea0158b4 src/client/http_client.rs: remove debug statement 2019-03-06 06:45:51 +01:00
Dietmar Maurer ba3a60b2a4 src/client/http_client.rs: store/load ticket in xdg runtime dir, depend on crate xdg 2019-03-05 12:56:21 +01:00
Dietmar Maurer 219ef0e6ca src/catar/encoder.rs: add verbose flags 2019-03-04 08:01:09 +01:00
Dietmar Maurer 6f62c9240a src/api2/admin/datastore.rs: imp delete_snapshot 2019-03-03 11:29:00 +01:00
Dietmar Maurer 184f17afad src/api2/admin/datastore.rs: impl list_snapshots 2019-03-02 16:20:50 +01:00
Dietmar Maurer 17d6979afd src/bin/proxmox-backup-client.rs: generate a log file 2019-03-01 09:35:41 +01:00
Dietmar Maurer 51144821f9 src/bin/proxmox-backup-client.rs: login before starting backup 2019-03-01 06:48:41 +01:00
Wolfgang Bumiller df67070b1c client/backup_repo: allow ip addresses for BackupRepository
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
Dietmar Maurer 0ffbccce5e rc/server/rest.rs: allow to pass parameters as application/json 2019-02-27 12:37:53 +01:00
Dietmar Maurer a477d6889a src/client/http_client.rs: login once, store and reuse ticket/token 2019-02-26 11:59:10 +01:00
Wolfgang Bumiller 56458d9764 http_client: on a tty, read password if no env var is set
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 15:03:34 +01:00
Dietmar Maurer 81da38c143 src/client/http_client.rs: implement post
without parameters for now...
2019-02-20 14:09:55 +01:00
Dietmar Maurer a4a5c78c01 src/client/http_client.rs: send csrf token 2019-02-18 16:59:23 +01:00
Wolfgang Bumiller fc7f03523c cleanup
Error::from is already a function taking 1 parameter,
there's no need to wrap it with `|e| Error::from(e)`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:21:27 +01:00
Dietmar Maurer 591f570b22 src/client/http_client.rs: trim path 2019-02-18 06:24:28 +01:00
Dietmar Maurer ef2f2efbcc improve api_schema module structure 2019-02-17 10:16:33 +01:00
Dietmar Maurer dc9a007b11 rename src/api to src/api_schema 2019-02-17 09:59:20 +01:00
Dietmar Maurer db4868ef4f src/client/backup_repo.rs - improve docu 2019-02-14 11:17:08 +01:00
Dietmar Maurer 151c6ce27d src/client.rs: improve library structure and docu 2019-02-14 11:11:39 +01:00
Dietmar Maurer 0dffe3f99a src/client/http_client.rs: try to login
use an environment var to store passphrase (PBS_PASSWORD)
2019-02-13 14:31:43 +01:00
Wolfgang Bumiller 4a3f65172c client: use hyper-tls for now
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
Dietmar Maurer 9f49fe1d5d avoid compiler warnings 2019-01-30 18:25:37 +01:00
Dietmar Maurer 1adb353d6f lient/http_client.rs: simplify code 2019-01-22 10:36:04 +01:00
Dietmar Maurer 1fdb4c6ffa client/http_client.rs: implement get, share common code 2019-01-21 18:56:48 +01:00
Dietmar Maurer 728797d0c1 reduce compiler warnings 2019-01-18 16:50:15 +01:00
Dietmar Maurer 83bdac1e3b api3/admin/datastore/upload_catar.rs: verify content type ("application/x-proxmox-backup-catar") 2019-01-17 12:43:29 +01:00
Dietmar Maurer 06aeb76a5c client/catar_backup_stream.rs: remove debug code 2019-01-17 12:16:36 +01:00
Dietmar Maurer 23bb8780d4 proxmox-backup-client: implement upload to server 2019-01-17 11:38:22 +01:00
Dietmar Maurer 597641fdd5 client/http_client.rs: new helper class 2019-01-17 11:38:03 +01:00
Dietmar Maurer e8edbbd49c client/catar_backup_stream.rs: new helper for catar uploads to server 2019-01-16 10:15:39 +01:00