0f0a35b390
src/backup/dynamic_index.rs: lock file inside new, code cleanup
2019-07-04 08:17:30 +02:00
a7c72ad9eb
src/backup/fixed_index.rs - FixedIndexReader: remove reference to chunk store
2019-07-04 08:13:16 +02:00
a660978c9a
src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats
2019-07-04 07:57:43 +02:00
86eda3eb0d
src/bin/proxmox-backup-client.rs: implement restore using BackupReader
2019-07-03 15:45:12 +02:00
b2a6897982
src/client/remote_chunk_reader.rs: remove dead code
2019-07-03 14:56:24 +02:00
7f99bf691a
src/client/remote_chunk_reader.rs: implement remote chunk reader
2019-07-03 14:39:13 +02:00
4f6aaf542c
src/client/http_client.rs: avoid compiler warning
2019-07-03 14:36:02 +02:00
3467cd91e9
src/client/http_client.rs: return Arc<Self>
...
Avoid to call canceller for each clone on drop ...
2019-07-03 14:26:07 +02:00
1e0784841c
src/backup/read_chunk.rs: use &mut self
2019-07-03 14:25:18 +02:00
0d70535a36
src/pxar/encoder.rs: refactor simple match statements to map_err()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-03 12:18:36 +02:00
6e1c31163b
src/pxar/sequential_decoder.rs: refactoring of simple match statements to map_err()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-03 12:18:24 +02:00
9d227350a3
src/pxar/sequential_decoder.rs: Cleanup and fix incorrect error message
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-03 12:16:59 +02:00
80af046794
src/server/rest.rs: avoid unwrap
2019-07-03 12:00:43 +02:00
7fb4f5642a
src/server/rest.rs: log peer address, use hyper MakeService
2019-07-03 11:54:35 +02:00
6d1f61b208
use hyper/tokio-openssl instead of hyper/tokio-tls
...
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
2019-07-02 13:36:28 +02:00
74d0a6bc23
src/bin/proxmox-backup-proxy.rs: set socket options for max. performance
2019-07-02 10:48:58 +02:00
93f8bb0445
src/bin/h2s-client.rs: avoid compiler warnings
2019-07-02 08:59:33 +02:00
877dd11133
src/backup/{fixed, dynamic}_index.rs: improve error messages
2019-07-02 08:56:56 +02:00
5c593a4d56
src/backup/read_chunk.rs: use Arc for DataStore
2019-07-02 08:49:16 +02:00
b850673634
src/backup/read_chunk.rs: move read chunk trait into extra file
...
And implement LocalChunkReader.
2019-07-02 08:22:29 +02:00
8fad30a4b1
use latest hyper master branch to include socket buffer size fix
2019-07-02 07:09:46 +02:00
f35197f449
replace tools::vec with proxmox::tools::vec
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 11:03:25 +02:00
d0162d53d3
tools: remove io module
...
now completely replaced by proxmox::tools::io.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:44:12 +02:00
9b2b627fe0
update remaining users of tools::io::ops
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:44:03 +02:00
9110a69bd8
tools: remove read/write
...
now completely replaced by
proxmox::tools::io::{ReadExt, WriteExt}
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:39:42 +02:00
ca3c3ce997
daemon: remove last use of tools::read/write
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:39:13 +02:00
5485b579a1
backup: use new proxmox::tools::io traits
...
This replaces all `write_value` calls with `write_le_value`
calls!
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:37:02 +02:00
dc9775d1b0
src/bin/h2s-client.rs: increase tokio tcp stream read buffer size
...
This improves download speed by factor 10.
2019-06-29 18:09:50 +02:00
34f7131160
src/bin/h2s-client.rs: avoid hyper, use h2 directly
...
But performance is still bad.
2019-06-29 15:58:18 +02:00
5ed2277f0e
src/bin/h2s-server.rs: test to reproduce slow h2 download
...
Simplified code, use with h2s-client.rs.
2019-06-29 14:57:54 +02:00
17243003b3
src/bin/download-speed.rs: h2 download speedtest for our server
...
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
a2b29b68d4
src/client/http_client.rs: use maximal h2 window sizes
2019-06-29 11:05:36 +02:00
872debdefa
src/bin/h2client.rs: tune h2 client parameters to get reasonable speed
...
We now get about 2.5GB/s (not really good, but better)
2019-06-29 10:41:24 +02:00
38294bcc56
src/client.rs: make module put_to_stream public
2019-06-29 10:40:43 +02:00
fded1f3154
add code to test H2 speed
...
I currently get about 1MB/s, which is ways too slow (must be a bug)?
2019-06-29 10:08:12 +02:00
d48a9955a5
src/backup/dynamic_index.rs: introduce ReadChunk trait
2019-06-28 16:35:00 +02:00
09d7dc5024
src/api2/reader.rs: add download chunk api
2019-06-28 16:27:01 +02:00
fcf5dea56d
src/client/http_client.rs: increase h2 window size, set tcp nodelay option
...
To increase h2 download speed.
2019-06-28 16:00:58 +02:00
81a6ce6fde
src/backup/chunk_store.rs: new method chunk_path()
...
Returns the absolute path.
2019-06-28 15:48:09 +02:00
fcfb84fedf
file download: avoid unnecessary copy
2019-06-28 07:07:52 +02:00
984a7c3502
src/client/http_client.rs - h2 download: implement flow control
2019-06-28 07:02:43 +02:00
dd066d28e2
src/api2/reader.rs: implement backup reader protocol
2019-06-27 09:01:41 +02:00
42a87f7b96
src/server/h2service.rs: implement generic h2 service
2019-06-26 17:38:33 +02:00
fb0470837b
src/client/http_client.rs: split out code to start new h2 connections
2019-06-26 12:09:18 +02:00
b208da8393
src/backup/data_{chunk,blob}.rs: add verify_crc method
2019-06-26 09:54:25 +02:00
fef44d4f78
src/bin/proxmox-backup-client.rs: decode blobs on download
...
No more need for that clumsy blob cli (removed).
2019-06-26 09:18:59 +02:00
c2b945341c
src/client/http_client.rs - download: use generic Write type, return writer.
...
Make it possible to write int Vec<u8>.
2019-06-26 09:17:13 +02:00
bb8231409e
src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json
...
Use the KeyConfig serialization with kdf = None.
2019-06-26 07:32:34 +02:00
3031e44c58
src/bin/proxmox-backup-client.rs: remove stale download file, start blob cli
2019-06-25 13:00:37 +02:00
f14a8c9a85
src/api2/admin/datastore.rs: use correct path for download
2019-06-25 12:59:36 +02:00
81820b0d4d
src/client/http_client.rs - download: fix error handling
2019-06-25 12:43:55 +02:00
45db6f89e1
src/bin/proxmox-backup-client.rs: implement download command
2019-06-25 11:17:24 +02:00
9e47c0a56e
src/api2/admin/datastore.rs: implement generic download method
...
We simply allow to download raw binary data.
2019-06-25 10:16:59 +02:00
da7d67213e
src/api2/admin/datastore/pxar.rs: remove pxar upload api
...
Upload needs to be done using the backup protocol.
2019-06-25 09:45:35 +02:00
dd8e744f9a
src/api2/admin/datastore.rs: use a worker log log prune actions
2019-06-25 08:12:25 +02:00
6899dbfb47
src/bin/proxmox-backup-client.rs: strip .blob extension
2019-06-25 07:11:02 +02:00
c6d203bbd3
src/backup/backup_info.rs: also list .blob files
2019-06-25 07:07:45 +02:00
045e8a568d
src/bin/proxmox-backup-client.rs: change key name to rsa-encrypted.key
2019-06-25 07:00:07 +02:00
79bc7345e8
src/cli/command.rs: fix ReST output format
2019-06-25 06:19:51 +02:00
9f46c7de4b
src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup
2019-06-24 13:56:37 +02:00
cd7dc87903
pxar: impl .pxarexclude parsing and exclude matching
...
.pxarexclude files allow to exclude or include parts of a subtree by matching
with a glob pattern. The globs are used according to the matches of fnmatch.
In addition '**' can be used to match multiple directories within the path.
Order of the entries matter, as later ones win over previous ones.
As the .pxarexclude files can be placed at any node of the directory hirarchy,
this implies that matching child entries win over parent entries.
The only exception to this behaviour is, when a parent entry already fully
matched the path, thereby excluding the child entries which would match
otherwise.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-06-24 10:26:20 +02:00
46bd880041
src/api2/backup/environment.rs: new helper add_blob()
2019-06-24 09:35:37 +02:00
cb08ac3efe
src/api2/backup.rs: replace upload_config with upload_blob
2019-06-23 11:44:30 +02:00
a38c5d4d12
src/backup/data_blob.rs: implement from_raw()
2019-06-23 09:35:44 +02:00
6f083b7a92
src/api2/backup/upload_chunk.rs: compute crc on upload
2019-06-23 09:04:42 +02:00
ba01828d38
try to use use proxmox::tools::io::ops::ReadExtOps
2019-06-22 16:29:10 +02:00
27042ce637
src/backup/file_formats.rs: improve docs
2019-06-22 15:45:36 +02:00
9f83e0f7ee
src/backup/crypt_config.rs: simplify decode interface
2019-06-22 13:24:29 +02:00
0066c6d972
src/backup/crypt_config.rs: remove encode_chunk, use encrypt_to instead
2019-06-22 13:02:53 +02:00
ee8a7e8080
src/backup/crypt_config.rs: implement encrypt_to (output to writer)
2019-06-22 12:25:04 +02:00
991abfa8b4
src/backup/file_formats.rs: split out file format data
2019-06-22 09:12:25 +02:00
b7f4f27d6c
add crc field for binary blobs formats
2019-06-21 18:49:04 +02:00
a7f67a9a9c
depend on crc32fast
...
We can use this to compute fast checksums to test file integrity (not crypto safe).
2019-06-21 15:58:00 +02:00
3025b3a53c
src/backup/data_blob.rs: new file format for binary blobs
2019-06-21 11:32:07 +02:00
077a8cae63
src/backup/crypt_config.rs: make code more flexible
...
Allow to pass magic numbers as parameters.
2019-06-21 10:50:48 +02:00
c38266c18d
renamed: src/backup/crypt_setup.rs -> src/backup/crypt_config.rs
2019-06-21 09:51:18 +02:00
3ea8bfc9c0
src/bin/proxmox-backup-client.rs: add import-master-pubkey command
2019-06-21 07:34:17 +02:00
37c5a17505
src/bin/proxmox-backup-client.rs: helper to generate RSA master key
2019-06-21 07:12:19 +02:00
ab44acff57
src/backup/key_derivation.rs: add modified field to key file
2019-06-20 07:58:33 +02:00
6d0983dbe1
src/bin/proxmox-backup-client.rs: add keyfile parameter for backup
2019-06-19 17:16:41 +02:00
181f097af9
src/bin/proxmox-backup-client.rs - key API: pass kdf parameter
...
Allow to store keys without password.
2019-06-18 18:21:01 +02:00
9b06db4563
src/bin/proxmox-backup-client.rs: use file system path to reference keys
2019-06-18 17:38:50 +02:00
091540298e
src/bin/proxmox-backup-client.rs - create_key: always read password from tty
2019-06-18 17:27:52 +02:00
826f309bf5
src/backup/key_derivation.rs: move kdf code into separate file
2019-06-18 11:17:22 +02:00
11515438cc
Cargo.toml: use serde feature derive
2019-06-18 06:23:25 +02:00
ac71623483
src/bin/proxmox-backup-client.rs: implement "key change-passphrase"
2019-06-17 13:10:00 +02:00
f2401311b0
src/bin/proxmox-backup-client.rs: start key management
2019-06-17 10:33:24 +02:00
a7dd483097
src/backup/*_index.rs: used generated magic numbers
2019-06-14 14:58:37 +02:00
bffd40d6b7
src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools
2019-06-14 11:40:04 +02:00
00388226e6
src/api2/backup/environment.rs: log index checksums
2019-06-14 10:36:20 +02:00
9335d74eba
src/backup/fixed_index.rs: compute checksum over the index
2019-06-14 10:34:29 +02:00
16ff6b7cd8
src/backup/dynamic_index.rs: compute checksum over the index
2019-06-14 10:33:59 +02:00
36075475a6
src/api2/backup/environment.rs: log duplicate chunks
2019-06-14 07:12:30 +02:00
fa148dbd11
src/api2/backup/upload_chunk.rs: verify chunk data if possible
2019-06-14 06:23:46 +02:00
0f37577092
src/bin/cipherbench.rs: add test for zstd compression
2019-06-13 17:24:57 +02:00
9399c98f82
src/bin/cipherbench.rs: cipher speed test with large blocks
2019-06-13 17:16:43 +02:00
f640f23ac5
src/backup/data_chunk.rs: fix typo
2019-06-13 16:42:55 +02:00
dba72d5016
src/backup/data_chunk.rs: only use compressed data if it is shorter than uncompressed
2019-06-13 12:56:50 +02:00
68ab376ae9
src/backup/crypt_setup.rs: only use compressed data if it is shorter than uncompressed
2019-06-13 12:47:54 +02:00
f98ac774ee
backup: Add support for client side encryption
...
first try ...
2019-06-13 11:47:23 +02:00
Dietmar Maurer
Christian Ebner
Wolfgang Bumiller