 Dietmar Maurer
|
29ae5c86a2
|
src/backup/fixed_index.rs: make chunk_size public
|
2019-05-28 06:18:24 +02:00 |
|
|
Dietmar Maurer
|
7dd1bcac58
|
src/client/http_client.rs: correctly parse error response in download_chunk_list
|
2019-05-27 09:35:30 +02:00 |
|
|
Dietmar Maurer
|
a1e7cff3ea
|
src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk
|
2019-05-27 07:57:43 +02:00 |
|
|
Dietmar Maurer
|
771953f9eb
|
src/api2/admin/datastore/backup.rs: use smaller window sizes
|
2019-05-27 07:53:40 +02:00 |
|
|
Dietmar Maurer
|
174ad378d8
|
src/client/http_client.rs: fix request pipelining
|
2019-05-27 07:24:32 +02:00 |
|
|
Dietmar Maurer
|
624362226e
|
src/client/merge_known_chunks.rs: use MergedChunkInfo as input
|
2019-05-26 10:52:56 +02:00 |
|
|
Dietmar Maurer
|
05cba08c9c
|
src/client/http_client.rs: append chunks in correct order
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
|
2019-05-26 09:57:34 +02:00 |
|
|
Dietmar Maurer
|
417cb0731e
|
src/api2/admin/datastore/backup.rs: verify chunk offset
This clearly shows that the current approach does not work - seems we get
chunks out of order.
|
2019-05-26 08:49:29 +02:00 |
|
|
Dietmar Maurer
|
34114e2606
|
src/server/rest.rs: correctly verify json parameters
|
2019-05-24 09:10:59 +02:00 |
|
|
Dietmar Maurer
|
eb2bdd1bcc
|
src/client/http_client.rs: encode parameters inside body
To avoid problems with long uris.
|
2019-05-24 08:32:55 +02:00 |
|
|
Dietmar Maurer
|
9bb675ec23
|
src/client/merge_known_chunks.rs: reduce RAM usage
There is no need to store chunk data for known chunks.
|
2019-05-24 07:42:58 +02:00 |
|
|
Dietmar Maurer
|
ecb21b4794
|
src/client/merge_known_chunks.rs: avoid VecDequeu - a single buffer is enough
|
2019-05-24 07:36:09 +02:00 |
|
|
Dietmar Maurer
|
aa1b2e04fe
|
src/client/merge_known_chunks.rs: merge known chunks
To decrease the number of api calls required...
|
2019-05-23 12:31:09 +02:00 |
|
|
Dietmar Maurer
|
91320f0879
|
src/client/http_client.rs: use ChunkInfo streams
This will make out of order uploads possible...
|
2019-05-23 12:31:09 +02:00 |
|
|
Dietmar Maurer
|
8bea85b42e
|
src/api2/admin/datastore/backup.rs: verify file size and chunk count on close
|
2019-05-23 08:50:36 +02:00 |
|
|
Dietmar Maurer
|
1179e15842
|
src/server/rest.rs: improve error handling
|
2019-05-23 08:15:32 +02:00 |
|
|
Dietmar Maurer
|
907f5bb0d8
|
src/api2/admin/datastore/backup/service.rs: improve error logging
|
2019-05-23 08:05:39 +02:00 |
|
|
Dietmar Maurer
|
32e1bf4260
|
src/bin/upload-speed.rs: code cleanup
|
2019-05-23 07:20:05 +02:00 |
|
|
Dietmar Maurer
|
d6f204eda6
|
src/client/http_client.rs: re-order code - public functions first
|
2019-05-23 06:28:10 +02:00 |
|
|
Dietmar Maurer
|
9af37c8f0e
|
src/client/http_client.rs: move low level H2 code into separate class
|
2019-05-22 17:28:25 +02:00 |
|
|
Dietmar Maurer
|
d2c48afc6e
|
src/client/http_client.rs: code cleanup
|
2019-05-22 13:24:33 +02:00 |
|
|
Dietmar Maurer
|
6ab34afa88
|
src/client/http_client.rs: cleanup h2 backup client
|
2019-05-22 13:05:51 +02:00 |
|
|
Dietmar Maurer
|
a55fcd740f
|
src/api2/admin/datastore/backup.rs: suse POST for finish() api
|
2019-05-22 13:04:58 +02:00 |
|
|
Dietmar Maurer
|
de6742c04f
|
src/bin/h2test.rs: removed
This test does no work anymore.
|
2019-05-22 11:14:56 +02:00 |
|
|
Dietmar Maurer
|
5a0b521809
|
src/api2/admin/datastore/backup.rs: remove test callbacks
|
2019-05-22 10:55:27 +02:00 |
|
|
Dietmar Maurer
|
553610b43e
|
src/client/http_client.rs: implement download_chunk_list
|
2019-05-22 09:46:02 +02:00 |
|
|
Dietmar Maurer
|
0cc0fffd1e
|
src/backup/chunk_stream.rs: use BytesMut in ChunkStream
|
2019-05-22 09:39:02 +02:00 |
|
|
Dietmar Maurer
|
c052be5c86
|
src/backup/chunk_stream.rs: use more generics
|
2019-05-22 09:18:05 +02:00 |
|
|
Dietmar Maurer
|
169c0e060f
|
src/backup/chunk_stream.rs: optimize FixedChunkStream (use BytesMut)
|
2019-05-22 09:05:35 +02:00 |
|
|
Dietmar Maurer
|
3be3f3dcc4
|
src/backup/chunk_stream.rs: fix behaviour for large input buffers
|
2019-05-22 08:05:27 +02:00 |
|
|
Dietmar Maurer
|
02fa54ff3c
|
src/backup/chunk_stream.rs: assemble data with correct order
|
2019-05-22 07:44:51 +02:00 |
|
|
Dietmar Maurer
|
d0afeaa831
|
src/api2/admin/datastore/backup.rs: improve error message
|
2019-05-22 07:43:54 +02:00 |
|
|
Dietmar Maurer
|
11867a2b54
|
src/tools.rs: fix hex_to_digest
|
2019-05-21 13:42:13 +02:00 |
|
|
Dietmar Maurer
|
7f3d2ffab9
|
src/backup/index.rs: rename ChunkListReader into DigestListEncoder
|
2019-05-21 12:28:44 +02:00 |
|
|
Dietmar Maurer
|
ee53955f52
|
src/backup/index.rs: implement DigestListDecoder
|
2019-05-21 12:21:22 +02:00 |
|
|
Dietmar Maurer
|
a95849321c
|
src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks
|
2019-05-21 09:54:11 +02:00 |
|
|
Dietmar Maurer
|
40f4e198a8
|
src/backup/dynamic_index.rs: add chunk_info method
|
2019-05-21 09:52:19 +02:00 |
|
|
Dietmar Maurer
|
a09c0e38d8
|
src/api2/admin/datastore/backup/environment.rs: register/lookup known chunks
|
2019-05-20 18:05:10 +02:00 |
|
|
Dietmar Maurer
|
97f03eff13
|
src/client/pxar_backup_stream.rs: fix docu
|
2019-05-20 17:29:00 +02:00 |
|
|
Dietmar Maurer
|
82ab72304e
|
src/client/http_client.rs: implement upload_stream
|
2019-05-20 14:19:24 +02:00 |
|
|
Dietmar Maurer
|
2698e8a514
|
src/client/pxar_backup_stream.rs: close pipe inside drop
|
2019-05-20 11:20:33 +02:00 |
|
|
Dietmar Maurer
|
f2269d8f99
|
src/tools.rs: implement hex_to_digest
|
2019-05-19 12:15:46 +02:00 |
|
|
Dietmar Maurer
|
751030aa7c
|
src/bin/test_chunk_speed2.rs: test max chunk size
|
2019-05-19 11:09:02 +02:00 |
|
|
Dietmar Maurer
|
ff77dbbea3
|
Revert "src/backup/chunk_stream.rs: simplify code"
This reverts commit e058744d8f.
The optimization was wrong, and produces wrong chunk boundaries.
|
2019-05-19 11:05:56 +02:00 |
|
|
Dietmar Maurer
|
178ac042c4
|
src/api_schema/schema.rs: improve error messages
|
2019-05-19 10:18:51 +02:00 |
|
|
Dietmar Maurer
|
0c516b123d
|
src/client/pxar_backup_stream.rs: use WrappedStreamReader
to make it fully async ...
|
2019-05-18 15:13:31 +02:00 |
|
|
Dietmar Maurer
|
c6e28b66c6
|
src/client/pxar_backup_stream.rs: use 1M pipe buffer size
|
2019-05-18 12:57:43 +02:00 |
|
|
Dietmar Maurer
|
6276448519
|
src/bin/test_chunk_speed2.rs: test chunker speed with real data
|
2019-05-18 11:59:17 +02:00 |
|
|
Dietmar Maurer
|
e058744d8f
|
src/backup/chunk_stream.rs: simplify code
|
2019-05-18 10:55:46 +02:00 |
|
|
Dietmar Maurer
|
8a7cc7565f
|
src/backup/chunk_stream.rs: implement fixed size chunk stream
|
2019-05-18 10:55:46 +02:00 |
|
|
Dietmar Maurer
|
9c9ad941c0
|
src/client/http_client.rs: improve speed test
pipeline responses to a separate async channel ...
|
2019-05-18 10:55:46 +02:00 |
|
|
Dietmar Maurer
|
adec8ea263
|
src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
|
2019-05-16 10:24:23 +02:00 |
|
|
Dietmar Maurer
|
372724afea
|
src/api2/admin/datastore/backup.rs: implement finish_backup
|
2019-05-15 12:58:55 +02:00 |
|
|
Dietmar Maurer
|
bb105f9dae
|
src/api2/admin/datastore/backup.rs: improve logging
|
2019-05-15 10:09:46 +02:00 |
|
|
Dietmar Maurer
|
25ddc15fe0
|
src/client/pipe_to_stream.rs: add missing file
|
2019-05-15 09:07:19 +02:00 |
|
|
Dietmar Maurer
|
a20772525f
|
src/api2/admin/datastore/backup.rs: implement close for dynamic writers
|
2019-05-15 07:58:05 +02:00 |
|
|
Dietmar Maurer
|
4e93f8c164
|
src/api2/types.rs: add schema/format for file names
|
2019-05-15 07:14:08 +02:00 |
|
|
Dietmar Maurer
|
e3dbd41b20
|
src/client/http_client.rs: add flow control for h2 upload
|
2019-05-14 14:54:21 +02:00 |
|
|
Dietmar Maurer
|
97f22ce57d
|
src/client/http_client.rs: implement h2 upload
without flow control so far ...
|
2019-05-14 12:58:59 +02:00 |
|
|
Dietmar Maurer
|
dafc27aeff
|
src/backup/chunk_stream.rs: async chunk stream
|
2019-05-14 10:05:29 +02:00 |
|
|
Dietmar Maurer
|
c0e22141ed
|
src/client/http_client.rs: derive Clone for H2Client
|
2019-05-14 09:00:24 +02:00 |
|
|
Dietmar Maurer
|
1133fe9a63
|
src/server/rest.rs: do not log 1xx status codes as errors
|
2019-05-14 06:23:22 +02:00 |
|
|
Dietmar Maurer
|
850ac6d0f0
|
src/client/http_client.rs: directly return H2Client on upgrade
|
2019-05-13 16:44:59 +02:00 |
|
|
Dietmar Maurer
|
10130cf48c
|
src/client/http_client.rs: cleanup, remove debug println
|
2019-05-13 12:11:18 +02:00 |
|
|
Dietmar Maurer
|
b57cb26406
|
src/client/http_client.rs: implement http2 client wrapper
|
2019-05-13 10:27:22 +02:00 |
|
|
Dietmar Maurer
|
9e391bb7f0
|
src/client/http_client.rs: allow to pass parameters (encode them as query string)
|
2019-05-13 09:12:03 +02:00 |
|
|
Dietmar Maurer
|
fbb798f6d2
|
src/api2/admin/datastore/backup.rs: ignore errors from last_backup
|
2019-05-13 07:06:56 +02:00 |
|
|
Dietmar Maurer
|
d36113665b
|
src/api2/admin/datastore/backup.rs: implement chunk index download from previous backup
|
2019-05-11 16:05:50 +02:00 |
|
|
Dietmar Maurer
|
457531e7fe
|
src/backup/index.rs: new ChunkListReader
|
2019-05-11 16:01:42 +02:00 |
|
|
Dietmar Maurer
|
af1d68185c
|
src/api2/admin/datastore/backup.rs: add missing 'archive-name' parameter to schema
|
2019-05-11 12:25:15 +02:00 |
|
|
Dietmar Maurer
|
6b95c7df07
|
src/api2/admin/datastore/backup/environment.rs: do not store path
|
2019-05-11 12:13:44 +02:00 |
|
|
Dietmar Maurer
|
51a4f63ff7
|
src/backup/datastore.rs: move last_backup to BackupInfo
|
2019-05-11 12:07:09 +02:00 |
|
|
Dietmar Maurer
|
b02a52e3a5
|
src/backup/datastore.rs: impl last_backup helper
|
2019-05-11 11:36:46 +02:00 |
|
|
Dietmar Maurer
|
c09775011a
|
src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups
|
2019-05-11 10:19:34 +02:00 |
|
|
Dietmar Maurer
|
f9578f3c79
|
backup-api: allow to create DynamicIndexWriter and add chunks to to
|
2019-05-10 10:25:40 +02:00 |
|
|
Dietmar Maurer
|
35a2d8a6a6
|
src/backup/backup_info.rs: Derive Clone
|
2019-05-10 06:59:23 +02:00 |
|
|
Dietmar Maurer
|
0506d1ed04
|
upload_chunk.rs: add comment about possible unnecessary code
|
2019-05-10 06:15:25 +02:00 |
|
|
Dietmar Maurer
|
5f42ef009a
|
cleanup: remove unused code
|
2019-05-09 18:11:39 +02:00 |
|
|
Dietmar Maurer
|
b4b63e520d
|
handle_async_api_request: put rpcenv into a Box
So that we can pass rpcenv into futures.
|
2019-05-09 18:01:24 +02:00 |
|
|
Dietmar Maurer
|
2c41096a4b
|
src/api2/admin/datastore/backup/upload_chunk.rs: fix maximal chunk size
|
2019-05-09 13:29:50 +02:00 |
|
|
Dietmar Maurer
|
78b5191550
|
src/server/formatter.rs: further cleanups and renaming ...
|
2019-05-09 13:28:26 +02:00 |
|
|
Dietmar Maurer
|
d55037e471
|
src/server/formatter.rs: rename format_result to format_data
To avoid confusions with Rust Result type.
|
2019-05-09 13:15:15 +02:00 |
|
|
Dietmar Maurer
|
21ee7912fa
|
src/api2/admin/datastore/backup.rs: implement upload chunk
|
2019-05-09 13:06:09 +02:00 |
|
|
Dietmar Maurer
|
4ebf0eabb0
|
src/api2/types.rs: use separate file for schema definitions
|
2019-05-09 07:44:09 +02:00 |
|
|
Dietmar Maurer
|
bd1507c4fa
|
src/api2/admin/datastore/backup/service.rs: move service code into extra file
|
2019-05-08 17:36:19 +02:00 |
|
|
Dietmar Maurer
|
0aadd40b4f
|
src/api2/admin/datastore/backup.rs: add required parameters, cleanups
|
2019-05-08 13:17:23 +02:00 |
|
|
Dietmar Maurer
|
d95ced64ad
|
src/api2/admin/datastore/backup/environment.rs: use separate file
|
2019-05-08 12:41:58 +02:00 |
|
|
Dietmar Maurer
|
ca60c371e2
|
api: rename h2upload to backup
|
2019-05-08 12:35:15 +02:00 |
|
|
Dietmar Maurer
|
d906b1124f
|
src/api2/admin/datastore/h2upload.rs: code cleanups
|
2019-05-08 11:36:43 +02:00 |
|
|
Dietmar Maurer
|
58c8d7d91f
|
rc/api2/admin/datastore/h2upload.rs: implement BackupEnvironment
To pass arbitrary information/state to api methods.
|
2019-05-08 11:26:54 +02:00 |
|
|
Dietmar Maurer
|
f757b30efc
|
src/server/rest.rs: use generics to pass RpcEnvironment
|
2019-05-08 11:09:01 +02:00 |
|
|
Dietmar Maurer
|
2edc341b29
|
src/tools.rs: Add AsAyn Trait
|
2019-05-08 11:07:36 +02:00 |
|
|
Dietmar Maurer
|
090ac9f79a
|
src/api2/admin/datastore/h2upload.rs: avoid unwrap
|
2019-05-08 08:21:18 +02:00 |
|
|
Dietmar Maurer
|
4e3da4b366
|
src/api2/admin/datastore/h2upload.rs: do not log client address
|
2019-05-08 08:15:55 +02:00 |
|
|
Dietmar Maurer
|
e3a44552ee
|
src/api2/admin/datastore/h2upload.rs: code cleanups
|
2019-05-08 08:12:49 +02:00 |
|
|
Dietmar Maurer
|
a66ab8ae5e
|
src/api2/admin/datastore/h2upload.rs: make worker abortable
|
2019-05-08 07:14:40 +02:00 |
|
|
Dietmar Maurer
|
72375ce6c6
|
src/api2/admin/datastore/h2upload.rs: log to worker task
|
2019-05-07 14:07:00 +02:00 |
|
|
Dietmar Maurer
|
d9bd06eae8
|
src/api2/admin/datastore/h2upload.rs: use WorkerTask
|
2019-05-07 13:42:00 +02:00 |
|
|
Dietmar Maurer
|
52cf506e48
|
src/api2/admin/datastore/h2upload.rs: implement async test api call
|
2019-05-07 12:26:55 +02:00 |
|
|
Dietmar Maurer
|
92ac375aff
|
src/api2/admin/datastore/h2upload.rs: improve http2 service
|
2019-05-07 11:24:44 +02:00 |
|