tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,925 Commits 2 Branches 54 Tags 14 MiB
71da3d6a1b
Commit Graph

260 Commits

Author SHA1 Message Date
Wolfgang Bumiller
2194824f77 api: BoxFut -> ApiFuture 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:16:37 +01:00
Wolfgang Bumiller
3d482025b3 move api schema into proxmox::api crate 
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:14:54 +01:00
Dietmar Maurer
552c225948 sort all property lookup tables 
Required, because we use binary sreach to find items.
 2019-11-21 13:32:09 +01:00
Dietmar Maurer
255f378a1b use const api definitions 2019-11-21 13:32:09 +01:00
Dietmar Maurer
0997967d65 do no restrict index names too much 2019-11-08 10:28:14 +01:00
Dietmar Maurer
62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer
11377a47bb avoid some clippy warnings 2019-10-25 18:04:37 +02:00
Dietmar Maurer
2c2097ee75 fix typo in error message 2019-10-07 12:48:56 +02:00
Dietmar Maurer
f7aa6f15d3 src/api2/reader.rs: use tokio::fs::read() to read file async 2019-10-07 12:24:06 +02:00
Dietmar Maurer
4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
Dietmar Maurer
c0b1b14c05 src/api2/reader.rs - download_chunk: use blocking io 
Turns out to be twice as fast as tokio async io...
 2019-10-05 11:41:19 +02:00
Dietmar Maurer
fb6026b66b backup writer api: verify checksum inside close 2019-09-23 10:56:53 +02:00
Dietmar Maurer
4a7de56e2f avoid injecting ENV vars from Makefile 
So that we can run "cargo build" without setting vars manually.
 2019-09-09 10:51:08 +02:00
Wolfgang Bumiller
ffb6434485 src/api2/reader.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
7622005574 src/api2/backup/upload_chunk.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
59b2baa0f6 src/api2/backup.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
6be147b78c src/api2/admin/datastore.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
f3a8d1d7e0 api2/node/time: replace mem::uninitialized 
and fixup use statements

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:14:41 +02:00
Dietmar Maurer
4af0ee055a api: always use complete file names (including add exctensions) 2019-08-07 10:10:14 +02:00
Dietmar Maurer
d7c243977d src/api2/admin/datastore.rs - list_snapshot_files: list all files 
Add files that are not mentioned in index.json.
 2019-08-07 09:16:14 +02:00
Dietmar Maurer
34066e2eeb src/api2/backup/environment.rs - add_blob: always verify blob crc 2019-08-07 09:15:13 +02:00
Dietmar Maurer
f889b15867 src/backup/data_blob.rs: always compute crc 2019-08-06 11:42:14 +02:00
Dietmar Maurer
a17a0e7a9f src/api2/admin/datastore.rs: extract backup size from index.json 2019-08-06 10:56:21 +02:00
Dietmar Maurer
8c70e3eb18 src/api2/admin/datastore.rs - files: return data from index.json 2019-08-05 13:22:19 +02:00
Dietmar Maurer
f69adc81a6 use new proxmox::tools::nodename 2019-08-03 17:06:23 +02:00
Dietmar Maurer
7f66c29e5c remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead 2019-08-03 16:44:31 +02:00
Dietmar Maurer
e18a6c9ee5 update to nix 0.14, use code from proxmox:tools 2019-08-03 13:05:38 +02:00
Dietmar Maurer
9fdc3ef46f src/api2/admin/datastore.rs: pass backup type/id (group) to prune 
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
 2019-07-27 08:49:14 +02:00
Dietmar Maurer
bbf9e7e951 src/api2/backup.rs: cleanup schema definitions 2019-07-26 09:07:29 +02:00
Dietmar Maurer
ca5d0b61ca src/api2/backup.rs: new required backup-time parameter 
The client should pass the time as parameter.
 2019-07-25 13:44:01 +02:00
Dietmar Maurer
e128d4e84f src/api2/admin/datastore.rs: backup logs may not be written twice 2019-07-25 12:29:04 +02:00
Dietmar Maurer
ef3254c086 src/api2/admin/datastore.rs: use correct .blob extension for log 2019-07-25 12:22:50 +02:00
Dietmar Maurer
07ee223590 src/api2/admin/datastore.rs: add api to upload backup client log file 2019-07-25 12:15:30 +02:00
Dietmar Maurer
15c847f136 src/api2/admin/datastore.rs: allow to list all snapshots 2019-07-18 09:11:43 +02:00
Dietmar Maurer
0eecf38fbf src/api2/admin/datastore.rs: add status api call 2019-07-16 13:34:38 +02:00
Dietmar Maurer
b46c3fad03 src/backup/fixed_index.rs: use correct size 
We need to consider that the last chunk may have smaller size.
 2019-07-04 15:13:22 +02:00
Dietmar Maurer
5e04ec707d src/api2/backup/environment.rs: improve chunk alignment/size checks 2019-07-04 13:40:43 +02:00
Dietmar Maurer
17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server 
I get about 30MB/s (much too slow)
 2019-06-29 13:43:10 +02:00
Dietmar Maurer
09d7dc5024 src/api2/reader.rs: add download chunk api 2019-06-28 16:27:01 +02:00
Dietmar Maurer
fcfb84fedf file download: avoid unnecessary copy 2019-06-28 07:07:52 +02:00
Dietmar Maurer
dd066d28e2 src/api2/reader.rs: implement backup reader protocol 2019-06-27 09:01:41 +02:00
Dietmar Maurer
42a87f7b96 src/server/h2service.rs: implement generic h2 service 2019-06-26 17:38:33 +02:00
Dietmar Maurer
f14a8c9a85 src/api2/admin/datastore.rs: use correct path for download 2019-06-25 12:59:36 +02:00
Dietmar Maurer
9e47c0a56e src/api2/admin/datastore.rs: implement generic download method 
We simply allow to download raw binary data.
 2019-06-25 10:16:59 +02:00
Dietmar Maurer
da7d67213e src/api2/admin/datastore/pxar.rs: remove pxar upload api 
Upload needs to be done using the backup protocol.
 2019-06-25 09:45:35 +02:00
Dietmar Maurer
dd8e744f9a src/api2/admin/datastore.rs: use a worker log log prune actions 2019-06-25 08:12:25 +02:00
Dietmar Maurer
46bd880041 src/api2/backup/environment.rs: new helper add_blob() 2019-06-24 09:35:37 +02:00
Dietmar Maurer
cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer
6f083b7a92 src/api2/backup/upload_chunk.rs: compute crc on upload 2019-06-23 09:04:42 +02:00
Dietmar Maurer
bffd40d6b7 src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools 2019-06-14 11:40:04 +02:00
Dietmar Maurer
00388226e6 src/api2/backup/environment.rs: log index checksums 2019-06-14 10:36:20 +02:00
Dietmar Maurer
36075475a6 src/api2/backup/environment.rs: log duplicate chunks 2019-06-14 07:12:30 +02:00
Dietmar Maurer
fa148dbd11 src/api2/backup/upload_chunk.rs: verify chunk data if possible 2019-06-14 06:23:46 +02:00
Dietmar Maurer
f98ac774ee backup: Add support for client side encryption 
first try ...
 2019-06-13 11:47:23 +02:00
Dietmar Maurer
6762db70d6 src/api2/types.rs: define CHUNK_DIGEST_SCHEMA 2019-06-13 07:30:42 +02:00
Wolfgang Bumiller
dd5495d6dc tree-wide: use 'dyn' for all trait objects 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-06-07 13:13:48 +02:00
Dietmar Maurer
4800df7ba6 src/api2/admin/datastore.rs: remove test-upload api 2019-06-07 07:52:11 +02:00
Dietmar Maurer
113898d7e9 src/api2/admin/datastore/upload.rs: remove unused upload api 2019-06-07 07:39:41 +02:00
Dietmar Maurer
986bef16be src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1 
So that we can include it in static doc strings.
 2019-06-05 08:41:20 +02:00
Dietmar Maurer
c9ec0956cf src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1 2019-06-05 08:12:13 +02:00
Dietmar Maurer
7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Dietmar Maurer
339ddfcbfa src/cli/command.rs: move doc generator code to src/api_schema/format.rs 2019-06-04 12:32:22 +02:00
Dietmar Maurer
39d6846e1e src/api2/admin/datastore/backup.rs: implement config file upload 2019-06-03 09:45:14 +02:00
Dietmar Maurer
36898ffce6 src/backup/chunk_stream.rs: add optional chunk_size parameter 2019-05-30 13:28:24 +02:00
Dietmar Maurer
96482891ae src/api2/admin/datastore/backup/environment.rs: log upload statistics 2019-05-30 09:21:19 +02:00
Dietmar Maurer
642322b433 src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload 
So that we can provide better statistics (patches follows).
 2019-05-30 09:21:19 +02:00
Dietmar Maurer
e6389f4e75 src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files) 2019-05-29 10:38:57 +02:00
Dietmar Maurer
39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Dietmar Maurer
a42d1f55c3 src/api2/admin/datastore/backup.rs: add debug flag 2019-05-29 09:35:21 +02:00
Dietmar Maurer
976595e1a9 src/backup/dynamic_index.rs: split class DynamicIndexWriter 2019-05-29 08:49:57 +02:00
Dietmar Maurer
60e589a111 src/api2/admin/datastore/backup.rs: log sucessful finish call 2019-05-28 12:55:22 +02:00
Dietmar Maurer
006f3ff407 src/api2/admin/datastore/backup/environment.rs: add more consistency checks 2019-05-28 09:21:31 +02:00
Dietmar Maurer
3dc5b2a203 src/client/http_client.rs: use start of chunk as offset 2019-05-28 09:21:31 +02:00
Dietmar Maurer
e3d525feb7 src/api2/admin/datastore/backup.rs: fix typo 2019-05-28 07:13:19 +02:00
Dietmar Maurer
a42fa400ee src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api 2019-05-28 06:18:55 +02:00
Dietmar Maurer
a1e7cff3ea src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk 2019-05-27 07:57:43 +02:00
Dietmar Maurer
771953f9eb src/api2/admin/datastore/backup.rs: use smaller window sizes 2019-05-27 07:53:40 +02:00
Dietmar Maurer
05cba08c9c src/client/http_client.rs: append chunks in correct order 
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
 2019-05-26 09:57:34 +02:00
Dietmar Maurer
417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset 
This clearly shows that the current approach does not work - seems we get
chunks out of order.
 2019-05-26 08:49:29 +02:00
Wolfgang Bumiller
cc84a830c5 RFC: schema: make enums static 
I don't see a reason to allow these to be dynamically
modifiable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 12:38:51 +02:00
Dietmar Maurer
aa1b2e04fe src/client/merge_known_chunks.rs: merge known chunks 
To decrease the number of api calls required...
 2019-05-23 12:31:09 +02:00
Dietmar Maurer
8bea85b42e src/api2/admin/datastore/backup.rs: verify file size and chunk count on close 2019-05-23 08:50:36 +02:00
Dietmar Maurer
907f5bb0d8 src/api2/admin/datastore/backup/service.rs: improve error logging 2019-05-23 08:05:39 +02:00
Dietmar Maurer
a55fcd740f src/api2/admin/datastore/backup.rs: suse POST for finish() api 2019-05-22 13:04:58 +02:00
Dietmar Maurer
5a0b521809 src/api2/admin/datastore/backup.rs: remove test callbacks 2019-05-22 10:55:27 +02:00
Dietmar Maurer
d0afeaa831 src/api2/admin/datastore/backup.rs: improve error message 2019-05-22 07:43:54 +02:00
Dietmar Maurer
7f3d2ffab9 src/backup/index.rs: rename ChunkListReader into DigestListEncoder 2019-05-21 12:28:44 +02:00
Dietmar Maurer
a95849321c src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks 2019-05-21 09:54:11 +02:00
Dietmar Maurer
a09c0e38d8 src/api2/admin/datastore/backup/environment.rs: register/lookup known chunks 2019-05-20 18:05:10 +02:00
Dietmar Maurer
82ab72304e src/client/http_client.rs: implement upload_stream 2019-05-20 14:19:24 +02:00
Dietmar Maurer
adec8ea263 src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed 
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
 2019-05-16 10:24:23 +02:00
Dietmar Maurer
372724afea src/api2/admin/datastore/backup.rs: implement finish_backup 2019-05-15 12:58:55 +02:00
Dietmar Maurer
bb105f9dae src/api2/admin/datastore/backup.rs: improve logging 2019-05-15 10:09:46 +02:00
Dietmar Maurer
a20772525f src/api2/admin/datastore/backup.rs: implement close for dynamic writers 2019-05-15 07:58:05 +02:00
Dietmar Maurer
4e93f8c164 src/api2/types.rs: add schema/format for file names 2019-05-15 07:14:08 +02:00
Dietmar Maurer
fbb798f6d2 src/api2/admin/datastore/backup.rs: ignore errors from last_backup 2019-05-13 07:06:56 +02:00
Dietmar Maurer
d36113665b src/api2/admin/datastore/backup.rs: implement chunk index download from previous backup 2019-05-11 16:05:50 +02:00
Dietmar Maurer
af1d68185c src/api2/admin/datastore/backup.rs: add missing 'archive-name' parameter to schema 2019-05-11 12:25:15 +02:00
Dietmar Maurer
6b95c7df07 src/api2/admin/datastore/backup/environment.rs: do not store path 2019-05-11 12:13:44 +02:00
Dietmar Maurer
51a4f63ff7 src/backup/datastore.rs: move last_backup to BackupInfo 2019-05-11 12:07:09 +02:00