Commit Graph

333 Commits

Author SHA1 Message Date
Dietmar Maurer
74d0a6bc23 src/bin/proxmox-backup-proxy.rs: set socket options for max. performance 2019-07-02 10:48:58 +02:00
Dietmar Maurer
93f8bb0445 src/bin/h2s-client.rs: avoid compiler warnings 2019-07-02 08:59:33 +02:00
Dietmar Maurer
dc9775d1b0 src/bin/h2s-client.rs: increase tokio tcp stream read buffer size
This improves download speed by factor 10.
2019-06-29 18:09:50 +02:00
Dietmar Maurer
34f7131160 src/bin/h2s-client.rs: avoid hyper, use h2 directly
But performance is still bad.
2019-06-29 15:58:18 +02:00
Dietmar Maurer
5ed2277f0e src/bin/h2s-server.rs: test to reproduce slow h2 download
Simplified code, use with h2s-client.rs.
2019-06-29 14:57:54 +02:00
Dietmar Maurer
17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
Dietmar Maurer
872debdefa src/bin/h2client.rs: tune h2 client parameters to get reasonable speed
We now get about 2.5GB/s (not really good, but better)
2019-06-29 10:41:24 +02:00
Dietmar Maurer
fded1f3154 add code to test H2 speed
I currently get about 1MB/s, which is ways too slow (must be a bug)?
2019-06-29 10:08:12 +02:00
Dietmar Maurer
b208da8393 src/backup/data_{chunk,blob}.rs: add verify_crc method 2019-06-26 09:54:25 +02:00
Dietmar Maurer
fef44d4f78 src/bin/proxmox-backup-client.rs: decode blobs on download
No more need for that clumsy blob cli (removed).
2019-06-26 09:18:59 +02:00
Dietmar Maurer
bb8231409e src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json
Use the KeyConfig serialization with kdf = None.
2019-06-26 07:32:34 +02:00
Dietmar Maurer
3031e44c58 src/bin/proxmox-backup-client.rs: remove stale download file, start blob cli 2019-06-25 13:00:37 +02:00
Dietmar Maurer
45db6f89e1 src/bin/proxmox-backup-client.rs: implement download command 2019-06-25 11:17:24 +02:00
Dietmar Maurer
6899dbfb47 src/bin/proxmox-backup-client.rs: strip .blob extension 2019-06-25 07:11:02 +02:00
Dietmar Maurer
045e8a568d src/bin/proxmox-backup-client.rs: change key name to rsa-encrypted.key 2019-06-25 07:00:07 +02:00
Dietmar Maurer
9f46c7de4b src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup 2019-06-24 13:56:37 +02:00
Dietmar Maurer
cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer
a7f67a9a9c depend on crc32fast
We can use this to compute fast checksums to test file integrity (not crypto safe).
2019-06-21 15:58:00 +02:00
Dietmar Maurer
3ea8bfc9c0 src/bin/proxmox-backup-client.rs: add import-master-pubkey command 2019-06-21 07:34:17 +02:00
Dietmar Maurer
37c5a17505 src/bin/proxmox-backup-client.rs: helper to generate RSA master key 2019-06-21 07:12:19 +02:00
Dietmar Maurer
ab44acff57 src/backup/key_derivation.rs: add modified field to key file 2019-06-20 07:58:33 +02:00
Dietmar Maurer
6d0983dbe1 src/bin/proxmox-backup-client.rs: add keyfile parameter for backup 2019-06-19 17:16:41 +02:00
Dietmar Maurer
181f097af9 src/bin/proxmox-backup-client.rs - key API: pass kdf parameter
Allow to store keys without password.
2019-06-18 18:21:01 +02:00
Dietmar Maurer
9b06db4563 src/bin/proxmox-backup-client.rs: use file system path to reference keys 2019-06-18 17:38:50 +02:00
Dietmar Maurer
091540298e src/bin/proxmox-backup-client.rs - create_key: always read password from tty 2019-06-18 17:27:52 +02:00
Dietmar Maurer
826f309bf5 src/backup/key_derivation.rs: move kdf code into separate file 2019-06-18 11:17:22 +02:00
Dietmar Maurer
ac71623483 src/bin/proxmox-backup-client.rs: implement "key change-passphrase" 2019-06-17 13:10:00 +02:00
Dietmar Maurer
f2401311b0 src/bin/proxmox-backup-client.rs: start key management 2019-06-17 10:33:24 +02:00
Dietmar Maurer
0f37577092 src/bin/cipherbench.rs: add test for zstd compression 2019-06-13 17:24:57 +02:00
Dietmar Maurer
9399c98f82 src/bin/cipherbench.rs: cipher speed test with large blocks 2019-06-13 17:16:43 +02:00
Dietmar Maurer
f98ac774ee backup: Add support for client side encryption
first try ...
2019-06-13 11:47:23 +02:00
Wolfgang Bumiller
dd5495d6dc tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
Dietmar Maurer
4247fccb0f src/client/http_client.rs: automatically close connection in finish 2019-06-05 11:41:50 +02:00
Wolfgang Bumiller
cb4426b348 make h2 client connection cancellable
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-05 09:45:59 +02:00
Dietmar Maurer
7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Christian Ebner
97eeea3b4a src/bin/proxmox-backup-proxy.rs: fix typo in pkcs12 error message
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-06-05 06:29:02 +02:00
Dietmar Maurer
22e5aa1d3e src/bin/dump-backup-api.rs: helper to generate backup API docs 2019-06-04 13:12:42 +02:00
Dietmar Maurer
ec8a9bb953 src/bin/proxmox-backup-client.rs: implement upload_config 2019-06-03 10:39:44 +02:00
Dietmar Maurer
1c0472e88f avoid compiler warnings 2019-06-03 06:43:47 +02:00
Dietmar Maurer
36898ffce6 src/backup/chunk_stream.rs: add optional chunk_size parameter 2019-05-30 13:28:24 +02:00
Dietmar Maurer
49ef316bcd src/bin/proxmox-backup-client.rs: backup_image() - pass correct chunk size 2019-05-30 12:52:12 +02:00
Dietmar Maurer
202cadc29e src/bin/proxmox-backup-client.rs: remove unused code 2019-05-30 12:47:35 +02:00
Dietmar Maurer
6af905c18f src/bin/proxmox-backup-client.rs: implement image/block-device upload 2019-05-30 12:46:01 +02:00
Dietmar Maurer
eb1804c567 src/bin/proxmox-backup-client.rs: use std lib instead of nix 2019-05-30 12:19:23 +02:00
Dietmar Maurer
6e907505aa src/bin/proxmox-backup-client.rs: fix file type test 2019-05-30 12:08:34 +02:00
Dietmar Maurer
39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Christian Ebner
9b38443314 pxar: change logic and impl all command line flags for xattrs/fcaps/acls
Allows to individually set the flags for storing/dumping/restoring of
xattrs/fcaps/acls in the cli of pxar.
Changes logic so that each of them can be threated individually.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-29 06:51:20 +02:00
Dietmar Maurer
c4ff3dcefd src/bin/proxmox-backup-client.rs: use new BackupClient to upload data 2019-05-28 10:12:44 +02:00
Christian Ebner
af309d4d5c src/bin/pxar.rs: set archive permissions to 0o640
As the archive can contain potentially sensitive data such as key files, it makes
sense to restrict the permissions.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-28 06:24:47 +02:00
Christian Ebner
b344461b33 pxar: pass feature_flags to encoder/decoder instead of individual flags
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:11:36 +02:00
Dietmar Maurer
32e1bf4260 src/bin/upload-speed.rs: code cleanup 2019-05-23 07:20:05 +02:00
Christian Ebner
0d9bab05ab src/bin/pxar.rs: impl CLI flags to disable archiving/restoring of xattrs and fcaps.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-23 06:40:50 +02:00
Dietmar Maurer
6ab34afa88 src/client/http_client.rs: cleanup h2 backup client 2019-05-22 13:05:51 +02:00
Dietmar Maurer
de6742c04f src/bin/h2test.rs: removed
This test does no work anymore.
2019-05-22 11:14:56 +02:00
Dietmar Maurer
751030aa7c src/bin/test_chunk_speed2.rs: test max chunk size 2019-05-19 11:09:02 +02:00
Dietmar Maurer
6276448519 src/bin/test_chunk_speed2.rs: test chunker speed with real data 2019-05-18 11:59:17 +02:00
Dietmar Maurer
adec8ea263 src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
2019-05-16 10:24:23 +02:00
Dietmar Maurer
850ac6d0f0 src/client/http_client.rs: directly return H2Client on upgrade 2019-05-13 16:44:59 +02:00
Dietmar Maurer
b57cb26406 src/client/http_client.rs: implement http2 client wrapper 2019-05-13 10:27:22 +02:00
Dietmar Maurer
9e391bb7f0 src/client/http_client.rs: allow to pass parameters (encode them as query string) 2019-05-13 09:12:03 +02:00
Dietmar Maurer
0aadd40b4f src/api2/admin/datastore/backup.rs: add required parameters, cleanups 2019-05-08 13:17:23 +02:00
Dietmar Maurer
ca60c371e2 api: rename h2upload to backup 2019-05-08 12:35:15 +02:00
Dietmar Maurer
45cdce069e src/client/http_client.rs: request password at creation time 2019-04-30 11:44:35 +02:00
Dietmar Maurer
3a9bea3918 src/bin/h2test.rs: test h2 upgrade 2019-04-29 11:59:18 +02:00
Dietmar Maurer
5a2df00004 src/client/http_client.rs: cleanup, make login fully async 2019-04-28 10:55:03 +02:00
Wolfgang Bumiller
e888155712 proxy: allow .key/.pem files instead of a pkcs12/.pfx file
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-26 10:47:33 +02:00
Wolfgang Bumiller
d98c9a7a38 use service Type=notify
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:01:33 +00:00
Dietmar Maurer
5f550fd99f start hyper server using with_graceful_shutdown()
Without, hyper keeps some futures running, and the server does not
correctly shutdown.
2019-04-10 08:24:32 +02:00
Dietmar Maurer
d607b8861b src/server/worker_task.rs: implement task control socket 2019-04-09 12:15:06 +02:00
Dietmar Maurer
e3f41f2165 src/tools/daemon.rs: use new ServerState handler 2019-04-08 14:00:23 +02:00
Dietmar Maurer
35950380a9 src/server/worker_task.rs: carefully handle file permissions 2019-04-06 17:53:12 +02:00
Dietmar Maurer
49cf9f3d0b src/tools.rs: file_get_json() - add new default parameter 2019-04-04 12:24:18 +02:00
Wolfgang Bumiller
a690ecacf0 daemon: simplify daemon creation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 12:12:54 +01:00
Wolfgang Bumiller
e4311382d9 tools: daemon: rename some structs
Reloadable resources are now 'Reloadable' instead of
'ReexecContinue'.

The struct handling the reload is a 'Reloader', not a
'ReexecStore'.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 11:09:46 +01:00
Wolfgang Bumiller
5e7bc50af1 add reload support to api daemon
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
a2ca713701 add reload support to proxy
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
1c042cdc6c proxy: listener error handling fixup
If the listening socket goes into some error state we'll get
std::io::Errors rather than higher level errors from the
native_tls::TlsAcceptor, those are usually fatal. (Ran into
this after performing a shutdown() on the file descriptor,
after which the future just endlessly loops in accept().)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Dietmar Maurer
ba10f2b0d4 src/bin/dynamic-files.rs: test case for file shrink during backup 2019-03-16 14:22:14 +01:00
Dietmar Maurer
3dbfe5b142 simplify pxar module structure 2019-03-15 08:24:32 +01:00
Dietmar Maurer
1ef46b81d3 src/bin/pxar.rs: implement extract 2019-03-15 08:03:44 +01:00
Dietmar Maurer
5defa71bf9 rename PxarBackupWriter into PxarDecodeWriter 2019-03-15 07:20:22 +01:00
Dietmar Maurer
37940aa1f9 src/bin/pxar.rs: implement dump/list 2019-03-14 17:43:11 +01:00
Dietmar Maurer
40360fde6e src/pxar/decoder.rs: implement dump_archive
use this to debug archive problems ...
2019-03-14 13:10:27 +01:00
Dietmar Maurer
8968258b66 rename catar into pxar
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
Dietmar Maurer
7c4dd94670 rename catar into pxar
Seems impossible to stay compatible with the original catar specification,
so we change the name to avoid confusions.
2019-03-14 10:14:32 +01:00
Dietmar Maurer
08dc340a12 src/bin/proxmox-backup-client.rs: complete archive names 2019-03-14 08:09:35 +01:00
Wolfgang Bumiller
6f90a6a764 protocol: cleanup finish_backup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-13 14:16:17 +01:00
Dietmar Maurer
b238851832 src/bin/proxmox-backup-client.rs: implement completion for snapshots 2019-03-13 13:31:29 +01:00
Dietmar Maurer
4981134708 src/bin/proxmox-backup-client.rs: implement chunk-size completion 2019-03-13 12:26:01 +01:00
Dietmar Maurer
98f0b97293 src/bin/proxmox-backup-client.rs: cleanup 2019-03-13 12:17:39 +01:00
Dietmar Maurer
024f11bb55 src/bin/proxmox-backup-client.rs: implement completion for backup groups 2019-03-13 11:56:37 +01:00
Dietmar Maurer
edd3c8c605 src/client/backup_repo.rs: implement FromStr trait 2019-03-13 10:09:39 +01:00
Dietmar Maurer
d0a03d40ce src/bin/proxmox-backup-client.rs: implement repository completions 2019-03-13 09:47:12 +01:00
Dietmar Maurer
bff110308b src/bin/proxmox-backup-client.rs: fix BACKUPSPEC_REGEX - correctly specify non-capture group
And improve bash completion.
2019-03-13 07:23:59 +01:00
Dietmar Maurer
496a67846f src/cli/command.rs: pass parsed parameters to completion function 2019-03-12 14:39:51 +01:00
Dietmar Maurer
d5c34d98c0 src/catar/decoder.rs: simplify public restore API 2019-03-11 14:31:01 +01:00
Dietmar Maurer
8e39232acc src/bin/proxmox-backup-client.rs: strip .didx file extensions 2019-03-11 10:54:02 +01:00
Dietmar Maurer
74cdb52107 src/bin/proxmox-backup-client.rs: include file extension in backupspec 2019-03-11 10:27:28 +01:00
Dietmar Maurer
e9c9409a99 src/catar/decoder.rs: implement real sequential decoder (no Seek)
I copied the old code to src/catar/inspector.rs. Will modify
that later to implement a random access decoder ...
2019-03-08 16:55:54 +01:00
Dietmar Maurer
eed6db3923 src/bin/proxmox-backup-client.rs: impl option all-file-systems 2019-03-08 09:36:40 +01:00
Dietmar Maurer
e3c30c50a6 src/bin/catar.rs: implement all-file-systems option 2019-03-08 09:28:12 +01:00
Dietmar Maurer
2689810c92 src/bin/catar.rs: implement verbose flags 2019-03-08 08:20:56 +01:00
Dietmar Maurer
79c281fdac src/catar/encoder.rs: use simply flag "all_file_systems" 2019-03-08 08:14:26 +01:00
Dietmar Maurer
2ae7d196b9 src/bin/proxmox-backup-client.rs: correctly strip file extension 2019-03-06 11:18:46 +01:00
Dietmar Maurer
9f91249351 src/bin/proxmox-backup-client.rs: impl restore 2019-03-06 11:02:08 +01:00
Wolfgang Bumiller
6716f30bbc add protocol test client
This uses futures for everything which is mostly useful as a
test to see if the protocol crate's non-blocking I/O support
can handle it...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
Dietmar Maurer
80822b95b1 src/bin/proxmox-backup-client.rs: sort list results 2019-03-06 07:20:51 +01:00
Dietmar Maurer
fba30411fb src/bin/proxmox-backup-client.rs: allow to set backup-id 2019-03-06 07:02:52 +01:00
Wolfgang Bumiller
391d310741 backup_info: BackupDir shortcut
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
Dietmar Maurer
3ec3ec3f9a src/bin/proxmox-backup-client.rs: print backup duration 2019-03-05 08:11:40 +01:00
Dietmar Maurer
875fb1c01a src/backup/datastore.rs: use unix epoch to create DateTime
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
Dietmar Maurer
9b492eb256 src/backup/datastore.rs: protect BackupDir fields, impl new() 2019-03-04 13:51:36 +01:00
Dietmar Maurer
1e9a94e579 src/backup/datastore.rs: protect BackupGroup fields, impl new() 2019-03-04 13:38:23 +01:00
Dietmar Maurer
cdebd467e6 src/bin/proxmox-backup-client.rs: simply log to stdout 2019-03-04 09:38:34 +01:00
Dietmar Maurer
219ef0e6ca src/catar/encoder.rs: add verbose flags 2019-03-04 08:01:09 +01:00
Dietmar Maurer
597a92039e src/bin/proxmox-backup-client.rs: rename create into backup 2019-03-03 11:46:53 +01:00
Dietmar Maurer
6f62c9240a src/api2/admin/datastore.rs: imp delete_snapshot 2019-03-03 11:29:00 +01:00
Dietmar Maurer
e77a02ed87 catar api: cleanup parameter names 2019-03-03 09:48:32 +01:00
Dietmar Maurer
421c9ba1d9 src/bin/proxmox-backup-client.rs: use new json_object_to_query 2019-03-03 09:44:25 +01:00
Dietmar Maurer
f5f13ebc5a src/tools.rs: new helper json_object_to_query 2019-03-03 08:51:37 +01:00
Dietmar Maurer
49dc074007 parameter name cleanups (use '-' instead of '_') 2019-03-02 16:34:10 +01:00
Dietmar Maurer
ad20d19877 src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups 2019-03-02 16:28:36 +01:00
Dietmar Maurer
184f17afad src/api2/admin/datastore.rs: impl list_snapshots 2019-03-02 16:20:50 +01:00
Dietmar Maurer
812c6f8716 src/api2/admin/datastore.rs: impl list_backup_groups 2019-03-02 11:29:05 +01:00
Dietmar Maurer
e909522f06 src/backup/datastore.rs: use timestamp with zone offset 2019-03-01 17:36:40 +01:00
Dietmar Maurer
17d6979afd src/bin/proxmox-backup-client.rs: generate a log file 2019-03-01 09:35:41 +01:00
Dietmar Maurer
5a2892f0ae src/bin/proxmox-backup-client.rs: use chrono::DateTime 2019-03-01 06:59:32 +01:00
Dietmar Maurer
51144821f9 src/bin/proxmox-backup-client.rs: login before starting backup 2019-03-01 06:48:41 +01:00
Dietmar Maurer
83b7db0209 src/api2/admin/datastore.rs: start prune api
just a dummy for now.
2019-02-27 16:53:17 +01:00
Dietmar Maurer
f98ea63de2 src/bin/proxmox-backup-client.rs: fix bash completion 2019-02-27 12:37:53 +01:00
Dietmar Maurer
25f1650b71 src/bin/proxmox-backup-client.rs: minor code cleanup 2019-02-27 07:21:57 +01:00
Dietmar Maurer
ae0be2dda9 src/bin/proxmox-backup-client.rs: allow to upload more that one directory 2019-02-26 12:27:28 +01:00
Dietmar Maurer
698d9d4402 src/cli/command.rs: add help command 2019-02-23 15:10:48 +01:00
Dietmar Maurer
8b6dd2240d src/cli/command.rs::allow to generate ReST docu 2019-02-23 11:29:18 +01:00
Dietmar Maurer
33256db6dd src/cli/command.rs: simplify cli handler 2019-02-22 11:15:10 +01:00
Dietmar Maurer
4de0e142a0 cleanup cli library structure 2019-02-21 09:08:03 +01:00
Dietmar Maurer
8cc0d6aff3 src/bin/proxmox-backup-client.rs: implement garbage-collection sub-command 2019-02-20 14:10:45 +01:00
Dietmar Maurer
8c75372b79 src/backup/datastore.rs: list_backups() - include list of index files 2019-02-20 13:37:44 +01:00
Dietmar Maurer
14691fc1f3 src/bin/test_chunk_size.rs: avoid compilöer warnings, exit after 1GB 2019-02-20 12:05:28 +01:00
Dietmar Maurer
1604ec2654 src/bin/test_chunk_size.rs: another chunker test - gather some stats 2019-02-20 11:50:10 +01:00
Dietmar Maurer
247cdbce72 src/api2/admin/datastore/catar.rs: allow to configure chunk-size 2019-02-19 15:19:12 +01:00
Wolfgang Bumiller
fc7f03523c cleanup
Error::from is already a function taking 1 parameter,
there's no need to wrap it with `|e| Error::from(e)`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:21:27 +01:00
Dietmar Maurer
51ebd07974 src/api_schema/router.rs: add docu for RpcEnvironmentType 2019-02-17 11:59:18 +01:00
Dietmar Maurer
ef2f2efbcc improve api_schema module structure 2019-02-17 10:16:33 +01:00
Dietmar Maurer
dc9a007b11 rename src/api to src/api_schema 2019-02-17 09:59:20 +01:00
Dietmar Maurer
a8f268afbb fix configuration dir permission
Carefully set and check permissions ...
2019-02-16 12:19:13 +01:00
Dietmar Maurer
151c6ce27d src/client.rs: improve library structure and docu 2019-02-14 11:11:39 +01:00
Dietmar Maurer
0dffe3f99a src/client/http_client.rs: try to login
use an environment var to store passphrase (PBS_PASSWORD)
2019-02-13 14:31:43 +01:00
Dietmar Maurer
33d64b81e0 src/bin/proxmox-backup-client.rs: define and use BackupRepository 2019-02-13 12:30:52 +01:00
Dietmar Maurer
e5064ba607 simplify backup lib structure (pub use xxx:*), improve doc 2019-02-12 14:13:31 +01:00