tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
1,598 Commits 2 Branches 54 Tags 14 MiB
bf6e321744
Commit Graph

333 Commits

Author SHA1 Message Date
Dietmar Maurer
977eeb24f6 src/bin/proxmox-backup-client.rs: avoid doubled file extensions (.blob.blob) in index.json 2019-08-07 10:23:08 +02:00
Dietmar Maurer
4af0ee055a api: always use complete file names (including add exctensions) 2019-08-07 10:10:14 +02:00
Dietmar Maurer
c4f025ebbf src/bin/proxmox-backup-client.rs: fix remote filename completion 2019-08-07 06:48:18 +02:00
Dietmar Maurer
c807d231eb src/client/http_client.rs: compute file checksums 2019-08-06 11:26:06 +02:00
Dietmar Maurer
a17a0e7a9f src/api2/admin/datastore.rs: extract backup size from index.json 2019-08-06 10:56:21 +02:00
Dietmar Maurer
8c70e3eb18 src/api2/admin/datastore.rs - files: return data from index.json 2019-08-05 13:22:19 +02:00
Dietmar Maurer
f69adc81a6 use new proxmox::tools::nodename 2019-08-03 17:06:23 +02:00
Dietmar Maurer
e18a6c9ee5 update to nix 0.14, use code from proxmox:tools 2019-08-03 13:05:38 +02:00
Christian Ebner
4d142ea79e pxar: cleanup: refactor and rename exclude pattern 
The original name PxarExcludePattern makes no sense anymore as the patterns are
also used to match filenames during restore of the archive.

Therefore, exclude_pattern.rs is moved to match_pattern.rs and PxarExcludePattern
rename to MatchPattern.
Further, since it makes more sense the MatchTypes are now declared as None,
Positive, Negative, PartialPositive or PartialNegative, as this makes more sense
and seems more readable.
Positive matches are those without '!' prefix, Negatives with '!' prefix.

This makes also the filename matching in the encoder/decoder more intuitive and
the logic was adapted accordingly.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:52:32 +02:00
Christian Ebner
47651f9530 pxar: cleanup: move feature flags to src/pxar/flags.rs and omit CA_FORMAT prefix on all of them 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:51:06 +02:00
Dietmar Maurer
b335f5b713 src/bin/proxmox-backup-client.rs: sign index.json 2019-08-02 10:00:15 +02:00
Christian Ebner
81a9905e0a pxar: implement feature flag support for device nodes, fifos and sockets 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:55:17 +02:00
Dietmar Maurer
2c3891d1c3 src/bin/proxmox-backup-client.rs: upload backup index.json 
The plan is to use this file to verify the backup content.
 2019-08-01 12:39:02 +02:00
Dietmar Maurer
cec17a3ec7 src/bin/proxmox-backup-client.rs - files: allow --repository parameter 2019-07-31 11:45:55 +02:00
Dietmar Maurer
52c171e49d src/bin/proxmox-backup-client.rs: implement files command 2019-07-31 11:23:53 +02:00
Dietmar Maurer
43a406fd3f src/bin/proxmox-backup-client.rs: supress useless output from prune command 2019-07-31 10:15:16 +02:00
Christian Ebner
6a87910949 pxar: implement allow_existing_dirs for pxar decoder 
By default, restoring an archive will fail if files with the same filename
already exist in the target directory.

By setting the allow_existing_dirs flag, the restore will not fail if an
existing directory is encountered.
The metadata (permissions, acls, ...) of the existing directory will be set
to the ones from the archive.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-30 09:40:30 +02:00
Dietmar Maurer
46d5aa0a09 src/bin/proxmox-backup-client.rs: add restore flag --allow-existing-dirs 
Not jet implemented ...
 2019-07-29 12:49:15 +02:00
Dietmar Maurer
ea7a7ef299 src/bin/proxmox-backup-client.rs: fixup - correctly pass prune parameters 2019-07-27 09:24:23 +02:00
Dietmar Maurer
9fdc3ef46f src/api2/admin/datastore.rs: pass backup type/id (group) to prune 
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
 2019-07-27 08:49:14 +02:00
Dietmar Maurer
7a6cfbd98d src/bin/proxmox-backup-client.rs: correctly compute duration 
And format time with SecondsFormat::Secs
 2019-07-26 11:13:49 +02:00
Dietmar Maurer
bbf9e7e951 src/api2/backup.rs: cleanup schema definitions 2019-07-26 09:07:29 +02:00
Dietmar Maurer
ca5d0b61ca src/api2/backup.rs: new required backup-time parameter 
The client should pass the time as parameter.
 2019-07-25 13:44:01 +02:00
Dietmar Maurer
ec34f7eba0 src/bin/proxmox-backup-client.rs: implement upload-log 2019-07-25 12:18:15 +02:00
Dietmar Maurer
79679c2d6e src/bin/proxmox-backup-client.rs: allow to uploag .log files (as binary blobs) 2019-07-24 13:07:02 +02:00
Dietmar Maurer
5b72c9b4f7 src/pxar/encoder.rs: add new skip_lost_and_found parameter 2019-07-24 12:21:25 +02:00
Dietmar Maurer
2eeaacb974 src/pxar/encoder.rs: allow to pass list of devices 
For better mount point include control...
 2019-07-24 08:11:59 +02:00
Dietmar Maurer
fa5d6977dd Revert previous, commit, use UTC RFC3339 without timezone (Z) 
We now have human readable data/time, and names are still sortable.
 2019-07-22 11:49:30 +02:00
Dietmar Maurer
40a13369a3 src/bin/pxar.rs - extract: print archive name with --verbose flag 2019-07-19 07:01:57 +02:00
Dietmar Maurer
bbd4365c61 src/bin/pxar.rs: remove wrong debug message 2019-07-19 07:00:17 +02:00
Christian Ebner
a0ec687cd7 src/bin/pxar.rs: allow to pass paths and match patterns as args to pxar extract 
To improve usability it is now possible to directly pass paths or match patterns
as arguments to pxar extract to partially restore an archive.
The patterns provided via CLI are appended to the ones read from file by the
--files-from option in order to have priority over those.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-19 06:50:39 +02:00
Dietmar Maurer
9aa3f68278 src/bin/proxmox-backup-client.rs: code cleanup 2019-07-18 13:16:32 +02:00
Dietmar Maurer
f6ede796ca src/bin/proxmox-backup-client.rs - status: only print result[data] 2019-07-18 09:52:11 +02:00
Dietmar Maurer
15c847f136 src/api2/admin/datastore.rs: allow to list all snapshots 2019-07-18 09:11:43 +02:00
Christian Ebner
129dda4729 src/bin/pxar.rs: add --files-from parameter to pxar extract 
Allows to pass a list of match pattern to the pxar extract call by reading them
from file.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:20:29 +02:00
Christian Ebner
fa7e957ca3 pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:19:31 +02:00
Christian Ebner
bdf0d82ced src/bin/pxar.rs: Make pxar extract target optional 
In order to improve usablity, the target on archive extraction will be the
current working directory by default.
A different target can be provided via the optional --target <PATH> parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 09:40:02 +02:00
Dietmar Maurer
34a816cc7b src/bin/proxmox-backup-client.rs: add output-fromat options, implement status api 2019-07-16 13:35:25 +02:00
Dietmar Maurer
aad2ee4947 src/bin/pxar.rs: only print archive name in verbose mode 2019-07-16 08:38:02 +02:00
Dietmar Maurer
6b9a071028 src/bin/pxar.rs: add --verbose parameter to list (replace dump) 2019-07-16 08:31:40 +02:00
Dietmar Maurer
2665cef703 src/bin/proxmox-backup-client.rs: make repository parameter optional 
And allow to read it from $ENV{PBS_REPOSITORY}
 2019-07-16 07:30:04 +02:00
Dietmar Maurer
9eae781ab8 src/bin/pxar.rs - extract: allow to read from stdin 2019-07-05 12:39:25 +02:00
Dietmar Maurer
c6c9e093d0 src/bin/pxar.rs - dump: allow to read from stdin 2019-07-05 12:32:15 +02:00
Dietmar Maurer
bf1252619a src/bin/proxmox-backup-client.rs - restore: allow to pipe output to stdout 2019-07-05 12:14:50 +02:00
Dietmar Maurer
f8100e9623 src/bin/proxmox-backup-client.rs: remove download cli, do everything in restore 2019-07-05 11:36:45 +02:00
Dietmar Maurer
f4bf7dfcc7 src/client/remote_chunk_reader.rs: implement simple caching 2019-07-05 10:42:46 +02:00
Dietmar Maurer
afb4cd28be src/backup/fixed_index.rs: implement BufferedFixedReader 2019-07-05 09:19:56 +02:00
Christian Ebner
7dcbe051e9 pxar: Refactor SequentialDecoder to store the callback function within the struct 
Reduces the number of arguments for the function calls within the decoder.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-05 06:38:22 +02:00
Dietmar Maurer
86eda3eb0d src/bin/proxmox-backup-client.rs: implement restore using BackupReader 2019-07-03 15:45:12 +02:00
Dietmar Maurer
6d1f61b208 use hyper/tokio-openssl instead of hyper/tokio-tls 
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
 2019-07-02 13:36:28 +02:00
Dietmar Maurer
74d0a6bc23 src/bin/proxmox-backup-proxy.rs: set socket options for max. performance 2019-07-02 10:48:58 +02:00
Dietmar Maurer
93f8bb0445 src/bin/h2s-client.rs: avoid compiler warnings 2019-07-02 08:59:33 +02:00
Dietmar Maurer
dc9775d1b0 src/bin/h2s-client.rs: increase tokio tcp stream read buffer size 
This improves download speed by factor 10.
 2019-06-29 18:09:50 +02:00
Dietmar Maurer
34f7131160 src/bin/h2s-client.rs: avoid hyper, use h2 directly 
But performance is still bad.
 2019-06-29 15:58:18 +02:00
Dietmar Maurer
5ed2277f0e src/bin/h2s-server.rs: test to reproduce slow h2 download 
Simplified code, use with h2s-client.rs.
 2019-06-29 14:57:54 +02:00
Dietmar Maurer
17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server 
I get about 30MB/s (much too slow)
 2019-06-29 13:43:10 +02:00
Dietmar Maurer
872debdefa src/bin/h2client.rs: tune h2 client parameters to get reasonable speed 
We now get about 2.5GB/s (not really good, but better)
 2019-06-29 10:41:24 +02:00
Dietmar Maurer
fded1f3154 add code to test H2 speed 
I currently get about 1MB/s, which is ways too slow (must be a bug)?
 2019-06-29 10:08:12 +02:00
Dietmar Maurer
b208da8393 src/backup/data_{chunk,blob}.rs: add verify_crc method 2019-06-26 09:54:25 +02:00
Dietmar Maurer
fef44d4f78 src/bin/proxmox-backup-client.rs: decode blobs on download 
No more need for that clumsy blob cli (removed).
 2019-06-26 09:18:59 +02:00
Dietmar Maurer
bb8231409e src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json 
Use the KeyConfig serialization with kdf = None.
 2019-06-26 07:32:34 +02:00
Dietmar Maurer
3031e44c58 src/bin/proxmox-backup-client.rs: remove stale download file, start blob cli 2019-06-25 13:00:37 +02:00
Dietmar Maurer
45db6f89e1 src/bin/proxmox-backup-client.rs: implement download command 2019-06-25 11:17:24 +02:00
Dietmar Maurer
6899dbfb47 src/bin/proxmox-backup-client.rs: strip .blob extension 2019-06-25 07:11:02 +02:00
Dietmar Maurer
045e8a568d src/bin/proxmox-backup-client.rs: change key name to rsa-encrypted.key 2019-06-25 07:00:07 +02:00
Dietmar Maurer
9f46c7de4b src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup 2019-06-24 13:56:37 +02:00
Dietmar Maurer
cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer
a7f67a9a9c depend on crc32fast 
We can use this to compute fast checksums to test file integrity (not crypto safe).
 2019-06-21 15:58:00 +02:00
Dietmar Maurer
3ea8bfc9c0 src/bin/proxmox-backup-client.rs: add import-master-pubkey command 2019-06-21 07:34:17 +02:00
Dietmar Maurer
37c5a17505 src/bin/proxmox-backup-client.rs: helper to generate RSA master key 2019-06-21 07:12:19 +02:00
Dietmar Maurer
ab44acff57 src/backup/key_derivation.rs: add modified field to key file 2019-06-20 07:58:33 +02:00
Dietmar Maurer
6d0983dbe1 src/bin/proxmox-backup-client.rs: add keyfile parameter for backup 2019-06-19 17:16:41 +02:00
Dietmar Maurer
181f097af9 src/bin/proxmox-backup-client.rs - key API: pass kdf parameter 
Allow to store keys without password.
 2019-06-18 18:21:01 +02:00
Dietmar Maurer
9b06db4563 src/bin/proxmox-backup-client.rs: use file system path to reference keys 2019-06-18 17:38:50 +02:00
Dietmar Maurer
091540298e src/bin/proxmox-backup-client.rs - create_key: always read password from tty 2019-06-18 17:27:52 +02:00
Dietmar Maurer
826f309bf5 src/backup/key_derivation.rs: move kdf code into separate file 2019-06-18 11:17:22 +02:00
Dietmar Maurer
ac71623483 src/bin/proxmox-backup-client.rs: implement "key change-passphrase" 2019-06-17 13:10:00 +02:00
Dietmar Maurer
f2401311b0 src/bin/proxmox-backup-client.rs: start key management 2019-06-17 10:33:24 +02:00
Dietmar Maurer
0f37577092 src/bin/cipherbench.rs: add test for zstd compression 2019-06-13 17:24:57 +02:00
Dietmar Maurer
9399c98f82 src/bin/cipherbench.rs: cipher speed test with large blocks 2019-06-13 17:16:43 +02:00
Dietmar Maurer
f98ac774ee backup: Add support for client side encryption 
first try ...
 2019-06-13 11:47:23 +02:00
Wolfgang Bumiller
dd5495d6dc tree-wide: use 'dyn' for all trait objects 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-06-07 13:13:48 +02:00
Dietmar Maurer
4247fccb0f src/client/http_client.rs: automatically close connection in finish 2019-06-05 11:41:50 +02:00
Wolfgang Bumiller
cb4426b348 make h2 client connection cancellable 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-06-05 09:45:59 +02:00
Dietmar Maurer
7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Christian Ebner
97eeea3b4a src/bin/proxmox-backup-proxy.rs: fix typo in pkcs12 error message 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-06-05 06:29:02 +02:00
Dietmar Maurer
22e5aa1d3e src/bin/dump-backup-api.rs: helper to generate backup API docs 2019-06-04 13:12:42 +02:00
Dietmar Maurer
ec8a9bb953 src/bin/proxmox-backup-client.rs: implement upload_config 2019-06-03 10:39:44 +02:00
Dietmar Maurer
1c0472e88f avoid compiler warnings 2019-06-03 06:43:47 +02:00
Dietmar Maurer
36898ffce6 src/backup/chunk_stream.rs: add optional chunk_size parameter 2019-05-30 13:28:24 +02:00
Dietmar Maurer
49ef316bcd src/bin/proxmox-backup-client.rs: backup_image() - pass correct chunk size 2019-05-30 12:52:12 +02:00
Dietmar Maurer
202cadc29e src/bin/proxmox-backup-client.rs: remove unused code 2019-05-30 12:47:35 +02:00
Dietmar Maurer
6af905c18f src/bin/proxmox-backup-client.rs: implement image/block-device upload 2019-05-30 12:46:01 +02:00
Dietmar Maurer
eb1804c567 src/bin/proxmox-backup-client.rs: use std lib instead of nix 2019-05-30 12:19:23 +02:00
Dietmar Maurer
6e907505aa src/bin/proxmox-backup-client.rs: fix file type test 2019-05-30 12:08:34 +02:00
Dietmar Maurer
39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Christian Ebner
9b38443314 pxar: change logic and impl all command line flags for xattrs/fcaps/acls 
Allows to individually set the flags for storing/dumping/restoring of
xattrs/fcaps/acls in the cli of pxar.
Changes logic so that each of them can be threated individually.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-29 06:51:20 +02:00
Dietmar Maurer
c4ff3dcefd src/bin/proxmox-backup-client.rs: use new BackupClient to upload data 2019-05-28 10:12:44 +02:00
Christian Ebner
af309d4d5c src/bin/pxar.rs: set archive permissions to 0o640 
As the archive can contain potentially sensitive data such as key files, it makes
sense to restrict the permissions.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-28 06:24:47 +02:00
Christian Ebner
b344461b33 pxar: pass feature_flags to encoder/decoder instead of individual flags 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-05-24 06:11:36 +02:00