2761d6a4f3
src/pxar/encoder.rs: use BackupCatalogWriter
2019-08-09 09:46:49 +02:00
977eeb24f6
src/bin/proxmox-backup-client.rs: avoid doubled file extensions (.blob.blob) in index.json
2019-08-07 10:23:08 +02:00
4af0ee055a
api: always use complete file names (including add exctensions)
2019-08-07 10:10:14 +02:00
c4f025ebbf
src/bin/proxmox-backup-client.rs: fix remote filename completion
2019-08-07 06:48:18 +02:00
c807d231eb
src/client/http_client.rs: compute file checksums
2019-08-06 11:26:06 +02:00
a17a0e7a9f
src/api2/admin/datastore.rs: extract backup size from index.json
2019-08-06 10:56:21 +02:00
8c70e3eb18
src/api2/admin/datastore.rs - files: return data from index.json
2019-08-05 13:22:19 +02:00
f69adc81a6
use new proxmox::tools::nodename
2019-08-03 17:06:23 +02:00
e18a6c9ee5
update to nix 0.14, use code from proxmox:tools
2019-08-03 13:05:38 +02:00
4d142ea79e
pxar: cleanup: refactor and rename exclude pattern
...
The original name PxarExcludePattern makes no sense anymore as the patterns are
also used to match filenames during restore of the archive.
Therefore, exclude_pattern.rs is moved to match_pattern.rs and PxarExcludePattern
rename to MatchPattern.
Further, since it makes more sense the MatchTypes are now declared as None,
Positive, Negative, PartialPositive or PartialNegative, as this makes more sense
and seems more readable.
Positive matches are those without '!' prefix, Negatives with '!' prefix.
This makes also the filename matching in the encoder/decoder more intuitive and
the logic was adapted accordingly.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-03 08:52:32 +02:00
47651f9530
pxar: cleanup: move feature flags to src/pxar/flags.rs and omit CA_FORMAT prefix on all of them
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-03 08:51:06 +02:00
b335f5b713
src/bin/proxmox-backup-client.rs: sign index.json
2019-08-02 10:00:15 +02:00
81a9905e0a
pxar: implement feature flag support for device nodes, fifos and sockets
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-02 07:55:17 +02:00
2c3891d1c3
src/bin/proxmox-backup-client.rs: upload backup index.json
...
The plan is to use this file to verify the backup content.
2019-08-01 12:39:02 +02:00
cec17a3ec7
src/bin/proxmox-backup-client.rs - files: allow --repository parameter
2019-07-31 11:45:55 +02:00
52c171e49d
src/bin/proxmox-backup-client.rs: implement files command
2019-07-31 11:23:53 +02:00
43a406fd3f
src/bin/proxmox-backup-client.rs: supress useless output from prune command
2019-07-31 10:15:16 +02:00
6a87910949
pxar: implement allow_existing_dirs for pxar decoder
...
By default, restoring an archive will fail if files with the same filename
already exist in the target directory.
By setting the allow_existing_dirs flag, the restore will not fail if an
existing directory is encountered.
The metadata (permissions, acls, ...) of the existing directory will be set
to the ones from the archive.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-30 09:40:30 +02:00
46d5aa0a09
src/bin/proxmox-backup-client.rs: add restore flag --allow-existing-dirs
...
Not jet implemented ...
2019-07-29 12:49:15 +02:00
ea7a7ef299
src/bin/proxmox-backup-client.rs: fixup - correctly pass prune parameters
2019-07-27 09:24:23 +02:00
9fdc3ef46f
src/api2/admin/datastore.rs: pass backup type/id (group) to prune
...
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
2019-07-27 08:49:14 +02:00
7a6cfbd98d
src/bin/proxmox-backup-client.rs: correctly compute duration
...
And format time with SecondsFormat::Secs
2019-07-26 11:13:49 +02:00
bbf9e7e951
src/api2/backup.rs: cleanup schema definitions
2019-07-26 09:07:29 +02:00
ca5d0b61ca
src/api2/backup.rs: new required backup-time parameter
...
The client should pass the time as parameter.
2019-07-25 13:44:01 +02:00
ec34f7eba0
src/bin/proxmox-backup-client.rs: implement upload-log
2019-07-25 12:18:15 +02:00
79679c2d6e
src/bin/proxmox-backup-client.rs: allow to uploag .log files (as binary blobs)
2019-07-24 13:07:02 +02:00
5b72c9b4f7
src/pxar/encoder.rs: add new skip_lost_and_found parameter
2019-07-24 12:21:25 +02:00
2eeaacb974
src/pxar/encoder.rs: allow to pass list of devices
...
For better mount point include control...
2019-07-24 08:11:59 +02:00
fa5d6977dd
Revert previous, commit, use UTC RFC3339 without timezone (Z)
...
We now have human readable data/time, and names are still sortable.
2019-07-22 11:49:30 +02:00
40a13369a3
src/bin/pxar.rs - extract: print archive name with --verbose flag
2019-07-19 07:01:57 +02:00
bbd4365c61
src/bin/pxar.rs: remove wrong debug message
2019-07-19 07:00:17 +02:00
a0ec687cd7
src/bin/pxar.rs: allow to pass paths and match patterns as args to pxar extract
...
To improve usability it is now possible to directly pass paths or match patterns
as arguments to pxar extract to partially restore an archive.
The patterns provided via CLI are appended to the ones read from file by the
--files-from option in order to have priority over those.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-19 06:50:39 +02:00
9aa3f68278
src/bin/proxmox-backup-client.rs: code cleanup
2019-07-18 13:16:32 +02:00
f6ede796ca
src/bin/proxmox-backup-client.rs - status: only print result[data]
2019-07-18 09:52:11 +02:00
15c847f136
src/api2/admin/datastore.rs: allow to list all snapshots
2019-07-18 09:11:43 +02:00
129dda4729
src/bin/pxar.rs: add --files-from parameter to pxar extract
...
Allows to pass a list of match pattern to the pxar extract call by reading them
from file.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-17 12:20:29 +02:00
fa7e957ca3
pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-17 12:19:31 +02:00
bdf0d82ced
src/bin/pxar.rs: Make pxar extract target optional
...
In order to improve usablity, the target on archive extraction will be the
current working directory by default.
A different target can be provided via the optional --target <PATH> parameter.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-17 09:40:02 +02:00
34a816cc7b
src/bin/proxmox-backup-client.rs: add output-fromat options, implement status api
2019-07-16 13:35:25 +02:00
aad2ee4947
src/bin/pxar.rs: only print archive name in verbose mode
2019-07-16 08:38:02 +02:00
6b9a071028
src/bin/pxar.rs: add --verbose parameter to list (replace dump)
2019-07-16 08:31:40 +02:00
2665cef703
src/bin/proxmox-backup-client.rs: make repository parameter optional
...
And allow to read it from $ENV{PBS_REPOSITORY}
2019-07-16 07:30:04 +02:00
9eae781ab8
src/bin/pxar.rs - extract: allow to read from stdin
2019-07-05 12:39:25 +02:00
c6c9e093d0
src/bin/pxar.rs - dump: allow to read from stdin
2019-07-05 12:32:15 +02:00
bf1252619a
src/bin/proxmox-backup-client.rs - restore: allow to pipe output to stdout
2019-07-05 12:14:50 +02:00
f8100e9623
src/bin/proxmox-backup-client.rs: remove download cli, do everything in restore
2019-07-05 11:36:45 +02:00
f4bf7dfcc7
src/client/remote_chunk_reader.rs: implement simple caching
2019-07-05 10:42:46 +02:00
afb4cd28be
src/backup/fixed_index.rs: implement BufferedFixedReader
2019-07-05 09:19:56 +02:00
7dcbe051e9
pxar: Refactor SequentialDecoder to store the callback function within the struct
...
Reduces the number of arguments for the function calls within the decoder.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-07-05 06:38:22 +02:00
86eda3eb0d
src/bin/proxmox-backup-client.rs: implement restore using BackupReader
2019-07-03 15:45:12 +02:00
6d1f61b208
use hyper/tokio-openssl instead of hyper/tokio-tls
...
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
2019-07-02 13:36:28 +02:00
74d0a6bc23
src/bin/proxmox-backup-proxy.rs: set socket options for max. performance
2019-07-02 10:48:58 +02:00
93f8bb0445
src/bin/h2s-client.rs: avoid compiler warnings
2019-07-02 08:59:33 +02:00
dc9775d1b0
src/bin/h2s-client.rs: increase tokio tcp stream read buffer size
...
This improves download speed by factor 10.
2019-06-29 18:09:50 +02:00
34f7131160
src/bin/h2s-client.rs: avoid hyper, use h2 directly
...
But performance is still bad.
2019-06-29 15:58:18 +02:00
5ed2277f0e
src/bin/h2s-server.rs: test to reproduce slow h2 download
...
Simplified code, use with h2s-client.rs.
2019-06-29 14:57:54 +02:00
17243003b3
src/bin/download-speed.rs: h2 download speedtest for our server
...
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
872debdefa
src/bin/h2client.rs: tune h2 client parameters to get reasonable speed
...
We now get about 2.5GB/s (not really good, but better)
2019-06-29 10:41:24 +02:00
fded1f3154
add code to test H2 speed
...
I currently get about 1MB/s, which is ways too slow (must be a bug)?
2019-06-29 10:08:12 +02:00
b208da8393
src/backup/data_{chunk,blob}.rs: add verify_crc method
2019-06-26 09:54:25 +02:00
fef44d4f78
src/bin/proxmox-backup-client.rs: decode blobs on download
...
No more need for that clumsy blob cli (removed).
2019-06-26 09:18:59 +02:00
bb8231409e
src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json
...
Use the KeyConfig serialization with kdf = None.
2019-06-26 07:32:34 +02:00
3031e44c58
src/bin/proxmox-backup-client.rs: remove stale download file, start blob cli
2019-06-25 13:00:37 +02:00
45db6f89e1
src/bin/proxmox-backup-client.rs: implement download command
2019-06-25 11:17:24 +02:00
6899dbfb47
src/bin/proxmox-backup-client.rs: strip .blob extension
2019-06-25 07:11:02 +02:00
045e8a568d
src/bin/proxmox-backup-client.rs: change key name to rsa-encrypted.key
2019-06-25 07:00:07 +02:00
9f46c7de4b
src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup
2019-06-24 13:56:37 +02:00
cb08ac3efe
src/api2/backup.rs: replace upload_config with upload_blob
2019-06-23 11:44:30 +02:00
a7f67a9a9c
depend on crc32fast
...
We can use this to compute fast checksums to test file integrity (not crypto safe).
2019-06-21 15:58:00 +02:00
3ea8bfc9c0
src/bin/proxmox-backup-client.rs: add import-master-pubkey command
2019-06-21 07:34:17 +02:00
37c5a17505
src/bin/proxmox-backup-client.rs: helper to generate RSA master key
2019-06-21 07:12:19 +02:00
ab44acff57
src/backup/key_derivation.rs: add modified field to key file
2019-06-20 07:58:33 +02:00
6d0983dbe1
src/bin/proxmox-backup-client.rs: add keyfile parameter for backup
2019-06-19 17:16:41 +02:00
181f097af9
src/bin/proxmox-backup-client.rs - key API: pass kdf parameter
...
Allow to store keys without password.
2019-06-18 18:21:01 +02:00
9b06db4563
src/bin/proxmox-backup-client.rs: use file system path to reference keys
2019-06-18 17:38:50 +02:00
091540298e
src/bin/proxmox-backup-client.rs - create_key: always read password from tty
2019-06-18 17:27:52 +02:00
826f309bf5
src/backup/key_derivation.rs: move kdf code into separate file
2019-06-18 11:17:22 +02:00
ac71623483
src/bin/proxmox-backup-client.rs: implement "key change-passphrase"
2019-06-17 13:10:00 +02:00
f2401311b0
src/bin/proxmox-backup-client.rs: start key management
2019-06-17 10:33:24 +02:00
0f37577092
src/bin/cipherbench.rs: add test for zstd compression
2019-06-13 17:24:57 +02:00
9399c98f82
src/bin/cipherbench.rs: cipher speed test with large blocks
2019-06-13 17:16:43 +02:00
f98ac774ee
backup: Add support for client side encryption
...
first try ...
2019-06-13 11:47:23 +02:00
dd5495d6dc
tree-wide: use 'dyn' for all trait objects
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
4247fccb0f
src/client/http_client.rs: automatically close connection in finish
2019-06-05 11:41:50 +02:00
cb4426b348
make h2 client connection cancellable
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-05 09:45:59 +02:00
7773ccc11f
src/api2.rs: move backup api to /backup
2019-06-05 07:23:21 +02:00
97eeea3b4a
src/bin/proxmox-backup-proxy.rs: fix typo in pkcs12 error message
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-06-05 06:29:02 +02:00
22e5aa1d3e
src/bin/dump-backup-api.rs: helper to generate backup API docs
2019-06-04 13:12:42 +02:00
ec8a9bb953
src/bin/proxmox-backup-client.rs: implement upload_config
2019-06-03 10:39:44 +02:00
1c0472e88f
avoid compiler warnings
2019-06-03 06:43:47 +02:00
36898ffce6
src/backup/chunk_stream.rs: add optional chunk_size parameter
2019-05-30 13:28:24 +02:00
49ef316bcd
src/bin/proxmox-backup-client.rs: backup_image() - pass correct chunk size
2019-05-30 12:52:12 +02:00
202cadc29e
src/bin/proxmox-backup-client.rs: remove unused code
2019-05-30 12:47:35 +02:00
6af905c18f
src/bin/proxmox-backup-client.rs: implement image/block-device upload
2019-05-30 12:46:01 +02:00
eb1804c567
src/bin/proxmox-backup-client.rs: use std lib instead of nix
2019-05-30 12:19:23 +02:00
6e907505aa
src/bin/proxmox-backup-client.rs: fix file type test
2019-05-30 12:08:34 +02:00
39e60bd6fa
src/api2/admin/datastore/backup.rs: improve debug logging
2019-05-29 10:17:38 +02:00
9b38443314
pxar: change logic and impl all command line flags for xattrs/fcaps/acls
...
Allows to individually set the flags for storing/dumping/restoring of
xattrs/fcaps/acls in the cli of pxar.
Changes logic so that each of them can be threated individually.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-29 06:51:20 +02:00
c4ff3dcefd
src/bin/proxmox-backup-client.rs: use new BackupClient to upload data
2019-05-28 10:12:44 +02:00
af309d4d5c
src/bin/pxar.rs: set archive permissions to 0o640
...
As the archive can contain potentially sensitive data such as key files, it makes
sense to restrict the permissions.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-28 06:24:47 +02:00
b344461b33
pxar: pass feature_flags to encoder/decoder instead of individual flags
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:11:36 +02:00
32e1bf4260
src/bin/upload-speed.rs: code cleanup
2019-05-23 07:20:05 +02:00
0d9bab05ab
src/bin/pxar.rs: impl CLI flags to disable archiving/restoring of xattrs and fcaps.
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-23 06:40:50 +02:00
6ab34afa88
src/client/http_client.rs: cleanup h2 backup client
2019-05-22 13:05:51 +02:00
de6742c04f
src/bin/h2test.rs: removed
...
This test does no work anymore.
2019-05-22 11:14:56 +02:00
751030aa7c
src/bin/test_chunk_speed2.rs: test max chunk size
2019-05-19 11:09:02 +02:00
6276448519
src/bin/test_chunk_speed2.rs: test chunker speed with real data
2019-05-18 11:59:17 +02:00
adec8ea263
src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed
...
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
2019-05-16 10:24:23 +02:00
850ac6d0f0
src/client/http_client.rs: directly return H2Client on upgrade
2019-05-13 16:44:59 +02:00
b57cb26406
src/client/http_client.rs: implement http2 client wrapper
2019-05-13 10:27:22 +02:00
9e391bb7f0
src/client/http_client.rs: allow to pass parameters (encode them as query string)
2019-05-13 09:12:03 +02:00
0aadd40b4f
src/api2/admin/datastore/backup.rs: add required parameters, cleanups
2019-05-08 13:17:23 +02:00
ca60c371e2
api: rename h2upload to backup
2019-05-08 12:35:15 +02:00
45cdce069e
src/client/http_client.rs: request password at creation time
2019-04-30 11:44:35 +02:00
3a9bea3918
src/bin/h2test.rs: test h2 upgrade
2019-04-29 11:59:18 +02:00
5a2df00004
src/client/http_client.rs: cleanup, make login fully async
2019-04-28 10:55:03 +02:00
e888155712
proxy: allow .key/.pem files instead of a pkcs12/.pfx file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-26 10:47:33 +02:00
d98c9a7a38
use service Type=notify
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:01:33 +00:00
5f550fd99f
start hyper server using with_graceful_shutdown()
...
Without, hyper keeps some futures running, and the server does not
correctly shutdown.
2019-04-10 08:24:32 +02:00
d607b8861b
src/server/worker_task.rs: implement task control socket
2019-04-09 12:15:06 +02:00
e3f41f2165
src/tools/daemon.rs: use new ServerState handler
2019-04-08 14:00:23 +02:00
35950380a9
src/server/worker_task.rs: carefully handle file permissions
2019-04-06 17:53:12 +02:00
49cf9f3d0b
src/tools.rs: file_get_json() - add new default parameter
2019-04-04 12:24:18 +02:00
a690ecacf0
daemon: simplify daemon creation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 12:12:54 +01:00
e4311382d9
tools: daemon: rename some structs
...
Reloadable resources are now 'Reloadable' instead of
'ReexecContinue'.
The struct handling the reload is a 'Reloader', not a
'ReexecStore'.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 11:09:46 +01:00
5e7bc50af1
add reload support to api daemon
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
a2ca713701
add reload support to proxy
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
1c042cdc6c
proxy: listener error handling fixup
...
If the listening socket goes into some error state we'll get
std::io::Errors rather than higher level errors from the
native_tls::TlsAcceptor, those are usually fatal. (Ran into
this after performing a shutdown() on the file descriptor,
after which the future just endlessly loops in accept().)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
ba10f2b0d4
src/bin/dynamic-files.rs: test case for file shrink during backup
2019-03-16 14:22:14 +01:00
3dbfe5b142
simplify pxar module structure
2019-03-15 08:24:32 +01:00
1ef46b81d3
src/bin/pxar.rs: implement extract
2019-03-15 08:03:44 +01:00
5defa71bf9
rename PxarBackupWriter into PxarDecodeWriter
2019-03-15 07:20:22 +01:00
37940aa1f9
src/bin/pxar.rs: implement dump/list
2019-03-14 17:43:11 +01:00
40360fde6e
src/pxar/decoder.rs: implement dump_archive
...
use this to debug archive problems ...
2019-03-14 13:10:27 +01:00
8968258b66
rename catar into pxar
...
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
7c4dd94670
rename catar into pxar
...
Seems impossible to stay compatible with the original catar specification,
so we change the name to avoid confusions.
2019-03-14 10:14:32 +01:00
08dc340a12
src/bin/proxmox-backup-client.rs: complete archive names
2019-03-14 08:09:35 +01:00
6f90a6a764
protocol: cleanup finish_backup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-13 14:16:17 +01:00
b238851832
src/bin/proxmox-backup-client.rs: implement completion for snapshots
2019-03-13 13:31:29 +01:00
4981134708
src/bin/proxmox-backup-client.rs: implement chunk-size completion
2019-03-13 12:26:01 +01:00
98f0b97293
src/bin/proxmox-backup-client.rs: cleanup
2019-03-13 12:17:39 +01:00
024f11bb55
src/bin/proxmox-backup-client.rs: implement completion for backup groups
2019-03-13 11:56:37 +01:00
edd3c8c605
src/client/backup_repo.rs: implement FromStr trait
2019-03-13 10:09:39 +01:00
d0a03d40ce
src/bin/proxmox-backup-client.rs: implement repository completions
2019-03-13 09:47:12 +01:00
bff110308b
src/bin/proxmox-backup-client.rs: fix BACKUPSPEC_REGEX - correctly specify non-capture group
...
And improve bash completion.
2019-03-13 07:23:59 +01:00
496a67846f
src/cli/command.rs: pass parsed parameters to completion function
2019-03-12 14:39:51 +01:00
d5c34d98c0
src/catar/decoder.rs: simplify public restore API
2019-03-11 14:31:01 +01:00
8e39232acc
src/bin/proxmox-backup-client.rs: strip .didx file extensions
2019-03-11 10:54:02 +01:00
74cdb52107
src/bin/proxmox-backup-client.rs: include file extension in backupspec
2019-03-11 10:27:28 +01:00
e9c9409a99
src/catar/decoder.rs: implement real sequential decoder (no Seek)
...
I copied the old code to src/catar/inspector.rs. Will modify
that later to implement a random access decoder ...
2019-03-08 16:55:54 +01:00
Dietmar Maurer
Christian Ebner
Wolfgang Bumiller