Commit Graph

35 Commits

Author SHA1 Message Date
Fabian Grünbichler e43b9175c0 client: factor out UploadOptions
to reduce function signature complexity.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:18 +01:00
Fabian Grünbichler 0bfcea6a11 cleanup: remove unnecessary 'mut' and '.clone()'
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Fabian Grünbichler 7c66701366 tokio 1.0: use ReceiverStream from tokio-stream
to wrap a Receiver in a Stream. this will likely move back into tokio
proper once we have a std Stream..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Fabian Grünbichler 8b7f8d3f3d expose previous backup time in backup env
and use this information to add more information to client backup log
and guide the download manifest decision.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
Wolfgang Bumiller 8db1468952 more clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-14 13:58:35 +02:00
Stefan Reiter 708fab3082 format: fix typo in function name
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:32:21 +02:00
Dominik Csapak ba20987ae7 client/remote: add support to specify port number
this adds the ability to add port numbers in the backup repo spec
as well as remotes, so that user that are behind a
NAT/Firewall/Reverse proxy can still use it

also adds some explanation and examples to the docs to make it clearer
for h2 client i left the localhost:8007 part, since it is not
configurable where we bind to

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 10:49:20 +02:00
Dietmar Maurer 6a7be83efe avoid chrono dependency, depend on proxmox 0.3.8
- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
2020-09-15 07:12:57 +02:00
Hannes Laimer 61d7b5013c add benchmark flag to backup creation for proper cleanup when running a benchmark
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-09-10 08:25:24 +02:00
Thomas Lamprecht 1ffe030123 various typo fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 18:52:31 +02:00
Fabian Grünbichler 8819d1f2f5 blobs: attempt to verify on decode when possible
regular chunks are only decoded when their contents are accessed, in
which case we need to have the key anyway and want to verify the digest.

for blobs we need to verify beforehand, since their checksums are always
calculated based on their raw content, and stored in the manifest.

manifests are also stored as blobs, but don't have a digest in the
traditional sense (they might have a signature covering parts of their
contents, but that is verified already when loading the manifest).

this commit does not cover pull/sync code which copies blobs and chunks
as-is without decoding them.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-04 07:27:56 +02:00
Stefan Reiter 505c5f0f76 fix typo: avgerage to average
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 07:08:08 +02:00
Dietmar Maurer 39f18b30b6 src/backup/data_blob.rs: new load_from_reader(), which verifies the CRC
And make verify_crc private for now. We always call load_from_reader() to
verify the CRC.

Also add load_chunk() to datastore.rs (from chunk_store::read_chunk())
2020-07-28 10:23:16 +02:00
Thomas Lamprecht ea9e559fc4 client: log archive upload duration more accurate, fix grammar
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 10:15:28 +02:00
Thomas Lamprecht 0cf14984cc client: avoid division by zero in avg speed calculation, be more accurate
using micros vs. as_secs_f64 allows to have it calculated as usize
bytes, easier to handle - this was also used when it still lived in
upload_chunk_info_stream

Co-authored-by: Stoiko Ivanov <s.ivanov@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 10:14:40 +02:00
Thomas Lamprecht f40b4fb05a client writer: do not output chunklist for now on verbose true
Verbosity needs to be a non binary level, as this now is just
debug/development info, for endusers normally to much.

We want to have it available, but with a much higher verbosity level.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 10:28:38 +02:00
Thomas Lamprecht 6e1deb158a client: rework logging upload size, bandwidth, ... info
Track reused size and chunk counts.
Log reused size and use pretty print for all sizes and bandwidth
metrics.
Calculate speed over the actually uploaded size, as else it can be
skewed really bad (showing like terabytes per second)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 10:28:38 +02:00
Dietmar Maurer dde18bbb85 proxmox-backup-client benchmark: improve output format 2020-07-10 09:13:52 +02:00
Dietmar Maurer 323b2f3dd6 proxmox-backup-client benchmark: add --verbose flag 2020-07-09 16:16:39 +02:00
Dietmar Maurer 3dacedce71 src/backup/manifest.rs: use serde_json::from_value() to deserialize data
Also modified from_data compute signature ditectly from json.
2020-07-09 09:50:28 +02:00
Dietmar Maurer 3638341aa4 src/backup/file_formats.rs: remove signed chunks
We can include signature in the manifest instead (patch will follow).
2020-07-08 16:23:26 +02:00
Dietmar Maurer 3bad3e6e52 src/client/backup_writer.rs - upload_stream: add crypt_mode 2020-07-08 10:43:28 +02:00
Wolfgang Bumiller f28d9088ed introduce a CryptMode enum
This also replaces the recently introduced --encryption
parameter on the client with a --crypt-mode parameter.

This can be "none", "encrypt" or "sign-only".

Note that this introduces various changes in the API types
which previously did not take the above distinction into
account properly:

Both `BackupContent` and the manifest's `FileInfo`:
    lose `encryption: Option<bool>`
    gain `crypt_mode: Option<CryptMode>`

Within the backup manifest itself, the "crypt-mode" property
will always be set.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-07 15:24:19 +02:00
Dietmar Maurer 4059285649 fix typo 2020-06-29 12:59:25 +02:00
Dietmar Maurer b957aa81bd update backup api for incremental backup
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-26 07:17:08 +02:00
Wolfgang Bumiller f7d4e4b506 switch from failure to anyhow
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-17 18:43:30 +02:00
Dietmar Maurer ca2dbb8af1 src/client/backup_writer.rs: also print archive size 2020-02-25 14:12:29 +01:00
Dietmar Maurer 6da73c823f src/client/backup_writer.rs: prefix upload stats with archive name 2020-02-24 13:24:46 +01:00
Dietmar Maurer e02c3d461f src/client/backup_writer.rs: implement verbose options 2020-02-24 12:48:40 +01:00
Dietmar Maurer ee5fe978e2 src/client/backup_writer.rs: reduce verbosity 2020-02-24 11:33:00 +01:00
Wolfgang Bumiller dc08934563 drop Cancellable future in favor of abortable
futures-0.3 has a futures::future::abortable() function
which does the exact same, returns an Abortable future with
an AbortHandle providing an abort() method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:52:07 +01:00
Wolfgang Bumiller db0cb9ce0b update a chunk of stuff to the hyper release
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
Dietmar Maurer e016f9ff2f src/client/backup_writer.rs: rename force_close to cancel
And do not consume self.
2019-11-08 10:34:16 +01:00
Dietmar Maurer 62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer cf9271e27c src/client/backup_writer.rs: move BackupClient code into extra file
And rename it to BackupWriter.
2019-10-12 13:53:11 +02:00