Commit Graph

66 Commits

Author SHA1 Message Date
Wolfgang Bumiller
8c74349b08 api-types: add namespace to BackupGroup
Make it easier by adding an helper accepting either group or
directory

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-05-12 09:33:50 +02:00
Dominik Csapak
07a683d266 pbs-client: extract: add top-level dir in tar.zst
when download a folder, include that folder as first entry (except '/')

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-22 11:35:55 +02:00
Dominik Csapak
7098f5d885 pbs-client: extract: rewrite create_zip with sequential decoder
instead of an async recursive function. Not only is it less code,
recursive futures are not really nice and it should be faster too.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-22 11:35:53 +02:00
Wolfgang Bumiller
db87d93efc make datastore BackupGroup/Dir ctors private
And use the api-types for their contents.

These are supposed to be instances for a datastore, the pure
specifications are the ones in pbs_api_types which should be
preferred in crates like clients which do not need to deal
with the datastore directly.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-20 11:56:23 +02:00
Wolfgang Bumiller
988d575dbb api-types: introduce BackupType enum and Group/Dir api types
The type is a real enum.

All are API types and implement Display and FromStr. The
ordering is the same as it is in pbs-datastore.

Also, they are now flattened into a few structs instead of
being copied manually.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 13:12:46 +02:00
Thomas Lamprecht
bdfa637058 client: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:25:05 +02:00
Wolfgang Bumiller
e3746a329e pbs-client: pxar: avoid some more clones
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:30:40 +02:00
Wolfgang Bumiller
7546e9c997 pbs-client: pxar: avoid some vec extensions
The `Components` `Iterator` has an `as_path()` method to get
the remainder as a borrowed path. This is more efficient
iterating and joining the components into a new `PathBuf`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:28:01 +02:00
Wolfgang Bumiller
0bb4036f25 pbs-client: pxar: drop link_to_pathbuf
pxar's Hardlink and Symlink structs implement `AsRef<OsStr>`
and have an `.as_os_str()` method.

Simply use `Path::new(link)`.

Also, the function was not very well written, and we don't
always need an owned copy.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:18:31 +02:00
Wolfgang Bumiller
84d3af3a0e pbs-client: pxar: fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:17:20 +02:00
Dominik Csapak
23af572d3f pbs-client: add 'create_tar' helper function
similar to create_zip, uses an accessor to write a tar into an output
that implements AsyncWrite, but we use a Decoder to iterate instead
of having a recursive function. This is done so that we get the
entries in the correct order, and it should be faster as well.

Includes files, directories, symlinks, hardlink, block/char devs, fifos
into the tar. If the hardlink points to outside the current dir to
archive, promote the first instance to a 'real' file, and use a
hardlink for the rest.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:05:08 +02:00
Wolfgang Bumiller
99f09fd3c1 bump proxmox-compression dependency to 0.1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:37:20 +02:00
Wolfgang Bumiller
ebb85c1ca3 bump proxmox-schema dependency to 1.3.1 for streaming attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:20:27 +02:00
Wolfgang Bumiller
fb6e48f402 bump proxmox-router dependency to 1.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:17:08 +02:00
Wolfgang Bumiller
38774184a9 tree-wide: replace serde_json::from_value(a_value.clone())
`&Value` itself implements `Deserializer` and can therefore
be passed directly to `T::deserialize` without requiring an
intermediate `clone()`. (This also enables optionally
borrowing strings if the result has a short enough lifetime)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 16:12:15 +02:00
Fabian Ebner
2b323a359d pxar: create: add entry: fix anchored path pattern matching
Similar to 874bd545 ("pxar: fix anchored exclusion at archive root"),
but this time for inclusion. Because of the inconsistency, it could
happen that a file included in generate_directory_file_list() got
excluded in add_entry(), e.g. with a .pxarexclude file like
> *
> !/supposed-to-be-included

Reported-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2022-04-06 16:58:08 +02:00
Dominik Csapak
7c22932c64 pbs-client: print error when we couldn't download previous fidx/didx
When we have a previous manifest, we try to download the fidx/didx files
to get the known chunks list. We continue if that fails (which is ok),
but we did not print any error, leading to a confusing backup output,
since the users would expect that chunks will be reused.

Printing the error should at least make it apparent that something did
not work correctly.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-06 10:12:29 +02:00
Wolfgang Bumiller
bd4562e4b1 bump proxmox-schema dep to 1.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:50:21 +01:00
Wolfgang Bumiller
9c890d72b9 bump proxmox-async dep to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:25:37 +01:00
Wolfgang Bumiller
229c1788c1 bump proxmox-lang dep to 1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:24 +01:00
Dominik Csapak
f26d7ca5c5 use io_format_err, io_bail, io_err_other from proxmox-lang
and move the comment from the local io_bail in pbs-client/src/pxar/fuse.rs
to the only use

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:13 +01:00
Dominik Csapak
b066586a47 depend on new 'proxmox-compression' crate
the compression utilities live there now

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:23:43 +01:00
Fabian Grünbichler
dcf5a0f62d misc clippy fixes
the trivial ones ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-02-08 14:57:16 +01:00
Fabian Grünbichler
aa174e8e8a tree-wide: drop redundant clones
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
Fabian Grünbichler
9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
Wolfgang Bumiller
c3b8e74fdf bump regex dep to 1.5
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-16 11:25:02 +01:00
Wolfgang Bumiller
821aa8eae6 bump proxmox-schema to 1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-12-16 11:25:02 +01:00
Dietmar Maurer
d4877712f8 pbs-client: avoid mut self in http_client methods.
It is not necessary, so avoid it. The client can now be used
with multiple threads (without using a Mutex).

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-12-04 14:44:05 +01:00
Wolfgang Bumiller
73fba2edea fix a warning in io_return macro
newer compilers warn about the semicolon there, so put
braces around it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-11-29 11:26:25 +01:00
Dietmar Maurer
726b9d4469 use proxmox-sys 0.2.1 and proxmox-io 1.0.1
And remove unused code from pbs-tools.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 12:30:03 +01:00
Dietmar Maurer
577095e2f7 move pbs-tools/src/percent_encoding.rs to pbs-api-types/src/percent_encoding.rs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 11:48:52 +01:00
Dietmar Maurer
92ef0b56d8 move pbs-tools/src/str.rs to pbs-client/src/pxar/create.rs
Code is only used there.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-25 10:43:22 +01:00
Dietmar Maurer
25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
Dietmar Maurer
2d5287fbbc use RateLimitConfig for HttpClient and pull
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-22 07:49:41 +01:00
Thomas Lamprecht
a58a5cf795 move HumanByte to pbs-abi-types crate
Originally-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-11-20 19:35:24 +01:00
Dietmar Maurer
92a8f0bc82 depend on proxmox-async 0.2 2021-11-20 17:14:02 +01:00
Dietmar Maurer
9a1b24b6b1 use new proxmox-async crate
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-19 18:03:22 +01:00
Dietmar Maurer
860eaec58f use proxmox::tools::fd::fd_change_cloexec from proxmox 0.15.3
Depend on proxmox 0.15.3

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-18 13:43:41 +01:00
Dietmar Maurer
09f999337a update to proxmox-http 0.5.4
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-14 08:27:45 +01:00
Dietmar Maurer
d5f58006d3 cached_traffic_control: use ShareableRateLimit trait object 2021-11-13 17:49:38 +01:00
Dietmar Maurer
2419dc0de9 pbs-client: add option to use the new RateLimiter
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-10 10:15:32 +01:00
Dietmar Maurer
e0a19d3313 use new fsync parameter to replace_file and atomic_open_or_create
Depend on proxmox 0.15.0 and proxmox-openid 0.8.1

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-10-21 07:28:32 +02:00
Dietmar Maurer
b3f279e2d9 use complete_file_name from proxmox-router 1.1 2021-10-13 14:10:02 +02:00
Wolfgang Bumiller
890b88cbef remove pbs-tools::ops::ControlFlow
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-12 14:36:40 +02:00
Wolfgang Bumiller
6ef1b649d9 update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
Wolfgang Bumiller
e3f3359c86 bump proxmox dependency to 0.14.0 and proxmox-http to 0.5.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-08 11:18:22 +02:00
Wolfgang Bumiller
f66d66aafe drop dynamic_index.rs duplicate in pbs-client
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-06 15:29:27 +02:00
Wolfgang Bumiller
347e0d4c57 fix deprecated use of std::u64/... modules
integer primitive type modules are deprecated, use
associated constants instead

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-04 15:02:30 +02:00
Wolfgang Bumiller
d7eedbd24b tools::format: avoid some string copies
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-30 12:43:33 +02:00
Wolfgang Bumiller
b62edce929 remove pbs_client::connect_to_localhost
It also used `CertInfo` from pbs-tools which is also server
specific.

The original helper is now in the main crate's
client_helpers instead.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-29 14:11:26 +02:00