Commit Graph

305 Commits

Author SHA1 Message Date
Stefan Reiter b13089cdf5 file-restore: add 'extract' command for VM file restore
The data on the restore daemon is either encoded into a pxar archive, to
provide the most accurate data for local restore, or encoded directly
into a zip file (or written out unprocessed for files), depending on the
'pxar' argument to the 'extract' API call.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 14:43:41 +02:00
Stefan Reiter dd9cef56fc file-restore-daemon: add binary with virtio-vsock API server
Implements the base of a small daemon to run within a file-restore VM.

The binary spawns an API server on a virtio-vsock socket, listening for
connections from the host. This happens mostly manually via the standard
Unix socket API, since tokio/hyper do not have support for vsock built
in. Once we have the accept'ed file descriptor, we can create a
UnixStream and use our tower service implementation for that.

The binary is deliberately not installed in the usual $PATH location,
since it shouldn't be executed on the host by a user anyway.

For now, only the API calls 'status' and 'stop' are implemented, to
demonstrate and test proxmox::api functionality.

Authorization is provided via a custom ApiAuth only checking a header
value against a static /ticket file.

Since the REST server implementation uses the log!() macro, we can
redirect its output to stdout by registering env_logger as the logging
target. env_logger is already in our dependency tree via zstd/bindgen.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-08 13:57:57 +02:00
Dominik Csapak 76425d84b3 file-restore: add binary and basic commands
For now it only supports 'list' and 'extract' commands for 'pxar.didx'
files. This should be the foundation for a general file-restore
interface that is shared with block-level snapshots.

This is packaged as a seperate .deb file, since for block level restore
it will need to depend on pve-qemu-kvm, which we want to seperate from
proxmox-backup-client.

[original code for proxmox-file-restore.rs]
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

[code cleanups/clippy, use helpers::list_dir_content/ArchiveEntry, no
/block subdir for .fidx files, seperate binary and package]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-04-08 13:57:57 +02:00
Dominik Csapak 79d841014e tools/compression: add DeflateEncoder and helpers
implements a deflate encoder that can compress anything that implements
AsyncRead + Unpin into a file with the helper 'compress'

if the inner type is a Stream, it implements Stream itself, this way
some streaming data can be streamed compressed

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-07 12:34:31 +02:00
Thomas Lamprecht a417c8a93e bump version to 1.0.13-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-04-02 15:32:27 +02:00
Wolfgang Bumiller a1d90719e4 bump pxar dep to 0.10.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-31 14:00:20 +02:00
Dietmar Maurer 6f63c29306 Cargo.toml: fix: set version to 1.0.12 2021-03-26 14:14:12 +01:00
Thomas Lamprecht c4430a937d bump version to 1.0.11-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-18 12:36:28 +01:00
Wolfgang Bumiller b31cdec225 update to pxar 0.10
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:48:09 +01:00
Thomas Lamprecht 355a41a763 bump version to 1.0.10-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-11 13:40:22 +01:00
Thomas Lamprecht 1825c1a9b7 bump version to 1.0.9-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 17:06:57 +01:00
Wolfgang Bumiller 46d53e3e90 bump proxmox dep to 0.11.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-25 09:23:30 +01:00
Wolfgang Bumiller e68c0e68bd bump proxmox dep to 0.10.7
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:44:59 +01:00
Dietmar Maurer 8616a4afe5 docgen: dump string format (property strings) 2021-02-21 15:54:14 +01:00
Wolfgang Bumiller 38a81c6b46 bump pxar to 0.9.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:56:56 +01:00
Dietmar Maurer 3084232cb5 depend on proxmox 0.10.5 2021-02-11 10:49:16 +01:00
Dietmar Maurer 51c80c5a52 depend on proxmox 0.10.4 2021-02-10 10:24:36 +01:00
Dietmar Maurer 8dc45e291a depend on proxmox 0.10.2 2021-02-05 12:42:20 +01:00
Thomas Lamprecht 8aa4842fa8 bump version to 1.0.8-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-04 12:39:52 +01:00
Thomas Lamprecht 64394b0de8 bump version to 1.0.7-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-03 10:36:18 +01:00
Dietmar Maurer f4ba2e3155 depend on proxmox 0.10.1 2021-02-03 08:53:34 +01:00
Wolfgang Bumiller b419050aa7 bump pxar to 0.8
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-02 11:02:08 +01:00
Fabian Grünbichler 35c95ca653 bump apt-pkg-native dependency
our patches got applied upstream, and a release was cut, so we no longer
need to depend on a manually patched version here.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 11:53:25 +01:00
Fabian Grünbichler 427d90e6c1 update to tokio 1.0
and various related crates

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
Wolfgang Bumiller 027ef213aa api: tfa management and login
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
Dietmar Maurer cb4865466e depend on proxmox 0.9.1 2020-12-22 13:30:41 +01:00
Dietmar Maurer 2012825913 depend on proxmox 0.9.0 2020-12-22 08:52:24 +01:00
Thomas Lamprecht 42150d263b update pxar dependency to 0.6.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-16 13:13:31 +01:00
Thomas Lamprecht 2d87f2fb73 bump version to 1.0.6-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-11 14:19:28 +01:00
Dietmar Maurer 7c9835465e tape: add helpers to emulate tape read/write behavior 2020-12-06 09:41:16 +01:00
Wolfgang Bumiller 54813c650e bump proxmox dep to 0.8.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
Thomas Lamprecht 96f35520a0 bump version to 1.0.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 15:30:06 +01:00
Dietmar Maurer 27b8a3f671 bump version to 1.0.4-1 2020-11-25 08:03:11 +01:00
Dietmar Maurer 731eeef25b cli: use new alias feature for "snapshots"
Now maps to "snapshot list".
2020-11-24 13:26:43 +01:00
Thomas Lamprecht 027eb2bbe6 bump version to 1.0.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 08:56:18 +01:00
Thomas Lamprecht 866c859a1e bump version to 1.0.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 08:33:20 +01:00
Thomas Lamprecht 08b3823025 bump dependency on proxmox to 0.7.1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-20 17:38:34 +01:00
Wolfgang Bumiller 5d1d0f5d6c use pin-project to remove more unsafe blocks
we already have it in our dependency tree, so use it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 09:43:38 +01:00
Thomas Lamprecht 709c15abaa bump version to 1.0.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 10:21:30 +01:00
Thomas Lamprecht deef6fbc0c cargo: extend authors list
this was mostly selected by executing

and adding those with more than a hand full of commits, so no hard
feelings here, this was definitively also a team effort to get stuff
polished!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 14:47:48 +01:00
Thomas Lamprecht 4ac529141f bump version to 1.0.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 14:47:48 +01:00
Thomas Lamprecht 80b0423d54 bump version to 0.9.7-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 07:37:05 +01:00
Thomas Lamprecht 9d79cec4d5 bump version to 0.9.6-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-04 19:13:04 +01:00
Thomas Lamprecht 641862ddad bump version to 0.9.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-03 17:41:26 +01:00
Wolfgang Bumiller 7a7368ee08 bump proxmox dependency to 0.7.0 for totp udpates
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-30 15:19:07 +01:00
Thomas Lamprecht c9299e76fc bump version to 0.9.3-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 17:20:04 +01:00
Wolfgang Bumiller e10c5c74f6 bump proxmox dependency to 0.6.0 for api tokens and tfa
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-29 15:11:39 +01:00
Thomas Lamprecht 0903403ce7 bump version to 0.9.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-29 14:58:21 +01:00
Thomas Lamprecht 4430f199c4 bump version to 0.9.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-28 21:27:15 +01:00
Wolfgang Bumiller 41255b4d95 bump proxmox dependency to 0.5.0 for nix 0.19
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-19 12:35:03 +02:00
Fabian Grünbichler 0c4c6a7b1c build: bump nix dependency
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-19 12:12:33 +02:00
Thomas Lamprecht e6475b09e0 cargo: bump dependency of proxmox crate
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-16 12:19:43 +02:00
Dietmar Maurer 39cd81de92 bump version to 0.9.1-1 2020-10-14 13:42:30 +02:00
Fabian Grünbichler b13da548b9 fix #3070: replace internal with public URLs
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-12 13:33:16 +02:00
Fabian Grünbichler 9a75eb11cb depend on proxmox 0.4.3
needed for constnamedbitmap macro

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-08 13:09:52 +02:00
Thomas Lamprecht 1a48cbf164 bump version to 0.9.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-01 16:19:49 +02:00
Dietmar Maurer b56c111e93 depend on proxmox 0.4.2 2020-09-28 10:50:44 +02:00
Dietmar Maurer cfe01b2e6a bump version to 0.8.21-1 2020-09-25 13:20:35 +02:00
Dietmar Maurer 510544770b depend on crossbeam-channel 2020-09-25 12:58:20 +02:00
Dietmar Maurer 17a1f579d0 bump version to 0.8.20-1 2020-09-24 13:17:06 +02:00
Dietmar Maurer a329324139 bump version to 0.8.19-1 2020-09-22 13:30:52 +02:00
Dietmar Maurer 5d7449a121 bump version to 0.8.18-1 2020-09-22 12:39:47 +02:00
Dietmar Maurer 4ae6aede60 bump version to 0.8.17-1 2020-09-21 14:09:20 +02:00
Dietmar Maurer 3fade35260 bump proxmox version to 0.4.1 2020-09-21 13:51:33 +02:00
Dietmar Maurer 0a15870a82 depend on proxmox 0.4.0 2020-09-19 06:40:44 +02:00
Dietmar Maurer 9cebc837d5 depend on pxar 0.6.1 2020-09-18 12:02:17 +02:00
Dietmar Maurer dda72456d7 depend on proxmox 0.3.9 2020-09-17 08:49:50 +02:00
Dietmar Maurer 6a7be83efe avoid chrono dependency, depend on proxmox 0.3.8
- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
2020-09-15 07:12:57 +02:00
Dietmar Maurer 3e4a67f350 bump version to 0.8.16-1 2020-09-11 15:55:37 +02:00
Fabian Grünbichler cf9ea3c4c7 server: set http2 max frame size
else we get the default of 16k, which is quite low for our use case.
this improves the TLS upload benchmark speed by about 30-40% for me.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-10 12:43:51 +02:00
Dietmar Maurer 04c2731349 bump version to 0.8.15-1 2020-09-10 09:26:16 +02:00
Dietmar Maurer cd6ddb5a69 depend on proxmox 0.3.5 2020-09-04 08:11:53 +02:00
Thomas Lamprecht 7397f4a390 bump version to 0.8.14-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-02 10:41:42 +02:00
Dietmar Maurer a7bc5d4eaf depend on proxmox 0.3.4 2020-08-28 06:32:33 +02:00
Thomas Lamprecht 97cd0a2a6d bump version to 0.8.13-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-27 16:15:31 +02:00
Wolfgang Bumiller 9bdeecaee4 bump pxar dep to 0.6.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-27 12:16:21 +02:00
Wolfgang Bumiller 5aa103c3c3 bump pxar dep to 0.5.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-25 12:37:11 +02:00
Dietmar Maurer 399e48a1ed bump version to 0.8.12-1 2020-08-25 08:57:12 +02:00
Wolfgang Bumiller 82b7adf90b bump pxar dep to 0.4.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-24 11:56:01 +02:00
Thomas Lamprecht 52991f239f bump version to 0.8.11-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-19 19:20:22 +02:00
Fabian Grünbichler c5ac2b9ddd bump version to 0.8.10-1
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 15:47:30 +02:00
Wolfgang Bumiller f667f49dab bump proxmox dependency to 0.3.3 for serde helpers
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 11:32:01 +02:00
Wolfgang Bumiller 799b3d88bc bump proxmox dependency to 0.3.2 for timer / file locking
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-04 11:27:44 +02:00
Dietmar Maurer 5e4b32706c depend on proxmox 0.3.1 2020-08-02 12:02:21 +02:00
Wolfgang Bumiller 8aa67ee758 bump proxmox to 0.3, cleanup http_err macro usage
Also swap the order of a couple of `.map_err().await` to
`.await.map_err()` since that's generally more efficient.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 09:38:36 +02:00
Wolfgang Bumiller f6c6e09a8a update to pxar 0.3 to support negative timestamps
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 08:31:37 +02:00
Thomas Lamprecht beaa683a52 bump version to 0.8.9-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 11:24:56 +02:00
Thomas Lamprecht 7d07b73def bump version to 0.8.8-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 13:12:18 +02:00
Thomas Lamprecht fdac28fcec update proxmox crate to get latest websocket implementation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 12:15:49 +02:00
Dominik Csapak 1c2f842a98 api2/nodes: add termproxy and vncwebsocket api calls
Even though it has nothing to do with vnc, we keep the name of the api
call for compatibility with our xtermjs client.

termproxy:
verifies that the user is allowed to open a console and starts
termproxy with the correct parameters

starts a TcpListener on "localhost:0" so that the kernel decides the
port (instead of trying to rerserving like in pve). Then it
leaves the fd open for termproxy and gives the number as port
and tells it via '--port-as-fd' that it should interpret this
as an open fd

the vncwebsocket api call checks the 'vncticket' (name for compatibility)
and connects the remote side (after an Upgrade) with a local TcpStream
connecting to the port given via WebSocket from the proxmox crate

to make sure that only the client can connect that called termproxy and
no one can connect to an arbitrary port on the host we have to include
the port in the ticket data

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-23 12:06:38 +02:00
Stefan Reiter a4e86972a4 add .../apt/update API call
Depends on patched apt-pkg-native-rs. Changelog-URL detection is
inspired by PVE perl code for now, though marked with fixme to use 'apt
changelog' later on, if/when our repos have APT-compatible changelogs
set up.

list_installed_apt_packages iterates all packages and creates an
APTUpdateInfo with detailed information for every package matched by the
given filter Fn.

Sadly, libapt-pkg has some questionable design choices regarding their
use of 'iterators', which means quite a bit of nesting...

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-23 10:41:14 +02:00
Fabian Grünbichler 295d4f4116 bump udev build-dependency
0.4 contains a fix for C chars on non-x86 architectures.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-07-20 12:11:54 +02:00
Wolfgang Bumiller ba5b8a3e76 bump pxar dependency to 0.2.1
Contains a fix for the check for the maximum allowed size of
acl group object entries.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-16 11:48:22 +02:00
Wolfgang Bumiller 4a55fa87d5 bump version to 0.8.7-1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 10:25:53 +02:00
Thomas Lamprecht cf063c1973 bump version to 0.8.6-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-10 11:35:04 +02:00
Wolfgang Bumiller 660a34892d update proxmox crate to 0.2.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 11:08:27 +02:00
Dietmar Maurer 7884e7ef4f bump version to 0.8.5-1 2020-07-09 15:35:07 +02:00
Wolfgang Bumiller 62f6a7e3d9 bump pathpatterns to 0.1.2
Fixes `**/foo` not matching "foo" without slashes.
(`**/lost+found` now matches the `lost+found` dir at the
root of our tree properly).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 14:34:10 +02:00
Dietmar Maurer c8bed1b4d7 bump version to 0.8.4-1 2020-07-09 14:28:44 +02:00
Dietmar Maurer e284073e4a bump version to 0.8.3-1 2020-07-09 13:55:15 +02:00