Commit Graph

6193 Commits

Author SHA1 Message Date
Thomas Lamprecht 7b125de3e1 datastore: add helper to get a iterator for backup groups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht de015ce7e1 datastore: implement Iterator for backup group listing
While currently it's still only used in a collected() way, most call
sites can be switched over to use the iterator directly, as often
they already convert the not-so-cheap, in-memory vector back in
.into_iter() anyway.

somewhat also preparatory (yak shaving) work for namespaces

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht 72f8154571 api datastore: some code cleanups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht 693f3285eb datastore: backup info: drop deprecated list_backup_groups
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht 7d9cb8c458 replace deprecated list_backup_group from BackupInfo with Datastore one
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht c90dbb5c7b datastore: move list_backup_groups into Datastore impl
Having that as static method in BackupInfo makes zero sense and just
complicates call sites, which need to extract the base_path from the
store manually upfront.

Mark old fn as deprecated so that we can do the move in a separate
step.

It's also planned to add an Iterator impl for this to allow more
efficient usage in the future.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-15 12:38:16 +02:00
Thomas Lamprecht bdfa637058 client: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:25:05 +02:00
Thomas Lamprecht f9a5beaa15 backup client: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:06:15 +02:00
Thomas Lamprecht 00ae34dfda tools: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:05:17 +02:00
Thomas Lamprecht 9531d2c570 rust fmt for pbs src
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:03:46 +02:00
Thomas Lamprecht ee0ea73500 server: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 14:01:25 +02:00
Thomas Lamprecht dc7a5b3491 api: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 13:33:01 +02:00
Thomas Lamprecht 35f151e010 config: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 13:32:04 +02:00
Thomas Lamprecht 42c2b5bec9 datastore: rustfmt whole package
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 13:27:53 +02:00
Thomas Lamprecht fb3c007f8a d/changelog: fixup released
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-14 09:48:00 +02:00
Thomas Lamprecht ff7568f1d9 bump version to 2.1.6-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-13 17:31:21 +02:00
Thomas Lamprecht 1fd46218ea cli: tape key-restore: print more info for better ux
as getting the marker error if we passed valid json (but not valid
key) is confusing

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-13 16:59:47 +02:00
Thomas Lamprecht ede9dc0d1a api: tape key restore: fix optional param handling and code refactoring
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-13 16:59:47 +02:00
Markus Frank ae60eed310 proxmox-tape: api: restore_key-code moved to tape-encryption-keys
The restore_key api-endpoint is tape/drive/{drive}/restore-key.
Since I cannot set the url parameter for the drivename to null or
undefined, when restoring by exported-key, I moved the
added restore_key-api-code to
"create_key aka POST api2/json/config/tape-encryption-keys" and
added an ApiHandler call in the cli's "restore_key" to call
"create_key" in the api.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2022-04-13 16:31:17 +02:00
Wolfgang Bumiller e3746a329e pbs-client: pxar: avoid some more clones
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:30:40 +02:00
Wolfgang Bumiller 7546e9c997 pbs-client: pxar: avoid some vec extensions
The `Components` `Iterator` has an `as_path()` method to get
the remainder as a borrowed path. This is more efficient
iterating and joining the components into a new `PathBuf`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:28:01 +02:00
Wolfgang Bumiller 0bb4036f25 pbs-client: pxar: drop link_to_pathbuf
pxar's Hardlink and Symlink structs implement `AsRef<OsStr>`
and have an `.as_os_str()` method.

Simply use `Path::new(link)`.

Also, the function was not very well written, and we don't
always need an owned copy.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:18:31 +02:00
Wolfgang Bumiller 84d3af3a0e pbs-client: pxar: fmt
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:17:20 +02:00
Dominik Csapak 055eab54ff ui: datastore/Content: enable tar download in ui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:08:34 +02:00
Dominik Csapak 984ddb2ff2 api: admin/datastore: add tar support for pxar_file_download
by using the newly added 'create_tar' and the 'ZstdEncoder'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:08:26 +02:00
Dominik Csapak 23af572d3f pbs-client: add 'create_tar' helper function
similar to create_zip, uses an accessor to write a tar into an output
that implements AsyncWrite, but we use a Decoder to iterate instead
of having a recursive function. This is done so that we get the
entries in the correct order, and it should be faster as well.

Includes files, directories, symlinks, hardlink, block/char devs, fifos
into the tar. If the hardlink points to outside the current dir to
archive, promote the first instance to a 'real' file, and use a
hardlink for the rest.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 10:05:08 +02:00
Wolfgang Bumiller 99f09fd3c1 bump proxmox-compression dependency to 0.1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 09:37:20 +02:00
Wolfgang Bumiller da7a71115c bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:21:18 +02:00
Wolfgang Bumiller ebb85c1ca3 bump proxmox-schema dependency to 1.3.1 for streaming attribute
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:20:27 +02:00
Wolfgang Bumiller fb6e48f402 bump proxmox-router dependency to 1.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:17:08 +02:00
Dominik Csapak b7c3eaa981 api: admin/datastore: enable streaming for some api calls
namely /admin/datastore/{store}/snapshots
and /nodes/{node}/tasks

since those are api calls where the result can get quite large
with this change, the serialization is now streaming instead of making
a `Value` in memory.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:13:42 +02:00
Dominik Csapak 32e2b5abe6 adapt to the new ApiHandler variants
namely 'StreamingSync' and 'StreamingAsync'
in rest-server by using the new formatter function,
and in the debug binary by using 'to_value'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:13:40 +02:00
Dominik Csapak 2ef2c0fe0c proxmox-rest-server: OutputFormatter: add new format_data_streaming method
that takes the data in form of a `Box<dyn SerializableReturn + Send>`
instead of a Value.

Implement it in json and extjs formatter, by starting a thread and
stream the serialized data via a `BufWriter<SenderWriter>` and use
the Receiver side as a stream for the response body.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-13 08:13:36 +02:00
Thomas Lamprecht 9c3b29bd8f ui: datastore options: maintenance mode related refactorings
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-12 16:54:56 +02:00
Thomas Lamprecht 3c8f240712 ui: datastore options: fix active-ops-tracking store leak
without this the store stayed active in the background and kept
updating every 3s for every datastore the ui was opened.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-12 16:21:41 +02:00
Thomas Lamprecht 6353e22c00 ui: datastore options: factor out update stop/start to controller
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-12 16:18:43 +02:00
Wolfgang Bumiller 38774184a9 tree-wide: replace serde_json::from_value(a_value.clone())
`&Value` itself implements `Deserializer` and can therefore
be passed directly to `T::deserialize` without requiring an
intermediate `clone()`. (This also enables optionally
borrowing strings if the result has a short enough lifetime)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 16:12:15 +02:00
Thomas Lamprecht 845baef61b ui: maintenance mode: also render message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-12 16:12:15 +02:00
Thomas Lamprecht 73ce2ae1c7 ui: maintenance mode: refactor renderer
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-12 16:12:12 +02:00
Hannes Laimer 556eda0537 ui: add option to change the maintenance type
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Hannes Laimer 5fd823c3f2 api: add get_active_operations endpoint
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Hannes Laimer 758c6ed588 api: make maintenance_type updatable
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Hannes Laimer 4bc84a6549 pbs-datastore: add active operations tracking
Saves the currently active read/write operation counts in a file. The
file is updated whenever a reference returned by lookup_datastore is
dropped and whenever a reference is returned by lookup_datastore. The
files are locked before every access, there is one file per datastore.

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Hannes Laimer e9d2fc9362 datastore: add check for maintenance in lookup
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Hannes Laimer 2a05c75ff1 api-types: add maintenance type
+ bump proxmox-schema dep to 1.2.1 (for quoted property string)

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2022-04-12 15:29:14 +02:00
Thomas Lamprecht 66b88dadba ui: node config: avoid split listeners
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-12 15:29:03 +02:00
Wolfgang Bumiller 9ee2ef2e55 client: drop unnecessary clone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-04-12 12:34:52 +02:00
Thomas Lamprecht 12558e0dde tree wide: some stylistic clippy fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-11 08:14:28 +02:00
Thomas Lamprecht b22d785c18 api types: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 18:00:18 +02:00
Thomas Lamprecht 4ad118c613 cli: backup manager: rust fmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-10 17:50:35 +02:00