Commit Graph

6322 Commits

Author SHA1 Message Date
Thomas Lamprecht 5525ec246f tape: key recovery: refcator and split string/file case for cli params
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-04-06 16:25:34 +02:00
Markus Frank b676dbce78 fix #3854 paperkey import to proxmox-tape
added a parameter to the cli for importing tape key via a json-parameter or
via reading a exported paperkey-file or json-file.
For this i also added a backupkey parameter to the api, but here it only
accepts json.

The cli interprets the parameter first as json-string, then json-file
and last as paperkey-file.

functionality:
proxmox-tape key paperkey [fingerprint of existing key] > paperkey.backup
proxmox-tape key restore --backupkey paperkey.backup # key from line above
proxmox-tape key restore --backupkey paperkey.json # only the json
proxmox-tape key restore --backupkey '{"kdf": {"Scrypt": ...' # json as string

for importing the key as paperkey-file it is irrelevant, if the paperkey got exported as html
or txt.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2022-04-06 13:39:56 +02:00
Dominik Csapak 7c22932c64 pbs-client: print error when we couldn't download previous fidx/didx
When we have a previous manifest, we try to download the fidx/didx files
to get the known chunks list. We continue if that fails (which is ok),
but we did not print any error, leading to a confusing backup output,
since the users would expect that chunks will be reused.

Printing the error should at least make it apparent that something did
not work correctly.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-04-06 10:12:29 +02:00
Stefan Sterz 2b422b82fb fix #3067: api: add support for multi-line comments in node.cfg
add support for multi-line comments to node.cfg and the api, similar to
how pve handles multi-line comments

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
Acked-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-23 10:43:43 +01:00
Wolfgang Bumiller 9e2b423e27 tools: improve PhantomData usage
The ticket doesn't contain a `T`, it's stringified. We only
produce a new T when verifying.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-22 12:41:14 +01:00
Dominik Csapak 39ffb75d91 api: datastore_status: restore api/gui compatibility
the latest changes to this api call changed/removed some things that
were actually necessary for the gui. Readd those and document them this
time.

The change from u64 to i64 limits us to 8EiB of Datastore sizes (instead if
16EiB) but if we reach that, we must adapt most other parts to use 128bit
sizes anyway

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-22 10:31:25 +01:00
Dietmar Maurer 762f7d15dc datastore status: factor out api type DataStoreStatusListItem
And use the rust type instead of json::Value.
2022-03-20 09:38:50 +01:00
Markus Frank 80ab05e40c fix #3934 tape owner-selector to Authid
changed pmxUserSelector to pbsAuthidSelector, because it is currently
not possible to restore with a api token via gui.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
2022-03-17 10:13:18 +01:00
Stefan Sterz e099bd0717 ui: fix panel height in the dashboard
this fixes an issue where the layout looks misaligned in three column
layouts

Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
2022-03-11 12:52:50 +01:00
Stefan Sterz 171a00ca97 tape, docs, api: fix miscellaneous typos
Signed-off-by: Stefan Sterz <s.sterz@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-11 12:52:06 +01:00
Wolfgang Bumiller c8322f8a33 config: don't manually track padding size
make ConfigVersionCacheData a #[repr(C)] union to fix its
size and let it transparently `Deref{,Mut}` to its actual
contents

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-10 10:32:46 +01:00
Thomas Lamprecht eb080f361a docs: tape: minor wording tweaks
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-09 10:44:37 +01:00
Dominik Csapak bd0300917e docs: improve tape-backup examples
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-09 10:44:37 +01:00
Wolfgang Bumiller 787c6550d4 proxmox-backup-debug api: fewer cloning
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 10:10:54 +01:00
Dominik Csapak c6140c62ab proxmox-backup-debug api: rustfmt fixes
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 10:10:54 +01:00
Dominik Csapak 9735f5de84 proxmox-backup-debug api: parse parameters before sending to api
when we use http to make the api call, we have to parse the parameters
before, else we might send the string "true" instead of the boolean true
and the api rejects it with a 'Parameter verification error'.

We already have all api call schemas here, so parsing is possible.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-09 10:10:54 +01:00
Fabian Grünbichler a07ace0d1e regex: bump to 1.5.5
to ensure CVE fix for DoS on untrusted RE is picked up where it matters

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2022-03-09 09:55:36 +01:00
Dominik Csapak 904ce33d9f tools: parse_objset_stat: drop the unecessary 'objset-' from the log
'objset_id' already contains that, so the error was
"could not parse 'objset-objset-0xFFFF' stat file"

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-08 09:13:05 +01:00
Dominik Csapak 6dd5944772 tools: zfs_dataset_stats: remove dataset <-> obset file mapping on error
this can only real fail for two reasons:
* the format is wrong:
    this should not happen unless the format changed, then it will
    happen every time
* the file can't be read:
    this can happen if a user deletes and recreates a dataset manually,
    since the mapped file does not exist anymore but the dataset does

for the second case, delete the mapping from the hashmap, so that the
next call will refresh the mapping with the correct file

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-08 09:12:52 +01:00
Dominik Csapak dcd1518e10 api/config: use http_bail for 'not found' errors
the api should return a 404 error for entries that do not exist

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-08 09:09:25 +01:00
Dominik Csapak 8d6425aa24 api/config: use param_bail for parameter errors
when using the 'extjs' formatter, it marks them in a way, so that
the gui can mark the form fields with the error

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-08 09:09:22 +01:00
Dietmar Maurer 4042eedf18 Username schema: set min_length to 1
Just to get a better error message (the regex already requires min_length 1)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-03-07 13:47:06 +01:00
Dietmar Maurer 1c8efc0062 cleanup: move BasicRealmInfo to pbs-api-types
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-03-07 08:06:55 +01:00
Wolfgang Bumiller a9a15a9ab4 bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:53:41 +01:00
Wolfgang Bumiller bd4562e4b1 bump proxmox-schema dep to 1.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-03-04 09:50:21 +01:00
Thomas Lamprecht e1f9553f2d pbs-config: improve semi-useful comment
commenting that version_cache.increase_datastore_generation increases
the, well, version is rather superfluous. Also avoid the use of "we",
which is always ambiguous in code comments.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-03-01 09:54:39 +01:00
Dominik Csapak 118deb4db8 pbs-datastore: use ConfigVersionCache for datastore
instead of relying on the content of some configs

previously, we always read and parsed the config file, and only
generated a new config object when the path or the 'verify-new' option
changed.

now, we increase the datastore generation on config save, and if that
changed (or the last load is 1 minute in the past), we always
generate a new object

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-01 08:16:27 +01:00
Dominik Csapak 9c96e5368a docs: add tape schedule examples
just a few examples how one could configure tape pools and jobs.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-01 08:10:53 +01:00
Dominik Csapak 83b5076dce docs: explain retention time for event allocation policy in more detail
'when the calendar event' triggers was too vague, it could mean for
the current media-set or the next time. Apart from that, it was not
technically correct all the time, since we take the start time of
the next media set if that exists first.

The idea here is that we begin the retention when the media set is
finished.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-03-01 08:08:18 +01:00
Dominik Csapak fef61684b4 datastore: add tuning option for chunk order
currently, we sort chunks by inode when verifying or backing up to tape.
we get the inode# by stat'ing each chunk, which may be more expensive
than the gains of reading the chunks in order

Since that is highly dependent on the underlying storage of the datastore,
introduce a tuning option  so that the admin can tune that behaviour
for each datastore.

The default stays the same (sorting by inode)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-02-23 09:06:03 +01:00
Thomas Lamprecht 118f8589a9 client cli: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-22 11:50:46 +01:00
Thomas Lamprecht c2f84841b6 bin: daily-update: use syslog/log crates instead of printing to stderr
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-22 10:58:44 +01:00
Thomas Lamprecht b0728103b6 bin: daily-update: make single checks/updates fail gracefully
avoid that the acme renewal is skipped due to bailing out earlier
from a subscription or apt update error.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-22 10:27:00 +01:00
Thomas Lamprecht 00d41438b9 bin: daily-update: use from_millis instead of big nanosecond value
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-22 10:25:40 +01:00
Thomas Lamprecht 50654b22df bin: daily-update: rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-21 15:52:28 +01:00
Wolfgang Bumiller d96c7da31f bump d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:25:54 +01:00
Wolfgang Bumiller 9c890d72b9 bump proxmox-async dep to 0.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:25:37 +01:00
Wolfgang Bumiller 229c1788c1 bump proxmox-lang dep to 1.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:24 +01:00
Dominik Csapak f26d7ca5c5 use io_format_err, io_bail, io_err_other from proxmox-lang
and move the comment from the local io_bail in pbs-client/src/pxar/fuse.rs
to the only use

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:24:13 +01:00
Dominik Csapak b066586a47 depend on new 'proxmox-compression' crate
the compression utilities live there now

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2022-02-21 14:23:43 +01:00
Thomas Lamprecht 0d7873cf09 cargo: bump schema dependency
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-18 15:04:22 +01:00
Thomas Lamprecht 667476f19d cli client: backup: better use of our api macro capabilities
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-18 15:04:22 +01:00
Thomas Lamprecht a1b800c232 cli client: backup: refactor/cleanup of (dry-run) logs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-18 15:04:22 +01:00
Markus Frank 4b8395ee0e fix #3323: cli client: add dry-run option for backup command
adds a dry-run parameter for "proxmox-backup-client backup".
With this parameter on it simply prints out what would be uploaded,
instead of uploading it.

Signed-off-by: Markus Frank <m.frank@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-18 15:04:22 +01:00
Dominik Csapak dcd9c17fff tape/pool_writer: skip already backed up chunks in iterator
currently, the iterator goes over *all* chunks of the index, even
those already backed up by a previous snapshots in the same tape
backup. this is bad since for each iterator, we stat each chunk to
sort by inode number. so to avoid stat'ing the same chunks over
and over for consecutive snapshots, add a 'skip_fn' to the iterator
and in the pool writer and check the catalog_set if we can skip it

this means we can drop the later check for the catalog_set
(since we don't modify that here)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-02-18 10:40:41 +01:00
Thomas Lamprecht f30757df50 rrd: extract data: avoid always calculating start-time fallback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-15 07:59:55 +01:00
Thomas Lamprecht ac20cb1f65 rrd: avoid intermediate index, directly loop over data
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-15 07:59:55 +01:00
Thomas Lamprecht c19af51ecb rrd cache: code style, avoid useless intermediate mutable
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2022-02-15 07:59:12 +01:00
Dietmar Maurer d6644e29fe move src/shared_rate_limiter.rs to src/tools/shared_rate_limiter.rs
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-02-14 14:57:56 +01:00
Dietmar Maurer 260147bd73 ParallelHandler: avoid re-export (cleanup)
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2022-02-14 14:12:39 +01:00