Commit Graph

3766 Commits

Author SHA1 Message Date
Hannes Laimer
477859662a api2: add optional verify-schdule field to create/update datastore endpoint
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-09-18 12:12:16 +02:00
Hannes Laimer
ccd7241e2f add verify_schedule field to DataStoreConfig
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-09-18 12:11:55 +02:00
Hannes Laimer
f37ef25bdd api2: add VERIFY_SCHEDULE_SCHEMA
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-09-18 12:11:39 +02:00
Fabian Grünbichler
b93bbab454 fix #3014: allow DataStoreAdmins to list DS config
filtered by those they are privileged enough to read individually. this
allows such users to configure prune/GC schedules via the GUI (the API
already allowed it previously).

permission-wise, a user with this privilege can already:
- list all stores they have access to (returns just name/comment)
- read the config of each store they have access to individually
(returns full config of that datastore + digest of whole config)

but combines them to
- read configs of all datastores they have access to (returns full
config of those datastores + digest of whole config)

user that have AUDIT on just /datastore without propagate can now no
longer read all configurations (but this could be added it back, it just
seems to make little sense to me).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-18 12:09:13 +02:00
Dietmar Maurer
9cebc837d5 depend on pxar 0.6.1 2020-09-18 12:02:17 +02:00
Dietmar Maurer
1bc1d81a00 move compute_file_csum to src/tools.rs 2020-09-17 10:27:04 +02:00
Dietmar Maurer
dda72456d7 depend on proxmox 0.3.9 2020-09-17 08:49:50 +02:00
Dominik Csapak
8f2f3dd710 fix #2942: implement lacp bond mode and bond_xmit_hash_policy
this was not yet implemented, should be compatible with pve and the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-17 08:36:25 +02:00
Dominik Csapak
85959a99ea api2/network: add bond-primary parameter
needed for 'active-backup' bond mode

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-17 08:36:14 +02:00
Dominik Csapak
36700a0a87 api2/pull: make pull worker abortable
by selecting between the pull_future and the abort future

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-17 06:11:33 +02:00
Dominik Csapak
dd4b42bac1 fix #2870: renew tickets in HttpClient
by packing the auth into a RwLock and starting a background
future that renews the ticket every 15 minutes

we still use the BroadcastFuture for the first ticket and only
if that is finished we start the scheduled future

we have to store an abort handle for the renewal future and abort it when
the http client is dropped, so we do not request new tickets forever

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-17 06:09:54 +02:00
Fabian Grünbichler
9626c28619 always allow retrieving (censored) subscription info
like we do for PVE. this is visible on the dashboard, and caused 403 on
each update which bothers me when looking at the dev console.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-17 06:03:25 +02:00
Fabian Grünbichler
463c03462a fix #2957: allow Sys.Audit access to node RRD
this is the same privilege needed to query the node status.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-17 06:03:25 +02:00
Thomas Lamprecht
a086427a7d docs: fix epilogs fixme comment
restructured text comment syntax, i.e., everything it cannot parse is
a comment, is a real PITA!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-16 16:36:54 +02:00
Dietmar Maurer
4d431383d3 src/backup/data_blob.rs: expose verify_crc again 2020-09-16 10:43:42 +02:00
Stefan Reiter
d10332a15d SnapshotVerifyState: use enum for state
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-15 13:06:04 +02:00
Stefan Reiter
43772efc6e backup: check all referenced chunks actually exist
A client can omit uploading chunks in the "known_chunks" list, those
then also won't be written on the server side. Check all those chunks
mentioned in the index but not uploaded for existance and report an
error if they don't exist instead of marking a potentially broken backup
as "successful".

This is only important if the base snapshot references corrupted chunks,
but has not been negatively verified. Also, it is important to only
verify this at the end, *after* all index writers are closed, since only
then can it be guaranteed that no GC will sweep referenced chunks away.

If a chunk is found missing, also mark the previous backup with a
verification failure, since we know the missing chunk has to referenced
in it (only way it could have been inserted to known_chunks with
checked=false). This has the benefit of automatically doing a
full-upload backup if the user attempts to retry after seeing the new
error, instead of requiring a manual verify or forget.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-15 10:00:05 +02:00
Stefan Reiter
0af2da0437 backup: check verify state of previous backup before allowing reuse
Do not allow clients to reuse chunks from the previous backup if it has
a failed validation result. This would result in a new "successful"
backup that potentially references broken chunks.

If the previous backup has not been verified, assume it is fine and
continue on.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-15 09:59:29 +02:00
Dietmar Maurer
d09db6c2e9 rename BackupDir::new_with_group to BackupDir::with_group 2020-09-15 09:40:03 +02:00
Dietmar Maurer
bc871bd19d src/backup/backup_info.rs: new BackupDir::with_rfc3339 2020-09-15 09:34:46 +02:00
Dietmar Maurer
b11a6a029d debian/control: update 2020-09-15 09:33:38 +02:00
Dietmar Maurer
6a7be83efe avoid chrono dependency, depend on proxmox 0.3.8
- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
2020-09-15 07:12:57 +02:00
Dietmar Maurer
58169da46a www/OnlineHelpInfo.js: update for syncjobs 2020-09-12 15:10:08 +02:00
Dietmar Maurer
158f49e246 debian/control: update hyper dependency 2020-09-11 16:03:38 +02:00
Dietmar Maurer
3e4a67f350 bump version to 0.8.16-1 2020-09-11 15:55:37 +02:00
Fabian Grünbichler
e0e5b4426a BackupDir: make constructor fallible
since converting from i64 epoch timestamp to DateTime is not always
possible. previously, passing invalid backup-time from client to server
(or vice-versa) panicked the corresponding tokio task. now we get proper
error messages including the invalid timestamp.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:49:35 +02:00
Fabian Grünbichler
7158b304f5 handle invalid mtime when formating entries
otherwise operations like catalog shell panic when viewing pxar archives
containing such entries, e.g. with mtime very far ahead into the future.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:48:43 +02:00
Fabian Grünbichler
833eca6d2f use non-panicky timestamp_opt where appropriate
by either printing the original, out-of-range timestamp as-is, or
bailing with a proper error message instead of panicking.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:48:24 +02:00
Fabian Grünbichler
151acf5d96 don't truncate DateTime nanoseconds
where we don't care about them anyway..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:48:10 +02:00
Fabian Grünbichler
4a363fb4a7 catalog dump: preserve original mtime
even if it can't be handled by chrono. silently replacing it with epoch
0 is confusing..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:43:54 +02:00
Dominik Csapak
229adeb746 ui/docs: add onlineHelp button for syncjobs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-11 12:17:54 +02:00
Dominik Csapak
1eff9a1e89 docs: add section for calendar events
and move the info defined in 'Schedules' there,
the explanation of calendar events is inspired by the systemd.time
manpage and the pve docs (especially the examples are mostly
copied/adapted from there)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-11 12:17:42 +02:00
Dominik Csapak
ed4f0a0edc ui: fix calendarevent examples
*/x is valid syntax for us, but not systemd, so to not confuse users
write it like systemd would accept it

also an timespec must at least have hours and minutes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-11 12:17:32 +02:00
Dominik Csapak
13bed6226e tools/systemd/parse_time: enable */x syntax for calendar events
we support this in pve, so also support it here to have a more
consistent syntax

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-11 12:17:22 +02:00
Thomas Lamprecht
d937daedb3 docs: set html img width limitation through css
avoid hardcoding width in the docs itself, so that other render
outputs can choose another size.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-11 11:10:08 +02:00
Thomas Lamprecht
8cce51135c docs: do not render TODOs in release builts
they are not useful for endusers...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-11 11:09:00 +02:00
Thomas Lamprecht
0cfe1b3f13 docs: set GmbH as copyright holder
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-11 11:08:36 +02:00
Thomas Lamprecht
05c16a6e59 docs: use alabaster theme
It's not all perfect (yet) but way cleaner and simpler to use than
the sphinx one.

Custom do the scrolling for the fixed side bar and make some other
slight adjustments.

Main issue for now is that the "Developer Appendix" is always shown
in the navigation tree, but we only include that toctree for
devbuilds...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-11 11:08:13 +02:00
Dylan Whyte
3294b516d3 faq: fix typo
In note block:
    Proxmox Packup Server -> Proxmox Backup Server

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-09-10 15:18:13 +02:00
Dietmar Maurer
139bcedc53 benchmark: update TLS reference speed
We are now faster with recent patches.
2020-09-10 12:55:43 +02:00
Fabian Grünbichler
cf9ea3c4c7 server: set http2 max frame size
else we get the default of 16k, which is quite low for our use case.
this improves the TLS upload benchmark speed by about 30-40% for me.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-10 12:43:51 +02:00
Thomas Lamprecht
e84fde3e14 docs: faq: spell out PBS
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-10 12:35:04 +02:00
Dylan Whyte
1de47507ff Add section "FAQ"
Adds an FAQ to the docs, based on question that have
been appearing on the forum.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-09-10 11:33:00 +02:00
Dietmar Maurer
1a9948a488 examples/upload-speed.rs: pass new benchmark parameter 2020-09-10 09:34:51 +02:00
Dietmar Maurer
04c2731349 bump version to 0.8.15-1 2020-09-10 09:26:16 +02:00
Dietmar Maurer
5656888cc9 verify: fix done count
We need to filter out benchmark group earlier
2020-09-10 09:06:33 +02:00
Dietmar Maurer
5fdc5a6f3d verify: skip benchmark directory 2020-09-10 08:44:18 +02:00
Hannes Laimer
61d7b5013c add benchmark flag to backup creation for proper cleanup when running a benchmark
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2020-09-10 08:25:24 +02:00
Dylan Whyte
871181d984 mount: fix mount subcommand
fixes the error, "manifest does not contain
file 'X.pxar'", that occurs when trying to mount
a pxar archive with 'proxmox-backup-client mount':

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-09-10 07:21:16 +02:00
Dominik Csapak
02939e178d ui: only mark backup encrypted if there are any files
if we have a stale backup without an manifest, we do not count
the remaining files in the backup dir anymore, but this means
we now have to check here if there are really any encrypted

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-10 07:18:51 +02:00