Commit Graph

3248 Commits

Author SHA1 Message Date
Thomas Lamprecht
729d41fe6a api: disks/zfs: check template exsits before enabling zfs-import service
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-30 09:34:21 +02:00
Stoiko Ivanov
905147a5ee api2/node/disks/zfs: instantiate import service
When creating a new zpool for a datastore, also instantiate an
import-unit for it. This helps in cases where '/etc/zfs/zool.cache'
get corrupted and thus the pool is not imported upon boot.

This patch needs the corresponding addition of 'zfs-import@.service' in
the zfsonlinux repository.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-09-30 08:43:38 +02:00
Dominik Csapak
0c41e0d06b ui: add task description for logrotation
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:17:07 +02:00
Dominik Csapak
b37b59b726 ui: RemoteEdit: make comment and fingerprint deletable
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:16:53 +02:00
Dominik Csapak
60b9b48e71 require square brackets for ipv6 addresses
we need this, because we append the port to this to get a target url
e.g. we print

format!("https://{}:8007/", address)

if address is now an ipv6 (e.g. fe80::1) it would become

https://fe80::1:8007/ which is a valid ipv6 on its own

by using square brackets we get:

https://[fe80::1]:8007/ which now connects to the correct ip/port

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:16:27 +02:00
Dominik Csapak
abf8b5d475 docs: fix wrong user in repository explanation
we use 'root@pam' by default, not 'root'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:14:36 +02:00
Dominik Csapak
7eebe1483e server/worker_task: fix panic on slice range when index is empty
since len() and MAX_INDEX_TASKS are both usize, they underflow
instead of getting negative values

instead check the sizes and set them accordingly

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:11:06 +02:00
Dominik Csapak
9a76091785 proxmox-backup-proxy: add task archive rotation
this starts a task once a day at "00:00" that rotates the task log
archive if it is bigger than 500k

if we want, we can make the schedule/size limit/etc. configurable,
but for now it's ok to set fixed values for that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:41:18 +02:00
Dominik Csapak
c386b06fc6 server/worker_task: remove unecessary read_task_list
since there are no users of this anymore and we now have a nicer
TaskListInfoIterator to use, we can drop this function

this also means that 'update_active_workers' does not need to return
a list anymore since we never used that result besides in
read_task_list

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:40:50 +02:00
Dominik Csapak
6bcfc5c1a4 api2/status: use the TaskListInfoIterator here
this means that limiting with epoch now works correctly
also change the api type to i64, since that is what the starttime is
saved as

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:40:24 +02:00
Dominik Csapak
768e10d0b3 api2/node/tasks: use TaskListInfoIterator instead of read_task_list
this makes the filtering/limiting much nicer and readable

since we now have potentially an 'infinite' amount of tasks we iterate over,
and cannot now beforehand how many there are, we return the total count
as always 1 higher then requested iff we are not at the end (this is
the case when the amount of entries is smaller than the requested limit)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:40:02 +02:00
Dominik Csapak
e7244387c7 server/worker_task: add TaskListInfoIterator
this is an iterator that reads/parses/updates the task list as
necessary and returns the tasks in descending order (newest first)

it does this by using our logrotate iterator and using a vecdeque

we can use this to iterate over all tasks, even if they are in the
archive and even if the archive is logrotated but only read
as much as we need

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:39:16 +02:00
Dominik Csapak
5ade6c25f3 server/worker_task: write older tasks into archive file
instead of removing tasks beyond the 1000 that are in the index
write them into an archive file by appending them at the end
this way we can later still read them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:38:44 +02:00
Dominik Csapak
784fa1c2e3 server/worker_task: split task list file into two
one for only the active tasks and one for up to 1000 finished tasks

factor out the parsing of a task file (we will later need this again)
and use iterator combinators for easier code

we now sort the tasks ascending (this will become important in a later patch)
but reverse (for now) it to keep compatibility

this code also omits the converting into an intermittent hash
since it cannot really happen that we have duplicate tasks in this list
(since the call is locked by an flock, and it is the only place where we
write into the lists)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:38:28 +02:00
Dominik Csapak
66f4e6a809 server/worker_task: refactor locking of the task list
also add the functionality of having a 'shared' (read) lock for the list
we will need this later

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:37:54 +02:00
Dominik Csapak
8074d2b0c3 tools: add logrotate module
this is a helper to rotate and iterate over log files
there is an iterator for open filehandles as well as
only the filename

also it has the possibilty to rotate them
for compression, zstd is used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:33:21 +02:00
Dietmar Maurer
b02d49ab26 proxmox_backup_client key: allow to generate paperkey for master key 2020-09-29 08:29:42 +02:00
Dietmar Maurer
82a0cd2ad4 proxmox_backup_client key: add new paper-key command 2020-09-29 08:29:42 +02:00
Wolfgang Bumiller
ee1a9c3230 parallel_handler: clippy: 'while_let_loop'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-29 08:13:51 +02:00
Wolfgang Bumiller
db24c01106 parallel_handler: explicit Arc::clone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-28 13:40:03 +02:00
Wolfgang Bumiller
ae3cfa8f0d parallel_handler: formatting cleanup, doc comment typo fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-28 13:40:03 +02:00
Dietmar Maurer
b56c111e93 depend on proxmox 0.4.2 2020-09-28 10:50:44 +02:00
Dominik Csapak
bbeb0256f1 server/worker_task: factor out task list rendering
we will need this later again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-28 07:31:27 +02:00
Dominik Csapak
005a5b9677 api2/node/tasks: move userfilter to function signature
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-28 07:18:13 +02:00
Dietmar Maurer
55bee04856 src/tools/parallel_handler.rs: remove unnecessary Sync bound 2020-09-26 16:16:11 +02:00
Dietmar Maurer
42fd40a124 src/bin/proxmox_backup_client/benchmark.rs: avoid compiler warning 2020-09-26 16:13:19 +02:00
Dietmar Maurer
f21508b9e1 src/backup/verify.rs: use ParallelHandler to verify chunks 2020-09-26 11:14:37 +02:00
Dietmar Maurer
ee7a308de4 src/backup/verify.rs: cleanup use clause 2020-09-26 10:23:44 +02:00
Dietmar Maurer
636e674ee7 src/client/pull.rs: simplify code 2020-09-26 10:09:51 +02:00
Dietmar Maurer
b02b374b46 src/tools/parallel_handler.rs: remove static lifetime bound from handler_fn 2020-09-26 09:26:06 +02:00
Dietmar Maurer
1c13afa8f9 src/tools/parallel_handler.rs: join all threads in drop handler 2020-09-26 08:47:56 +02:00
Dietmar Maurer
69b92fab7e src/tools/parallel_handler.rs: remove unnecessary Sync trait bound 2020-09-26 07:38:44 +02:00
Thomas Lamprecht
6ab77df3f5 ui: some more eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:47:25 +02:00
Thomas Lamprecht
264c19582b ui: some more eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:36:58 +02:00
Thomas Lamprecht
8acd4d9afc ui: some more eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:34:54 +02:00
Thomas Lamprecht
65b0cea6bd ui: some eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:29:42 +02:00
Dietmar Maurer
cfe01b2e6a bump version to 0.8.21-1 2020-09-25 13:20:35 +02:00
Dietmar Maurer
b19b032be3 debian/control: update 2020-09-25 13:17:49 +02:00
Dietmar Maurer
5441708634 src/client/pull.rs: use new ParallelHandler 2020-09-25 12:58:20 +02:00
Dietmar Maurer
3c9b370255 src/tools/parallel_handler.rs: execute closure inside a thread pool 2020-09-25 12:58:20 +02:00
Dietmar Maurer
510544770b depend on crossbeam-channel 2020-09-25 12:58:20 +02:00
Thomas Lamprecht
e8293841c2 docs: html: show "Proxmox Backup" in navi for small devices
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 20:03:17 +02:00
Thomas Lamprecht
46114bf28e docs: html: improve css for small displays
fixed-width navi/toc links were not switched in color for small width
displays, and thus they were barely readable as the background
switches to dark for small widths.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 20:03:17 +02:00
Thomas Lamprecht
0d7e61f06f docs: buildsys: add more dependencies to html target
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 19:45:23 +02:00
Thomas Lamprecht
fd6a54dfbc docs: conf: fix conf for new alabaster theme version
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 19:44:50 +02:00
Thomas Lamprecht
1ea5722b8f docs: html: adapt custom css
highlighting the current chapter and some other small formatting
improvements

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 19:44:00 +02:00
Thomas Lamprecht
bc8fadf494 docs: index: hide todo list toctree and genindex
I do not found another way to disable inclusion in the sidebar...

The genindex information is alredy provided through glossary

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 19:43:18 +02:00
Thomas Lamprecht
a76934ad33 docs: html: adapt sidebar in index page
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-24 19:41:19 +02:00
Dominik Csapak
d7a122a026 use jobstate mechanism for verify/garbage_collection schedules
also changes:
* correct comment about reset (replace 'sync' with 'action')
* check schedule change correctly (only when it is actually changed)

with this changes, we can drop the 'lookup_last_worker' method

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-24 17:06:12 +02:00
Dominik Csapak
6c25588e63 proxy: fix error handling in prune scheduling
we rely on the jobstate handling to write the error of the worker
into its state file, but we used '?' here in a block which does not
return the error to the block, but to the function/closure instead

so if a prune job failed because of such an '?', we did not write
into the statefile and got a wrong state there

instead use our try_block! macro that wraps the code in a closure

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-24 17:06:09 +02:00