Commit Graph

4312 Commits

Author SHA1 Message Date
Dietmar Maurer
124b93f31c upload_chunk: use block_in_place 2020-10-01 11:00:23 +02:00
Dominik Csapak
0f22f53b36 ui: RemoteEdit: remove port field and parse it from host field
use our hostport regexes to parse out a potential port from the host field
and send it individually

this makes for a simpler and cleaner ui

this additionally checks the field for valid input before sending it to
the backend

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-01 10:12:04 +02:00
Dominik Csapak
3784dbf029 ui: RemoteView: improve host columns
do not show the default (8007) port
and only add brackets [] to ipv6 addresses if there is a port

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-01 10:11:31 +02:00
Dominik Csapak
4c95d58c41 api2/types: fix DNS_NAME Regexes
We forgot to put braces around the DNS_NAME regex, and in
DNS_NAME_OR_IP_REGEX

this is wrong because the regex

 ^foo|bar$

matches 'foo' at the beginning and 'bar' at the end, so either

 foobaz
 bazbar

would match. only

 ^(foo|bar)$

 matches only 'foo' and 'bar'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-01 06:09:34 +02:00
Dominik Csapak
38d4675921 fix ipv6 handling for remotes/sync jobs
* add square brackets to ipv6 adresses in BackupRepository if they not
already have some (we save them without in the remote config)

* in get_pull_parameters, we now create a BackupRepository first and use
  those values (which does the [] mapping), this also has the advantage
  that we have one place less were we hardcode 8007 as port

* in the ui, add square brackets for ipv6 adresses for remotes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 13:40:03 +02:00
Dietmar Maurer
7b8aa893fa src/client/pull.rs: log progress 2020-09-30 13:35:09 +02:00
Dietmar Maurer
fb2678f96e www/index.hbs: add nodename to title 2020-09-30 12:10:04 +02:00
Dominik Csapak
486ed27299 ui: improve running task overlay
by setting a maxHeight+scrollable
(i used 500px to be still visible on our 'min screen size' 1280x720)

and by disabling emptyText deferral, which now shows the text instantly

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 11:07:08 +02:00
Dominik Csapak
df4827f2c0 tasks: improve behaviour on upgrade
when upgrading from a version where we stored all tasks in the 'active' file,
we did not completly account for finished tasks still there

we should update the file when encountering any finished task in
'active' as well as filter them out on the api call (if they get through)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 11:05:50 +02:00
Dietmar Maurer
ef1b436350 paperkey: add html output 2020-09-30 10:49:20 +02:00
Dietmar Maurer
b19b4bfcb0 examples: fix HttpClient::new usage 2020-09-30 10:49:20 +02:00
Dietmar Maurer
e64b9f9204 src/tools.rs: make command_output return Vec<u8>
And add a new helper to return output as string.
2020-09-30 10:49:20 +02:00
Dominik Csapak
9c33683c25 ui: add port support for remotes
by adding a field to RemoteEdit and showing it in the grid

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 10:49:20 +02:00
Dominik Csapak
ba20987ae7 client/remote: add support to specify port number
this adds the ability to add port numbers in the backup repo spec
as well as remotes, so that user that are behind a
NAT/Firewall/Reverse proxy can still use it

also adds some explanation and examples to the docs to make it clearer
for h2 client i left the localhost:8007 part, since it is not
configurable where we bind to

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 10:49:20 +02:00
Thomas Lamprecht
729d41fe6a api: disks/zfs: check template exsits before enabling zfs-import service
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-30 09:34:21 +02:00
Stoiko Ivanov
905147a5ee api2/node/disks/zfs: instantiate import service
When creating a new zpool for a datastore, also instantiate an
import-unit for it. This helps in cases where '/etc/zfs/zool.cache'
get corrupted and thus the pool is not imported upon boot.

This patch needs the corresponding addition of 'zfs-import@.service' in
the zfsonlinux repository.

Suggested-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-09-30 08:43:38 +02:00
Dominik Csapak
0c41e0d06b ui: add task description for logrotation
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:17:07 +02:00
Dominik Csapak
b37b59b726 ui: RemoteEdit: make comment and fingerprint deletable
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:16:53 +02:00
Dominik Csapak
60b9b48e71 require square brackets for ipv6 addresses
we need this, because we append the port to this to get a target url
e.g. we print

format!("https://{}:8007/", address)

if address is now an ipv6 (e.g. fe80::1) it would become

https://fe80::1:8007/ which is a valid ipv6 on its own

by using square brackets we get:

https://[fe80::1]:8007/ which now connects to the correct ip/port

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:16:27 +02:00
Dominik Csapak
abf8b5d475 docs: fix wrong user in repository explanation
we use 'root@pam' by default, not 'root'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:14:36 +02:00
Dominik Csapak
7eebe1483e server/worker_task: fix panic on slice range when index is empty
since len() and MAX_INDEX_TASKS are both usize, they underflow
instead of getting negative values

instead check the sizes and set them accordingly

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-30 06:11:06 +02:00
Dominik Csapak
9a76091785 proxmox-backup-proxy: add task archive rotation
this starts a task once a day at "00:00" that rotates the task log
archive if it is bigger than 500k

if we want, we can make the schedule/size limit/etc. configurable,
but for now it's ok to set fixed values for that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:41:18 +02:00
Dominik Csapak
c386b06fc6 server/worker_task: remove unecessary read_task_list
since there are no users of this anymore and we now have a nicer
TaskListInfoIterator to use, we can drop this function

this also means that 'update_active_workers' does not need to return
a list anymore since we never used that result besides in
read_task_list

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:40:50 +02:00
Dominik Csapak
6bcfc5c1a4 api2/status: use the TaskListInfoIterator here
this means that limiting with epoch now works correctly
also change the api type to i64, since that is what the starttime is
saved as

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:40:24 +02:00
Dominik Csapak
768e10d0b3 api2/node/tasks: use TaskListInfoIterator instead of read_task_list
this makes the filtering/limiting much nicer and readable

since we now have potentially an 'infinite' amount of tasks we iterate over,
and cannot now beforehand how many there are, we return the total count
as always 1 higher then requested iff we are not at the end (this is
the case when the amount of entries is smaller than the requested limit)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:40:02 +02:00
Dominik Csapak
e7244387c7 server/worker_task: add TaskListInfoIterator
this is an iterator that reads/parses/updates the task list as
necessary and returns the tasks in descending order (newest first)

it does this by using our logrotate iterator and using a vecdeque

we can use this to iterate over all tasks, even if they are in the
archive and even if the archive is logrotated but only read
as much as we need

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:39:16 +02:00
Dominik Csapak
5ade6c25f3 server/worker_task: write older tasks into archive file
instead of removing tasks beyond the 1000 that are in the index
write them into an archive file by appending them at the end
this way we can later still read them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:38:44 +02:00
Dominik Csapak
784fa1c2e3 server/worker_task: split task list file into two
one for only the active tasks and one for up to 1000 finished tasks

factor out the parsing of a task file (we will later need this again)
and use iterator combinators for easier code

we now sort the tasks ascending (this will become important in a later patch)
but reverse (for now) it to keep compatibility

this code also omits the converting into an intermittent hash
since it cannot really happen that we have duplicate tasks in this list
(since the call is locked by an flock, and it is the only place where we
write into the lists)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:38:28 +02:00
Dominik Csapak
66f4e6a809 server/worker_task: refactor locking of the task list
also add the functionality of having a 'shared' (read) lock for the list
we will need this later

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:37:54 +02:00
Dominik Csapak
8074d2b0c3 tools: add logrotate module
this is a helper to rotate and iterate over log files
there is an iterator for open filehandles as well as
only the filename

also it has the possibilty to rotate them
for compression, zstd is used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:33:21 +02:00
Dietmar Maurer
b02d49ab26 proxmox_backup_client key: allow to generate paperkey for master key 2020-09-29 08:29:42 +02:00
Dietmar Maurer
82a0cd2ad4 proxmox_backup_client key: add new paper-key command 2020-09-29 08:29:42 +02:00
Wolfgang Bumiller
ee1a9c3230 parallel_handler: clippy: 'while_let_loop'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-29 08:13:51 +02:00
Wolfgang Bumiller
db24c01106 parallel_handler: explicit Arc::clone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-28 13:40:03 +02:00
Wolfgang Bumiller
ae3cfa8f0d parallel_handler: formatting cleanup, doc comment typo fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-28 13:40:03 +02:00
Dietmar Maurer
b56c111e93 depend on proxmox 0.4.2 2020-09-28 10:50:44 +02:00
Dominik Csapak
bbeb0256f1 server/worker_task: factor out task list rendering
we will need this later again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-28 07:31:27 +02:00
Dominik Csapak
005a5b9677 api2/node/tasks: move userfilter to function signature
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-28 07:18:13 +02:00
Dietmar Maurer
55bee04856 src/tools/parallel_handler.rs: remove unnecessary Sync bound 2020-09-26 16:16:11 +02:00
Dietmar Maurer
42fd40a124 src/bin/proxmox_backup_client/benchmark.rs: avoid compiler warning 2020-09-26 16:13:19 +02:00
Dietmar Maurer
f21508b9e1 src/backup/verify.rs: use ParallelHandler to verify chunks 2020-09-26 11:14:37 +02:00
Dietmar Maurer
ee7a308de4 src/backup/verify.rs: cleanup use clause 2020-09-26 10:23:44 +02:00
Dietmar Maurer
636e674ee7 src/client/pull.rs: simplify code 2020-09-26 10:09:51 +02:00
Dietmar Maurer
b02b374b46 src/tools/parallel_handler.rs: remove static lifetime bound from handler_fn 2020-09-26 09:26:06 +02:00
Dietmar Maurer
1c13afa8f9 src/tools/parallel_handler.rs: join all threads in drop handler 2020-09-26 08:47:56 +02:00
Dietmar Maurer
69b92fab7e src/tools/parallel_handler.rs: remove unnecessary Sync trait bound 2020-09-26 07:38:44 +02:00
Thomas Lamprecht
6ab77df3f5 ui: some more eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:47:25 +02:00
Thomas Lamprecht
264c19582b ui: some more eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:36:58 +02:00
Thomas Lamprecht
8acd4d9afc ui: some more eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:34:54 +02:00
Thomas Lamprecht
65b0cea6bd ui: some eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-09-25 18:29:42 +02:00