Commit Graph

3117 Commits

Author SHA1 Message Date
Dominik Csapak c67b1fa72f syncjob: change worker type for sync jobs
'sync' is used for manually pulling a remote datastore
changing it for a scheduled sync to 'syncjob' so that we can
differentiate between both types of syncs

this also adds a seperate task description for it

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:28:04 +02:00
Dominik Csapak 268687ddf0 api2/pull: refactor priv checking and creating pull parameters
we want to reuse those in the api call for manually running a sync job

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:27:43 +02:00
Dominik Csapak 426c1e353b api2/config/sync: fix id parameter
'name' is not the correct parameter for get/post

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:54 +02:00
Dominik Csapak 2888b27f4c create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:25 +02:00
Dominik Csapak f5d00373f3 ui: add missing comment field to remote model
when using a diffstore, we have to add all used columns to the model,
else they will not refresh on a load

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:17:15 +02:00
Dietmar Maurer 934f5bb8ac src/bin/proxmox-backup-proxy.rs: cleanup, move code to src/tools/disks.rs
And simplify find_mounted_device by using stat.st_dev
2020-05-29 11:13:36 +02:00
Dominik Csapak 9857472211 fix removing of remotes
we have to save the remote config after removing the section

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 10:48:26 +02:00
Dietmar Maurer 013fa7bbcb rrd: reduce io by saving data only once a minute 2020-05-29 09:16:13 +02:00
Dietmar Maurer a8d7033cb2 src/bin/proxmox-backup-proxy.rs: add test if last prune job is still running 2020-05-29 08:06:48 +02:00
Dietmar Maurer 04ad7bc436 src/bin/proxmox-backup-proxy.rs: test if last sync job is still running 2020-05-29 08:06:48 +02:00
Dietmar Maurer 77ebbefc1a src/server/worker_task.rs: make worker_is_active_local pub 2020-05-29 08:06:48 +02:00
Dietmar Maurer 750252ba2f src/tools/systemd/time.rs: add test for "daily" schedule 2020-05-29 07:52:09 +02:00
Dietmar Maurer dc58194ebe src/bin/proxmox-backup-proxy.rs: use correct id to lookup sync jobs 2020-05-29 07:50:59 +02:00
Dietmar Maurer c6887a8a4d remote config gui: add comment field 2020-05-29 06:46:56 +02:00
Thomas Lamprecht 090decbe76 BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names
The repo URL consists of
* optional userid
* optional host
* datastore name

All three have defined regex or format, but none of that is used, so
for example not all valid datastore names are accepted.

Move definition of the regex over to api2::types where we can access
all required regexes easily.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:29:23 +02:00
Thomas Lamprecht c32186595e api2::types: factor out USER_ID regex
allows for better reuse in a next patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:27:38 +02:00
Dietmar Maurer 947f45252d www/ServerStatus.js: use term "IO wait" for CPU iowait
Because we already use "IO delay" for the storage statistics.
2020-05-29 06:12:49 +02:00
Dietmar Maurer c94e1f655e rrd stats: improve io delay stats 2020-05-28 19:12:13 +02:00
Thomas Lamprecht d80d1f9a2b bump version to 0.2.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-28 17:39:41 +02:00
Thomas Lamprecht 6161ac18a4 ui: remotes: fix remote remove buttons base url
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-28 17:29:54 +02:00
Dominik Csapak 6bba120d14 ui: fix RemoteEdit password change
we have to remove the password from the submitvalues if it did not
change

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 17:24:06 +02:00
Dietmar Maurer 91e5bb49f5 src/bin/proxmox-backup-proxy.rs: simplify code
and gather all stats for the root disk
2020-05-28 12:30:54 +02:00
Dietmar Maurer 547e3c2f6c src/tools/disks/zfs.rs: use wtime + rtime (wait + run time) 2020-05-28 11:45:34 +02:00
Dietmar Maurer 4bf26be3bb www/DataStoreStatistic.js: add transfer rate 2020-05-28 10:20:29 +02:00
Dietmar Maurer 25c550bc28 src/bin/proxmox-backup-proxy.rs: gather zpool io stats 2020-05-28 10:09:13 +02:00
Dietmar Maurer 0146133b4b src/tools/disks/zfs.rs: helper to read zfs pool io stats 2020-05-28 10:07:52 +02:00
Dietmar Maurer 3eeba68785 depend on proxmox 0.1.38, use new fs helper functions 2020-05-28 10:06:44 +02:00
Dominik Csapak f5056656b2 use the sync id for the scheduled sync worker task
this way, multiple sync jobs with the same local store, can get scheduled

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:26:03 +02:00
Dominik Csapak 8c87743642 fix 'remove_vanished' cli arg again
since the target side wants this to be a boolean and
serde interprets a None Value as 'null' we have to only
add this when it is really set via cli

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:25:30 +02:00
Dominik Csapak 05d755b282 fix inserting of worker tasks
when starting a new task, we do two things to keep track of tasks
(in that order):
* updating the 'active' file with a list of tasks with
  'update_active_workers'
* updating the WORKER_TASK_LIST

the second also updates the status of running tasks in the file by
checking if it is still running by checking the WORKER_TASK_LIST

since those two things are not locked, it can happend that
we update the file, and before updating the WORKER_TASK_LIST,
another thread calls update_active_workers and tries to
get the status from the task log, which won't have any data yet
so the status is 'unknown'

(we do not update that status ever, likely for performance reasons,
so we have to fix this here)

by switching the order of the two operations, we make sure that only
tasks reach the 'active' file which are inserted in the WORKER_TASK_LIST

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:24:42 +02:00
Dietmar Maurer 143b654550 src/tools.rs - command_output: add parameter to check exit code 2020-05-27 07:25:39 +02:00
Dietmar Maurer 97fab7aa11 src/tools.rs: new helper to handle command_output (std::process::Output) 2020-05-27 06:53:25 +02:00
Thomas Lamprecht ed216fd773 ui: acl view: only update if component is activated
Avoid triggering non-required background updates during browsing a
datastores content or statistics panels. They're not expensive, but I
do not like such behavior at all (having traveled with trains and
spotty network to often)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 18:58:21 +02:00
Thomas Lamprecht 0f13623443 ui: tasks: add sync description+
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 18:36:58 +02:00
Thomas Lamprecht dbd959d43f ui: tasks: render reader with full info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 18:36:58 +02:00
Thomas Lamprecht f68ae22cc0 ui: factor out render_datetime_utc
will be reused in the next patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 18:36:48 +02:00
Thomas Lamprecht 06c3dc8a8e ui: task: improve rendering of backup/prune worker entries
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 13:37:57 +02:00
Dietmar Maurer a6fbbd03c8 depend on proxmox 0.1.37 2020-05-26 13:00:34 +02:00
Thomas Lamprecht 26956d73a2 ui: datastore prune: remove debug logging
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 12:50:06 +02:00
Thomas Lamprecht 3f98b34705 ui: rework datastore content panel controller
Mostly refactoring, but actually fixes an issue where one seldom run
into a undefined dereference due to the store onLoad callback getting
triggered after some of the componet was destroyed - on quick
switching through the datastores.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 12:46:48 +02:00
Dominik Csapak 40dc103103 fix cli pull api call
there is no 'delete' parameter, only 'remove-vanished', so fix that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:39:19 +02:00
Dominik Csapak 12710fd3c3 ui: add missing monStoreErrors
to actually show api errors on the list call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:57 +02:00
Dominik Csapak 9e2a4653b4 ui: add crud for remotes
listing/adding/editing/removing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:39 +02:00
Dominik Csapak de4db62c57 remotes: save passwords as base64
to avoid having arbitrary characters in the config (e.g. newlines)
note that this breaks existings configs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:06 +02:00
Dietmar Maurer 1a0d3d11d2 src/api2/admin/datastore.rs: add rrd api 2020-05-26 12:26:14 +02:00
Dietmar Maurer 8c03041a2c src/bin/proxmox-backup-proxy.rs: gather block device stats on datastore 2020-05-26 11:20:59 +02:00
Dietmar Maurer 3fcc4b4e5c src/tools/disks.rs: add helper to read block device stats 2020-05-26 11:20:22 +02:00
Dietmar Maurer 3ed07ed2cd src/tools/disks.rs: export read_sys 2020-05-26 09:49:13 +02:00
Thomas Lamprecht 75410d65ef d/control: proxmox-backup-server: depend on proxmox-backup-docs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-26 09:37:03 +02:00
Dominik Csapak 83fd4b3b1b remote: try to use Struct for api
with a catch: password is in the struct but we do not want it to return
via the api, so we only 'serialize' it when the string is not empty
(this can only happen when the format is not checked by us, iow.
when its returned from the api) and setting it manually to ""
when we return remotes from the api

this way we can still use the type but do not return the password

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:55:07 +02:00