a95a3fb893
fix csum calculation of not 'chunk_size' aligned images
...
the last chunk does not have to be as big as the chunk_size,
just use the already available 'chunk_end' function which does the
correct thing
this fixes restoration of images whose sizes are not a multiple of
'chunk_size' as well
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 10:18:30 +02:00
620911b426
src/tools/disks/lvm.rs: implement get_lvm_devices()
2020-06-04 09:12:19 +02:00
5c264c8d80
src/tools/disks.rs: add/use get_partition_type_info
2020-06-04 07:48:22 +02:00
8d78589969
improve display of 'next run' for sync jobs
...
if the last sync job is too far in the past (or there was none at all
for now) we run it at the next iteration, so we want to show that
we now calculate the next_run by using either the real last endtime
as time or 0
then in the frontend, we check if the next_run is < now and show 'pending'
(we do it this way also for replication on pve)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:03:54 +02:00
eed8a5ad79
tools/systemd/time: fix compute_next_event for weekdays
...
two things were wrong here:
* the range (x..y) does not include y, so the range
(day_num+1..6) goes from (day_num+1) to 5 (but sunday is 6)
* WeekDays.bits() does not return the 'day_num' of that day, but
the bit value (e.g. 64 for SUNDAY) but was treated as the index of
the day of the week
to fix this, we drop the map to WeekDays and use the 'indices'
directly
this patch makes the test work again
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:02:33 +02:00
538b9c1c27
systemd/time: add tests for all weekdays
...
this fails for now
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:02:23 +02:00
55919bf141
verify_file: add missing closing parenthesis in error message
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 19:10:01 +02:00
456ad0c478
src/tools/disks/zfs.rs: add parser for zpool list output
2020-06-03 12:16:08 +02:00
c48aa39f3b
src/bin/proxmox-backup-client.rs: implement quite flag
2020-06-03 10:11:37 +02:00
2d32fe2c04
client restore: don't add server file ending if already specified
...
If one executes a client command like
# proxmox-backup-client files <snapshot> --repository ...
the files shown have already the '.fidx' or '.blob' file ending, so
if a user would just copy paste that one the client would always add
.blob, and the server would not find that file.
So avoid adding file endings if it is already a known OK one.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 07:03:55 +02:00
dc155e9bd7
client restore: factor out archive/type parsing
...
will be extended in a next patch.
Also drop a dead else branch, can never get hit as we always add
.blob as fallback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 07:03:12 +02:00
4e14781aec
fix typo
2020-06-03 06:59:43 +02:00
a595f0fee0
client: improve connection/new fingerprint query
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-02 10:40:31 +02:00
add5861e8d
typo fixes all over the place
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 16:39:08 +02:00
1610c45a86
src/client/pull.rs: also download client.log.blob
2020-05-30 14:51:33 +02:00
b2387eaa45
avoid compiler warnings
2020-05-30 14:05:33 +02:00
96d65fbcd0
cleanup: define/use const for predefined blob file names.
2020-05-30 14:04:15 +02:00
7cc3473a4e
src/client/backup_specification.rs: split code into extra file
2020-05-30 10:54:38 +02:00
4856a21836
src/client/pull.rs: more verbose logging
2020-05-30 08:12:43 +02:00
04b0ca8b59
add owner to group and snapshot listings
...
while touching it, make columns and tbar in DataStoreContent.js
declarative members and remove the (now) unnecessary initComponent
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-30 07:24:12 +02:00
d43f86f3f3
api2: add admin/sync endpoint
...
this returns the list of syncjobs with status, as opposed to
config/sync (which is just the config)
also adds an api call where users can run the job manually under
/admin/sync/$ID/run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:31:32 +02:00
997d7e19fc
config/sync: add SyncJobStatus Struct/Schema
...
contains the config + status
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:29:39 +02:00
c67b1fa72f
syncjob: change worker type for sync jobs
...
'sync' is used for manually pulling a remote datastore
changing it for a scheduled sync to 'syncjob' so that we can
differentiate between both types of syncs
this also adds a seperate task description for it
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:28:04 +02:00
268687ddf0
api2/pull: refactor priv checking and creating pull parameters
...
we want to reuse those in the api call for manually running a sync job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:27:43 +02:00
426c1e353b
api2/config/sync: fix id parameter
...
'name' is not the correct parameter for get/post
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:54 +02:00
2888b27f4c
create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:25 +02:00
934f5bb8ac
src/bin/proxmox-backup-proxy.rs: cleanup, move code to src/tools/disks.rs
...
And simplify find_mounted_device by using stat.st_dev
2020-05-29 11:13:36 +02:00
9857472211
fix removing of remotes
...
we have to save the remote config after removing the section
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 10:48:26 +02:00
013fa7bbcb
rrd: reduce io by saving data only once a minute
2020-05-29 09:16:13 +02:00
a8d7033cb2
src/bin/proxmox-backup-proxy.rs: add test if last prune job is still running
2020-05-29 08:06:48 +02:00
04ad7bc436
src/bin/proxmox-backup-proxy.rs: test if last sync job is still running
2020-05-29 08:06:48 +02:00
77ebbefc1a
src/server/worker_task.rs: make worker_is_active_local pub
2020-05-29 08:06:48 +02:00
750252ba2f
src/tools/systemd/time.rs: add test for "daily" schedule
2020-05-29 07:52:09 +02:00
dc58194ebe
src/bin/proxmox-backup-proxy.rs: use correct id to lookup sync jobs
2020-05-29 07:50:59 +02:00
090decbe76
BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names
...
The repo URL consists of
* optional userid
* optional host
* datastore name
All three have defined regex or format, but none of that is used, so
for example not all valid datastore names are accepted.
Move definition of the regex over to api2::types where we can access
all required regexes easily.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:29:23 +02:00
c32186595e
api2::types: factor out USER_ID regex
...
allows for better reuse in a next patch
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:27:38 +02:00
c94e1f655e
rrd stats: improve io delay stats
2020-05-28 19:12:13 +02:00
91e5bb49f5
src/bin/proxmox-backup-proxy.rs: simplify code
...
and gather all stats for the root disk
2020-05-28 12:30:54 +02:00
547e3c2f6c
src/tools/disks/zfs.rs: use wtime + rtime (wait + run time)
2020-05-28 11:45:34 +02:00
25c550bc28
src/bin/proxmox-backup-proxy.rs: gather zpool io stats
2020-05-28 10:09:13 +02:00
0146133b4b
src/tools/disks/zfs.rs: helper to read zfs pool io stats
2020-05-28 10:07:52 +02:00
3eeba68785
depend on proxmox 0.1.38, use new fs helper functions
2020-05-28 10:06:44 +02:00
f5056656b2
use the sync id for the scheduled sync worker task
...
this way, multiple sync jobs with the same local store, can get scheduled
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:26:03 +02:00
8c87743642
fix 'remove_vanished' cli arg again
...
since the target side wants this to be a boolean and
serde interprets a None Value as 'null' we have to only
add this when it is really set via cli
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:25:30 +02:00
05d755b282
fix inserting of worker tasks
...
when starting a new task, we do two things to keep track of tasks
(in that order):
* updating the 'active' file with a list of tasks with
'update_active_workers'
* updating the WORKER_TASK_LIST
the second also updates the status of running tasks in the file by
checking if it is still running by checking the WORKER_TASK_LIST
since those two things are not locked, it can happend that
we update the file, and before updating the WORKER_TASK_LIST,
another thread calls update_active_workers and tries to
get the status from the task log, which won't have any data yet
so the status is 'unknown'
(we do not update that status ever, likely for performance reasons,
so we have to fix this here)
by switching the order of the two operations, we make sure that only
tasks reach the 'active' file which are inserted in the WORKER_TASK_LIST
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:24:42 +02:00
143b654550
src/tools.rs - command_output: add parameter to check exit code
2020-05-27 07:25:39 +02:00
97fab7aa11
src/tools.rs: new helper to handle command_output (std::process::Output)
2020-05-27 06:53:25 +02:00
40dc103103
fix cli pull api call
...
there is no 'delete' parameter, only 'remove-vanished', so fix that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:39:19 +02:00
de4db62c57
remotes: save passwords as base64
...
to avoid having arbitrary characters in the config (e.g. newlines)
note that this breaks existings configs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:06 +02:00
1a0d3d11d2
src/api2/admin/datastore.rs: add rrd api
2020-05-26 12:26:14 +02:00
8c03041a2c
src/bin/proxmox-backup-proxy.rs: gather block device stats on datastore
2020-05-26 11:20:59 +02:00
3fcc4b4e5c
src/tools/disks.rs: add helper to read block device stats
2020-05-26 11:20:22 +02:00
3ed07ed2cd
src/tools/disks.rs: export read_sys
2020-05-26 09:49:13 +02:00
83fd4b3b1b
remote: try to use Struct for api
...
with a catch: password is in the struct but we do not want it to return
via the api, so we only 'serialize' it when the string is not empty
(this can only happen when the format is not checked by us, iow.
when its returned from the api) and setting it manually to ""
when we return remotes from the api
this way we can still use the type but do not return the password
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:55:07 +02:00
5dcdcea293
api2/config/remote: remove password from read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:49:12 +02:00
99f443c6ae
api2/config/remote: lock and use digest for removal
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:45 +02:00
4f966d0592
api2/config/remote: use rpcenv for digest for read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:28 +02:00
db0c228719
config/remote: add 'name' to Remote struct
...
and use it as section id, like with User
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:05 +02:00
309ef20d6d
src/bin/proxmox-backup-proxy.rs: simplify code
2020-05-25 16:20:32 +02:00
d0833a70f7
src/bin/proxmox-backup-proxy.rs: gather datastore usage stats
2020-05-25 16:20:32 +02:00
d8dc281992
www/DataStoreStatus.js: display loadavg stats
2020-05-25 11:54:15 +02:00
2c66a590c0
src/bin/proxmox-backup-proxy.rs: gather iowait stats
2020-05-25 11:54:15 +02:00
485841da2c
src/bin/proxmox-backup-proxy.rs: gather loadavg stats
2020-05-25 11:40:20 +02:00
3e930f2bdc
www/DataStoreStatus.js: display root disk stats
2020-05-25 11:34:24 +02:00
dd15c0aa3b
src/bin/proxmox-backup-proxy.rs: gather root disk stats
2020-05-25 11:10:07 +02:00
c1b24fbf0b
www/DataStoreStatus.js: display swap stats
2020-05-25 10:39:54 +02:00
3f23b17298
src/rrd/rrd.rs: do not wrap error and return ErrorKind::NotFound
2020-05-25 10:30:04 +02:00
c25c9d8dd1
src/bin/proxmox-backup-proxy.rs: gather swap usage stats
2020-05-25 10:25:58 +02:00
84dc6adcc1
src/rrd/cache.rs: display/log error when RRD load fails
2020-05-25 10:18:53 +02:00
0c4344650d
src/rrd/rrd.rs: store/verify magic number
2020-05-25 09:21:54 +02:00
4f9513996c
src/bin/proxmox-backup-proxy.rs: use block_in_place for rrd update
2020-05-25 08:30:59 +02:00
736edc7a7e
src/rrd/rrd.rs: implement DST_COUNTER
2020-05-25 08:14:30 +02:00
2b55de407e
src/rrd/rrd.rs: correctly compute derived values
...
use f64 for time.
2020-05-25 07:02:04 +02:00
a608806f65
www/DataStoreStatus.js: display netin/netout
2020-05-24 19:02:35 +02:00
8f0cec2642
src/bin/proxmox-backup-proxy.rs: gather netin/netout stats
2020-05-24 19:02:35 +02:00
0ed9a2b3ae
src/config/network.rs: implement is_physical_nic() helper
2020-05-24 19:02:35 +02:00
58edd33d2b
src/rrd/rrd.rs: implement DST_DERIVE
2020-05-24 19:02:35 +02:00
4fb05fde17
src/rrd/rrd.rs: restructure whole code
2020-05-24 16:51:28 +02:00
daca4f7888
src/rrd/rrd.rs: reduce size by using f64:NAN as UNKNOWN
2020-05-24 09:09:09 +02:00
4e6585839b
src/rrd/rrd.rs: simplify an fix old value deletion
2020-05-24 06:44:06 +02:00
8c981ae379
rrd: fix display interval, try to avoid numeric errors
2020-05-23 16:03:43 +02:00
803ab12ad4
rrd: simplify code
2020-05-23 15:37:17 +02:00
a4a3f7ca5e
rrd: pack multiple rrd values into th estat list
2020-05-23 14:03:44 +02:00
a2f862eed6
add experimental rrd api to get cpu stats
2020-05-23 11:50:53 +02:00
eaeda365e0
start gathering stats using new rrd module
2020-05-23 10:43:08 +02:00
6359dc891a
add simple rrd implementation
2020-05-23 10:42:48 +02:00
07ad6470ca
src/client/pull.rs: split out pull related code
2020-05-22 08:04:20 +02:00
a6160cdfeb
src/bin/proxmox-backup-proxy.rs: schedule sync jobs
2020-05-22 07:50:59 +02:00
183125d576
src/api2/pull.rs: aquire try_shared_chunk_store_lock inside pull_store
2020-05-22 07:24:17 +02:00
a3016d6583
proxmox-backup-manager: add sync-job cli
2020-05-21 11:44:45 +02:00
b29d046e89
proxmox-backup-manager: split out cert.rs
2020-05-21 11:22:20 +02:00
380bd7df97
proxmox-backup-manager: split out datastore.rs
2020-05-21 11:14:34 +02:00
ea6f404e55
proxmox-backup-manager: split out dns.rs
2020-05-21 11:10:58 +02:00
a35a211d9e
proxmox-backup-manager: split out network.rs
2020-05-21 11:08:38 +02:00
53e14507c1
proxmox-backup-manager: split out acl.rs
2020-05-21 10:56:46 +02:00
6fa39e53e0
proxmox-backup-manager: split out users.rs
2020-05-21 10:53:06 +02:00
a220a4564a
roxmox-backup-manager: start splitting command into several files
2020-05-21 10:46:07 +02:00
6f652b1b3a
rename 'job' to 'sync'
2020-05-21 10:29:25 +02:00
b1d4edc769
src/api2/config/job.rs: add job api
2020-05-21 10:16:35 +02:00
b4900286ce
src/config/jobs.rs: use SectionConfig for jobs
2020-05-21 10:16:35 +02:00
c681885227
src/bin/proxmox-backup-manager.rs: format output of show commands
2020-05-20 16:47:37 +02:00
ee8b464466
src/tools/systemd.rs: avoid compiler warnings
2020-05-20 16:47:08 +02:00
ce55db66d6
proxmox-backup-manager: add show command for remote and datastore
...
to show the data for a single item
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 16:33:07 +02:00
2882c881e9
api2/access/acl: add path and exact parameter to list_acl
...
so that we can get only a subset of the acls, filtered by the backed
also return the digest here
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:44:36 +02:00
12e3895399
api2/access/acl: make update_acl a protected api call
...
since we want to set the owner of the acl config to 'root'
which is only possible when using a protected api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:22:41 +02:00
11b6391c83
add 'exact' parameter to extract_acl_node_data
...
so that we can return acls for a single path
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:22:10 +02:00
b05672579e
api2/roles: change return field of role to roleid
...
to be compatible with the pve api
with this, we can reuse the ui parts (RoleSelector)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:47 +02:00
5160c0e986
api2/acl: add privs array to roles
...
so that an admin can see which roles have which privileges
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:37 +02:00
1ad9dd08f4
acls: use constnamemap macro for privileges
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:28 +02:00
25829a879b
src/bin/proxmox-backup-proxy.rs: schedule prune jobs
2020-05-20 13:00:53 +02:00
872062ee9f
src/config/datastore.rs_ change prune types from i64 to u64
2020-05-20 13:00:13 +02:00
67f7ffd0db
src/config/datastore.rs: add prune settings
2020-05-20 11:29:59 +02:00
0fafac2492
src/api2/access/user.rs: remove useless description
...
The description is not used at all if we refer to a type.
2020-05-20 11:27:58 +02:00
49ff10921c
src/api2/types.rs: define PRUNE_SCHEMA_KEEP_*
2020-05-20 10:13:38 +02:00
479e4932b5
src/tools/systemd/parse_time.rs: improve error message
2020-05-20 09:43:16 +02:00
dd7a7eae8f
src/bin/proxmox-backup-manager.rs: add completion helper for gc-schedule
2020-05-20 09:42:51 +02:00
8545480a31
src/bin/proxmox-backup-proxy.rs: add simple task scheduler for garbage collection
2020-05-20 08:59:45 +02:00
d6c28ddf84
src/tools/systemd/time.rs: export parse/verify
2020-05-20 08:38:39 +02:00
42fdbe5112
src/config/datastore.rs: add gc-schedule property
2020-05-20 08:38:10 +02:00
9c5c383bff
user: create default root user as typed struct
...
we added a userid attribute to the User struct, but missed that we
created the default user without that attribuet via the json! macro
which lead to a runtime panic on the deserialization
by using the struct directly, such errors will be caught by the compiler
in the future
with this change, we can remove the serde_json import here
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 06:09:08 +02:00
7d4e362993
depend on proxmox 0.1.32, src/api2/access/user.rs: simplify code
2020-05-19 12:58:46 +02:00
522c0da0a0
use new 'id_property' for user::User and use it in api calls
...
this allows us to return a user::User (or Vec<> of it)
instead of a generic serde value
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-19 09:33:56 +02:00
16c75c580b
adapt to changes of SectionConfigPlugin
...
it requires not an Option<String> for the optional id_property
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-19 09:28:45 +02:00
07ce44a633
avoid compiler warnings
2020-05-19 07:03:41 +02:00
e8d1da6a15
depend on proxmox 0.1.31 - use Value to store result metadata
2020-05-18 09:57:35 +02:00
7a314d18f7
src/tools/systemd/parse_time.rs: check max values
2020-05-16 13:13:50 +02:00
2d08c97ae2
CalendarEvent - compute_next_value: use change tracking to avoid repeated testing
2020-05-16 10:32:27 +02:00
50ce1f987d
CalendarEvent - compute_next_value: support seconds
2020-05-16 10:21:24 +02:00
d1a5ffdf78
src/tools/systemd/tm_editor.rs: new helper class
2020-05-16 10:09:41 +02:00
99baf7afcc
CalendarEvent: test and fix repeated values
2020-05-16 07:43:51 +02:00
fed270bf3f
CalendarEvent: speedup/simplify repetition tests
2020-05-16 07:09:53 +02:00
e05b637c73
src/tools/systemd/parse_time.rs: move parser into separate file
2020-05-16 06:53:15 +02:00
2ee6b3fdb9
src/tools/systemd/time.rs: implement compute_next_event
2020-05-16 06:33:03 +02:00
f3a96b2cdb
renamed: src/tools/systemd/parser.rs -> src/tools/systemd/config.rs
2020-05-16 06:32:28 +02:00
a260c74a12
src/tools/systemd/time.rs: add helpers to compute CalendarEvents
2020-05-15 17:55:54 +02:00
30f577248b
src/api2/node/time.rs: avoid custom unsafe readlink implementations
2020-05-15 06:50:07 +02:00
00491c0230
src/tools/systemd/parser.rs: use different setups for service and timer files, code cleanup
2020-05-14 13:55:13 +02:00
2ebdbac1c4
depend on nom, add parser for systemd calendar enents and time span
2020-05-14 12:18:30 +02:00
f486e9e50e
add systemd configuration file parser/writer, start job configuration
2020-05-12 13:07:49 +02:00
65dab0266c
proxmox-backup-manager: add completion helper for port list
2020-05-08 17:28:04 +02:00
525008f7ad
proxmox-backup-manager - network list: render ports/slaves
...
And render interface name as first comumn.
2020-05-08 16:07:23 +02:00
5bef0f43da
src/config/network.rs - check_bridge_ports: correctly check vlan ports
2020-05-08 15:51:47 +02:00
0f6bdbb01f
src/config/network.rs - write_config: add more consistency checks
2020-05-08 14:31:38 +02:00
a4ccb46176
src/config/network.rs: avoid duplicate port usage
2020-05-08 11:15:00 +02:00
80bf084876
src/config/network.rs: do not combine entries
...
It is unclear when and how to write combined entries ...
2020-05-08 10:20:57 +02:00
db5672e83e
src/config/network.rs: always write bridge_ports and bond_slaves
...
So that we can reliable detect the interface type.
2020-05-08 09:58:03 +02:00
86a5d56c4e
proxmox-backup-manager: add network create command
2020-05-08 09:55:56 +02:00
3dd27a3bf8
src/api2/node/network.rs: add protected flag to revert
2020-05-08 09:30:25 +02:00
3aedb73816
src/api2/node/network.rs: pass bridge_ports and slaves a property strings
...
To make it compatible with pve.
2020-05-08 08:49:17 +02:00
bab5d18c3d
src/config/network.rs: implement bond_mode
...
and rename bond_slaves to slaves to make it compatible with pve.
2020-05-07 14:07:45 +02:00
Dominik Csapak
Dietmar Maurer
Thomas Lamprecht
Wolfgang Bumiller