Commit Graph

369 Commits

Author SHA1 Message Date
Stefan Reiter
dec00364b3 ParallelHandler: check for errors during thread join
Fix a potential bug where errors that happen after the SendHandle has
been dropped while doing the thread join might have been ignored.
Requires internal check_abort to be moved out of 'impl SendHandle' since
we only have the Mutex left, not the SendHandle.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-01 14:30:32 +02:00
Dominik Csapak
8074d2b0c3 tools: add logrotate module
this is a helper to rotate and iterate over log files
there is an iterator for open filehandles as well as
only the filename

also it has the possibilty to rotate them
for compression, zstd is used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-29 08:33:21 +02:00
Wolfgang Bumiller
ee1a9c3230 parallel_handler: clippy: 'while_let_loop'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-29 08:13:51 +02:00
Wolfgang Bumiller
db24c01106 parallel_handler: explicit Arc::clone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-28 13:40:03 +02:00
Wolfgang Bumiller
ae3cfa8f0d parallel_handler: formatting cleanup, doc comment typo fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-28 13:40:03 +02:00
Dietmar Maurer
55bee04856 src/tools/parallel_handler.rs: remove unnecessary Sync bound 2020-09-26 16:16:11 +02:00
Dietmar Maurer
b02b374b46 src/tools/parallel_handler.rs: remove static lifetime bound from handler_fn 2020-09-26 09:26:06 +02:00
Dietmar Maurer
1c13afa8f9 src/tools/parallel_handler.rs: join all threads in drop handler 2020-09-26 08:47:56 +02:00
Dietmar Maurer
69b92fab7e src/tools/parallel_handler.rs: remove unnecessary Sync trait bound 2020-09-26 07:38:44 +02:00
Dietmar Maurer
3c9b370255 src/tools/parallel_handler.rs: execute closure inside a thread pool 2020-09-25 12:58:20 +02:00
Dietmar Maurer
6a7be83efe avoid chrono dependency, depend on proxmox 0.3.8
- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
2020-09-15 07:12:57 +02:00
Fabian Grünbichler
833eca6d2f use non-panicky timestamp_opt where appropriate
by either printing the original, out-of-range timestamp as-is, or
bailing with a proper error message instead of panicking.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:48:24 +02:00
Fabian Grünbichler
151acf5d96 don't truncate DateTime nanoseconds
where we don't care about them anyway..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:48:10 +02:00
Dominik Csapak
13bed6226e tools/systemd/parse_time: enable */x syntax for calendar events
we support this in pve, so also support it here to have a more
consistent syntax

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-11 12:17:22 +02:00
Dominik Csapak
1db4cfb308 tools/sytemd/time: add tests for multivalue fields
we did this wrong earlier, so it makes sense to add regression tests

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-08 07:09:43 +02:00
Dietmar Maurer
32afd60336 src/tools/systemd/time.rs: derive Clone 2020-09-07 12:37:08 +02:00
Dietmar Maurer
02e47b8d6e SYSTEMD_CALENDAR_EVENT_SCHEMA: fix wrong schema description 2020-09-07 09:07:55 +02:00
Dominik Csapak
44055cac4d tools/systemd/time: enable dates for calendarevents
this implements parsing and calculating calendarevents that have a
basic date component (year-mon-day) with the usual syntax options
(*, ranges, lists)

and some special events:
monthly
yearly/annually (like systemd)
quarterly
semiannually,semi-annually (like systemd)

includes some regression tests

the ~ syntax for days (the last x days of the month) is not yet
implemented

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:36:29 +02:00
Dominik Csapak
1dfc09cb6b tools/systemd/time: fix signed conversion
instead of using 'as' and silently converting wrong,
use the TryInto trait and raise an error if we cannot convert

this should only happen if we have a negative year,
but this is expected (we do not want schedules from before the year 0)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:35:38 +02:00
Dominik Csapak
48c56024aa tools/systemd/tm_editor: add setter/getter for months/years/days
add_* are modeled after add_days

subtract one for set_mon to have a consistent interface for all fields
(i.e. getter/setter return/expect the 'real' number, not the ones
in the tm struct)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:34:27 +02:00
Dominik Csapak
cf103266b3 tools/systemd/tm_editor: move conversion of the year into getter and setter
the tm struct contains the year - 1900 but we added that

if we want to use the libc normalization correctly, the tm struct
must have the correct year in it, else the computations for timezones,
etc. fail

instead add a getter that adds the years and a setter that subtracts it again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:34:04 +02:00
Dominik Csapak
d5cf8f606c tools/systemd/time: fix selection for multiple options
if we give multiple options/ranges for a value, e.g.
2,4,8
we always choose the biggest, instead of the smallest that is next

this happens because in DateTimeValue::find_next(value)
'next' can be set multiple times and we set it when the new
value was *bigger* than the last found 'next' value, when in reality
we have to choose the *smallest* next we can find

reverse the comparison operator to fix this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:33:42 +02:00
Dominik Csapak
ce7ab28cfa tools/systemd/parse_time: error out on invalid ranges
if the range is reverse (bigger..smaller) we will never find a value,
so error out during parsing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:28:48 +02:00
Dominik Csapak
07ca6f6e66 tools/systemd/tm_editor: remove reset_time from add_days and document it
we never passed 'false' to it anyway so remove it
(we can add it again if we should ever need it)

also remove the adding of wday (gets normalized anyway)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:28:24 +02:00
Dominik Csapak
15ec790a40 tools/systemd/time: convert the resulting timestamp into an option
we want to use dates for the calendarspec, and with that there are some
impossible combinations that cannot be detected during parsing
(e.g. some datetimes do not exist in some timezones, and the timezone
can change after setting the schedule)

so finding no timestamp is not an error anymore but a valid result

we omit logging in that case (since it is not an error anymore)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:28:05 +02:00
Dominik Csapak
cb73b2d69c tools/systemd/time: move continue out of the if/else
will be called anyway

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:27:20 +02:00
Dominik Csapak
c931c87173 tools/systemd/time: let libc normalize time for us
mktime/gmtime can normalize time and even can handle special timezone
cases like the fact that the time 2:30 on specific day/timezone combos
do not exists

we have to convert the signature of all functions that use
normalize_time since mktime/gmtime can return an EOVERFLOW
but if this happens there is no way we can find a good time anyway

since normalize_time will always set wday according to the rest of the
time, remove set_wday

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:26:40 +02:00
Dominik Csapak
28a0a9343c tools/systemd/tm_editor: remove TMChanges optimization
while it was correct, there was no measurable speed gain
(a benchmark yielded 2.8 ms for a spec that did not find a timestamp either way)
so remove it for simpler code

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:26:04 +02:00
Dietmar Maurer
cd6ddb5a69 depend on proxmox 0.3.5 2020-09-04 08:11:53 +02:00
Thomas Lamprecht
1ffe030123 various typo fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 18:52:31 +02:00
Thomas Lamprecht
c86b6f40d7 tools/format: implement from u64 for HumanByte helper type
Could be problematic for systems where usize is 32 bit, but we do not
really support those.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 14:18:49 +02:00
Dominik Csapak
934deeff2d fix #2904: zpool status: parse vdevs with state but without statistics
some vdevs (e.g. spares) have a 'state' (e.g. AVAIL), but
not statistics like READ/WRITE/etc.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 11:41:32 +02:00
Dominik Csapak
c162df60c8 zfs status: add test with spares
this will fail for now, fixed in the next commit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 11:41:32 +02:00
Hannes Laimer
be614c625f api2/node/../disks/directory: added DELETE endpoint for removal of mount-units
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-08-14 07:06:10 +02:00
Wolfgang Bumiller
df30017ff8 remove unused import
rustc doesn't warn about this kind of import, however,
clippy does

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-13 09:05:15 +02:00
Wolfgang Bumiller
3f3ae19d63 formatting fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-12 14:30:03 +02:00
Wolfgang Bumiller
72dc68323c replace and remove old ticket functions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-12 14:28:21 +02:00
Wolfgang Bumiller
593f917742 introduce Ticket struct
and add tests and compatibility tests

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-12 14:28:21 +02:00
Dietmar Maurer
3dc1a2d5b6 src/tools/fs.rs: new helper lock_dir_noblock
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-11 10:57:48 +02:00
Wolfgang Bumiller
e7cb4dc50d introduce Username, Realm and Userid api types
and begin splitting up types.rs as it has grown quite large
already

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:05:01 +02:00
Wolfgang Bumiller
98c259b4c1 remove timer and lock functions, fix building with proxmox 0.3.2
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-04 11:33:02 +02:00
Wolfgang Bumiller
1cafbdc70d more whitespace fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-03 12:02:19 +02:00
Wolfgang Bumiller
a3eb7b2cea whitespace fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-03 12:00:59 +02:00
Dominik Csapak
eeb19aeb2d systemd/time: fix weekday wrapping on month
the weekday does not change depending on the month, so remove that wrapping

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-03 08:18:42 +02:00
Dominik Csapak
6c96ec418d systemd/time: add tests for weekday month wrapping
this will fail for now, gets fixed in the next commit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-03 08:15:26 +02:00
Dominik Csapak
be10cdb122 fix #2856: also check whole device for device mapper
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-28 11:03:45 +02:00
Dominik Csapak
a4d1675513 api2/access: implement term ticket
modeled after pves/pmgs vncticket (i substituted the vnc with term)
by putting the path and username as secret data in the ticket

when sending the ticket to /access/ticket it only verifies it,
checks the privs on the path and does not generate a new ticket

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-23 11:55:00 +02:00
Thomas Lamprecht
50ec1a8712 tools/format: add struct to pretty print bytes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 09:36:02 +02:00
Thomas Lamprecht
a74b026baa systemd/time: document CalendarEvent struct and add TODOs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 07:55:42 +02:00
Thomas Lamprecht
956295cefe parse_calendar_event: support the weekly special expression
While we do not yet support the date specs for CalendarEvent the left
out "weekly" special expression[0] dies not requires that support.
It is specified to be equivalent with `Mon *-*-* 00:00:00` [0] and
this can be implemented with the weekday and time support we already
have.

[0]: https://www.freedesktop.org/software/systemd/man/systemd.time.html#Calendar%20Events

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-21 13:24:51 +02:00
Thomas Lamprecht
449e4a66fe tools/xattr: a char from C is not universally a rust i8
Make it actually do the correct cast by using `libc::c_char`.

Fixes issues when building on other platforms, e.g., the aarch64
client only build on Arch Linux ARM I tested in my free time.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-19 19:46:27 +02:00
Wolfgang Bumiller
032cd1b862 pxar: restore file attributes, improve errors
and use the correct integer types for these operations

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 10:25:45 +02:00
Dominik Csapak
ec01eeadc6 refactor CertInfo to tools
we want to reuse some of the functionality elsewhere

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-10 11:08:53 +02:00
Dominik Csapak
3ec99affc8 get_disks: don't fail on zfs_devices
zfs does not have to be installed, so simply log an error and
continue, users still get an error when clicking directly on
ZFS

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:47:31 +02:00
Dominik Csapak
a9649ddc44 disks/zpool_status: add test for pool with special character
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
Dominik Csapak
4f9096a211 disks/zpool_list: allow some more characters for pool list
not exhaustive of what zfs allows (space is missing), but this
can be done easily without problems

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
Dominik Csapak
c3a4b5e2e1 zpool_list: add tests for special pool names
those names are allowed for zpools

these will fail for now, but it will be fixed in the next commit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
Dietmar Maurer
f91d58e157 src/tools/runtime.rs: implement get_runtime_with_builder 2020-07-07 10:11:04 +02:00
Dominik Csapak
4eb4e94918 fix test output
field separator for pools is always a tab when using -H

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 10:31:11 +02:00
Dominik Csapak
402c8861d8 fix typo
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 09:12:29 +02:00
Dominik Csapak
cbef49bf4f remove absolute paths when executing binaries
we set the paths manually, so this is ok

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 07:09:06 +02:00
Dominik Csapak
f386f512d0 add AsyncReaderStream
and replace AsyncIndexReader's stream implementation with that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 06:33:31 +02:00
Dietmar Maurer
3ddb14889a src/tools/daemon.rs: reopen STDOUT/STDERR journald streams to get correct PID in logs 2020-06-22 13:06:53 +02:00
Wolfgang Bumiller
c08fac4d69 tools::daemon: sync with child after MainPid message
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-22 10:58:04 +02:00
Wolfgang Bumiller
dc2ef2b54f tools::daemon: fetch exe name in the beginning
We get the path to our executable via a readlink() on
"/proc/self/exe", which appends a " (deleted)" during
package reloads.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-22 10:31:54 +02:00
Dietmar Maurer
f28cfb322a avoid compiler warnings 2020-06-20 07:24:02 +02:00
Dietmar Maurer
3bbe291c51 zpool_status.rs - indented_list_to_tree: do not set name property
This is no necessary. We only touch/set 'children' and 'leaf' properties.
2020-06-20 07:19:25 +02:00
Dietmar Maurer
215968e033 src/tools/disks/zpool_status.rs: add 'leaf' attribute to root node, rename 'prev' into 'parent' 2020-06-20 06:49:06 +02:00
Dietmar Maurer
eddd1a1b9c src/tools/disks/zpool_status.rs: move use clause top of file 2020-06-20 06:17:22 +02:00
Dietmar Maurer
d2ce211899 fixup for previous commit 2020-06-20 06:15:26 +02:00
Dietmar Maurer
1cb46c6f65 src/tools/disks/zpool_status.rs - cleanup: use struct StackItem instead of tuple 2020-06-19 18:58:57 +02:00
Dietmar Maurer
5d88c3a1c8 src/tools/disks/zpool_status.rs: remove unnecessary checks
Thos things can never happen, so simply use unwrap().
2020-06-19 18:27:39 +02:00
Dietmar Maurer
07fb504943 src/tools/disks/zpool_status.rs: simplify code by using serde_json::to_value 2020-06-19 17:51:13 +02:00
Dietmar Maurer
f675c5e978 src/tools/disks/zpool_status.rs - add all attributes to the tree 2020-06-19 16:55:28 +02:00
Wolfgang Bumiller
4e37d9ce67 add general indented_list_to_tree implementation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 14:37:40 +02:00
Wolfgang Bumiller
e303077132 lru_cache: restrict and annotate Send impl
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 09:37:34 +02:00
Dietmar Maurer
3352ee5656 parse_zpool_status_field: handle tabs without copying input 2020-06-18 19:40:01 +02:00
Dietmar Maurer
b29cbc414d parse_zpool_status_vdev: consider tabs as 8 spaces 2020-06-18 18:38:56 +02:00
Dietmar Maurer
9438aca6c9 src/tools/disks/zpool_status.rs: improve parser 2020-06-18 14:55:22 +02:00
Dietmar Maurer
547f0c97e4 src/tools/nom.rs: new helper parse_complete_line() for single line parsers
Like parse_complete(), but generates simpler error messages.
2020-06-18 12:57:55 +02:00
Dietmar Maurer
177a2de992 src/tools/nom.rs: move nom helpers into separate file 2020-06-18 12:41:13 +02:00
Dietmar Maurer
0686b1f4db src/tools/disks/zpool_list.rs: split code into separate file 2020-06-18 10:31:07 +02:00
Dietmar Maurer
0727e56a06 src/tools/disks/zpool_status.rs: parse zpool status output 2020-06-18 10:23:15 +02:00
Dietmar Maurer
2fd3d57490 src/tools/disks/zfs.rs: rename ZFSPoolStatus into ZFSPoolInfo, fix error message 2020-06-17 09:08:26 +02:00
Dietmar Maurer
d0eccae37d avoid compiler warning 2020-06-17 08:07:42 +02:00
Dietmar Maurer
a34154d900 src/tools/disks/zfs.rs: cleanup parse_pool_header 2020-06-17 07:47:11 +02:00
Dietmar Maurer
c2cc32b4dd src/tools/disks/zfs.rs: add more parser tests 2020-06-17 07:38:19 +02:00
Dietmar Maurer
46405fa35d src/tools/disks/zfs.rs: add comment 2020-06-17 07:14:26 +02:00
Dietmar Maurer
66af7f51bc src/tools/disks/zfs.rs: make zfs list parser private 2020-06-17 07:00:54 +02:00
Dietmar Maurer
c72ccd4e33 src/tools/disks/zfs.rs: add regression tests for parse_zfs_list 2020-06-16 18:14:35 +02:00
Dietmar Maurer
902b2cc278 src/tools/disks/zfs.rs: simplify code 2020-06-16 17:51:17 +02:00
Dietmar Maurer
36c65ee0b0 src/tools/disks/zfs.rs: cleanup (rename usage properties)
And allow to parse zpool list output without -v flag.
2020-06-16 13:25:53 +02:00
Dietmar Maurer
3378fd9fe5 src/tools/disks/zfs.rs: parse more infos (dedup, fragmentation, health) 2020-06-16 13:25:53 +02:00
Dietmar Maurer
58c51cf3d9 avoid compiler warnings 2020-06-16 13:25:53 +02:00
Dietmar Maurer
5509b199fb use new run_command helper 2020-06-16 13:25:53 +02:00
Wolfgang Bumiller
4482f3fe11 pxar, acl: cleanup acl helper usage
use NixPath for Acl::set_file to avoid memduping the c
string

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:52:18 +02:00
Dominik Csapak
e693818afc refactor time functions to tools
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:10 +02:00
Dietmar Maurer
d4f2397d4c add api to format disks and create datastores 2020-06-10 11:03:36 +02:00
Dietmar Maurer
669c137fec src/tools/systemd.rs: implement daemon_reload, start_unit, stop_unit and enable_unit 2020-06-10 08:56:04 +02:00
Dietmar Maurer
144006fade src/tools.rs: add new run_command helper 2020-06-10 07:16:47 +02:00