Commit Graph

387 Commits

Author SHA1 Message Date
Dietmar Maurer 0727e56a06 src/tools/disks/zpool_status.rs: parse zpool status output 2020-06-18 10:23:15 +02:00
Dietmar Maurer 2fd3d57490 src/tools/disks/zfs.rs: rename ZFSPoolStatus into ZFSPoolInfo, fix error message 2020-06-17 09:08:26 +02:00
Dietmar Maurer d0eccae37d avoid compiler warning 2020-06-17 08:07:42 +02:00
Dietmar Maurer a34154d900 src/tools/disks/zfs.rs: cleanup parse_pool_header 2020-06-17 07:47:11 +02:00
Dietmar Maurer c2cc32b4dd src/tools/disks/zfs.rs: add more parser tests 2020-06-17 07:38:19 +02:00
Dietmar Maurer 46405fa35d src/tools/disks/zfs.rs: add comment 2020-06-17 07:14:26 +02:00
Dietmar Maurer 66af7f51bc src/tools/disks/zfs.rs: make zfs list parser private 2020-06-17 07:00:54 +02:00
Dietmar Maurer c72ccd4e33 src/tools/disks/zfs.rs: add regression tests for parse_zfs_list 2020-06-16 18:14:35 +02:00
Dietmar Maurer 902b2cc278 src/tools/disks/zfs.rs: simplify code 2020-06-16 17:51:17 +02:00
Dietmar Maurer 36c65ee0b0 src/tools/disks/zfs.rs: cleanup (rename usage properties)
And allow to parse zpool list output without -v flag.
2020-06-16 13:25:53 +02:00
Dietmar Maurer 3378fd9fe5 src/tools/disks/zfs.rs: parse more infos (dedup, fragmentation, health) 2020-06-16 13:25:53 +02:00
Dietmar Maurer 58c51cf3d9 avoid compiler warnings 2020-06-16 13:25:53 +02:00
Dietmar Maurer 5509b199fb use new run_command helper 2020-06-16 13:25:53 +02:00
Wolfgang Bumiller 4482f3fe11 pxar, acl: cleanup acl helper usage
use NixPath for Acl::set_file to avoid memduping the c
string

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:52:18 +02:00
Dominik Csapak e693818afc refactor time functions to tools
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:10 +02:00
Dietmar Maurer d4f2397d4c add api to format disks and create datastores 2020-06-10 11:03:36 +02:00
Dietmar Maurer 669c137fec src/tools/systemd.rs: implement daemon_reload, start_unit, stop_unit and enable_unit 2020-06-10 08:56:04 +02:00
Dietmar Maurer 144006fade src/tools.rs: add new run_command helper 2020-06-10 07:16:47 +02:00
Dietmar Maurer b9cf6ee797 src/tools/systemd/types.rs: add Mount config 2020-06-09 18:47:10 +02:00
Wolfgang Bumiller cdde66d277 statistics: covariance(): avoid allocation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 13:57:27 +02:00
Dominik Csapak ba97479848 add statistics module
provides some basic statistics functions (sum, mean, etc.)
and a function to return the parameters of the linear regression of
two variables

implemented using num_traits to be more flexible for the types

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:19:51 +02:00
Dominik Csapak 33070956af let disk_usage return StorageStatus and use it for datastores/nodes
disk_usage returned the same values as defined in StorageStatus,
so simply use that

with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:05:39 +02:00
Dietmar Maurer da84cc52f4 src/tools/systemd.rs: implement escape_unit and unescape_unit 2020-06-09 11:52:06 +02:00
Dietmar Maurer 9bb161c881 src/tools/disks.rs: add create_single_linux_partition and create_file_system 2020-06-08 17:43:01 +02:00
Dietmar Maurer 297e600730 cleanup comment 2020-06-08 17:43:01 +02:00
Dietmar Maurer ed7b3a7de2 src/tools/disks.rs: add get_fs_uuid helper 2020-06-08 17:43:01 +02:00
Dietmar Maurer 0f358204bd src/tools/disks.rs: add helper to list partitions 2020-06-08 17:43:01 +02:00
Dietmar Maurer ca6124d5fa src/tools/disks.rs: make helpers pub
So that I can use them with my test code.
2020-06-08 17:43:01 +02:00
Wolfgang Bumiller c443f58b09 switch to external pxar and fuse crates
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Wolfgang Bumiller 1e3d9b103d xattr: make xattr_name_fcaps public
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Dietmar Maurer bc853b028f src/tools/disks.rs: cleanup, remove unused DiskUse bitflag type 2020-06-08 09:43:07 +02:00
Dietmar Maurer d406de299b src/tools/disks.rs: use dev_t to index zfs/lvm device sets 2020-06-08 09:01:34 +02:00
Dietmar Maurer 7c3aa258f8 src/tools/disks/zfs.rs: allow empty zpool list output 2020-06-08 07:23:04 +02:00
Dietmar Maurer 044055062c src/tools/disks.rs: new helper to reread partition table 2020-06-08 07:22:06 +02:00
Dietmar Maurer 707974fdb3 src/api2/node/disks.rs: implement initgpt API 2020-06-07 10:30:34 +02:00
Dietmar Maurer 9069debcd8 src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA 2020-06-07 07:20:25 +02:00
Dietmar Maurer d2522b2db6 src/tools/disks.rs: fix disk size, add completion helper 2020-06-06 15:39:25 +02:00
Dietmar Maurer 042afd6e52 src/tools/disks.rs: new helper disk_by_name() 2020-06-06 12:22:38 +02:00
Dietmar Maurer de1e1a9d95 src/tools/disks.rs: use api macro so that we can use those types with the api 2020-06-06 11:37:24 +02:00
Dietmar Maurer 91960d6162 src/tools/disks.rs - get_disks: query smart status 2020-06-06 09:18:20 +02:00
Dietmar Maurer 4c24a48eb3 src/tools/disks/smart.rs: use model.to_string_lossy() to simplify code 2020-06-06 09:05:22 +02:00
Dietmar Maurer 484e761dab src/tools/disks/smart.rs: try to get correct wearout for ATA devices 2020-06-06 09:01:15 +02:00
Dietmar Maurer 059b7a252e src/tools/disks/smart.rs - get_smart_data: use &Disk instead of &str
So that we can query other device infos easily (model, vendor, ..)
2020-06-06 08:24:58 +02:00
Dietmar Maurer eb80aac288 src/tools/disks/smart.rs: parse output from smartctl 2020-06-05 18:30:06 +02:00
Dietmar Maurer c26aad405f src/tools/disks.rs: implement get_disks (similar to the one in PVE::Diskmanage)
But no ceph support for now. Also no support for old cciss block devices.
2020-06-05 10:33:53 +02:00
Dietmar Maurer f03a0e509e src/tools/disks.rs; use correct subdir to check holders 2020-06-05 10:33:53 +02:00
Dietmar Maurer 4c1e8855cc src/tools/disks.rs: fix disk type detection, remove newline from vendor string 2020-06-05 08:09:52 +02:00
Dietmar Maurer 620911b426 src/tools/disks/lvm.rs: implement get_lvm_devices() 2020-06-04 09:12:19 +02:00
Dietmar Maurer 5c264c8d80 src/tools/disks.rs: add/use get_partition_type_info 2020-06-04 07:48:22 +02:00
Dominik Csapak eed8a5ad79 tools/systemd/time: fix compute_next_event for weekdays
two things were wrong here:
* the range (x..y) does not include y, so the range
  (day_num+1..6) goes from (day_num+1) to 5 (but sunday is 6)

* WeekDays.bits() does not return the 'day_num' of that day, but
  the bit value (e.g. 64 for SUNDAY) but was treated as the index of
  the day of the week
  to fix this, we drop the map to WeekDays and use the 'indices'
  directly

this patch makes the test work again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:02:33 +02:00
Dominik Csapak 538b9c1c27 systemd/time: add tests for all weekdays
this fails for now

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:02:23 +02:00
Dietmar Maurer 456ad0c478 src/tools/disks/zfs.rs: add parser for zpool list output 2020-06-03 12:16:08 +02:00
Thomas Lamprecht add5861e8d typo fixes all over the place
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 16:39:08 +02:00
Dietmar Maurer 934f5bb8ac src/bin/proxmox-backup-proxy.rs: cleanup, move code to src/tools/disks.rs
And simplify find_mounted_device by using stat.st_dev
2020-05-29 11:13:36 +02:00
Dietmar Maurer 750252ba2f src/tools/systemd/time.rs: add test for "daily" schedule 2020-05-29 07:52:09 +02:00
Dietmar Maurer c94e1f655e rrd stats: improve io delay stats 2020-05-28 19:12:13 +02:00
Dietmar Maurer 547e3c2f6c src/tools/disks/zfs.rs: use wtime + rtime (wait + run time) 2020-05-28 11:45:34 +02:00
Dietmar Maurer 0146133b4b src/tools/disks/zfs.rs: helper to read zfs pool io stats 2020-05-28 10:07:52 +02:00
Dietmar Maurer 3fcc4b4e5c src/tools/disks.rs: add helper to read block device stats 2020-05-26 11:20:22 +02:00
Dietmar Maurer 3ed07ed2cd src/tools/disks.rs: export read_sys 2020-05-26 09:49:13 +02:00
Dietmar Maurer ee8b464466 src/tools/systemd.rs: avoid compiler warnings 2020-05-20 16:47:08 +02:00
Dietmar Maurer 479e4932b5 src/tools/systemd/parse_time.rs: improve error message 2020-05-20 09:43:16 +02:00
Dietmar Maurer d6c28ddf84 src/tools/systemd/time.rs: export parse/verify 2020-05-20 08:38:39 +02:00
Dominik Csapak 16c75c580b adapt to changes of SectionConfigPlugin
it requires not an Option<String> for the optional id_property

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-19 09:28:45 +02:00
Dietmar Maurer 7a314d18f7 src/tools/systemd/parse_time.rs: check max values 2020-05-16 13:13:50 +02:00
Dietmar Maurer 2d08c97ae2 CalendarEvent - compute_next_value: use change tracking to avoid repeated testing 2020-05-16 10:32:27 +02:00
Dietmar Maurer 50ce1f987d CalendarEvent - compute_next_value: support seconds 2020-05-16 10:21:24 +02:00
Dietmar Maurer d1a5ffdf78 src/tools/systemd/tm_editor.rs: new helper class 2020-05-16 10:09:41 +02:00
Dietmar Maurer 99baf7afcc CalendarEvent: test and fix repeated values 2020-05-16 07:43:51 +02:00
Dietmar Maurer fed270bf3f CalendarEvent: speedup/simplify repetition tests 2020-05-16 07:09:53 +02:00
Dietmar Maurer e05b637c73 src/tools/systemd/parse_time.rs: move parser into separate file 2020-05-16 06:53:15 +02:00
Dietmar Maurer 2ee6b3fdb9 src/tools/systemd/time.rs: implement compute_next_event 2020-05-16 06:33:03 +02:00
Dietmar Maurer f3a96b2cdb renamed: src/tools/systemd/parser.rs -> src/tools/systemd/config.rs 2020-05-16 06:32:28 +02:00
Dietmar Maurer a260c74a12 src/tools/systemd/time.rs: add helpers to compute CalendarEvents 2020-05-15 17:55:54 +02:00
Dietmar Maurer 00491c0230 src/tools/systemd/parser.rs: use different setups for service and timer files, code cleanup 2020-05-14 13:55:13 +02:00
Dietmar Maurer 2ebdbac1c4 depend on nom, add parser for systemd calendar enents and time span 2020-05-14 12:18:30 +02:00
Dietmar Maurer f486e9e50e add systemd configuration file parser/writer, start job configuration 2020-05-12 13:07:49 +02:00
Wolfgang Bumiller 10effc9849 add tools/disks.rs (work in progress...)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-05-05 10:14:42 +02:00
Dietmar Maurer b080583ba8 src/bin/proxmox-backup-manager.rs: improve user list output 2020-05-01 16:22:50 +02:00
Wolfgang Bumiller 9094186a57 xattr: cleanup: don't use pxar types in the API
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-24 11:23:48 +02:00
Wolfgang Bumiller 27a3decbfe xattr: api cleanup
Make `flistxattr()` return a `ListXAttr` helper which
provides an iterator over `&CStr`.

This exposes the property that xattr names are a
zero-terminated string without simply being an opaque
"byte vector". Using &[u8] as a type here is too lax.

Also let `fgetxattr` take a `CStr`. While this may be a
burden on the caller, we usually already have
zero-terminated strings on the call site. Currently we only
use this method coming from `flistxattr` after all.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-24 10:56:52 +02:00
Wolfgang Bumiller 9af76ef075 xattr: use checked_mul to increase size
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-24 10:56:52 +02:00
Wolfgang Bumiller f7d4e4b506 switch from failure to anyhow
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-17 18:43:30 +02:00
Christian Ebner 536683e73b src/backup/dynamic_index.rs: Add LruCache for chunks.
In order to improve non-sequential reads of chunks as e.g. in FUSE.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-28 08:47:16 +01:00
Dietmar Maurer 4939255fb4 src/tools/format.rs: move output rendering code to this new file 2020-02-28 07:30:35 +01:00
Christian Ebner af934f8cf6 tools::lru_cache: Improve access() and insert() by using HashMap::entry().
entry() allows to lookup the position where and entry belongs and update/insert
it in the HashMap more efficiently than get_mut() and insert().
Details: https://gankra.github.io/blah/hashbrown-insert/

In addition, use the struct LinkedList and remove the outdated code.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-27 06:56:25 +01:00
Christian Ebner e3ab9a383c tools::lru_cache: Separate LinkedList part of LruCache into own struct.
In order to keep the separation more aparent and avoid mut borrow conflics.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-27 06:55:48 +01:00
Christian Ebner 75c2ee7bab tools::lru_cache: Make key generic.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-02-27 06:55:16 +01:00
Dietmar Maurer dd04383bb8 src/tools/runtime.rs: call OPENSSL_thread_stop to avoid race with openssl cleanup handlers 2020-02-26 10:38:55 +01:00
Wolfgang Bumiller 0bcbb5434e cleanup unused module
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-02-25 11:38:27 +01:00
Wolfgang Bumiller 9e003074cb runtime: fix blocking strategy:
- do not "double"-block_in_place() (it may not be nested)
- do not call block_in_place() in non-worker threads

is_in_tokio() isn't sufficient, we need to actually know
that we're in a worker-thread, so we do this by remembering
that we're blocking.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-02-17 09:48:22 +01:00
Wolfgang Bumiller 1283d58ca9 runtime: remove IN_TOKIO thread local variable
tokio now has Handle::try_current() allowing us to
generally check for a tokio runtime even if spawned by
someone else

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-02-13 08:56:38 +01:00
Dietmar Maurer 501f4fa220 depend on proxmox 0.1.13, use new tty helpers from there 2020-01-31 08:16:00 +01:00
Christian Ebner 0502ce6da3 tools::acl: Add helpers to create extended attribute buffer.
This helpers are used to construct the extended attributes values from
the ACLs stored in the pxar archive.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-29 09:34:57 +01:00
Christian Ebner bcf0d452c9 pxar: move acl helper functions to src/tools/acl.rs
They are not only needed by the pxar::sequential_decoder but also for the fuse
xattr impl, so it makes more sense to have them there.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-29 09:34:54 +01:00
Wolfgang Bumiller 6f763ae673 tools: LruCache: add dropcheck marker
See https://doc.rust-lang.org/nomicon/phantom-data.html

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-29 09:18:27 +01:00
Christian Ebner 04350b4c08 tools: add access method to LruCache
This will return a mutable reference just like get_mut, but on a cache miss
it will get and insert the missing value via the fetch method provided via the
Cacher trait.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-24 09:32:09 +01:00
Christian Ebner 1685c2e341 tools: LRU cache to use for pxar FUSE implementation.
Implements a cache with least recently used cache replacement policy.
Internally the state is tracked by a HashMap (for fast access) and a doubly
linked list (for the access order).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-22 13:11:42 +01:00
Dietmar Maurer f1d99e3f6a src/bin/proxmox-backup-client.rs: use a std channel to write the catalog 2020-01-22 12:49:08 +01:00
Dietmar Maurer dcd033a53c src/tools/std_channel_writer.rs: new StdChannelWriter helper class 2020-01-22 11:33:57 +01:00
Wolfgang Bumiller 9ea4bce444 bump proxmox crate to 0.1.7
The -sys, -tools and -api crate have now been merged into
the proxmx crate directly. Only macro crates are separate
(but still reexported by the proxmox crate in their
designated locations).

When we need to depend on "parts" of the crate later on
we'll just have to use features.

The reason is mostly that these modules had
inter-dependencies which really make them not independent
enough to be their own crates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 13:48:37 +01:00
Wolfgang Bumiller 077ac6f8b3 runtime: actually park on Pending...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 10:37:16 +01:00
Wolfgang Bumiller 2d51569396 runtime: custom block_on implementation
futures::executor::block_on does not allow nesting, but we
need that...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 10:24:52 +01:00
Wolfgang Bumiller 650e052299 runtime: drop now not required Send and static restrictions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-20 14:09:24 +01:00
Wolfgang Bumiller d973aa827c introduce new runtime tokio helpers
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-20 13:12:40 +01:00
Oguz Bektas cbe01dc5c7 tools: add read_and_verify_password helper
add a helper to perform some basic checks on password prompts.
- verification (asks for a 2nd time)
- check length

also use the new helper where password input in tty is taken to reduce
duplicate code.

this helper should be used when creating keys, changing passphrases etc.

note: this helper can be extended later on to provide better checks for
password strength.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-01-15 10:52:57 +01:00
Dietmar Maurer 8bf4559b4e update to nix 0.16 2019-12-19 09:29:44 +01:00
Wolfgang Bumiller dc08934563 drop Cancellable future in favor of abortable
futures-0.3 has a futures::future::abortable() function
which does the exact same, returns an Abortable future with
an AbortHandle providing an abort() method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:52:07 +01:00
Wolfgang Bumiller 8554ac5ec3 fix broadcast_future test case
We used to await all the futures via the runtime's shutdown
method, which doesn't exist anymore, so await all the join
handles instead.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:47:12 +01:00
Wolfgang Bumiller f374ba4c0b update tests
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 12:42:02 +01:00
Wolfgang Bumiller b5c9a706ce fix cancellable future
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 12:10:04 +01:00
Wolfgang Bumiller db0cb9ce0b update a chunk of stuff to the hyper release
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
Dietmar Maurer 62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer 834a2f95a0 avoid some clippy warnings 2019-10-25 18:44:51 +02:00
Wolfgang Bumiller 175eeb870d clippy: use write_all in file logger
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 13:56:09 +02:00
Wolfgang Bumiller 653b1ca10e some clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-11 12:14:05 +02:00
Wolfgang Bumiller c77563146b comment fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-03 11:17:33 +02:00
Wolfgang Bumiller daef93f481 tools: add tokio::main() replacement
to deal with block_on() not allowing blocking()

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-03 11:16:30 +02:00
Wolfgang Bumiller 66fbf5bad0 add WrappedReaderStream test
Mostly to just document the rt.block_on() issue with
blocking().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-03 10:05:45 +02:00
Wolfgang Bumiller 083ff3fd5d update to tokio 0.2.0-alpha.4
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 556eb70ea3 add generic either-or AsyncRead/Write type
The HttpsConnector will use this. Instead of implementing a
specialized MaybeTlsStream, this is simply a generic "either
this or that kind of Async Read/Write type".

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 0cdb8d9c5b src/tools/wrapped_reader_stream.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller e668912a99 src/tools/futures.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 0f5856acca src/tools/daemon.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 56e351c911 src/tools/broadcast_future.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 627bb7d114 src/tools/async_mutex.rs: switch to async
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:21:26 +02:00
Wolfgang Bumiller 40f8680e37 tools: remove SignalFd
We don't use it anymore and it'll just be more work when
switching to async-await.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-23 13:44:53 +02:00
Wolfgang Bumiller 48b85e8e3b tools: tty: replace mem::uninitialized and style fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:14:19 +02:00
Wolfgang Bumiller b528fddfc6 tools: timer: style fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:07:56 +02:00
Wolfgang Bumiller b7f44ace92 timer: replace mem::uninitialized
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:07:10 +02:00
Dietmar Maurer 7f66c29e5c remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead 2019-08-03 16:44:31 +02:00
Dietmar Maurer fd329bdcd7 remove src/tools/procfs.rs, use proxmox::sys::linux::procfs instead 2019-08-03 16:26:44 +02:00
Dietmar Maurer 779e4c90b1 src/tools/vec.rs: remove unused file
We already moved that to proxmox::tools
2019-08-03 16:11:47 +02:00
Dietmar Maurer e18a6c9ee5 update to nix 0.14, use code from proxmox:tools 2019-08-03 13:05:38 +02:00
Dietmar Maurer b86f263ced use new proxmox::sys::linux::magic 2019-08-03 09:28:57 +02:00
Christian Ebner 5e50c606b0 pxar: cleanup: s/CA_FORMAT/PXAR/g and s/CaFormat/Pxar/g
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-03 08:51:33 +02:00
Christian Ebner 91451c4355 pxar: cleanup: move filesystem related magic numbers to tools/fs/magic.rs
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-03 08:48:12 +02:00
Christian Ebner 2be3eff5c3 pxar: avoid taking ownership of xattrs, fcaps and quota_projid on corresponding restore functions.
By borrowing these objects we preserve the functionality but make sure
that ownership doesn't change, avoiding problems when contained within other
structs such as e.g. a buffer storing these attributes.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-08-02 06:44:00 +02:00
Dietmar Maurer c698636a5d src/tools/fs.rs - read_dir: return nix::Result 2019-07-04 10:47:16 +02:00
Wolfgang Bumiller f35197f449 replace tools::vec with proxmox::tools::vec
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 11:03:25 +02:00
Wolfgang Bumiller d0162d53d3 tools: remove io module
now completely replaced by proxmox::tools::io.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:44:12 +02:00
Wolfgang Bumiller 9110a69bd8 tools: remove read/write
now completely replaced by
proxmox::tools::io::{ReadExt, WriteExt}

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:39:42 +02:00
Wolfgang Bumiller ca3c3ce997 daemon: remove last use of tools::read/write
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-07-01 10:39:13 +02:00
Wolfgang Bumiller dd5495d6dc tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
Dietmar Maurer 44c54845b3 src/tools/xattr.rs: fix test when run as root 2019-06-05 07:57:42 +02:00
Christian Ebner 042babe4e7 src/tools/fs.rs: impl ioctl calls to get/set fsxattr
This implements fs_ioc_fsgetxattr/fs_ioc_fssetxattr calls in order to read or
write fsxattr for a given file descriptor.
This is needed in order to read or write the quota project id for filesystems
which support project quotas (EXT4/XFS/FUSE).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-06-05 06:22:49 +02:00
Wolfgang Bumiller 390e83c9b9 typo fix and doc improvement
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:57:28 +02:00
Wolfgang Bumiller 7fb49397d5 doc fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:56:14 +02:00
Wolfgang Bumiller 382609b04e tools: add Cancellable, start a futures submodule
To make a future cancellable, use:
let (future, canceller) =
    crate::tools::futures::cancellable(future);

Proceed with using `future` as usual, `canceller` is
clonable and can cancel the future via the `.cancel()`
method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:18:29 +02:00
Wolfgang Bumiller 0d32d71fb7 tools: add AsyncMutex::new_locked
Allows creating a pre-locked mutex, returning the mutex and
a guard.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-29 11:18:29 +02:00
Christian Ebner 687407741b src/tools/xattr.rs: add test for is_valid_xattr_name()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:11:36 +02:00
Christian Ebner 357e4614e2 Minor refactoring of pxars xattr encoder/decoder, mostly reformatting and renaming.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-24 06:11:36 +02:00
Wolfgang Bumiller c05a8c8d18 doc-test fixup
cargo test by default compiles and runs all code snippets
found in the documentation...

oops...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 15:22:33 +02:00
Wolfgang Bumiller 8ea3b1d188 tree-wide: use the new vec/io tools modules
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 09:49:10 +02:00
Wolfgang Bumiller 9cdda3f7c7 tools: add helpful vector and read operations
After importing the I/O ops trait via:
    use crate::tools::io::ops::*;

Instead of:
    let mut buffer = vec![0u8; 65536];
    file.read_exact(&mut buffer)?;
use:
    let buffer = file.read_exact_allocated(65536)?;

After importing the vector helpers via:
    use crate::tools::vec::{self, ops::*};

For a buffer which *could* be uninitialized but you prefer
zero-initialization anyway for security reasons, instead of:
    let mut buffer = vec![0u8; len];
use:
    let mut buffer = vec::undefined(len);
which zero-initializes, but, if the `valgrind` feature flag
is enabled, marks the vector as having undefined contents,
so reading from it will cause valgrind errors.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 09:49:10 +02:00
Christian Ebner de61bc9264 src/tools/xattr.rs: impl unit tests for fsetattr and fgetattr
As git does not preserve xattrs they have to be set before they can be read.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-22 10:01:10 +02:00
Christian Ebner bee8d8ea78 src/tools/xattr.rs: add functions name_store and security_capability
These functions allow to check if the name for xattrs/fcaps are valid.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-21 06:05:31 +02:00
Christian Ebner 2dcdd3b427 src/tools/xattrs.rs: impl libc wrapper functions to get/set xattrs and fcaps
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-16 11:14:21 +02:00
Christian Ebner dec1ff18b8 src/tools/acl.rs: impl add_entry_full for ACL and set_permissions for ACLEntry
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-16 11:10:06 +02:00
Christian Ebner d22096ef4b src/tools/acl.rs: impl POSIX access control list (ACL) manipulation tools
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-05-14 06:28:42 +02:00
Dietmar Maurer 824b5ee4ee src/tools/broadcast_future.rs: cleanup, decompose into two classes
In order to make it more usable.
2019-04-30 08:57:20 +02:00
Wolfgang Bumiller d82ed9b0f5 tools: add AsyncMutex
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-29 14:19:03 +02:00
Dietmar Maurer 3dceb9b304 src/tools/broadcast_future.rs: add new constructor new_oneshot()
To simplify usage. Also added a test case.
2019-04-27 10:56:49 +02:00
Dietmar Maurer 490be29ed6 src/tools/broadcast_future.rs: new helper class 2019-04-26 17:57:57 +02:00
Christian Ebner 8b7c0def84 src/tools/procfs.rs: clean up tab indentation and use whitespaces instead
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-25 13:14:52 +02:00
Wolfgang Bumiller 5e5eed5c3b use double-fork for reload
To ensure the new process' parent is pid 1, so systemd won't
complain about supervising a process it does not own.

Fixes the following log spam on reloads:
Apr 25 10:50:54 deb-dev systemd[1]: proxmox-backup.service: Supervising process 1625 which is not our child. We'll most likely not notice when it exits.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:02:12 +00:00
Wolfgang Bumiller 897982e237 tools: add read/write convenience helpers
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:02:12 +00:00
Wolfgang Bumiller d98c9a7a38 use service Type=notify
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:01:33 +00:00
Wolfgang Bumiller 9c351a36c4 tools/daemon: add sd_notify wrapper
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:01:28 +00:00
Dietmar Maurer d8871f2f18 src/tools/procfs.rs: avoid useless allocations 2019-04-24 09:35:15 +02:00
Christian Ebner ae694f44ef src/tools/procfs.rs: add tests for read_proc_net_route() and read_proc_net_ipv6_route()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-24 06:43:52 +02:00
Christian Ebner ec8db837fb src/tools/procfs.rs: refactor read_proc_net_route() and read_proc_net_ipv6_route()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-24 06:43:32 +02:00
Christian Ebner eb46c3d1ec src/tools/procfs.rs: implement read_proc_net_ipv6_route()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-23 14:31:23 +02:00
Christian Ebner 286f0d4099 src/tools/procfs.rs: implement read_proc_net_route()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-15 11:13:16 +02:00
Christian Ebner d2981e2738 src/tools/procfs.rs: implement read_proc_net_dev()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-13 11:52:59 +02:00
Christian Ebner 13abfdc57a src/tools/procfs.rs: implement read_memory_usage()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-13 11:52:01 +02:00
Christian Ebner 3005ef48b6 src/tools/procfs.rs: Rename variable and struct to be more consistent
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-12 12:13:34 +02:00
Christian Ebner cc93bacb72 src/tools/procfs.rs: simplify read_cpuinfo()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-12 12:13:14 +02:00
Wolfgang Bumiller 620dccf1a1 tools/daemon: dup the TcpListener file descriptor
Now that we let hyper shutdown gracefully we need an owned
version of the listening socket to prevent it from closing
before running the reload preparations.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-10 15:20:10 +02:00
Christian Ebner 1cd33633aa /src/tools: Cleanup tempfiles in folder
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-10 13:56:22 +02:00
Christian Ebner 2b6a3e1396 src/tools/procfs.rs: implement read_cpuinfo()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-10 13:11:11 +02:00
Dietmar Maurer 5f550fd99f start hyper server using with_graceful_shutdown()
Without, hyper keeps some futures running, and the server does not
correctly shutdown.
2019-04-10 08:24:32 +02:00
Dietmar Maurer 7f2078de59 src/tools/procfs.rs: avoid compiler warning 2019-04-10 06:40:29 +02:00
Christian Ebner 1f6fbf1c57 src/tools/procfs.rs: implement read_meminfo()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-09 15:21:49 +02:00
Dietmar Maurer e3f41f2165 src/tools/daemon.rs: use new ServerState handler 2019-04-08 14:00:23 +02:00
Dietmar Maurer eea8131952 src/tools.rs: implement file_set_contents_full() 2019-04-06 13:53:43 +02:00
Dietmar Maurer 244abab769 src/tools/file_logger.rs: fix test 2019-04-06 11:24:37 +02:00
Christian Ebner a2474d328f src/tools/procfs.rs: implement read_proc_uptime and read_proc_uptime_ticks
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-05 06:22:21 +02:00
Christian Ebner 08daa81af6 procfs: implement check_process_running and check_process_running_pstart
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-04 13:54:17 +02:00
Dietmar Maurer 79ad7b7ec2 src/tools/procfs.rs: split regex into several lines 2019-04-04 09:25:19 +02:00
Dietmar Maurer 52ddc2522d src/tools/procfs.rs: use libc::pid_t 2019-04-04 08:05:43 +02:00
Dietmar Maurer d3d7ab1273 src/tools/file_logger.rs: new - accept AsRef<Path> 2019-04-03 14:13:33 +02:00
Dietmar Maurer 3c2012f97a src/tools/procfs.rs: start helper tools to read procfs 2019-04-03 13:39:20 +02:00
Dietmar Maurer 3489936ebb src/tools/file_logger.rs: change timestamp format to rfc3339 2019-04-03 08:58:43 +02:00
Dietmar Maurer 9136f857cc add global var to indicate server shutdown requests 2019-04-01 12:05:11 +02:00
Dietmar Maurer 11861a482d src/backup/chunk_store.rs: fix GC
Added option to get oldest_writer timestamp from ProcessLocker.
2019-03-31 17:21:36 +02:00
Dietmar Maurer abfc001f25 src/tools/process_locker.rs: new() - use generic path parameter 2019-03-22 09:41:32 +01:00
Dietmar Maurer a650f50301 src/tools/process_locker.rs: implement inter-process reader-writer locks 2019-03-22 08:05:51 +01:00
Wolfgang Bumiller a690ecacf0 daemon: simplify daemon creation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 12:12:54 +01:00
Wolfgang Bumiller e4311382d9 tools: daemon: rename some structs
Reloadable resources are now 'Reloadable' instead of
'ReexecContinue'.

The struct handling the reload is a 'Reloader', not a
'ReexecStore'.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 11:09:46 +01:00