Commit Graph

71 Commits

Author SHA1 Message Date
7c069e82d1 fix #3743: extract zfs dataset io stats from /proc/spl/kstat/zfs/POOL/objset-*
Recently, ZFS removed the pool global io stats from
/proc/spl/kstat/zfs/POOL/io with no replacement.

To gather stats about the datastores, access now the objset specific
entries there. To be able to make that efficient, cache a map of
dataset <-> obset ids, so that we do not have to parse all files each time.

We update the cache each time we try to get the info for a dataset
where we do not have a mapping.

We cannot update it on datastore add/remove since that happens in the
proxmox-backup daemon, while we need the info here in proxmox-backup-proxy.

Sadly with this we lose the io wait metric, but it seems that this is no
longer tracked in zfs at all, so nothing we can do for that.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2022-01-11 08:45:55 +01:00
540fca5c9e tree-wide: cleanup manual map/flatten
found with clippy, best viewed with `-w` ;)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 15:02:07 +01:00
9a37bd6c84 tree-wide: fix needless borrows
found and fixed via clippy

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-12-30 13:55:33 +01:00
25877d05ac update to proxmox-sys 0.2 crate
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-11-24 10:32:27 +01:00
6ef1b649d9 update to first proxmox crate split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-10-11 11:58:49 +02:00
af2eb422d5 tools: smart: only throw error for smartctl fatal errors
only bit 0-2 are fatal errors, bit 3-7 are used to indicate
some drive conditions. for details see the manpage of smartctl(8)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 [ Thomas: resolved merge-conflict due to moved run_command ]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-09-27 08:59:13 +02:00
ccc3896ff3 avoid type re-exports 2021-09-14 08:35:43 +02:00
4c1b776168 another import cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-09-01 14:46:01 +02:00
20429238e0 disks: also check for file systems with lsblk
Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-09 13:02:30 +02:00
364299740f disks: refactor partition type handling
in preparation to also get the file system type from lsblk.

Co-developed-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2021-07-09 13:02:29 +02:00
770a36e53a add pbs-tools subcrate
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 15:10:37 +02:00
85beb7d875 tree-wide: switch to using mod.rs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-06 12:04:52 +02:00
f960fc3b6f fix #3433: use PVE's wearout logic in PBS
in PVE, the logic how wearout gets read from the smartctl output was
changed from a vendor -> id map to a sorted list of specific
attribute field names.

copy that list to pbs (in the same order), and use that to get the
wearout

in the future we might want to split the disk logic into its own crate
and reuse it in pve

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-06-08 08:31:37 +02:00
368f4c5416 fix gathering io stats for zpools
if a datastore or root is not used directly on the pool dir
(e.g. the installer creates 2 sub datasets ROOT/pbs-1), info in
/proc/self/mountinfo returns not the pool, but the path to the
dataset, which has no iostats itself in /proc/spl/kstat/zfs/
but only the pool itself

so instead of not gathering data at all, gather the info from the
underlying pool instead. if one has multiple datastores on the same
pool those rrd stats will be the same for all those datastores now
(instead of empty) similar to 'normal' directories

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-04-12 13:35:38 +02:00
3bb7e62e88 zpool parsers: small code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
3b060167f6 zpool parsers: fixup test value
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
d64226efee disks/zfs: fix test input and enforce check for config key
we test for the config key in the API so it makes sense to have as
test here too. Actually it would be better if we'd have a expect
Value defined here and enforce that it matches, but better than
nothing.

Fix the input for test 1, where tabs got replaced by spaces, as else
it fails

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-01 17:36:16 +01:00
2440eaa2df disks/zfs: pull common test code out in helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-01 14:48:35 +01:00
365915da9a clippy: use strip_prefix instead of manual stripping
it's less error-prone (off-by-one!)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
1ffe030123 various typo fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 18:52:31 +02:00
934deeff2d fix #2904: zpool status: parse vdevs with state but without statistics
some vdevs (e.g. spares) have a 'state' (e.g. AVAIL), but
not statistics like READ/WRITE/etc.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 11:41:32 +02:00
c162df60c8 zfs status: add test with spares
this will fail for now, fixed in the next commit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 11:41:32 +02:00
1cafbdc70d more whitespace fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-03 12:02:19 +02:00
a3eb7b2cea whitespace fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-03 12:00:59 +02:00
a9649ddc44 disks/zpool_status: add test for pool with special character
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
4f9096a211 disks/zpool_list: allow some more characters for pool list
not exhaustive of what zfs allows (space is missing), but this
can be done easily without problems

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
c3a4b5e2e1 zpool_list: add tests for special pool names
those names are allowed for zpools

these will fail for now, but it will be fixed in the next commit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
4eb4e94918 fix test output
field separator for pools is always a tab when using -H

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 10:31:11 +02:00
cbef49bf4f remove absolute paths when executing binaries
we set the paths manually, so this is ok

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 07:09:06 +02:00
f28cfb322a avoid compiler warnings 2020-06-20 07:24:02 +02:00
3bbe291c51 zpool_status.rs - indented_list_to_tree: do not set name property
This is no necessary. We only touch/set 'children' and 'leaf' properties.
2020-06-20 07:19:25 +02:00
215968e033 src/tools/disks/zpool_status.rs: add 'leaf' attribute to root node, rename 'prev' into 'parent' 2020-06-20 06:49:06 +02:00
eddd1a1b9c src/tools/disks/zpool_status.rs: move use clause top of file 2020-06-20 06:17:22 +02:00
d2ce211899 fixup for previous commit 2020-06-20 06:15:26 +02:00
1cb46c6f65 src/tools/disks/zpool_status.rs - cleanup: use struct StackItem instead of tuple 2020-06-19 18:58:57 +02:00
5d88c3a1c8 src/tools/disks/zpool_status.rs: remove unnecessary checks
Thos things can never happen, so simply use unwrap().
2020-06-19 18:27:39 +02:00
07fb504943 src/tools/disks/zpool_status.rs: simplify code by using serde_json::to_value 2020-06-19 17:51:13 +02:00
f675c5e978 src/tools/disks/zpool_status.rs - add all attributes to the tree 2020-06-19 16:55:28 +02:00
4e37d9ce67 add general indented_list_to_tree implementation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 14:37:40 +02:00
3352ee5656 parse_zpool_status_field: handle tabs without copying input 2020-06-18 19:40:01 +02:00
b29cbc414d parse_zpool_status_vdev: consider tabs as 8 spaces 2020-06-18 18:38:56 +02:00
9438aca6c9 src/tools/disks/zpool_status.rs: improve parser 2020-06-18 14:55:22 +02:00
177a2de992 src/tools/nom.rs: move nom helpers into separate file 2020-06-18 12:41:13 +02:00
0686b1f4db src/tools/disks/zpool_list.rs: split code into separate file 2020-06-18 10:31:07 +02:00
0727e56a06 src/tools/disks/zpool_status.rs: parse zpool status output 2020-06-18 10:23:15 +02:00
2fd3d57490 src/tools/disks/zfs.rs: rename ZFSPoolStatus into ZFSPoolInfo, fix error message 2020-06-17 09:08:26 +02:00
d0eccae37d avoid compiler warning 2020-06-17 08:07:42 +02:00
a34154d900 src/tools/disks/zfs.rs: cleanup parse_pool_header 2020-06-17 07:47:11 +02:00
c2cc32b4dd src/tools/disks/zfs.rs: add more parser tests 2020-06-17 07:38:19 +02:00
46405fa35d src/tools/disks/zfs.rs: add comment 2020-06-17 07:14:26 +02:00