tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,797 Commits 2 Branches 54 Tags 14 MiB
98eb435d90
Commit Graph

24 Commits

Author SHA1 Message Date
Dietmar Maurer
98eb435d90 proxmox-rrd: use syncfs after writing rrd files 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer
7f381a6246 proxmox-rrd: use fine grained locking in commit_journal_impl 
Aquire the rrd_map lock for each file (else we block access for a long time)
 2021-10-18 14:55:47 +02:00
Dietmar Maurer
c17fbbbc07 proxmox-rrd: log all errors from apply_and_commit_journal_thread (but only once) 2021-10-18 11:57:19 +02:00
Dietmar Maurer
2b05008a11 proxmox-rrd: cleanup - use struct instead of tuple 2021-10-16 12:45:03 +02:00
Dietmar Maurer
45700e2ecf proxmox-rrd: move RRDMap into extra file 2021-10-16 12:45:03 +02:00
Dietmar Maurer
f84304235b proxmox-rrd: move JournalState into extra file 2021-10-16 12:45:03 +02:00
Dietmar Maurer
0ca41155b2 proxmox-rrd: implement non blocking journal 
Do not block while applying the journal.
 2021-10-16 12:45:03 +02:00
Dietmar Maurer
a291ab59ba proxmox-rrd: rename RRDCacheState to JournalState 2021-10-15 09:35:44 +02:00
Dietmar Maurer
fce7cd0d36 proxmox-rrd: avoild blocking readers while applying the journal 
By using and extra RwLock<RRDMap> on the rrd data.
 2021-10-15 09:22:07 +02:00
Dietmar Maurer
658357c5a8 proxmox-rrd: log journal apply/flush times, split apply and flush 
We need to apply the journal only once.
 2021-10-15 07:16:41 +02:00
Dietmar Maurer
a9017805b7 proxmox-rrd: improve dev docs 2021-10-14 11:53:54 +02:00
Dietmar Maurer
2e3f94e12f proxmox-rrd: make rrd load callback configurable 2021-10-14 11:41:26 +02:00
Dietmar Maurer
58f70bccbb proxmox-rrd: pass time and value to update function 2021-10-14 08:12:56 +02:00
Dietmar Maurer
400f081487 proxmox-rrd: split out load_rrd (cleanup) 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
eb37d4ece2 proxmox-rrd: remove dependency to proxmox-rrd-api-types 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
1198f8d4e6 proxmox-rrd: implement new CBOR based format 
Storing much more data points now got get better graphs.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar Maurer
1d44f175c6 proxmox-rrd: use a journal to reduce amount of bytes written 
Append pending changes in a simple text based format that allows for
lockless appends as long as we stay below 4 KiB data per write.

Apply the journal every 30 minutes and on daemon startup.

Note that we do not ensure that the journal is synced, this is a
perfomance optimization we can make as the kernel defaults to
writeback in-flight data every 30s (sysctl vm/dirty_expire_centisecs)
anyway, so we lose at max half a minute of data on a crash, here one
should have in mind that we normally expose 1 minute as finest
granularity anyway, so not really much lost.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Wolfgang Bumiller
6ef1b649d9 update to first proxmox crate split 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-10-11 11:58:49 +02:00
Dietmar Maurer
b3a67f1f14 proxmox-rrd: use correct directory options in create_rrdb_dir 2021-10-07 08:50:50 +02:00
Dietmar Maurer
3def6bfc64 proxmox-rrd: use log crate instead of eprintln, avoid duplicate logs 2021-10-06 18:19:22 +02:00
Dietmar Maurer
0191759316 proxmox-rrd: improve developer docs 2021-10-06 12:19:54 +02:00
Dietmar Maurer
d1c3bc5350 split out RRD api types into proxmox-rrd-api-types crate 2021-10-06 09:49:51 +02:00
Dietmar Maurer
a97301350f proxmox-rrd: use create_path instead of std::fs::create_dir_all 
To ensure correct file ownership.
 2021-10-06 08:37:14 +02:00
Dietmar Maurer
09340f28f5 move RRD code into proxmox-rrd crate 2021-10-06 08:13:28 +02:00