3b62116ce6
implement AsyncReadChunk for LocalChunkReader
...
same as the sync ReadChunk but uses tokio::fs::read instead
of file_get_contents
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:54:23 +02:00
1c090810f5
api2/admin/datastore/snapshos: show encrypted and size info per file
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:39:56 +02:00
e181d2f6da
add encrypted info to Manifest
...
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:35:39 +02:00
16021f6ab7
use the existing async method for read_raw_chunk
...
does the same, except the manual drop, but thats handled there by
letting the value go out of scope
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:23:17 +02:00
ba694720fc
api2/admin/datastore: log stream error during file download
...
the client cannot get an error during an chunked http transfer, so at
least log it server side
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:58:55 +02:00
bde8e243cf
remove unsafe copy code
...
copy_nonoverlapping is basically a memcpy which can also be done
via copy_from_slice which is not unsafe
(copy_from_slice uses copy_nonoverlapping internally)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:56:15 +02:00
3352ee5656
parse_zpool_status_field: handle tabs without copying input
2020-06-18 19:40:01 +02:00
b29cbc414d
parse_zpool_status_vdev: consider tabs as 8 spaces
2020-06-18 18:38:56 +02:00
026dc1d11f
src/api2/node/disks/zfs.rs: add zpool_details api
2020-06-18 15:04:46 +02:00
9438aca6c9
src/tools/disks/zpool_status.rs: improve parser
2020-06-18 14:55:22 +02:00
547f0c97e4
src/tools/nom.rs: new helper parse_complete_line() for single line parsers
...
Like parse_complete(), but generates simpler error messages.
2020-06-18 12:57:55 +02:00
177a2de992
src/tools/nom.rs: move nom helpers into separate file
2020-06-18 12:41:13 +02:00
0686b1f4db
src/tools/disks/zpool_list.rs: split code into separate file
2020-06-18 10:31:07 +02:00
0727e56a06
src/tools/disks/zpool_status.rs: parse zpool status output
2020-06-18 10:23:15 +02:00
2fd3d57490
src/tools/disks/zfs.rs: rename ZFSPoolStatus into ZFSPoolInfo, fix error message
2020-06-17 09:08:26 +02:00
3f851d1321
src/api2/node/disks/directory.rs: add early check if disk is unused
2020-06-17 08:31:11 +02:00
1aef491e24
src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks
2020-06-17 08:07:54 +02:00
d0eccae37d
avoid compiler warning
2020-06-17 08:07:42 +02:00
a34154d900
src/tools/disks/zfs.rs: cleanup parse_pool_header
2020-06-17 07:47:11 +02:00
c2cc32b4dd
src/tools/disks/zfs.rs: add more parser tests
2020-06-17 07:38:19 +02:00
46405fa35d
src/tools/disks/zfs.rs: add comment
2020-06-17 07:14:26 +02:00
66af7f51bc
src/tools/disks/zfs.rs: make zfs list parser private
2020-06-17 07:00:54 +02:00
c72ccd4e33
src/tools/disks/zfs.rs: add regression tests for parse_zfs_list
2020-06-16 18:14:35 +02:00
902b2cc278
src/tools/disks/zfs.rs: simplify code
2020-06-16 17:51:17 +02:00
8ecd7c9c21
move api dump binaries back to src/bin for package building
...
they're required for docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 14:48:01 +02:00
fb5a066500
src/api2/node/disks.rs: expose directory api
2020-06-16 13:36:32 +02:00
d19c96d507
move test binaries to examples/
...
These aren't installed and are only used for manual testing,
so there's no reason to force them to be built all the time.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 13:32:24 +02:00
929a13b357
src/api2/node/disks/zfs.rs: add zpool api
2020-06-16 13:25:53 +02:00
36c65ee0b0
src/tools/disks/zfs.rs: cleanup (rename usage properties)
...
And allow to parse zpool list output without -v flag.
2020-06-16 13:25:53 +02:00
3378fd9fe5
src/tools/disks/zfs.rs: parse more infos (dedup, fragmentation, health)
2020-06-16 13:25:53 +02:00
58c51cf3d9
avoid compiler warnings
2020-06-16 13:25:53 +02:00
5509b199fb
use new run_command helper
2020-06-16 13:25:53 +02:00
bb59df9134
catalog: don't panic on invalid file mtimes
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:25:54 +02:00
2564b0834f
fix file timestamps in catalog
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:25:31 +02:00
9321bbd1f5
pxar: fix missing subdirectories in catalogs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:04:56 +02:00
4264e52220
reuse some extractor code in catalog shell
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:54:54 +02:00
6988b29bdc
use O_EXCL when creating files during extraction
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:33:27 +02:00
98c54240e6
pxar: make extractor state more reusable
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:32:13 +02:00
d30c192589
AsyncReadChunk: require Send
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 09:50:29 +02:00
ac7513e368
src/tools.rs: add setup_safe_path_env()
2020-06-15 10:38:30 +02:00
fbbcd85839
src/api2/node/disks/directory.rs: implement add-datastore feature
2020-06-15 10:01:50 +02:00
7a6b549270
dynamic index: make it hard to mess up endianess
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-15 09:19:35 +02:00
0196b9bf5b
remove unnecessary .into
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:57:58 +02:00
20b3094bcb
api2/status: add task list endpoint
...
for now mostly copy/paste from nodes/nodename/tasks
(without the parameters)
but we should replace the 'read_task_list' with a method
that gives us the tasks since some timestamp
so that we can get a longer list of tasks than for the node
(we could of course embed this then in the nodes/node/task api call and
remove this again as long as the api is not stable)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:17 +02:00
df528ee6fa
implement From<TaskListInfo> for TaskListItem
...
and use it where its convenient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:12 +02:00
57e50fb906
use new Mmap helper for dynamic index
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 13:57:56 +02:00
3d571d5509
some internal combinator-influenced api cleanup
...
The download methods used to take the destination by value
and return them again, since this was required when using
combinators before we had `async fn`.
But this is just an ugly left-over now.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:46:42 +02:00
8e6e18b77c
client: make dump_image async, use async chunk reader
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:40:18 +02:00
4d16badf6f
add an AsyncReadChunk trait
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:38:21 +02:00
a609cf210e
more cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:01:04 +02:00
1498659b4e
cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:59:34 +02:00
4482f3fe11
pxar, acl: cleanup acl helper usage
...
use NixPath for Acl::set_file to avoid memduping the c
string
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:52:18 +02:00
5d85847f91
client: only start catalog upload if we have one
...
else we start a dynamic writer and never close it, leading to a backup error
this fixes an issue with backing up vm templates
(and possibly vms without disks)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:38:25 +02:00
476b4acadc
BackupEnvironment: do not set finished flag prematurely
...
we check if all dynamic_writers are closed and if the backup contains
any valid files, we can only mark the backup finished after those
checks, else the backup task gets marked as OK, even though it
is not finished and no cleanups run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:52 +02:00
cf1bd08131
pxar: fcaps in fuse
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:48 +02:00
ec8f042459
api2/status: use new rrd::extract_cached_data
...
and drop the now unused extract_lists function
this also fixes a bug, where we did not add the datastore to the list at
all when there was no rrd data
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:16 +02:00
431cc7b185
rrd: move creation of serde value into api
...
there is now a 'extract_cached_data' which just returns
the data of the specified field, and an api function that converts
a list of fields to the correct serde value
this way we do not have to create a serde value in rrd/cache.rs
(makes for a better interface)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:14 +02:00
e693818afc
refactor time functions to tools
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:10 +02:00
3d68536fc2
pxar: support .pxareclude files, error report updates
...
Report vanished files (instead of erroring out on them),
also only warn about files inaccessible due to permissions
instead of bailing out.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 12:22:18 +02:00
26e78a2efb
downgrade some FIXMEs to TODOs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:09:23 +02:00
5444fa940b
turn pxar::flags into bitflags, pxar::Flags
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:05:53 +02:00
d4f2397d4c
add api to format disks and create datastores
2020-06-10 11:03:36 +02:00
fab2413741
catalog: remove unused SenderWriter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 10:42:42 +02:00
669c137fec
src/tools/systemd.rs: implement daemon_reload, start_unit, stop_unit and enable_unit
2020-06-10 08:56:04 +02:00
fc6047fcb1
pxar: don't skip list+found by default
...
This used to be default-off and was accidentally set to
on-by-default with the pxar crate update.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 08:53:10 +02:00
3014088684
pxar: sort .pxareclude-cli file and fix its mode
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 08:49:14 +02:00
144006fade
src/tools.rs: add new run_command helper
2020-06-10 07:16:47 +02:00
b9cf6ee797
src/tools/systemd/types.rs: add Mount config
2020-06-09 18:47:10 +02:00
cdde66d277
statistics: covariance(): avoid allocation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 13:57:27 +02:00
239e49f927
pxar: create .pxarexclude-cli file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 13:17:59 +02:00
bda48e04da
api2: add status/datastore-usages api call
...
returns a list of the datastores and their usages, a list of usages of
the past month (for the gui) and an estimation of when its full
(using the linear regression)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:20:25 +02:00
ba97479848
add statistics module
...
provides some basic statistics functions (sum, mean, etc.)
and a function to return the parameters of the linear regression of
two variables
implemented using num_traits to be more flexible for the types
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:19:51 +02:00
6cad8ce4ce
rrd: add 'extract_lists'
...
this is an interface to simply get the Vec<Option<f64>> out of rrd
without going through serde values
we return a list of timestamps and a HashMap with the lists we could find
(otherwise it is not in the map)
if no lists could be extracted, the time list is also empty
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:19:06 +02:00
33070956af
let disk_usage return StorageStatus and use it for datastores/nodes
...
disk_usage returned the same values as defined in StorageStatus,
so simply use that
with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:05:39 +02:00
da84cc52f4
src/tools/systemd.rs: implement escape_unit and unescape_unit
2020-06-09 11:52:06 +02:00
2179359f40
move src/pxar.rs -> src/pxar/mod.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 10:49:59 +02:00
9bb161c881
src/tools/disks.rs: add create_single_linux_partition and create_file_system
2020-06-08 17:43:01 +02:00
297e600730
cleanup comment
2020-06-08 17:43:01 +02:00
ed7b3a7de2
src/tools/disks.rs: add get_fs_uuid helper
2020-06-08 17:43:01 +02:00
0f358204bd
src/tools/disks.rs: add helper to list partitions
2020-06-08 17:43:01 +02:00
ca6124d5fa
src/tools/disks.rs: make helpers pub
...
So that I can use them with my test code.
2020-06-08 17:43:01 +02:00
7eacdc765b
pxar: split assert_relative_path
...
the check for a single component is only required in the dir
stack atm
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 15:02:52 +02:00
c443f58b09
switch to external pxar and fuse crates
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
1e3d9b103d
xattr: make xattr_name_fcaps public
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
386990ba09
tools: add file_get_non_comment_lines
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
bc853b028f
src/tools/disks.rs: cleanup, remove unused DiskUse bitflag type
2020-06-08 09:43:07 +02:00
d406de299b
src/tools/disks.rs: use dev_t to index zfs/lvm device sets
2020-06-08 09:01:34 +02:00
dfb31de8f0
proxmox_backup_manager disk list: display gpt column
2020-06-08 07:35:44 +02:00
7c3aa258f8
src/tools/disks/zfs.rs: allow empty zpool list output
2020-06-08 07:23:04 +02:00
044055062c
src/tools/disks.rs: new helper to reread partition table
2020-06-08 07:22:06 +02:00
2b388026f8
src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node
2020-06-08 07:20:59 +02:00
707974fdb3
src/api2/node/disks.rs: implement initgpt API
2020-06-07 10:30:34 +02:00
9069debcd8
src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA
2020-06-07 07:20:25 +02:00
fa2bdc1309
src/config/acl.rs: add /system/disks to valid acl paths
2020-06-06 15:48:15 +02:00
8e40aa63c1
src/bin/proxmox-backup-manager.rs: add disk subcommand
2020-06-06 15:40:28 +02:00
d2522b2db6
src/tools/disks.rs: fix disk size, add completion helper
2020-06-06 15:39:25 +02:00
ce8e3de401
move disks api to /node/<node>/disks
2020-06-06 14:43:36 +02:00
7fa2779559
src/api2/disks.rs: implement smart api
2020-06-06 12:23:11 +02:00
042afd6e52
src/tools/disks.rs: new helper disk_by_name()
2020-06-06 12:22:38 +02:00
ff30caeaf8
src/api2/disks.rs - list-disks: add usage-type filter
2020-06-06 11:48:58 +02:00
Dominik Csapak
Dietmar Maurer
Wolfgang Bumiller