Dietmar Maurer
2107a5aebc
src/backup/manifest.rs: include signature inside the manifest
...
This is more flexible, because we can choose what fileds we want to sign.
2020-07-08 16:23:26 +02:00
Dietmar Maurer
3638341aa4
src/backup/file_formats.rs: remove signed chunks
...
We can include signature in the manifest instead (patch will follow).
2020-07-08 16:23:26 +02:00
Wolfgang Bumiller
0351f23ba4
client: introduce --keyfd parameter
...
This is a more convenient way to pass along the key when
creating encrypted backups of unprivileged containers in PVE
where the unprivileged user namespace cannot access
`/etc/pve/priv`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 13:56:38 +02:00
Dietmar Maurer
c1ff544eff
src/backup/crypt_config.rs - compute_digest: make it more secure
2020-07-08 12:53:04 +02:00
Wolfgang Bumiller
05389a0109
more xdg cleanup and encryption parameter improvements
...
Have a single common function to get the BaseDirectories
instance and a wrapper for `find()` and `place()` which
wrap the error with some context.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 10:57:28 +02:00
Wolfgang Bumiller
b65390ebc9
client: xdg usage: place() vs find()
...
place() is used when creating a file, as it will create
intermediate directories, only use it when actually placing
a new file.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 10:57:28 +02:00
Dietmar Maurer
3bad3e6e52
src/client/backup_writer.rs - upload_stream: add crypt_mode
2020-07-08 10:43:28 +02:00
Wolfgang Bumiller
24be37e3f6
client: fix schema to include --crypt-mode parameter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 10:09:15 +02:00
Wolfgang Bumiller
1008a69a13
pxar: less confusing logic
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:58:29 +02:00
Wolfgang Bumiller
521a0acb2e
DataStore::load_manifest: also return CryptMode
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:19:53 +02:00
Wolfgang Bumiller
3b66040de6
add DataBlob::crypt_mode
...
and move use statements up
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:19:53 +02:00
Wolfgang Bumiller
af3a0ae7b1
remove CryptMode::sign_only special method
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:19:53 +02:00
Dietmar Maurer
4e36f78438
src/backup/manifest.rs: support old encrypted property
...
Just to avoid confusion.
2020-07-08 08:52:27 +02:00
Wolfgang Bumiller
f28d9088ed
introduce a CryptMode enum
...
This also replaces the recently introduced --encryption
parameter on the client with a --crypt-mode parameter.
This can be "none", "encrypt" or "sign-only".
Note that this introduces various changes in the API types
which previously did not take the above distinction into
account properly:
Both `BackupContent` and the manifest's `FileInfo`:
lose `encryption: Option<bool>`
gain `crypt_mode: Option<CryptMode>`
Within the backup manifest itself, the "crypt-mode" property
will always be set.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-07 15:24:19 +02:00
Dietmar Maurer
f91d58e157
src/tools/runtime.rs: implement get_runtime_with_builder
2020-07-07 10:11:04 +02:00
Dietmar Maurer
7443a6e092
src/client/remote_chunk_reader.rs: implement clone for RemoteChunkReader
2020-07-07 07:34:58 +02:00
Wolfgang Bumiller
96ee857752
client: add --encryption boolen parameter
...
This can be used to explicitly disable encryption even if a
default key file exists in ~/.config.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-06 14:36:04 +02:00
Wolfgang Bumiller
887018bb79
client: use default encryption key if it is available
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-06 14:36:04 +02:00
Wolfgang Bumiller
9696f5193b
client: move key management into separate module
...
and use api macro for methods and Kdf type
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-06 14:36:04 +02:00
Wolfgang Bumiller
e13c4f66bb
minor style & whitespace fixups
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-06 10:55:25 +02:00
Thomas Lamprecht
08cb2038bd
api: disks: indentation fixup
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 17:59:30 +02:00
Dietmar Maurer
9de69cdb1a
src/bin/proxmox_backup_client/catalog.rs: split out catalog code
2020-07-03 16:45:47 +02:00
Dietmar Maurer
e9764238df
make ReadChunk not require mutable self.
...
That way we can reduce lock contentions because we lock for much shorter
times.
2020-07-03 07:37:29 +02:00
Dietmar Maurer
cc7995ac40
src/bin/proxmox_backup_client/task.rs: split out task command
2020-07-02 18:04:29 +02:00
Dietmar Maurer
43abba4b4f
src/bin/proxmox_backup_client/mount.rs: split out mount code
2020-07-02 17:49:59 +02:00
Dietmar Maurer
caea8d611f
proxmox-backup-client: add benchmark command
...
This is just a start, We need to add more useful things here...
2020-07-02 14:01:57 +02:00
Wolfgang Bumiller
7d0754a6d2
pxar: fixup 'vanished-file' logic a bit
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-30 14:41:42 +02:00
Wolfgang Bumiller
5afa0755ea
pxar: fix missing newlines in warnings
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-30 14:37:20 +02:00
Dietmar Maurer
2162e2c15d
src/api2/admin/datastore.rs: avoid slash in UPID strings
2020-06-30 13:11:22 +02:00
Dietmar Maurer
4059285649
fix typo
2020-06-29 12:59:25 +02:00
Dietmar Maurer
2e079b8bf2
partially revert commit 1f82f9b7b5
...
do it backwards compatible. Also, code was wrong because FixedIndexWriter
still computed old style csums...
2020-06-29 12:44:45 +02:00
Wolfgang Bumiller
a8e2940ff3
pxar: deal with files changing size during archiving
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-26 11:49:51 +02:00
Dominik Csapak
4eb4e94918
fix test output
...
field separator for pools is always a tab when using -H
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 10:31:11 +02:00
Dietmar Maurer
817bcda848
src/backup/verify.rs: do not stop on server shutdown
...
This is a read-only task, so there is no need to stop.
2020-06-26 09:45:59 +02:00
Dominik Csapak
f6de2c7359
WorkerTask: add warnings and count them
...
so that we have one level more between errors and OK
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 09:42:11 +02:00
Dominik Csapak
402c8861d8
fix typo
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 09:12:29 +02:00
Dominik Csapak
3f683799a8
improve 'debug' parameter
...
instead of checking on '1' or 'true', check that it is there and not
'0' and 'false'. this allows using simply
https://foo:8007/?debug
instead of
https://foo:8007/?debug=1
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-26 09:12:14 +02:00
Dietmar Maurer
1f82f9b7b5
src/backup/index.rs: add compute_csum
...
And use it for fixed and dynamic index. Please note that this
changes checksums for fixed indexes, so restore older backups
will fails now (not backward compatible).
2020-06-26 09:00:34 +02:00
Dietmar Maurer
fe3e65c3ea
src/api2/backup.rs: call register_chunk in previous download api
2020-06-26 08:22:46 +02:00
Dietmar Maurer
fdaab0df4e
src/backup/index.rs: add chunk_info method
2020-06-26 08:14:45 +02:00
Dietmar Maurer
b957aa81bd
update backup api for incremental backup
...
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-26 07:17:08 +02:00
Dietmar Maurer
8ea00f6e49
allow to abort verify jobs
...
And improve job description rendering on gui.
2020-06-25 12:56:36 +02:00
Stefan Reiter
e22f4882e7
extract create_download_response API helper
...
and put it into a new "api2::helpers" module.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 11:57:37 +02:00
Dietmar Maurer
355c055e81
src/bin/proxmox-backup-manager.rs: implement verify
2020-06-24 13:35:21 +02:00
Dietmar Maurer
c2009e5309
src/api2/admin/datastore.rs: add verify api
2020-06-24 13:35:21 +02:00
Dietmar Maurer
23f74c190e
src/backup/backup_info.rs: impl Display for BackupGroup
2020-06-24 13:35:21 +02:00
Wolfgang Bumiller
a6f8728339
update to pxar 0.1.9, update ReadAt implementations
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-24 11:57:12 +02:00
Stefan Reiter
facd9801cf
add incremental backup support
...
To support incremental backups (where not all chunks are sent to the
server), a new parameter "reuse-csum" is introduced on the
"create_fixed_index" API call. When set and equal to last backups'
checksum, the backup writer clones the data from the last index of this
archive file, and only updates chunks it actually receives.
In incremental mode some checks usually done on closing an index cannot
be made, since they would be inaccurate.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 10:01:25 +02:00
Dietmar Maurer
21302088de
remove debug println
2020-06-24 09:15:13 +02:00
Stefan Reiter
8268c9d161
fix overflow panic during upload
...
if *only* data chunks are registered (high chance during incremental
backup), then chunk_count might be one lower then upload_stat.count
because of the zero chunk being unconditionally uploaded but not used.
Thus when subtracting the two, an overflow would occur.
In general, don't let the client make the server panic, instead just set
duplicates to 0.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 09:07:22 +02:00
Dominik Csapak
b91b7d9ffd
api2/node/disks/zfs: check if default zfs mount path exists
...
and if it does bail, because otherwise we would get an
error on mounting and have a zpool that is not imported
and disks that are used
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:22:39 +02:00
Dominik Csapak
d33d8f4e6a
api2/admin/datastore: add pxar-file-download api call
...
streams a file from a pxar file of an unencrypted backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:21:15 +02:00
Dominik Csapak
5b1cfa01f1
api2/admin/datastore: add 'catalog' api call
...
returns the dir listing of the given filepath of the backup snapshot
the filepath has to be base64 encoded or 'root'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:16:12 +02:00
Dominik Csapak
05d18b907a
add From<&DirEntryAttribute to CatalogEntryType and make it pub(crate)
...
we want to get a string representation of the DirEntryAttribute
like 'f' for file, etc. and since we have such a mapping already
in the CatalogEntryType, use that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:08:50 +02:00
Dominik Csapak
e44fe0c9f5
derive Clone for the LocalChunkReader
...
this will be necessary for accessing local pxar behind didx files
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:07:28 +02:00
Dominik Csapak
4cf0ced950
add LocalDynamicReadAt
...
mostly copied from BufferedDynamicReadAt from proxmox-backup-client
but the reader is wrapped in an Arc in addition to the Mutex
we will use this for local access to a pxar behind a didx file
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:05:31 +02:00
Dietmar Maurer
60f9a6ea8f
src/backup/datastore.rs: add new helpers to load blobs and verify chunks
2020-06-24 06:58:14 +02:00
Dietmar Maurer
1090fd4424
src/backup/data_blob.rs: cleanup - improve code reuse
2020-06-24 06:56:48 +02:00
Dietmar Maurer
92c3fd2e22
src/backup/chunk_store.rs: allow to read name()
...
This is helpful for logging ...
2020-06-24 06:54:21 +02:00
Dietmar Maurer
d6d3b353be
cleanup: implement FromStr for BackupGroup
2020-06-23 08:16:56 +02:00
Dietmar Maurer
a67f7d0a07
cleanup: implement FromStr for BackupDir
2020-06-23 08:09:52 +02:00
Dietmar Maurer
c8137518fe
src/bin/proxmox_backup_manager/disk.rs: add renderer for wearout
...
So that we display the same value as the GUI.
2020-06-23 07:44:09 +02:00
Dominik Csapak
cbef49bf4f
remove absolute paths when executing binaries
...
we set the paths manually, so this is ok
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 07:09:06 +02:00
Dominik Csapak
0b99e5aebc
remove debug prints
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 06:33:58 +02:00
Dominik Csapak
f386f512d0
add AsyncReaderStream
...
and replace AsyncIndexReader's stream implementation with that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 06:33:31 +02:00
Dietmar Maurer
3ddb14889a
src/tools/daemon.rs: reopen STDOUT/STDERR journald streams to get correct PID in logs
2020-06-22 13:06:53 +02:00
Wolfgang Bumiller
c08fac4d69
tools::daemon: sync with child after MainPid message
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-22 10:58:04 +02:00
Wolfgang Bumiller
c40440092d
tools: add socketpair helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-22 10:49:56 +02:00
Wolfgang Bumiller
dc2ef2b54f
tools::daemon: fetch exe name in the beginning
...
We get the path to our executable via a readlink() on
"/proc/self/exe", which appends a " (deleted)" during
package reloads.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-22 10:31:54 +02:00
Dietmar Maurer
f28cfb322a
avoid compiler warnings
2020-06-20 07:24:02 +02:00
Dietmar Maurer
3bbe291c51
zpool_status.rs - indented_list_to_tree: do not set name property
...
This is no necessary. We only touch/set 'children' and 'leaf' properties.
2020-06-20 07:19:25 +02:00
Dietmar Maurer
42d19fdf69
src/api2/node/disks/zfs.rs: always set pool name
2020-06-20 07:15:32 +02:00
Dietmar Maurer
215968e033
src/tools/disks/zpool_status.rs: add 'leaf' attribute to root node, rename 'prev' into 'parent'
2020-06-20 06:49:06 +02:00
Dietmar Maurer
eddd1a1b9c
src/tools/disks/zpool_status.rs: move use clause top of file
2020-06-20 06:17:22 +02:00
Dietmar Maurer
d2ce211899
fixup for previous commit
2020-06-20 06:15:26 +02:00
Dietmar Maurer
1cb46c6f65
src/tools/disks/zpool_status.rs - cleanup: use struct StackItem instead of tuple
2020-06-19 18:58:57 +02:00
Dietmar Maurer
5d88c3a1c8
src/tools/disks/zpool_status.rs: remove unnecessary checks
...
Thos things can never happen, so simply use unwrap().
2020-06-19 18:27:39 +02:00
Dietmar Maurer
07fb504943
src/tools/disks/zpool_status.rs: simplify code by using serde_json::to_value
2020-06-19 17:51:13 +02:00
Dietmar Maurer
f675c5e978
src/tools/disks/zpool_status.rs - add all attributes to the tree
2020-06-19 16:55:28 +02:00
Wolfgang Bumiller
4e37d9ce67
add general indented_list_to_tree implementation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 14:37:40 +02:00
Wolfgang Bumiller
e303077132
lru_cache: restrict and annotate Send impl
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 09:37:34 +02:00
Dominik Csapak
6ef9bb59eb
api2/admin/datastore: add download-decoded endpoint
...
similar to 'download', but streams the decoded file
when it is not encrypted
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:39:15 +02:00
Dominik Csapak
eeaa2c212b
impl Sync for DataBlobReader
...
this is safe for the reason explained in the comment
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:37:47 +02:00
Dominik Csapak
4a3adc3de8
add AsyncIndexReader
...
implements AsyncRead as well as Stream for an IndexFile and a store
that implements AsyncReadChunk
we can use this to asyncread or stream the content of a FixedIndex or
DynamicIndex
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:32:33 +02:00
Dominik Csapak
abdb976340
add Display trait to BackupDir
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:28:35 +02:00
Dominik Csapak
3b62116ce6
implement AsyncReadChunk for LocalChunkReader
...
same as the sync ReadChunk but uses tokio::fs::read instead
of file_get_contents
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:54:23 +02:00
Dominik Csapak
1c090810f5
api2/admin/datastore/snapshos: show encrypted and size info per file
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:39:56 +02:00
Dominik Csapak
e181d2f6da
add encrypted info to Manifest
...
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:35:39 +02:00
Dominik Csapak
16021f6ab7
use the existing async method for read_raw_chunk
...
does the same, except the manual drop, but thats handled there by
letting the value go out of scope
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:23:17 +02:00
Dominik Csapak
ba694720fc
api2/admin/datastore: log stream error during file download
...
the client cannot get an error during an chunked http transfer, so at
least log it server side
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:58:55 +02:00
Dominik Csapak
bde8e243cf
remove unsafe copy code
...
copy_nonoverlapping is basically a memcpy which can also be done
via copy_from_slice which is not unsafe
(copy_from_slice uses copy_nonoverlapping internally)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:56:15 +02:00
Dietmar Maurer
3352ee5656
parse_zpool_status_field: handle tabs without copying input
2020-06-18 19:40:01 +02:00
Dietmar Maurer
b29cbc414d
parse_zpool_status_vdev: consider tabs as 8 spaces
2020-06-18 18:38:56 +02:00
Dietmar Maurer
026dc1d11f
src/api2/node/disks/zfs.rs: add zpool_details api
2020-06-18 15:04:46 +02:00
Dietmar Maurer
9438aca6c9
src/tools/disks/zpool_status.rs: improve parser
2020-06-18 14:55:22 +02:00
Dietmar Maurer
547f0c97e4
src/tools/nom.rs: new helper parse_complete_line() for single line parsers
...
Like parse_complete(), but generates simpler error messages.
2020-06-18 12:57:55 +02:00
Dietmar Maurer
177a2de992
src/tools/nom.rs: move nom helpers into separate file
2020-06-18 12:41:13 +02:00
Dietmar Maurer
0686b1f4db
src/tools/disks/zpool_list.rs: split code into separate file
2020-06-18 10:31:07 +02:00
Dietmar Maurer
0727e56a06
src/tools/disks/zpool_status.rs: parse zpool status output
2020-06-18 10:23:15 +02:00
Dietmar Maurer
2fd3d57490
src/tools/disks/zfs.rs: rename ZFSPoolStatus into ZFSPoolInfo, fix error message
2020-06-17 09:08:26 +02:00
Dietmar Maurer
3f851d1321
src/api2/node/disks/directory.rs: add early check if disk is unused
2020-06-17 08:31:11 +02:00
Dietmar Maurer
1aef491e24
src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks
2020-06-17 08:07:54 +02:00
Dietmar Maurer
d0eccae37d
avoid compiler warning
2020-06-17 08:07:42 +02:00
Dietmar Maurer
a34154d900
src/tools/disks/zfs.rs: cleanup parse_pool_header
2020-06-17 07:47:11 +02:00
Dietmar Maurer
c2cc32b4dd
src/tools/disks/zfs.rs: add more parser tests
2020-06-17 07:38:19 +02:00
Dietmar Maurer
46405fa35d
src/tools/disks/zfs.rs: add comment
2020-06-17 07:14:26 +02:00
Dietmar Maurer
66af7f51bc
src/tools/disks/zfs.rs: make zfs list parser private
2020-06-17 07:00:54 +02:00
Dietmar Maurer
c72ccd4e33
src/tools/disks/zfs.rs: add regression tests for parse_zfs_list
2020-06-16 18:14:35 +02:00
Dietmar Maurer
902b2cc278
src/tools/disks/zfs.rs: simplify code
2020-06-16 17:51:17 +02:00
Wolfgang Bumiller
8ecd7c9c21
move api dump binaries back to src/bin for package building
...
they're required for docs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 14:48:01 +02:00
Dietmar Maurer
fb5a066500
src/api2/node/disks.rs: expose directory api
2020-06-16 13:36:32 +02:00
Wolfgang Bumiller
d19c96d507
move test binaries to examples/
...
These aren't installed and are only used for manual testing,
so there's no reason to force them to be built all the time.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 13:32:24 +02:00
Dietmar Maurer
929a13b357
src/api2/node/disks/zfs.rs: add zpool api
2020-06-16 13:25:53 +02:00
Dietmar Maurer
36c65ee0b0
src/tools/disks/zfs.rs: cleanup (rename usage properties)
...
And allow to parse zpool list output without -v flag.
2020-06-16 13:25:53 +02:00
Dietmar Maurer
3378fd9fe5
src/tools/disks/zfs.rs: parse more infos (dedup, fragmentation, health)
2020-06-16 13:25:53 +02:00
Dietmar Maurer
58c51cf3d9
avoid compiler warnings
2020-06-16 13:25:53 +02:00
Dietmar Maurer
5509b199fb
use new run_command helper
2020-06-16 13:25:53 +02:00
Wolfgang Bumiller
bb59df9134
catalog: don't panic on invalid file mtimes
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:25:54 +02:00
Wolfgang Bumiller
2564b0834f
fix file timestamps in catalog
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:25:31 +02:00
Wolfgang Bumiller
9321bbd1f5
pxar: fix missing subdirectories in catalogs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:04:56 +02:00
Wolfgang Bumiller
4264e52220
reuse some extractor code in catalog shell
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:54:54 +02:00
Wolfgang Bumiller
6988b29bdc
use O_EXCL when creating files during extraction
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:33:27 +02:00
Wolfgang Bumiller
98c54240e6
pxar: make extractor state more reusable
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:32:13 +02:00
Wolfgang Bumiller
d30c192589
AsyncReadChunk: require Send
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 09:50:29 +02:00
Dietmar Maurer
ac7513e368
src/tools.rs: add setup_safe_path_env()
2020-06-15 10:38:30 +02:00
Dietmar Maurer
fbbcd85839
src/api2/node/disks/directory.rs: implement add-datastore feature
2020-06-15 10:01:50 +02:00
Wolfgang Bumiller
7a6b549270
dynamic index: make it hard to mess up endianess
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-15 09:19:35 +02:00
Wolfgang Bumiller
0196b9bf5b
remove unnecessary .into
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:57:58 +02:00
Dominik Csapak
20b3094bcb
api2/status: add task list endpoint
...
for now mostly copy/paste from nodes/nodename/tasks
(without the parameters)
but we should replace the 'read_task_list' with a method
that gives us the tasks since some timestamp
so that we can get a longer list of tasks than for the node
(we could of course embed this then in the nodes/node/task api call and
remove this again as long as the api is not stable)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:17 +02:00
Dominik Csapak
df528ee6fa
implement From<TaskListInfo> for TaskListItem
...
and use it where its convenient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:12 +02:00
Wolfgang Bumiller
57e50fb906
use new Mmap helper for dynamic index
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 13:57:56 +02:00
Wolfgang Bumiller
3d571d5509
some internal combinator-influenced api cleanup
...
The download methods used to take the destination by value
and return them again, since this was required when using
combinators before we had `async fn`.
But this is just an ugly left-over now.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:46:42 +02:00
Wolfgang Bumiller
8e6e18b77c
client: make dump_image async, use async chunk reader
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:40:18 +02:00
Wolfgang Bumiller
4d16badf6f
add an AsyncReadChunk trait
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:38:21 +02:00
Wolfgang Bumiller
a609cf210e
more cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 11:01:04 +02:00
Wolfgang Bumiller
1498659b4e
cleanup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:59:34 +02:00
Wolfgang Bumiller
4482f3fe11
pxar, acl: cleanup acl helper usage
...
use NixPath for Acl::set_file to avoid memduping the c
string
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:52:18 +02:00
Dominik Csapak
5d85847f91
client: only start catalog upload if we have one
...
else we start a dynamic writer and never close it, leading to a backup error
this fixes an issue with backing up vm templates
(and possibly vms without disks)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:38:25 +02:00
Dominik Csapak
476b4acadc
BackupEnvironment: do not set finished flag prematurely
...
we check if all dynamic_writers are closed and if the backup contains
any valid files, we can only mark the backup finished after those
checks, else the backup task gets marked as OK, even though it
is not finished and no cleanups run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:52 +02:00
Wolfgang Bumiller
cf1bd08131
pxar: fcaps in fuse
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:48 +02:00
Dominik Csapak
ec8f042459
api2/status: use new rrd::extract_cached_data
...
and drop the now unused extract_lists function
this also fixes a bug, where we did not add the datastore to the list at
all when there was no rrd data
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:16 +02:00
Dominik Csapak
431cc7b185
rrd: move creation of serde value into api
...
there is now a 'extract_cached_data' which just returns
the data of the specified field, and an api function that converts
a list of fields to the correct serde value
this way we do not have to create a serde value in rrd/cache.rs
(makes for a better interface)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:14 +02:00
Dominik Csapak
e693818afc
refactor time functions to tools
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:10 +02:00
Wolfgang Bumiller
3d68536fc2
pxar: support .pxareclude files, error report updates
...
Report vanished files (instead of erroring out on them),
also only warn about files inaccessible due to permissions
instead of bailing out.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 12:22:18 +02:00
Wolfgang Bumiller
26e78a2efb
downgrade some FIXMEs to TODOs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:09:23 +02:00
Wolfgang Bumiller
5444fa940b
turn pxar::flags into bitflags, pxar::Flags
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:05:53 +02:00
Dietmar Maurer
d4f2397d4c
add api to format disks and create datastores
2020-06-10 11:03:36 +02:00
Wolfgang Bumiller
fab2413741
catalog: remove unused SenderWriter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 10:42:42 +02:00
Dietmar Maurer
669c137fec
src/tools/systemd.rs: implement daemon_reload, start_unit, stop_unit and enable_unit
2020-06-10 08:56:04 +02:00
Wolfgang Bumiller
fc6047fcb1
pxar: don't skip list+found by default
...
This used to be default-off and was accidentally set to
on-by-default with the pxar crate update.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 08:53:10 +02:00
Wolfgang Bumiller
3014088684
pxar: sort .pxareclude-cli file and fix its mode
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 08:49:14 +02:00
Dietmar Maurer
144006fade
src/tools.rs: add new run_command helper
2020-06-10 07:16:47 +02:00
Dietmar Maurer
b9cf6ee797
src/tools/systemd/types.rs: add Mount config
2020-06-09 18:47:10 +02:00
Wolfgang Bumiller
cdde66d277
statistics: covariance(): avoid allocation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 13:57:27 +02:00
Wolfgang Bumiller
239e49f927
pxar: create .pxarexclude-cli file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 13:17:59 +02:00
Dominik Csapak
bda48e04da
api2: add status/datastore-usages api call
...
returns a list of the datastores and their usages, a list of usages of
the past month (for the gui) and an estimation of when its full
(using the linear regression)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:20:25 +02:00
Dominik Csapak
ba97479848
add statistics module
...
provides some basic statistics functions (sum, mean, etc.)
and a function to return the parameters of the linear regression of
two variables
implemented using num_traits to be more flexible for the types
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:19:51 +02:00
Dominik Csapak
6cad8ce4ce
rrd: add 'extract_lists'
...
this is an interface to simply get the Vec<Option<f64>> out of rrd
without going through serde values
we return a list of timestamps and a HashMap with the lists we could find
(otherwise it is not in the map)
if no lists could be extracted, the time list is also empty
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:19:06 +02:00
Dominik Csapak
33070956af
let disk_usage return StorageStatus and use it for datastores/nodes
...
disk_usage returned the same values as defined in StorageStatus,
so simply use that
with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:05:39 +02:00
Dietmar Maurer
da84cc52f4
src/tools/systemd.rs: implement escape_unit and unescape_unit
2020-06-09 11:52:06 +02:00
Wolfgang Bumiller
2179359f40
move src/pxar.rs -> src/pxar/mod.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 10:49:59 +02:00
Dietmar Maurer
9bb161c881
src/tools/disks.rs: add create_single_linux_partition and create_file_system
2020-06-08 17:43:01 +02:00
Dietmar Maurer
297e600730
cleanup comment
2020-06-08 17:43:01 +02:00
Dietmar Maurer
ed7b3a7de2
src/tools/disks.rs: add get_fs_uuid helper
2020-06-08 17:43:01 +02:00
Dietmar Maurer
0f358204bd
src/tools/disks.rs: add helper to list partitions
2020-06-08 17:43:01 +02:00
Dietmar Maurer
ca6124d5fa
src/tools/disks.rs: make helpers pub
...
So that I can use them with my test code.
2020-06-08 17:43:01 +02:00
Wolfgang Bumiller
7eacdc765b
pxar: split assert_relative_path
...
the check for a single component is only required in the dir
stack atm
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 15:02:52 +02:00
Wolfgang Bumiller
c443f58b09
switch to external pxar and fuse crates
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Wolfgang Bumiller
1e3d9b103d
xattr: make xattr_name_fcaps public
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Wolfgang Bumiller
386990ba09
tools: add file_get_non_comment_lines
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Dietmar Maurer
bc853b028f
src/tools/disks.rs: cleanup, remove unused DiskUse bitflag type
2020-06-08 09:43:07 +02:00
Dietmar Maurer
d406de299b
src/tools/disks.rs: use dev_t to index zfs/lvm device sets
2020-06-08 09:01:34 +02:00
Dietmar Maurer
dfb31de8f0
proxmox_backup_manager disk list: display gpt column
2020-06-08 07:35:44 +02:00
Dietmar Maurer
7c3aa258f8
src/tools/disks/zfs.rs: allow empty zpool list output
2020-06-08 07:23:04 +02:00
Dietmar Maurer
044055062c
src/tools/disks.rs: new helper to reread partition table
2020-06-08 07:22:06 +02:00
Dietmar Maurer
2b388026f8
src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node
2020-06-08 07:20:59 +02:00
Dietmar Maurer
707974fdb3
src/api2/node/disks.rs: implement initgpt API
2020-06-07 10:30:34 +02:00
Dietmar Maurer
9069debcd8
src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA
2020-06-07 07:20:25 +02:00
Dietmar Maurer
fa2bdc1309
src/config/acl.rs: add /system/disks to valid acl paths
2020-06-06 15:48:15 +02:00
Dietmar Maurer
8e40aa63c1
src/bin/proxmox-backup-manager.rs: add disk subcommand
2020-06-06 15:40:28 +02:00
Dietmar Maurer
d2522b2db6
src/tools/disks.rs: fix disk size, add completion helper
2020-06-06 15:39:25 +02:00
Dietmar Maurer
ce8e3de401
move disks api to /node/<node>/disks
2020-06-06 14:43:36 +02:00
Dietmar Maurer
7fa2779559
src/api2/disks.rs: implement smart api
2020-06-06 12:23:11 +02:00
Dietmar Maurer
042afd6e52
src/tools/disks.rs: new helper disk_by_name()
2020-06-06 12:22:38 +02:00
Dietmar Maurer
ff30caeaf8
src/api2/disks.rs - list-disks: add usage-type filter
2020-06-06 11:48:58 +02:00
Dietmar Maurer
553cd12ba6
src/api2/disks.rs: start disks api
2020-06-06 11:38:47 +02:00
Dietmar Maurer
de1e1a9d95
src/tools/disks.rs: use api macro so that we can use those types with the api
2020-06-06 11:37:24 +02:00
Dietmar Maurer
91960d6162
src/tools/disks.rs - get_disks: query smart status
2020-06-06 09:18:20 +02:00
Dietmar Maurer
4c24a48eb3
src/tools/disks/smart.rs: use model.to_string_lossy() to simplify code
2020-06-06 09:05:22 +02:00
Dietmar Maurer
484e761dab
src/tools/disks/smart.rs: try to get correct wearout for ATA devices
2020-06-06 09:01:15 +02:00
Dietmar Maurer
059b7a252e
src/tools/disks/smart.rs - get_smart_data: use &Disk instead of &str
...
So that we can query other device infos easily (model, vendor, ..)
2020-06-06 08:24:58 +02:00
Dietmar Maurer
eb80aac288
src/tools/disks/smart.rs: parse output from smartctl
2020-06-05 18:30:06 +02:00
Dietmar Maurer
c26aad405f
src/tools/disks.rs: implement get_disks (similar to the one in PVE::Diskmanage)
...
But no ceph support for now. Also no support for old cciss block devices.
2020-06-05 10:33:53 +02:00
Dietmar Maurer
f03a0e509e
src/tools/disks.rs; use correct subdir to check holders
2020-06-05 10:33:53 +02:00
Dietmar Maurer
4c1e8855cc
src/tools/disks.rs: fix disk type detection, remove newline from vendor string
2020-06-05 08:09:52 +02:00
Thomas Lamprecht
a720894ff0
rrd: fix off-by-one in save interval calculation
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-04 10:30:47 +02:00
Dominik Csapak
a95a3fb893
fix csum calculation of not 'chunk_size' aligned images
...
the last chunk does not have to be as big as the chunk_size,
just use the already available 'chunk_end' function which does the
correct thing
this fixes restoration of images whose sizes are not a multiple of
'chunk_size' as well
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 10:18:30 +02:00
Dietmar Maurer
620911b426
src/tools/disks/lvm.rs: implement get_lvm_devices()
2020-06-04 09:12:19 +02:00
Dietmar Maurer
5c264c8d80
src/tools/disks.rs: add/use get_partition_type_info
2020-06-04 07:48:22 +02:00
Dominik Csapak
8d78589969
improve display of 'next run' for sync jobs
...
if the last sync job is too far in the past (or there was none at all
for now) we run it at the next iteration, so we want to show that
we now calculate the next_run by using either the real last endtime
as time or 0
then in the frontend, we check if the next_run is < now and show 'pending'
(we do it this way also for replication on pve)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:03:54 +02:00
Dominik Csapak
eed8a5ad79
tools/systemd/time: fix compute_next_event for weekdays
...
two things were wrong here:
* the range (x..y) does not include y, so the range
(day_num+1..6) goes from (day_num+1) to 5 (but sunday is 6)
* WeekDays.bits() does not return the 'day_num' of that day, but
the bit value (e.g. 64 for SUNDAY) but was treated as the index of
the day of the week
to fix this, we drop the map to WeekDays and use the 'indices'
directly
this patch makes the test work again
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:02:33 +02:00
Dominik Csapak
538b9c1c27
systemd/time: add tests for all weekdays
...
this fails for now
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:02:23 +02:00
Thomas Lamprecht
55919bf141
verify_file: add missing closing parenthesis in error message
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 19:10:01 +02:00
Dietmar Maurer
456ad0c478
src/tools/disks/zfs.rs: add parser for zpool list output
2020-06-03 12:16:08 +02:00
Dietmar Maurer
c48aa39f3b
src/bin/proxmox-backup-client.rs: implement quite flag
2020-06-03 10:11:37 +02:00
Thomas Lamprecht
2d32fe2c04
client restore: don't add server file ending if already specified
...
If one executes a client command like
# proxmox-backup-client files <snapshot> --repository ...
the files shown have already the '.fidx' or '.blob' file ending, so
if a user would just copy paste that one the client would always add
.blob, and the server would not find that file.
So avoid adding file endings if it is already a known OK one.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 07:03:55 +02:00
Thomas Lamprecht
dc155e9bd7
client restore: factor out archive/type parsing
...
will be extended in a next patch.
Also drop a dead else branch, can never get hit as we always add
.blob as fallback
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-06-03 07:03:12 +02:00
Dietmar Maurer
4e14781aec
fix typo
2020-06-03 06:59:43 +02:00
Wolfgang Bumiller
a595f0fee0
client: improve connection/new fingerprint query
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-02 10:40:31 +02:00
Thomas Lamprecht
add5861e8d
typo fixes all over the place
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 16:39:08 +02:00
Dietmar Maurer
1610c45a86
src/client/pull.rs: also download client.log.blob
2020-05-30 14:51:33 +02:00
Dietmar Maurer
b2387eaa45
avoid compiler warnings
2020-05-30 14:05:33 +02:00
Dietmar Maurer
96d65fbcd0
cleanup: define/use const for predefined blob file names.
2020-05-30 14:04:15 +02:00
Dietmar Maurer
7cc3473a4e
src/client/backup_specification.rs: split code into extra file
2020-05-30 10:54:38 +02:00
Dietmar Maurer
4856a21836
src/client/pull.rs: more verbose logging
2020-05-30 08:12:43 +02:00
Dominik Csapak
04b0ca8b59
add owner to group and snapshot listings
...
while touching it, make columns and tbar in DataStoreContent.js
declarative members and remove the (now) unnecessary initComponent
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-30 07:24:12 +02:00
Dominik Csapak
d43f86f3f3
api2: add admin/sync endpoint
...
this returns the list of syncjobs with status, as opposed to
config/sync (which is just the config)
also adds an api call where users can run the job manually under
/admin/sync/$ID/run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:31:32 +02:00
Dominik Csapak
997d7e19fc
config/sync: add SyncJobStatus Struct/Schema
...
contains the config + status
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:29:39 +02:00
Dominik Csapak
c67b1fa72f
syncjob: change worker type for sync jobs
...
'sync' is used for manually pulling a remote datastore
changing it for a scheduled sync to 'syncjob' so that we can
differentiate between both types of syncs
this also adds a seperate task description for it
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:28:04 +02:00
Dominik Csapak
268687ddf0
api2/pull: refactor priv checking and creating pull parameters
...
we want to reuse those in the api call for manually running a sync job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:27:43 +02:00
Dominik Csapak
426c1e353b
api2/config/sync: fix id parameter
...
'name' is not the correct parameter for get/post
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:54 +02:00
Dominik Csapak
2888b27f4c
create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:25 +02:00
Dietmar Maurer
934f5bb8ac
src/bin/proxmox-backup-proxy.rs: cleanup, move code to src/tools/disks.rs
...
And simplify find_mounted_device by using stat.st_dev
2020-05-29 11:13:36 +02:00
Dominik Csapak
9857472211
fix removing of remotes
...
we have to save the remote config after removing the section
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 10:48:26 +02:00
Dietmar Maurer
013fa7bbcb
rrd: reduce io by saving data only once a minute
2020-05-29 09:16:13 +02:00
Dietmar Maurer
a8d7033cb2
src/bin/proxmox-backup-proxy.rs: add test if last prune job is still running
2020-05-29 08:06:48 +02:00
Dietmar Maurer
04ad7bc436
src/bin/proxmox-backup-proxy.rs: test if last sync job is still running
2020-05-29 08:06:48 +02:00
Dietmar Maurer
77ebbefc1a
src/server/worker_task.rs: make worker_is_active_local pub
2020-05-29 08:06:48 +02:00
Dietmar Maurer
750252ba2f
src/tools/systemd/time.rs: add test for "daily" schedule
2020-05-29 07:52:09 +02:00
Dietmar Maurer
dc58194ebe
src/bin/proxmox-backup-proxy.rs: use correct id to lookup sync jobs
2020-05-29 07:50:59 +02:00
Thomas Lamprecht
090decbe76
BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names
...
The repo URL consists of
* optional userid
* optional host
* datastore name
All three have defined regex or format, but none of that is used, so
for example not all valid datastore names are accepted.
Move definition of the regex over to api2::types where we can access
all required regexes easily.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:29:23 +02:00
Thomas Lamprecht
c32186595e
api2::types: factor out USER_ID regex
...
allows for better reuse in a next patch
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:27:38 +02:00
Dietmar Maurer
c94e1f655e
rrd stats: improve io delay stats
2020-05-28 19:12:13 +02:00
Dietmar Maurer
91e5bb49f5
src/bin/proxmox-backup-proxy.rs: simplify code
...
and gather all stats for the root disk
2020-05-28 12:30:54 +02:00
Dietmar Maurer
547e3c2f6c
src/tools/disks/zfs.rs: use wtime + rtime (wait + run time)
2020-05-28 11:45:34 +02:00
Dietmar Maurer
25c550bc28
src/bin/proxmox-backup-proxy.rs: gather zpool io stats
2020-05-28 10:09:13 +02:00
Dietmar Maurer
0146133b4b
src/tools/disks/zfs.rs: helper to read zfs pool io stats
2020-05-28 10:07:52 +02:00
Dietmar Maurer
3eeba68785
depend on proxmox 0.1.38, use new fs helper functions
2020-05-28 10:06:44 +02:00
Dominik Csapak
f5056656b2
use the sync id for the scheduled sync worker task
...
this way, multiple sync jobs with the same local store, can get scheduled
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:26:03 +02:00
Dominik Csapak
8c87743642
fix 'remove_vanished' cli arg again
...
since the target side wants this to be a boolean and
serde interprets a None Value as 'null' we have to only
add this when it is really set via cli
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:25:30 +02:00
Dominik Csapak
05d755b282
fix inserting of worker tasks
...
when starting a new task, we do two things to keep track of tasks
(in that order):
* updating the 'active' file with a list of tasks with
'update_active_workers'
* updating the WORKER_TASK_LIST
the second also updates the status of running tasks in the file by
checking if it is still running by checking the WORKER_TASK_LIST
since those two things are not locked, it can happend that
we update the file, and before updating the WORKER_TASK_LIST,
another thread calls update_active_workers and tries to
get the status from the task log, which won't have any data yet
so the status is 'unknown'
(we do not update that status ever, likely for performance reasons,
so we have to fix this here)
by switching the order of the two operations, we make sure that only
tasks reach the 'active' file which are inserted in the WORKER_TASK_LIST
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-28 06:24:42 +02:00
Dietmar Maurer
143b654550
src/tools.rs - command_output: add parameter to check exit code
2020-05-27 07:25:39 +02:00
Dietmar Maurer
97fab7aa11
src/tools.rs: new helper to handle command_output (std::process::Output)
2020-05-27 06:53:25 +02:00
Dominik Csapak
40dc103103
fix cli pull api call
...
there is no 'delete' parameter, only 'remove-vanished', so fix that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:39:19 +02:00
Dominik Csapak
de4db62c57
remotes: save passwords as base64
...
to avoid having arbitrary characters in the config (e.g. newlines)
note that this breaks existings configs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:06 +02:00
Dietmar Maurer
1a0d3d11d2
src/api2/admin/datastore.rs: add rrd api
2020-05-26 12:26:14 +02:00
Dietmar Maurer
8c03041a2c
src/bin/proxmox-backup-proxy.rs: gather block device stats on datastore
2020-05-26 11:20:59 +02:00
Dietmar Maurer
3fcc4b4e5c
src/tools/disks.rs: add helper to read block device stats
2020-05-26 11:20:22 +02:00
Dietmar Maurer
3ed07ed2cd
src/tools/disks.rs: export read_sys
2020-05-26 09:49:13 +02:00
Dominik Csapak
83fd4b3b1b
remote: try to use Struct for api
...
with a catch: password is in the struct but we do not want it to return
via the api, so we only 'serialize' it when the string is not empty
(this can only happen when the format is not checked by us, iow.
when its returned from the api) and setting it manually to ""
when we return remotes from the api
this way we can still use the type but do not return the password
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:55:07 +02:00
Dominik Csapak
5dcdcea293
api2/config/remote: remove password from read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:49:12 +02:00
Dominik Csapak
99f443c6ae
api2/config/remote: lock and use digest for removal
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:45 +02:00
Dominik Csapak
4f966d0592
api2/config/remote: use rpcenv for digest for read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:28 +02:00
Dominik Csapak
db0c228719
config/remote: add 'name' to Remote struct
...
and use it as section id, like with User
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:05 +02:00
Dietmar Maurer
309ef20d6d
src/bin/proxmox-backup-proxy.rs: simplify code
2020-05-25 16:20:32 +02:00
Dietmar Maurer
d0833a70f7
src/bin/proxmox-backup-proxy.rs: gather datastore usage stats
2020-05-25 16:20:32 +02:00
Dietmar Maurer
d8dc281992
www/DataStoreStatus.js: display loadavg stats
2020-05-25 11:54:15 +02:00
Dietmar Maurer
2c66a590c0
src/bin/proxmox-backup-proxy.rs: gather iowait stats
2020-05-25 11:54:15 +02:00
Dietmar Maurer
485841da2c
src/bin/proxmox-backup-proxy.rs: gather loadavg stats
2020-05-25 11:40:20 +02:00
Dietmar Maurer
3e930f2bdc
www/DataStoreStatus.js: display root disk stats
2020-05-25 11:34:24 +02:00
Dietmar Maurer
dd15c0aa3b
src/bin/proxmox-backup-proxy.rs: gather root disk stats
2020-05-25 11:10:07 +02:00
Dietmar Maurer
c1b24fbf0b
www/DataStoreStatus.js: display swap stats
2020-05-25 10:39:54 +02:00
Dietmar Maurer
3f23b17298
src/rrd/rrd.rs: do not wrap error and return ErrorKind::NotFound
2020-05-25 10:30:04 +02:00
Dietmar Maurer
c25c9d8dd1
src/bin/proxmox-backup-proxy.rs: gather swap usage stats
2020-05-25 10:25:58 +02:00
Dietmar Maurer
84dc6adcc1
src/rrd/cache.rs: display/log error when RRD load fails
2020-05-25 10:18:53 +02:00
Dietmar Maurer
0c4344650d
src/rrd/rrd.rs: store/verify magic number
2020-05-25 09:21:54 +02:00
Dietmar Maurer
4f9513996c
src/bin/proxmox-backup-proxy.rs: use block_in_place for rrd update
2020-05-25 08:30:59 +02:00
Dietmar Maurer
736edc7a7e
src/rrd/rrd.rs: implement DST_COUNTER
2020-05-25 08:14:30 +02:00
Dietmar Maurer
2b55de407e
src/rrd/rrd.rs: correctly compute derived values
...
use f64 for time.
2020-05-25 07:02:04 +02:00
Dietmar Maurer
a608806f65
www/DataStoreStatus.js: display netin/netout
2020-05-24 19:02:35 +02:00
Dietmar Maurer
8f0cec2642
src/bin/proxmox-backup-proxy.rs: gather netin/netout stats
2020-05-24 19:02:35 +02:00
Dietmar Maurer
0ed9a2b3ae
src/config/network.rs: implement is_physical_nic() helper
2020-05-24 19:02:35 +02:00
Dietmar Maurer
58edd33d2b
src/rrd/rrd.rs: implement DST_DERIVE
2020-05-24 19:02:35 +02:00
Dietmar Maurer
4fb05fde17
src/rrd/rrd.rs: restructure whole code
2020-05-24 16:51:28 +02:00
Dietmar Maurer
daca4f7888
src/rrd/rrd.rs: reduce size by using f64:NAN as UNKNOWN
2020-05-24 09:09:09 +02:00
Dietmar Maurer
4e6585839b
src/rrd/rrd.rs: simplify an fix old value deletion
2020-05-24 06:44:06 +02:00
Dietmar Maurer
8c981ae379
rrd: fix display interval, try to avoid numeric errors
2020-05-23 16:03:43 +02:00
Dietmar Maurer
803ab12ad4
rrd: simplify code
2020-05-23 15:37:17 +02:00
Dietmar Maurer
a4a3f7ca5e
rrd: pack multiple rrd values into th estat list
2020-05-23 14:03:44 +02:00
Dietmar Maurer
a2f862eed6
add experimental rrd api to get cpu stats
2020-05-23 11:50:53 +02:00
Dietmar Maurer
eaeda365e0
start gathering stats using new rrd module
2020-05-23 10:43:08 +02:00
Dietmar Maurer
6359dc891a
add simple rrd implementation
2020-05-23 10:42:48 +02:00
Dietmar Maurer
07ad6470ca
src/client/pull.rs: split out pull related code
2020-05-22 08:04:20 +02:00
Dietmar Maurer
a6160cdfeb
src/bin/proxmox-backup-proxy.rs: schedule sync jobs
2020-05-22 07:50:59 +02:00
Dietmar Maurer
183125d576
src/api2/pull.rs: aquire try_shared_chunk_store_lock inside pull_store
2020-05-22 07:24:17 +02:00
Dietmar Maurer
a3016d6583
proxmox-backup-manager: add sync-job cli
2020-05-21 11:44:45 +02:00
Dietmar Maurer
b29d046e89
proxmox-backup-manager: split out cert.rs
2020-05-21 11:22:20 +02:00
Dietmar Maurer
380bd7df97
proxmox-backup-manager: split out datastore.rs
2020-05-21 11:14:34 +02:00
Dietmar Maurer
ea6f404e55
proxmox-backup-manager: split out dns.rs
2020-05-21 11:10:58 +02:00
Dietmar Maurer
a35a211d9e
proxmox-backup-manager: split out network.rs
2020-05-21 11:08:38 +02:00
Dietmar Maurer
53e14507c1
proxmox-backup-manager: split out acl.rs
2020-05-21 10:56:46 +02:00
Dietmar Maurer
6fa39e53e0
proxmox-backup-manager: split out users.rs
2020-05-21 10:53:06 +02:00
Dietmar Maurer
a220a4564a
roxmox-backup-manager: start splitting command into several files
2020-05-21 10:46:07 +02:00
Dietmar Maurer
6f652b1b3a
rename 'job' to 'sync'
2020-05-21 10:29:25 +02:00
Dietmar Maurer
b1d4edc769
src/api2/config/job.rs: add job api
2020-05-21 10:16:35 +02:00
Dietmar Maurer
b4900286ce
src/config/jobs.rs: use SectionConfig for jobs
2020-05-21 10:16:35 +02:00
Dietmar Maurer
c681885227
src/bin/proxmox-backup-manager.rs: format output of show commands
2020-05-20 16:47:37 +02:00
Dietmar Maurer
ee8b464466
src/tools/systemd.rs: avoid compiler warnings
2020-05-20 16:47:08 +02:00
Dominik Csapak
ce55db66d6
proxmox-backup-manager: add show command for remote and datastore
...
to show the data for a single item
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 16:33:07 +02:00
Dominik Csapak
2882c881e9
api2/access/acl: add path and exact parameter to list_acl
...
so that we can get only a subset of the acls, filtered by the backed
also return the digest here
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:44:36 +02:00