Commit Graph

500 Commits

Author SHA1 Message Date
Stefan Reiter
a4e86972a4 add .../apt/update API call
Depends on patched apt-pkg-native-rs. Changelog-URL detection is
inspired by PVE perl code for now, though marked with fixme to use 'apt
changelog' later on, if/when our repos have APT-compatible changelogs
set up.

list_installed_apt_packages iterates all packages and creates an
APTUpdateInfo with detailed information for every package matched by the
given filter Fn.

Sadly, libapt-pkg has some questionable design choices regarding their
use of 'iterators', which means quite a bit of nesting...

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-23 10:41:14 +02:00
Fabian Grünbichler
91f5594c08 api: translate ENOTFOUND to 404 for downloads
and percolate the HttpError back up on the client side

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-07-22 09:19:29 +02:00
Stefan Reiter
113c9b5981 move subscription API path to /nodes
This aligns it with PVE and allows the widget toolkit's update window
"refresh" to work without modifications once POST /apt/update is
implemented.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-21 19:33:52 +02:00
Dominik Csapak
a26c27c8e6 api2/status: fix estimation bug
when a datastore has enough data to calculate the estimated full date,
but always has exactly the same usage, the factor b of the regression
is '0'

return 0 for that case so that the gui can show 'never' instead of
'not enough data'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-21 13:02:08 +02:00
Dominik Csapak
bfcef26a99 api2/node/status: add fingerprint
and rename get_usage to get_status (since its not usage only anymore)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-10 11:08:53 +02:00
Wolfgang Bumiller
660a34892d update proxmox crate to 0.2.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 11:08:27 +02:00
Dominik Csapak
7957fabff2 api: add ZPOOL_NAME_SCHEMA and regex
poolnames can containe spaces and some other special characters

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
Wolfgang Bumiller
521a0acb2e DataStore::load_manifest: also return CryptMode
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:19:53 +02:00
Wolfgang Bumiller
f28d9088ed introduce a CryptMode enum
This also replaces the recently introduced --encryption
parameter on the client with a --crypt-mode parameter.

This can be "none", "encrypt" or "sign-only".

Note that this introduces various changes in the API types
which previously did not take the above distinction into
account properly:

Both `BackupContent` and the manifest's `FileInfo`:
    lose `encryption: Option<bool>`
    gain `crypt_mode: Option<CryptMode>`

Within the backup manifest itself, the "crypt-mode" property
will always be set.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-07 15:24:19 +02:00
Thomas Lamprecht
08cb2038bd api: disks: indentation fixup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 17:59:30 +02:00
Dietmar Maurer
2162e2c15d src/api2/admin/datastore.rs: avoid slash in UPID strings 2020-06-30 13:11:22 +02:00
Dietmar Maurer
fe3e65c3ea src/api2/backup.rs: call register_chunk in previous download api 2020-06-26 08:22:46 +02:00
Dietmar Maurer
b957aa81bd update backup api for incremental backup
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-26 07:17:08 +02:00
Dietmar Maurer
8ea00f6e49 allow to abort verify jobs
And improve job description rendering on gui.
2020-06-25 12:56:36 +02:00
Stefan Reiter
e22f4882e7 extract create_download_response API helper
and put it into a new "api2::helpers" module.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 11:57:37 +02:00
Dietmar Maurer
c2009e5309 src/api2/admin/datastore.rs: add verify api 2020-06-24 13:35:21 +02:00
Stefan Reiter
facd9801cf add incremental backup support
To support incremental backups (where not all chunks are sent to the
server), a new parameter "reuse-csum" is introduced on the
"create_fixed_index" API call. When set and equal to last backups'
checksum, the backup writer clones the data from the last index of this
archive file, and only updates chunks it actually receives.

In incremental mode some checks usually done on closing an index cannot
be made, since they would be inaccurate.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 10:01:25 +02:00
Dietmar Maurer
21302088de remove debug println 2020-06-24 09:15:13 +02:00
Stefan Reiter
8268c9d161 fix overflow panic during upload
if *only* data chunks are registered (high chance during incremental
backup), then chunk_count might be one lower then upload_stat.count
because of the zero chunk being unconditionally uploaded but not used.
Thus when subtracting the two, an overflow would occur.

In general, don't let the client make the server panic, instead just set
duplicates to 0.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 09:07:22 +02:00
Dominik Csapak
b91b7d9ffd api2/node/disks/zfs: check if default zfs mount path exists
and if it does bail, because otherwise we would get an
error on mounting and have a zpool that is not imported
and disks that are used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:22:39 +02:00
Dominik Csapak
d33d8f4e6a api2/admin/datastore: add pxar-file-download api call
streams a file from a pxar file of an unencrypted backup

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:21:15 +02:00
Dominik Csapak
5b1cfa01f1 api2/admin/datastore: add 'catalog' api call
returns the dir listing of the given filepath of the backup snapshot
the filepath has to be base64 encoded or 'root'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:16:12 +02:00
Dietmar Maurer
60f9a6ea8f src/backup/datastore.rs: add new helpers to load blobs and verify chunks 2020-06-24 06:58:14 +02:00
Dominik Csapak
cbef49bf4f remove absolute paths when executing binaries
we set the paths manually, so this is ok

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 07:09:06 +02:00
Dominik Csapak
f386f512d0 add AsyncReaderStream
and replace AsyncIndexReader's stream implementation with that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 06:33:31 +02:00
Dietmar Maurer
42d19fdf69 src/api2/node/disks/zfs.rs: always set pool name 2020-06-20 07:15:32 +02:00
Wolfgang Bumiller
4e37d9ce67 add general indented_list_to_tree implementation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 14:37:40 +02:00
Dominik Csapak
6ef9bb59eb api2/admin/datastore: add download-decoded endpoint
similar to 'download', but streams the decoded file
when it is not encrypted

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:39:15 +02:00
Dominik Csapak
abdb976340 add Display trait to BackupDir
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:28:35 +02:00
Dominik Csapak
1c090810f5 api2/admin/datastore/snapshos: show encrypted and size info per file
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:39:56 +02:00
Dominik Csapak
e181d2f6da add encrypted info to Manifest
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:35:39 +02:00
Dominik Csapak
ba694720fc api2/admin/datastore: log stream error during file download
the client cannot get an error during an chunked http transfer, so at
least log it server side

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:58:55 +02:00
Dietmar Maurer
026dc1d11f src/api2/node/disks/zfs.rs: add zpool_details api 2020-06-18 15:04:46 +02:00
Dietmar Maurer
3f851d1321 src/api2/node/disks/directory.rs: add early check if disk is unused 2020-06-17 08:31:11 +02:00
Dietmar Maurer
1aef491e24 src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks 2020-06-17 08:07:54 +02:00
Dietmar Maurer
66af7f51bc src/tools/disks/zfs.rs: make zfs list parser private 2020-06-17 07:00:54 +02:00
Dietmar Maurer
fb5a066500 src/api2/node/disks.rs: expose directory api 2020-06-16 13:36:32 +02:00
Dietmar Maurer
929a13b357 src/api2/node/disks/zfs.rs: add zpool api 2020-06-16 13:25:53 +02:00
Dietmar Maurer
fbbcd85839 src/api2/node/disks/directory.rs: implement add-datastore feature 2020-06-15 10:01:50 +02:00
Wolfgang Bumiller
0196b9bf5b remove unnecessary .into
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:57:58 +02:00
Dominik Csapak
20b3094bcb api2/status: add task list endpoint
for now mostly copy/paste from nodes/nodename/tasks
(without the parameters)
but we should replace the 'read_task_list' with a method
that gives us the tasks since some timestamp

so that we can get a longer list of tasks than for the node
(we could of course embed this then in the nodes/node/task api call and
remove this again as long as the api is not stable)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:17 +02:00
Dominik Csapak
df528ee6fa implement From<TaskListInfo> for TaskListItem
and use it where its convenient

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:12 +02:00
Dominik Csapak
476b4acadc BackupEnvironment: do not set finished flag prematurely
we check if all dynamic_writers are closed and if the backup contains
any valid files, we can only mark the backup finished after those
checks, else the backup task gets marked as OK, even though it
is not finished and no cleanups run

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:52 +02:00
Dominik Csapak
ec8f042459 api2/status: use new rrd::extract_cached_data
and drop the now unused extract_lists function

this also fixes a bug, where we did not add the datastore to the list at
all when there was no rrd data

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:16 +02:00
Dominik Csapak
431cc7b185 rrd: move creation of serde value into api
there is now a 'extract_cached_data' which just returns
the data of the specified field, and an api function that converts
a list of fields to the correct serde value

this way we do not have to create a serde value in rrd/cache.rs
(makes for a better interface)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:14 +02:00
Dietmar Maurer
d4f2397d4c add api to format disks and create datastores 2020-06-10 11:03:36 +02:00
Dominik Csapak
bda48e04da api2: add status/datastore-usages api call
returns a list of the datastores and their usages, a list of usages of
the past month (for the gui) and an estimation of when its full
(using the linear regression)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:20:25 +02:00
Dominik Csapak
33070956af let disk_usage return StorageStatus and use it for datastores/nodes
disk_usage returned the same values as defined in StorageStatus,
so simply use that

with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:05:39 +02:00
Wolfgang Bumiller
c443f58b09 switch to external pxar and fuse crates
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
Dietmar Maurer
2b388026f8 src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node 2020-06-08 07:20:59 +02:00