02543a5c7f
api2/pull: extend do_sync_job to also handle schedule and jobstate
...
so that we can log if triggered by a schedule, and writing to a jobstatefile
also correctly polls now the abort_future of the worker, so that
users can stop a sync
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:49:28 +02:00
42b68f72e6
api/{pull, sync}: refactor to do_sync_job
...
and move the pull parameters into the worker, so that the task log
contains the error if there is one
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:40:52 +02:00
664d8a2765
api2/admin/sync: use JobState for faster access to state info
...
and delete the statefile again on syncjob removal
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:40:00 +02:00
ae197dda23
server/worker_task: let upid_read_status also return the endtime
...
the endtime should be the timestamp of the last log line
or if there is no log at all, the starttime
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:35:44 +02:00
4c116bafb8
server: change status of a task from a string to an enum
...
representing a state via an enum makes more sense in this case
we also implement FromStr and Display to make it easy to convet from/to
a string
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:35:19 +02:00
72dc68323c
replace and remove old ticket functions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-12 14:28:21 +02:00
81f293513e
backup: lock base snapshot and ensure existance on finish
...
To prevent forgetting the base snapshot of a running backup, and catch
the case when it still happens (e.g. via manual rm) to at least error
out instead of storing a potentially invalid backup.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-11 11:04:47 +02:00
8b5f72b176
Revert "backup: ensure base snapshots are still available after backup"
...
This reverts commit d53fbe2474
2020-08-11 11:03:53 +02:00
f23f75433f
backup: flock snapshot on backup start
...
An flock on the snapshot dir itself is used in addition to the group dir
lock. The lock is used to avoid races with forget and prune, while
having more granularity than the group lock (i.e. the group lock is
necessary to prevent more than one backup per group, but the snapshot
lock still allows backups unrelated to the currently running to be
forgotten/pruned).
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-11 11:02:21 +02:00
882c082369
mark signed manifests as such
...
for less-confusing display in the web interface
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:56:53 +02:00
14f6c9cb8b
chunk readers: ensure chunk/index CryptMode matches
...
an encrypted Index should never reference a plain-text chunk, and an
unencrypted Index should never reference an encrypted chunk.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:54:22 +02:00
2d55beeca0
datastore api: verify blob/index csum from manifest
...
when dowloading decoded files.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:52:45 +02:00
9238cdf50d
datastore api: only decode unencrypted indices
...
these checks were already in place for regular downloading of backed up
files, also do them when attempting to decode a catalog, or when
downloading decoded files referenced by a pxar index.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:51:20 +02:00
5d30f03826
impl PartialEq between Realm and RealmRef
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:23:36 +02:00
14263ef989
assert that Username does not impl PartialEq
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:21:12 +02:00
e7cb4dc50d
introduce Username, Realm and Userid api types
...
and begin splitting up types.rs as it has grown quite large
already
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:05:01 +02:00
866c556faf
move types.rs to types/mod.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 10:32:31 +02:00
4dbe129284
backup: only allow finished backups as base snapshot
...
If the datastore holds broken backups for some reason, do not attempt to
base following snapshots on those. This would lead to an error on
/previous, leaving the client no choice but to upload all chunks, even
though there might be potential for incremental savings.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-07 07:32:56 +02:00
d74edc3d89
finish_backup: mark backup as finished only after checks have passed
...
Commit 9fa55e09
2020-08-06 06:39:34 +02:00
98c259b4c1
remove timer and lock functions, fix building with proxmox 0.3.2
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-04 11:33:02 +02:00
70030b43d0
list_snapshots: Returns new "comment" property (fisrt line from notes)
2020-07-31 11:34:42 +02:00
ff86ef00a7
cleanup: manifest is always CryptMode::None
2020-07-31 10:25:30 +02:00
912b3f5bc9
src/api2/admin/datastore.rs: add API to get/set Notes for backus
2020-07-31 10:17:35 +02:00
d7ee07d838
src/api2/backup/environment.rs: remove debug code
2020-07-31 07:48:53 +02:00
53705acece
src/api2/backup/environment.rs: remove debug code
2020-07-31 07:47:08 +02:00
c8fff67d88
finish_backup: add chunk_upload_stats to manifest
2020-07-31 07:45:47 +02:00
9fa55e09a7
finish_backup: test/verify manifest at server side
...
We want to make sure that the client uploaded a readable manifest.
2020-07-31 07:45:47 +02:00
f39a900722
api2/node/termproxy: fix user in worker task
...
'username' here is without realm, but we really want to use user@realm
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-30 11:57:43 +02:00
1fc82c41f2
src/api2/backup.rs: aquire backup lock earlier in create_locked_backup_group()
2020-07-30 11:03:05 +02:00
d2b0c78e23
api2/node/termproxy: fix zombies on worker abort
...
tokios kill_on_drop sometimes leaves zombies around, especially
when there is not another tokio::process::Command spawned after
so instead of relying on the 'kill_on_drop' feature, we explicitly
kill the child on a worker abort. to be able to do this
we have to use 'tokio::select' instead of 'futures::select' since
the latter requires the future to be fused, which consumes the
child handle, leaving us no possibility to kill it after fusing.
(tokio::select does not need the futures to be fused, so we
can reuse the child future after the select again)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-30 10:38:14 +02:00
adfdc36936
verify: keep track and log which dirs failed the verification
...
so that we can print a list at the end of the worker which backups
are corrupt.
this is useful if there are many snapshots and some in between had an
error. Before this patch, the task log simply says to 'look in the logs'
but if the log is very long it makes it hard to see what exactly failed.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-30 09:39:37 +02:00
d8594d87f1
verify: keep also track of corrupt chunks
...
so that we do not have to verify a corrupt one multiple times
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-30 09:39:37 +02:00
d53fbe2474
backup: ensure base snapshots are still available after backup
...
This should never trigger if everything else works correctly, but it is
still a very cheap check to avoid wrongly marking a backup as "OK" when
in fact some chunks might be missing.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 08:28:54 +02:00
95bda2f25d
backup: use flock on backup group to forbid multiple backups at once
...
Multiple backups within one backup group don't really make sense, but
break all sorts of guarantees (e.g. a second backup started after a
first would use a "known-chunks" list from the previous unfinished one,
which would be empty - but using the list from the last finished one is
not a fix either, as that one could be deleted or pruned once the first
simultaneous backup is finished).
Fix it by only allowing one backup per backup group at one time. This is
done via a flock on the backup group directory, thus remaining intact
even after a reload.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 08:26:26 +02:00
c9756b40d1
datastore: prevent deletion of snaps in use as "previous backup"
...
To prevent a race with a background GC operation, do not allow deletion
of backups who's index might currently be referenced as the "known chunk
list" for successive backups. Otherwise the GC could delete chunks it
thinks are no longer referenced, while at the same time telling the
client that it doesn't need to upload said chunks because they already
exist.
Additionally, prevent deletion of whole backup groups, if there are
snapshots contained that appear to be currently in-progress. This is
currently unlikely to trigger, as that function is only used for sync
jobs, but it's a useful safeguard either way.
Deleting a single snapshot has a 'force' parameter, which is necessary
to allow deleting incomplete snapshots on an aborted backup. Pruning
also sets force=true to avoid the check, since it calculates which
snapshots to keep on its own.
To avoid code duplication, the is_finished method is factored out.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 08:26:01 +02:00
2aaae9705e
src/backup/verify.rs: try to verify chunks only once
...
We use a HashSet (per BackupGroup) to track already verified chunks.
2020-07-29 13:29:13 +02:00
8aa67ee758
bump proxmox to 0.3, cleanup http_err macro usage
...
Also swap the order of a couple of `.map_err().await` to
`.await.map_err()` since that's generally more efficient.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 09:38:36 +02:00
3865e27e96
src/api2/node.rs: 'mod' statement cleanup
...
split them into groups: `pub`, `pub(crate)` and non-pub
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 09:19:57 +02:00
7fde1a71ca
upload_chunk: allow upload of empty blobs
...
a blob can be empty (e.g. an empty pct fw conf), so we
have to set the minimum size to the header size
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-28 11:03:36 +02:00
39f18b30b6
src/backup/data_blob.rs: new load_from_reader(), which verifies the CRC
...
And make verify_crc private for now. We always call load_from_reader() to
verify the CRC.
Also add load_chunk() to datastore.rs (from chunk_store::read_chunk())
2020-07-28 10:23:16 +02:00
e6b599aa6c
services: make reload safer and default to it in gui
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-25 20:23:12 +02:00
ee15af6bb8
api: service command: fix test for essential service
...
makes no sense to disallow reload or start (even if start cannot
really happen)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:35:19 +02:00
33a88dafb9
server/state: add spawn_internal_task and use it for websockets
...
is a helper to spawn an internal tokio task without it showing up
in the task list
it is still tracked for reload and notifies the last_worker_listeners
this enables the console to survive a reload of proxmox-backup-proxy
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-24 11:17:33 +02:00
224c65f8de
termproxy: let users stop the termproxy task
...
for that we have to do a select on the workers abort_future
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-24 11:17:33 +02:00
3d3670d786
termproxy: cmd: support upgrade
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 13:12:18 +02:00
e744de0eb0
api: termproxy: fix ACL as /nodes is /system
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 12:57:11 +02:00
98b1733760
api: apt: use schema default const for quiet param
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 12:25:28 +02:00
1c2f842a98
api2/nodes: add termproxy and vncwebsocket api calls
...
Even though it has nothing to do with vnc, we keep the name of the api
call for compatibility with our xtermjs client.
termproxy:
verifies that the user is allowed to open a console and starts
termproxy with the correct parameters
starts a TcpListener on "localhost:0" so that the kernel decides the
port (instead of trying to rerserving like in pve). Then it
leaves the fd open for termproxy and gives the number as port
and tells it via '--port-as-fd' that it should interpret this
as an open fd
the vncwebsocket api call checks the 'vncticket' (name for compatibility)
and connects the remote side (after an Upgrade) with a local TcpStream
connecting to the port given via WebSocket from the proxmox crate
to make sure that only the client can connect that called termproxy and
no one can connect to an arbitrary port on the host we have to include
the port in the ticket data
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-23 12:06:38 +02:00
a4d1675513
api2/access: implement term ticket
...
modeled after pves/pmgs vncticket (i substituted the vnc with term)
by putting the path and username as secret data in the ticket
when sending the ticket to /access/ticket it only verifies it,
checks the privs on the path and does not generate a new ticket
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-23 11:55:00 +02:00
27fde64794
api: apt update must run protected
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 11:45:52 +02:00
fa3f0584bb
api: apt: support refreshing package index
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 11:21:54 +02:00
a4e86972a4
add .../apt/update API call
...
Depends on patched apt-pkg-native-rs. Changelog-URL detection is
inspired by PVE perl code for now, though marked with fixme to use 'apt
changelog' later on, if/when our repos have APT-compatible changelogs
set up.
list_installed_apt_packages iterates all packages and creates an
APTUpdateInfo with detailed information for every package matched by the
given filter Fn.
Sadly, libapt-pkg has some questionable design choices regarding their
use of 'iterators', which means quite a bit of nesting...
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-23 10:41:14 +02:00
91f5594c08
api: translate ENOTFOUND to 404 for downloads
...
and percolate the HttpError back up on the client side
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-07-22 09:19:29 +02:00
113c9b5981
move subscription API path to /nodes
...
This aligns it with PVE and allows the widget toolkit's update window
"refresh" to work without modifications once POST /apt/update is
implemented.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-21 19:33:52 +02:00
a26c27c8e6
api2/status: fix estimation bug
...
when a datastore has enough data to calculate the estimated full date,
but always has exactly the same usage, the factor b of the regression
is '0'
return 0 for that case so that the gui can show 'never' instead of
'not enough data'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-21 13:02:08 +02:00
bfcef26a99
api2/node/status: add fingerprint
...
and rename get_usage to get_status (since its not usage only anymore)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-10 11:08:53 +02:00
660a34892d
update proxmox crate to 0.2.0
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 11:08:27 +02:00
7957fabff2
api: add ZPOOL_NAME_SCHEMA and regex
...
poolnames can containe spaces and some other special characters
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
521a0acb2e
DataStore::load_manifest: also return CryptMode
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:19:53 +02:00
f28d9088ed
introduce a CryptMode enum
...
This also replaces the recently introduced --encryption
parameter on the client with a --crypt-mode parameter.
This can be "none", "encrypt" or "sign-only".
Note that this introduces various changes in the API types
which previously did not take the above distinction into
account properly:
Both `BackupContent` and the manifest's `FileInfo`:
lose `encryption: Option<bool>`
gain `crypt_mode: Option<CryptMode>`
Within the backup manifest itself, the "crypt-mode" property
will always be set.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-07 15:24:19 +02:00
08cb2038bd
api: disks: indentation fixup
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 17:59:30 +02:00
2162e2c15d
src/api2/admin/datastore.rs: avoid slash in UPID strings
2020-06-30 13:11:22 +02:00
fe3e65c3ea
src/api2/backup.rs: call register_chunk in previous download api
2020-06-26 08:22:46 +02:00
b957aa81bd
update backup api for incremental backup
...
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-26 07:17:08 +02:00
8ea00f6e49
allow to abort verify jobs
...
And improve job description rendering on gui.
2020-06-25 12:56:36 +02:00
e22f4882e7
extract create_download_response API helper
...
and put it into a new "api2::helpers" module.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 11:57:37 +02:00
c2009e5309
src/api2/admin/datastore.rs: add verify api
2020-06-24 13:35:21 +02:00
facd9801cf
add incremental backup support
...
To support incremental backups (where not all chunks are sent to the
server), a new parameter "reuse-csum" is introduced on the
"create_fixed_index" API call. When set and equal to last backups'
checksum, the backup writer clones the data from the last index of this
archive file, and only updates chunks it actually receives.
In incremental mode some checks usually done on closing an index cannot
be made, since they would be inaccurate.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 10:01:25 +02:00
21302088de
remove debug println
2020-06-24 09:15:13 +02:00
8268c9d161
fix overflow panic during upload
...
if *only* data chunks are registered (high chance during incremental
backup), then chunk_count might be one lower then upload_stat.count
because of the zero chunk being unconditionally uploaded but not used.
Thus when subtracting the two, an overflow would occur.
In general, don't let the client make the server panic, instead just set
duplicates to 0.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 09:07:22 +02:00
b91b7d9ffd
api2/node/disks/zfs: check if default zfs mount path exists
...
and if it does bail, because otherwise we would get an
error on mounting and have a zpool that is not imported
and disks that are used
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:22:39 +02:00
d33d8f4e6a
api2/admin/datastore: add pxar-file-download api call
...
streams a file from a pxar file of an unencrypted backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:21:15 +02:00
5b1cfa01f1
api2/admin/datastore: add 'catalog' api call
...
returns the dir listing of the given filepath of the backup snapshot
the filepath has to be base64 encoded or 'root'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:16:12 +02:00
60f9a6ea8f
src/backup/datastore.rs: add new helpers to load blobs and verify chunks
2020-06-24 06:58:14 +02:00
cbef49bf4f
remove absolute paths when executing binaries
...
we set the paths manually, so this is ok
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 07:09:06 +02:00
f386f512d0
add AsyncReaderStream
...
and replace AsyncIndexReader's stream implementation with that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 06:33:31 +02:00
42d19fdf69
src/api2/node/disks/zfs.rs: always set pool name
2020-06-20 07:15:32 +02:00
4e37d9ce67
add general indented_list_to_tree implementation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 14:37:40 +02:00
6ef9bb59eb
api2/admin/datastore: add download-decoded endpoint
...
similar to 'download', but streams the decoded file
when it is not encrypted
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:39:15 +02:00
abdb976340
add Display trait to BackupDir
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:28:35 +02:00
1c090810f5
api2/admin/datastore/snapshos: show encrypted and size info per file
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:39:56 +02:00
e181d2f6da
add encrypted info to Manifest
...
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:35:39 +02:00
ba694720fc
api2/admin/datastore: log stream error during file download
...
the client cannot get an error during an chunked http transfer, so at
least log it server side
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:58:55 +02:00
026dc1d11f
src/api2/node/disks/zfs.rs: add zpool_details api
2020-06-18 15:04:46 +02:00
3f851d1321
src/api2/node/disks/directory.rs: add early check if disk is unused
2020-06-17 08:31:11 +02:00
1aef491e24
src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks
2020-06-17 08:07:54 +02:00
66af7f51bc
src/tools/disks/zfs.rs: make zfs list parser private
2020-06-17 07:00:54 +02:00
fb5a066500
src/api2/node/disks.rs: expose directory api
2020-06-16 13:36:32 +02:00
929a13b357
src/api2/node/disks/zfs.rs: add zpool api
2020-06-16 13:25:53 +02:00
fbbcd85839
src/api2/node/disks/directory.rs: implement add-datastore feature
2020-06-15 10:01:50 +02:00
0196b9bf5b
remove unnecessary .into
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:57:58 +02:00
20b3094bcb
api2/status: add task list endpoint
...
for now mostly copy/paste from nodes/nodename/tasks
(without the parameters)
but we should replace the 'read_task_list' with a method
that gives us the tasks since some timestamp
so that we can get a longer list of tasks than for the node
(we could of course embed this then in the nodes/node/task api call and
remove this again as long as the api is not stable)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:17 +02:00
df528ee6fa
implement From<TaskListInfo> for TaskListItem
...
and use it where its convenient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:12 +02:00
476b4acadc
BackupEnvironment: do not set finished flag prematurely
...
we check if all dynamic_writers are closed and if the backup contains
any valid files, we can only mark the backup finished after those
checks, else the backup task gets marked as OK, even though it
is not finished and no cleanups run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:52 +02:00
ec8f042459
api2/status: use new rrd::extract_cached_data
...
and drop the now unused extract_lists function
this also fixes a bug, where we did not add the datastore to the list at
all when there was no rrd data
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:16 +02:00
431cc7b185
rrd: move creation of serde value into api
...
there is now a 'extract_cached_data' which just returns
the data of the specified field, and an api function that converts
a list of fields to the correct serde value
this way we do not have to create a serde value in rrd/cache.rs
(makes for a better interface)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:14 +02:00
d4f2397d4c
add api to format disks and create datastores
2020-06-10 11:03:36 +02:00
bda48e04da
api2: add status/datastore-usages api call
...
returns a list of the datastores and their usages, a list of usages of
the past month (for the gui) and an estimation of when its full
(using the linear regression)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:20:25 +02:00
33070956af
let disk_usage return StorageStatus and use it for datastores/nodes
...
disk_usage returned the same values as defined in StorageStatus,
so simply use that
with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:05:39 +02:00
c443f58b09
switch to external pxar and fuse crates
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
2b388026f8
src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node
2020-06-08 07:20:59 +02:00
707974fdb3
src/api2/node/disks.rs: implement initgpt API
2020-06-07 10:30:34 +02:00
9069debcd8
src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA
2020-06-07 07:20:25 +02:00
ce8e3de401
move disks api to /node/<node>/disks
2020-06-06 14:43:36 +02:00
7fa2779559
src/api2/disks.rs: implement smart api
2020-06-06 12:23:11 +02:00
ff30caeaf8
src/api2/disks.rs - list-disks: add usage-type filter
2020-06-06 11:48:58 +02:00
553cd12ba6
src/api2/disks.rs: start disks api
2020-06-06 11:38:47 +02:00
8d78589969
improve display of 'next run' for sync jobs
...
if the last sync job is too far in the past (or there was none at all
for now) we run it at the next iteration, so we want to show that
we now calculate the next_run by using either the real last endtime
as time or 0
then in the frontend, we check if the next_run is < now and show 'pending'
(we do it this way also for replication on pve)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:03:54 +02:00
add5861e8d
typo fixes all over the place
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 16:39:08 +02:00
96d65fbcd0
cleanup: define/use const for predefined blob file names.
2020-05-30 14:04:15 +02:00
04b0ca8b59
add owner to group and snapshot listings
...
while touching it, make columns and tbar in DataStoreContent.js
declarative members and remove the (now) unnecessary initComponent
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-30 07:24:12 +02:00
d43f86f3f3
api2: add admin/sync endpoint
...
this returns the list of syncjobs with status, as opposed to
config/sync (which is just the config)
also adds an api call where users can run the job manually under
/admin/sync/$ID/run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:31:32 +02:00
268687ddf0
api2/pull: refactor priv checking and creating pull parameters
...
we want to reuse those in the api call for manually running a sync job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:27:43 +02:00
426c1e353b
api2/config/sync: fix id parameter
...
'name' is not the correct parameter for get/post
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:54 +02:00
2888b27f4c
create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:25 +02:00
9857472211
fix removing of remotes
...
we have to save the remote config after removing the section
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 10:48:26 +02:00
090decbe76
BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names
...
The repo URL consists of
* optional userid
* optional host
* datastore name
All three have defined regex or format, but none of that is used, so
for example not all valid datastore names are accepted.
Move definition of the regex over to api2::types where we can access
all required regexes easily.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:29:23 +02:00
c32186595e
api2::types: factor out USER_ID regex
...
allows for better reuse in a next patch
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:27:38 +02:00
c94e1f655e
rrd stats: improve io delay stats
2020-05-28 19:12:13 +02:00
91e5bb49f5
src/bin/proxmox-backup-proxy.rs: simplify code
...
and gather all stats for the root disk
2020-05-28 12:30:54 +02:00
de4db62c57
remotes: save passwords as base64
...
to avoid having arbitrary characters in the config (e.g. newlines)
note that this breaks existings configs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:06 +02:00
1a0d3d11d2
src/api2/admin/datastore.rs: add rrd api
2020-05-26 12:26:14 +02:00
83fd4b3b1b
remote: try to use Struct for api
...
with a catch: password is in the struct but we do not want it to return
via the api, so we only 'serialize' it when the string is not empty
(this can only happen when the format is not checked by us, iow.
when its returned from the api) and setting it manually to ""
when we return remotes from the api
this way we can still use the type but do not return the password
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:55:07 +02:00
5dcdcea293
api2/config/remote: remove password from read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:49:12 +02:00
99f443c6ae
api2/config/remote: lock and use digest for removal
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:45 +02:00
4f966d0592
api2/config/remote: use rpcenv for digest for read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:28 +02:00
db0c228719
config/remote: add 'name' to Remote struct
...
and use it as section id, like with User
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:05 +02:00
d8dc281992
www/DataStoreStatus.js: display loadavg stats
2020-05-25 11:54:15 +02:00
3e930f2bdc
www/DataStoreStatus.js: display root disk stats
2020-05-25 11:34:24 +02:00
c1b24fbf0b
www/DataStoreStatus.js: display swap stats
2020-05-25 10:39:54 +02:00
a608806f65
www/DataStoreStatus.js: display netin/netout
2020-05-24 19:02:35 +02:00
803ab12ad4
rrd: simplify code
2020-05-23 15:37:17 +02:00
a4a3f7ca5e
rrd: pack multiple rrd values into th estat list
2020-05-23 14:03:44 +02:00
a2f862eed6
add experimental rrd api to get cpu stats
2020-05-23 11:50:53 +02:00
07ad6470ca
src/client/pull.rs: split out pull related code
2020-05-22 08:04:20 +02:00
183125d576
src/api2/pull.rs: aquire try_shared_chunk_store_lock inside pull_store
2020-05-22 07:24:17 +02:00
6f652b1b3a
rename 'job' to 'sync'
2020-05-21 10:29:25 +02:00
b1d4edc769
src/api2/config/job.rs: add job api
2020-05-21 10:16:35 +02:00
b4900286ce
src/config/jobs.rs: use SectionConfig for jobs
2020-05-21 10:16:35 +02:00
2882c881e9
api2/access/acl: add path and exact parameter to list_acl
...
so that we can get only a subset of the acls, filtered by the backed
also return the digest here
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:44:36 +02:00
12e3895399
api2/access/acl: make update_acl a protected api call
...
since we want to set the owner of the acl config to 'root'
which is only possible when using a protected api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:22:41 +02:00
11b6391c83
add 'exact' parameter to extract_acl_node_data
...
so that we can return acls for a single path
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:22:10 +02:00
b05672579e
api2/roles: change return field of role to roleid
...
to be compatible with the pve api
with this, we can reuse the ui parts (RoleSelector)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:47 +02:00
5160c0e986
api2/acl: add privs array to roles
...
so that an admin can see which roles have which privileges
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:37 +02:00
872062ee9f
src/config/datastore.rs_ change prune types from i64 to u64
2020-05-20 13:00:13 +02:00
67f7ffd0db
src/config/datastore.rs: add prune settings
2020-05-20 11:29:59 +02:00
0fafac2492
src/api2/access/user.rs: remove useless description
...
The description is not used at all if we refer to a type.
2020-05-20 11:27:58 +02:00
49ff10921c
src/api2/types.rs: define PRUNE_SCHEMA_KEEP_*
2020-05-20 10:13:38 +02:00
42fdbe5112
src/config/datastore.rs: add gc-schedule property
2020-05-20 08:38:10 +02:00
7d4e362993
depend on proxmox 0.1.32, src/api2/access/user.rs: simplify code
2020-05-19 12:58:46 +02:00
522c0da0a0
use new 'id_property' for user::User and use it in api calls
...
this allows us to return a user::User (or Vec<> of it)
instead of a generic serde value
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-19 09:33:56 +02:00
e8d1da6a15
depend on proxmox 0.1.31 - use Value to store result metadata
2020-05-18 09:57:35 +02:00
30f577248b
src/api2/node/time.rs: avoid custom unsafe readlink implementations
2020-05-15 06:50:07 +02:00
f486e9e50e
add systemd configuration file parser/writer, start job configuration
2020-05-12 13:07:49 +02:00
3dd27a3bf8
src/api2/node/network.rs: add protected flag to revert
2020-05-08 09:30:25 +02:00
3aedb73816
src/api2/node/network.rs: pass bridge_ports and slaves a property strings
...
To make it compatible with pve.
2020-05-08 08:49:17 +02:00
bab5d18c3d
src/config/network.rs: implement bond_mode
...
and rename bond_slaves to slaves to make it compatible with pve.
2020-05-07 14:07:45 +02:00
c2ffc68554
src/api2/node/network.rs: cleanup - factor out check_duplicate_gateway
2020-05-07 11:26:30 +02:00
9651833130
src/api2/node/network.rs: allow to create bridge and bond
2020-05-07 11:09:12 +02:00
7b22acd0c2
src/config/network.rs: make it compatible with pve
...
and depend on proxmox 0.1.26
2020-05-07 09:28:25 +02:00
5751e49566
src/server/worker_task.rs: implement and use status command
2020-05-07 09:27:33 +02:00
99641a6bbb
garbage_collect: call fail_on_abort to abort GV when requested.
2020-05-05 09:06:34 +02:00
db1e061dcb
src/bin/proxmox-backup-client.rs: correctly format prune result list.
2020-05-05 06:45:37 +02:00
ed751dc2ab
src/api2/node/status.rs: rework api, implement reboot and shutdown
2020-04-30 11:52:40 +02:00
ca9dfe5fa4
src/api2/node/tasks.rs: use api macro features for default values
2020-04-30 11:51:56 +02:00
720af9f69b
src/api2/node/tasks.rs: allow users to list/access there own tasks
2020-04-30 10:05:50 +02:00
f1490da82a
use resonable acl paths (fixup)
2020-04-30 09:32:13 +02:00
74c08a5782
use reasonable acl paths
2020-04-30 09:30:00 +02:00
bc0d03885c
use proxmox 0.1.25, use new EnumEntry feature
2020-04-29 13:01:24 +02:00
30fb602578
src/api2/admin/datastore.rs - get_datastore_list: only return name and comment
...
We dont want to leak the full configuration to users with limited access permission.
Please use the api2::config::datastore api to get the full configuration.
2020-04-29 09:21:34 +02:00
0a00f6e01c
src/api2/config/datastore.rs_ add delete property to update method
2020-04-29 09:09:59 +02:00
30003baaa4
src/api2/config/remote.rs: fix white space
2020-04-29 09:09:39 +02:00
5211705ff1
src/api2/config/remote.rs: add delete parameter to update method
2020-04-29 09:04:17 +02:00
ec67af9af3
src/api2/pull.rs: require Datastore.Prune if delete flag is set.
2020-04-29 07:19:32 +02:00
8247db5b39
src/config/acl.rs: introduice privileges and roles for remotes
2020-04-29 07:03:44 +02:00
409f44247b
fix api2::types::ACL_ROLE_SCHEMA
...
make sure we list all roles ...
2020-04-28 13:25:02 +02:00
54552dda59
implemnt backup ownership, improve datastore access permissions
2020-04-28 10:22:25 +02:00
d00e1a216f
src/config/acl.rs: introduce more/better datastore privileges
2020-04-27 07:13:50 +02:00
9c7fe29dfc
src/config/acl.rs: rtename PRTIV_DATASTORE_ALLOCATE to PRIV_DATASTORE_MODIFY
2020-04-27 06:50:35 +02:00
14627d671a
src/bin/proxmox-backup-manager.rs: add dns sub command
...
Also improved the DNS api, added a --delete option.
2020-04-26 08:23:23 +02:00
26d9aebc28
move src/api2/config/network.rs to src/api2/node/network.rs
...
So that we have the same api path for network config as pve.
2020-04-25 17:00:38 +02:00
fd7c0979b4
src/bin/proxmox-backup-manager.rs: implement netwerk revert
2020-04-24 10:45:49 +02:00
2eefd9aee1
src/config/network.rs: implement network reload, set "changes" attribute
2020-04-24 09:55:46 +02:00
8a6b86b8a7
src/config/network.rs: use a simple String for comments
2020-04-24 07:46:08 +02:00
10a9be45bd
src/api2/config/network.rs: implement update/delete comments
2020-04-23 16:08:35 +02:00
5f60a58fd5
src/config/network.rs; support interface comments, cleanups
2020-04-23 15:54:30 +02:00
5e4e88e83f
src/api2/config/network.rs: implement update/delete for bridge_ports and bond_slaves
2020-04-23 11:21:27 +02:00
339965d720
src/api2/config/network.rs: only allow one default gateway
2020-04-23 10:37:40 +02:00
42fbe91a34
src/config/network.rs: parse bond-slaves
2020-04-23 09:31:10 +02:00
1d9a68c2fc
src/config/network.rs: parse bridge-ports
2020-04-23 09:24:17 +02:00
02269f3dba
src/config/network.rs: introduce NetworkInterfaceType
2020-04-23 08:45:03 +02:00
2c18efd902
src/config/network.rs: use a single mtu setting (instead of mtu_v4 and mtu_v6)
2020-04-23 07:07:14 +02:00
b1564af25a
src/bin/proxmox-backup-manager.rs: format datastore list output
2020-04-22 17:37:20 +02:00
f1026a5aa9
src/api2/config/network.rs: allow to update 'auto' flag
2020-04-22 16:46:46 +02:00
3fce3bc36e
src/config/network/parser.rs: parse MTU settings
2020-04-22 13:44:51 +02:00
c016482c7a
src/api2/config/network.rs: implement delete property
2020-04-22 12:19:31 +02:00
27f2c23049
src/api2/config/network.rs: allow to update configuration method
2020-04-22 11:32:36 +02:00
df6bb03d0e
src/api2/config/network.rs: improve network api
2020-04-22 10:54:07 +02:00
76cf5208cf
src/api2/types.rs: add schemas for IP/CIDR
2020-04-22 10:28:53 +02:00
2ea7bf1b3d
src/api2/config/datastore.rs_ fix method docs
2020-04-22 08:53:16 +02:00
8b57cd4441
src/config/network.rs: remove netmask support
...
rely on cidr instead.
2020-04-22 08:45:13 +02:00
68da20bf62
src/api2/types.rs: define NETWORK_INTERFACE_NAME_SCHEMA
2020-04-21 17:54:52 +02:00
c357260d09
src/config/network.rs: move type definitions to src/api2/types.rs
2020-04-21 17:25:05 +02:00
ca0e534796
src/api2/config/network.rs: start network configuration api
2020-04-21 14:28:26 +02:00
365f0f720c
fix permission tests using non-uri parameters
...
We nood to do those tests inside the fuction body instead...
2020-04-18 08:23:04 +02:00
bb072ba49c
src/api2/access.rs: cleanup
2020-04-18 07:28:25 +02:00
ff329f970b
src/api2/types.rs: use anyhow::Error in test cases
2020-04-18 07:05:31 +02:00
f7d4e4b506
switch from failure to anyhow
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-17 18:43:30 +02:00
404d78c41e
src/api2/pull.rs: add access permission
2020-04-17 15:27:04 +02:00
1bfc1efa50
src/api2/subscription.rs: add access permissions
2020-04-17 15:14:28 +02:00
73ce1d1146
src/api2/reader.rs: add access permissions
2020-04-17 15:01:56 +02:00
70e5f2461d
src/api2/config/remote.rs: add access permissions
2020-04-17 14:57:26 +02:00
c0ef209aeb
src/api2/config/datastore.rs: impl digest check for delete, add access permissions
2020-04-17 14:51:29 +02:00
9f9f7eefa3
src/api2/backup.rs: add access permissions
2020-04-17 14:40:20 +02:00
bb34b58910
src/api2/admin/datastore.rs: add access permissions - first try
...
We need to refine this later (introduce backup owner concept?)
2020-04-17 14:36:27 +02:00
5972def5ec
acl: change path "storage" to "datastore"
2020-04-17 14:15:44 +02:00
aa90ced3bf
src/api2/access/role.rs: use schema ACL_ROLE_SCHEMA
2020-04-17 14:14:06 +02:00
ca257c8097
move type defs from src/api2/access/acl.rs to src/api2/types.rs
2020-04-17 14:13:15 +02:00
3fff55b293
src/api2/access/role.rs: new api to list roles
2020-04-17 14:03:24 +02:00
4f66423fcc
src/api2/access/user.rs: add access permissions
2020-04-17 11:04:36 +02:00
d4f020f4c5
src/api2/access/user.rs: add access permissions
2020-04-17 10:08:45 +02:00
d28ddb8e04
src/api2/access/acl.rs: add access permissions
2020-04-17 10:03:09 +02:00
83b6a7cf71
src/api2/node/tasks.rs: use api macro, implement access permissions
2020-04-16 17:47:21 +02:00
e4681f9f71
src/api2/node/syslog.rs: add access permissions
2020-04-16 17:08:19 +02:00
b5037fa8ed
src/api2/node/status.rs: add access permissions
2020-04-16 17:05:09 +02:00
1cf7bbf412
src/api2/node/services.rs: add access permissions
2020-04-16 12:47:16 +02:00
68ed0c629d
src/api2/node/journal.rs: add access permissions
2020-04-16 12:47:16 +02:00
4b40148caa
start impl. access permissions
2020-04-16 12:47:16 +02:00
109d7817cd
src/config/user.rs - cached_config: do not store/return digest
2020-04-15 11:35:57 +02:00
bd098a7f77
src/api2/node/dns.rs: use api macro (cleanup)
2020-04-15 10:09:18 +02:00
9c06f6c292
fix previous commit - use result.
2020-04-14 17:48:10 +02:00
9f4e47dd93
acl update: check path
2020-04-14 17:23:48 +02:00
d83175dd69
acl update: check if user exist.
2020-04-14 13:46:27 +02:00
9765092ede
acl api: implement update
2020-04-14 10:16:49 +02:00
ed3e60ae69
start ACL api
2020-04-13 11:09:44 +02:00
b88f9c5b1e
PASSWORD_SCHEMA: set max_length to 1024 (for tickets)
2020-04-09 13:35:58 +02:00
879546aff6
api: add default property to domain list
2020-04-09 13:35:08 +02:00
73b40e9b46
api: correctly sort access subdirmap
2020-04-09 13:34:07 +02:00
708db4b3ae
api: add list_domains
2020-04-09 11:36:45 +02:00
685e13347e
api: move config/user to access/users, implement change_password
...
To make it similar to the pve api
2020-04-09 10:21:24 +02:00
7d817b0358
implement auth framework
2020-04-08 14:06:15 +02:00
579728c641
add user configiguration
2020-04-08 14:06:15 +02:00
cf459b1982
gc: log pending removals
2020-04-06 09:50:40 +02:00
dda7015497
prune api: return a usable result (we run synchronous anyways)
2020-04-01 12:24:28 +02:00
6650a242fb
rewrite future select in upgrade_to_backup_protocol using select macro
...
and handle all ok/err cases with at least logging
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-03-18 11:33:59 +01:00
f24fc1166b
src/bin/proxmox-backup-client.rs - list_snapshots: use format_and_print_result_full()
...
Depend on proxmox v0.1.14.
2020-02-27 11:28:44 +01:00
ea5f547fdc
src/bin/proxmox-backup-client.rs - list_snapshot_files: use format_and_print_result_full()
2020-02-26 13:49:47 +01:00
8528fce874
src/api2/node/tasks.rs; make list_tasks public
2020-01-30 13:31:20 +01:00
70fa1aa3fc
fixup
2020-01-30 13:31:20 +01:00
5eeea607ae
src/section_config.rs - convert_to_array: add option to skip properties
2020-01-30 13:26:46 +01:00
99384f7933
src/api2/types.rs: define and use struct TaskListItem
2020-01-28 11:23:08 +01:00
2c4b303c62
src/api2/node/tasks.rs: use api macro for list_tasks
2020-01-28 11:23:08 +01:00
d59dbeca1b
src/client/http_client.rs: new struct HttpClientOptions
2020-01-25 12:18:00 +01:00
6afbe1d846
src/config/remote.rs: add fingerprint
2020-01-25 09:49:45 +01:00
dcb8db66d9
src/api2/types.rs: add CERT_FINGERPRINT_SHA256_SCHEMA
2020-01-25 09:49:45 +01:00
4191018c79
src/api2/admin/datastore.rs - download_file: use BACKUP_ARCHIVE_NAME_SCHEMA for file-name
2020-01-23 13:49:30 +01:00
a92830dc39
src/api2/types.rs: define and use api type GarbageCollectionStatus
2020-01-23 13:40:12 +01:00
dfc58d47ad
src/api2/admin/datastore.rs: use api macro for start_garbage_collection
2020-01-23 13:01:21 +01:00
1dc117bbba
src/api2/types.rs: define and use struct StorageStatus
2020-01-23 12:51:30 +01:00
09b1f7b202
src/api2/types.rs: define struct BackupContent, and use it with list_snapshot_files
2020-01-23 11:17:38 +01:00
68a6a0ee03
src/api2/admin/datastore.rs: use api macro for delete_snapshot
2020-01-23 10:16:45 +01:00
c425bdc9b2
src/api2/pull.rs: delete fanished snapshots
2020-01-23 09:58:57 +01:00
18cc66ee85
src/api2/pull.rs: use new Datastore::last_successful_backup helper
...
Do not fail if local group does not exists.
2020-01-22 15:06:28 +01:00
1ca037d81c
followup: commit all changes
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-22 14:14:52 +01:00
44b65b4ec7
api/datastore: do not allow updating path for now
...
It's a bit dangerous as it points to all the saved backups, so they
would be seemingly lost after updating the path.
Follow our logic from other products, e.g. in PVE we do not allow to
update the backing path/location of a storage either for similar
reasons.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-22 13:55:37 +01:00
f1d99e3f6a
src/bin/proxmox-backup-client.rs: use a std channel to write the catalog
2020-01-22 12:49:08 +01:00
9ea4bce444
bump proxmox crate to 0.1.7
...
The -sys, -tools and -api crate have now been merged into
the proxmx crate directly. Only macro crates are separate
(but still reexported by the proxmox crate in their
designated locations).
When we need to depend on "parts" of the crate later on
we'll just have to use features.
The reason is mostly that these modules had
inter-dependencies which really make them not independent
enough to be their own crates.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 13:48:37 +01:00
4b4eba0b9e
src/api2/pull.rs: implement delete flag for vanished groups
2020-01-17 11:24:55 +01:00
b31c8019d7
src/api2/types.rs: define and use GroupListItem
2020-01-17 10:17:18 +01:00
f357390c15
renamed: src/config/remotes.rs -> src/config/remote.rs
...
And use 'remote' instead of 'remotes' everywhere.
2020-01-16 14:32:06 +01:00
94609e2380
src/api2/pull.rs: use reference to remote configuration
2020-01-16 13:55:19 +01:00
002a191abf
src/api2/config: check digest
2020-01-15 12:27:05 +01:00
347834df25
src/api2/config: correctly lock files
2020-01-15 11:57:12 +01:00
1c79e1017f
services: fix indentation
...
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-01-15 10:55:04 +01:00
f3ec5dae7f
src/api2/config: improve api schema by adding return type
2020-01-14 14:55:58 +01:00
c5799e4043
src/api2/config/datastore.rs: improve api, implement update and read
2020-01-14 14:45:56 +01:00
08195ac823
src/api2/config/remotes.rs: improve api, implement update and read
2020-01-14 14:20:16 +01:00
d0187a51a9
src/section_config.rs - convert_to_array: optionally add digest
...
datastore::config() -> also return digest
remotes::config() -> also return digest
2020-01-14 12:57:03 +01:00
7e7b781a18
src/api2/types.rs: also define PASSWORD_FORMAT and use it correctly
2020-01-14 11:32:02 +01:00
da4a15a351
src/api2/types.rs: define and use PASSWORD_REGEX
2020-01-14 11:22:42 +01:00
ff620a3d1a
src/api2/types.rs: add regression test for PROXMOX_USER_ID_SCHEMA
2020-01-13 17:26:43 +01:00
ae62c4fe59
src/api2/types.rs: fix regex patterns
2020-01-13 14:38:31 +01:00
163dc16c0b
src/api2/types.rs: define PROXMOX_AUTH_REALM_SCHEMA and PROXMOX_USER_ID_SCHEMA
...
And try to use nbew schemas with config api...
2020-01-13 14:18:19 +01:00
b25f313d66
src/api2/types.rs: define HOSTNAME_FORMAT, DNS_NAME_FORMAT and DNS_NAME_OR_IP_FORMAT
2020-01-13 13:10:52 +01:00
454c13edce
src/api2/types.rs: define SINGLE_LINE_COMMENT_SCHEMA
2020-01-13 12:02:13 +01:00
167971ed49
src/api2/types.rs: define REMOTE_ID_SCHEMA here
2020-01-13 11:47:07 +01:00
50af953e1b
src/section_config.rs: make set_data generic
2020-01-11 11:09:27 +01:00
688fbe07a1
cleanup config api, add remotes config cli interface
2020-01-11 10:42:09 +01:00
ca44172404
src/api2/config/remotes.rs: cleanup - avoid compiler warnings
2020-01-10 19:25:26 +01:00
141304d64e
src/api2/config/remotes.rs: new API to configure remotes
2020-01-10 13:28:15 +01:00
eb506c830c
rename src/api2/sync.rs -> src/api2/pull.rs
...
Use "pull" instead of "sync", because this also indicates a
direction (like "sync from").
2020-01-10 11:15:16 +01:00
71da3d6a1b
src/api2/types.rs: fix SnapshotListItem schema definition
2020-01-09 10:36:19 +01:00
de8ec04123
src/api2/sync.rs: implement remote sync
2020-01-08 14:03:52 +01:00
fc189b198c
src/api2/types.rs: define and use new api type SnapshotListItem
2020-01-07 12:52:52 +01:00
550e0d8870
src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate
...
Without calling external openssl binary.
2019-12-22 17:35:06 +01:00
5e62d19cfb
fix create_datastore parameters:
...
comment was declared as optional but unwrap()ed, causing
panics when not using it
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-20 12:30:25 +01:00
645995634a
src/api2/config/datastore.rs - create: pass uid and gid instead of User
2019-12-20 09:23:58 +01:00
e67770d496
src/backup/chunk_store.rs - create: pass User instead of CreateOptions
2019-12-20 09:11:40 +01:00
bca9093520
api/ui: datastore: allow to set simple comment
...
for now forbid all control characters[0] in the comment value, the
section config writer cannot cope with newlines in the value, it
writes them out literally, allowing "injection" or breaking the whole
config.
In the webinterface use also a textfield, not a textarea.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-19 17:58:01 +01:00
afdcfb5bc9
let ChunkStore::create take CreateOptions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-19 13:14:49 +01:00
2337df7b3d
api2: add very basic 'status/usage' endpoint for nodes
...
For returning the nodes basic "usage status", for now one gets memory
and CPU utilization.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-18 17:24:32 +01:00
feaa1ad35f
replace file_set_contents with replace_file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-18 11:16:04 +01:00
a66e7920dc
src/api2/config/datastore.rs: set protected flags for create/delete API
2019-12-18 10:52:45 +01:00
f0db500808
src/api2/config/datastore.rs: fix api docu
2019-12-18 10:48:19 +01:00
81cc71c0d7
use proxmox-mini-journalreader to display syslog
2019-12-17 14:06:48 +01:00
20197f7cf5
src/api2/node/syslog.rs: use api macro
2019-12-17 13:26:49 +01:00
bb084b9c91
api2: update for latest proxmox-api changes
...
- rename ApiFuture into ApiResponseFuture
- impl. ApiHandler::Async
2019-12-16 10:01:51 +01:00
236a396aa1
src/api2/admin/datastore.rs - prune: log retention options
2019-12-14 16:32:16 +01:00
503995c767
src/api2/admin/datastore.rs: fix prune worker id
2019-12-14 16:05:21 +01:00
db0cb9ce0b
update a chunk of stuff to the hyper release
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
1ae5677de0
src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT
2019-12-12 14:15:40 +01:00
d0adf270fb
src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA
2019-12-12 12:37:11 +01:00
66c49c21c3
src/api2/types.rs: factor out DATASTORE_SCHEMA
2019-12-11 13:45:23 +01:00
567d3e00fb
src/api2/node/tasks.rs: new filter "store" to filter tasks for one store
2019-12-11 12:53:34 +01:00
6d04612955
src/api2/node/tasks.rs: add "running" filter to list only running tasks
2019-12-11 10:52:27 +01:00
8560fe3e5f
src/api2/node/tasks.rs - extract_upid: avoid duplicate error message
2019-12-10 13:45:56 +01:00
5830c20560
src/bin/proxmox-backup-client.rs: add simple task management cli
2019-12-10 13:43:53 +01:00
3b03abfe14
src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune
...
And use display_task_log on the client side.
2019-12-08 10:59:47 +01:00
163e9bbe91
src/bin/proxmox-backup-client.rs: new display task log helper
...
New helper to display worker task logs. Use it for prune (experimental).
2019-12-07 16:11:26 +01:00
6b508dd563
src/server/formatter.rs: impl. new result attribute "active"
2019-12-07 15:29:42 +01:00
102d8d4136
src/backup/prune.rs: implement --keep-hourly
2019-12-07 11:23:33 +01:00
1f604f1a5a
src/api2/admin/datastore.rs: add new api test_prune()
2019-12-06 13:06:56 +01:00
0ab08ac980
src/api2/admin/datastore.rs: cleanup - move schema defininition in front of rust method
2019-12-06 12:36:41 +01:00
1a0678b601
src/api2/admin/datastore.rs: remove prune parameters from status call
...
Not idea why they are added there.
2019-12-06 12:32:53 +01:00
9e3f008804
src/backup/prune.rs: add new helper keeps_something()
2019-12-06 12:28:31 +01:00
9b78352188
src/backup/prune.rs: define new struct PruneOptions
2019-12-06 08:56:27 +01:00
92acbd69f7
src/backup/backup_info.rs: remove old prune code
2019-12-06 08:29:27 +01:00
8f0b4c1f90
src/api2/admin/datastore.rs - prune: log info about all snapshots
2019-12-05 15:35:11 +01:00
9ce42759ec
src/backup/backup_info.rs - compute_prune_list: remove unfinished backups
2019-12-05 11:18:10 +01:00
aeeac29bb1
src/backup/backup_info.rs: new compute_prune_list helper
...
So that we can write regression tests for this.
2019-12-04 15:49:11 +01:00
3d229a4a01
Revert "test another api macro use"
...
This reverts commit a9aa52e6a8
2019-12-04 14:18:11 +01:00
4f1e40a29e
src/api2/admin/datastore.rs - read_backup_index: add manifest with size
...
So that the client can display the size of the index file.
2019-12-04 10:03:52 +01:00
a9aa52e6a8
test another api macro use
...
The api macro now supports hyphens in parameter names and
referencing externally defined `Schema`s, so here's an
example.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-03 13:41:40 +01:00
41874331ed
whitespace fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:52:15 +01:00
2905f2b5e6
update api macro example usage
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:51:27 +01:00
7b6c41078b
update api macro invocation to new style
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 14:19:46 +01:00
6486cb853f
first api macro usage test/example
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 10:05:37 +01:00
329d40b50b
rename ApiHandler::Async into ApiHandler::AsyncHttp
2019-11-23 09:03:21 +01:00
ad51d02aa9
src/server/rest.rs: cleanup async code
2019-11-22 13:02:05 +01:00
cad540e969
api/compat: remove remaining api_schema references
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:53:15 +01:00
a2479cfa1a
api/compat: drop more compat imports from api_schema.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:36:28 +01:00
2194824f77
api: BoxFut -> ApiFuture
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:16:37 +01:00
3d482025b3
move api schema into proxmox::api crate
...
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:14:54 +01:00
552c225948
sort all property lookup tables
...
Required, because we use binary sreach to find items.
2019-11-21 13:32:09 +01:00
255f378a1b
use const api definitions
2019-11-21 13:32:09 +01:00
0997967d65
do no restrict index names too much
2019-11-08 10:28:14 +01:00
62ee2eb405
avoid some clippy warnings
2019-10-26 11:42:05 +02:00
11377a47bb
avoid some clippy warnings
2019-10-25 18:04:37 +02:00
2c2097ee75
fix typo in error message
2019-10-07 12:48:56 +02:00
f7aa6f15d3
src/api2/reader.rs: use tokio::fs::read() to read file async
2019-10-07 12:24:06 +02:00
4ee8f53d07
remove DataChunk file format - use DataBlob instead
2019-10-06 10:31:06 +02:00
c0b1b14c05
src/api2/reader.rs - download_chunk: use blocking io
...
Turns out to be twice as fast as tokio async io...
2019-10-05 11:41:19 +02:00
fb6026b66b
backup writer api: verify checksum inside close
2019-09-23 10:56:53 +02:00
4a7de56e2f
avoid injecting ENV vars from Makefile
...
So that we can run "cargo build" without setting vars manually.
2019-09-09 10:51:08 +02:00
ffb6434485
src/api2/reader.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
7622005574
src/api2/backup/upload_chunk.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
59b2baa0f6
src/api2/backup.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
6be147b78c
src/api2/admin/datastore.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
f3a8d1d7e0
api2/node/time: replace mem::uninitialized
...
and fixup use statements
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:14:41 +02:00
4af0ee055a
api: always use complete file names (including add exctensions)
2019-08-07 10:10:14 +02:00
d7c243977d
src/api2/admin/datastore.rs - list_snapshot_files: list all files
...
Add files that are not mentioned in index.json.
2019-08-07 09:16:14 +02:00
34066e2eeb
src/api2/backup/environment.rs - add_blob: always verify blob crc
2019-08-07 09:15:13 +02:00
f889b15867
src/backup/data_blob.rs: always compute crc
2019-08-06 11:42:14 +02:00
a17a0e7a9f
src/api2/admin/datastore.rs: extract backup size from index.json
2019-08-06 10:56:21 +02:00
8c70e3eb18
src/api2/admin/datastore.rs - files: return data from index.json
2019-08-05 13:22:19 +02:00
f69adc81a6
use new proxmox::tools::nodename
2019-08-03 17:06:23 +02:00
7f66c29e5c
remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead
2019-08-03 16:44:31 +02:00
e18a6c9ee5
update to nix 0.14, use code from proxmox:tools
2019-08-03 13:05:38 +02:00
9fdc3ef46f
src/api2/admin/datastore.rs: pass backup type/id (group) to prune
...
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
2019-07-27 08:49:14 +02:00
bbf9e7e951
src/api2/backup.rs: cleanup schema definitions
2019-07-26 09:07:29 +02:00
ca5d0b61ca
src/api2/backup.rs: new required backup-time parameter
...
The client should pass the time as parameter.
2019-07-25 13:44:01 +02:00
e128d4e84f
src/api2/admin/datastore.rs: backup logs may not be written twice
2019-07-25 12:29:04 +02:00
ef3254c086
src/api2/admin/datastore.rs: use correct .blob extension for log
2019-07-25 12:22:50 +02:00
07ee223590
src/api2/admin/datastore.rs: add api to upload backup client log file
2019-07-25 12:15:30 +02:00
15c847f136
src/api2/admin/datastore.rs: allow to list all snapshots
2019-07-18 09:11:43 +02:00
0eecf38fbf
src/api2/admin/datastore.rs: add status api call
2019-07-16 13:34:38 +02:00
b46c3fad03
src/backup/fixed_index.rs: use correct size
...
We need to consider that the last chunk may have smaller size.
2019-07-04 15:13:22 +02:00
5e04ec707d
src/api2/backup/environment.rs: improve chunk alignment/size checks
2019-07-04 13:40:43 +02:00
17243003b3
src/bin/download-speed.rs: h2 download speedtest for our server
...
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
09d7dc5024
src/api2/reader.rs: add download chunk api
2019-06-28 16:27:01 +02:00
fcfb84fedf
file download: avoid unnecessary copy
2019-06-28 07:07:52 +02:00
dd066d28e2
src/api2/reader.rs: implement backup reader protocol
2019-06-27 09:01:41 +02:00
42a87f7b96
src/server/h2service.rs: implement generic h2 service
2019-06-26 17:38:33 +02:00
f14a8c9a85
src/api2/admin/datastore.rs: use correct path for download
2019-06-25 12:59:36 +02:00
9e47c0a56e
src/api2/admin/datastore.rs: implement generic download method
...
We simply allow to download raw binary data.
2019-06-25 10:16:59 +02:00
da7d67213e
src/api2/admin/datastore/pxar.rs: remove pxar upload api
...
Upload needs to be done using the backup protocol.
2019-06-25 09:45:35 +02:00
dd8e744f9a
src/api2/admin/datastore.rs: use a worker log log prune actions
2019-06-25 08:12:25 +02:00
46bd880041
src/api2/backup/environment.rs: new helper add_blob()
2019-06-24 09:35:37 +02:00
cb08ac3efe
src/api2/backup.rs: replace upload_config with upload_blob
2019-06-23 11:44:30 +02:00
6f083b7a92
src/api2/backup/upload_chunk.rs: compute crc on upload
2019-06-23 09:04:42 +02:00
bffd40d6b7
src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools
2019-06-14 11:40:04 +02:00
00388226e6
src/api2/backup/environment.rs: log index checksums
2019-06-14 10:36:20 +02:00
36075475a6
src/api2/backup/environment.rs: log duplicate chunks
2019-06-14 07:12:30 +02:00
fa148dbd11
src/api2/backup/upload_chunk.rs: verify chunk data if possible
2019-06-14 06:23:46 +02:00
f98ac774ee
backup: Add support for client side encryption
...
first try ...
2019-06-13 11:47:23 +02:00
6762db70d6
src/api2/types.rs: define CHUNK_DIGEST_SCHEMA
2019-06-13 07:30:42 +02:00
dd5495d6dc
tree-wide: use 'dyn' for all trait objects
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
4800df7ba6
src/api2/admin/datastore.rs: remove test-upload api
2019-06-07 07:52:11 +02:00
113898d7e9
src/api2/admin/datastore/upload.rs: remove unused upload api
2019-06-07 07:39:41 +02:00
986bef16be
src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
...
So that we can include it in static doc strings.
2019-06-05 08:41:20 +02:00
Dominik Csapak
Wolfgang Bumiller
Stefan Reiter
Fabian Grünbichler
Dietmar Maurer
Thomas Lamprecht
Oguz Bektas