a9767cf7de
gc: remove .bad files on garbage collect
...
The iterator of get_chunk_iterator is extended with a third parameter
indicating whether the current file is a chunk (false) or a .bad file
(true).
Count their sizes to the total of removed bytes, since it also frees
disk space.
.bad files are only deleted if the corresponding chunk exists, i.e. has
been rewritten. Otherwise we might delete data only marked bad because
of transient errors.
While at it, also clean up and use nix::unistd::unlinkat instead of
unsafe libc calls.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-09-08 12:43:13 +02:00
15ec790a40
tools/systemd/time: convert the resulting timestamp into an option
...
we want to use dates for the calendarspec, and with that there are some
impossible combinations that cannot be detected during parsing
(e.g. some datetimes do not exist in some timezones, and the timezone
can change after setting the schedule)
so finding no timestamp is not an error anymore but a valid result
we omit logging in that case (since it is not an error anymore)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-09-04 15:28:05 +02:00
ecd55041a2
fix #2978 : allow non-root to view datastore usage
...
for datastores where the requesting user has read or write permissions,
since the API method itself filters by that already. this is the same
permission setting and filtering that the datastore list API endpoint
does.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-04 06:18:20 +02:00
63d9aca96f
verify: log progress
2020-09-02 07:43:28 +02:00
4f09d31085
src/backup/verify.rs: use global hashes (instead of per group)
...
This makes verify more predictable.
2020-09-01 13:33:04 +02:00
6b809ff59b
src/backup/verify.rs: use separate thread to load data
2020-09-01 12:56:25 +02:00
3b2046d263
save last verify result in snapshot manifest
...
Save the state ("ok" or "failed") and the UPID of the respective
verify task. With this we can easily allow to open the relevant task
log and show when the last verify happened.
As we already load the manifest when listing the snapshots, just add
it there directly.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-26 07:35:13 +02:00
1ffe030123
various typo fixes
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 18:52:31 +02:00
5255e641fa
SnapshotListItem: add comment field also to schema
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 16:24:36 +02:00
5a718dce17
api datastore: fix typo in error message
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-08-25 14:16:40 +02:00
a1e9c05738
api2/node/services: turn service api calls into workers
...
to be in line with pve/pmg and be able to show the progress in the gui
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 12:37:17 +02:00
98161fddb5
cleanup last patch
2020-08-14 07:30:05 +02:00
be614c625f
api2/node/../disks/directory: added DELETE endpoint for removal of mount-units
...
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-08-14 07:06:10 +02:00
93bb51fe7e
config/jobstate: replace Job:load with create_state_file
...
it really is not necessary, since the only time we are interested in
loading the state from the file is when we list it, and there
we use JobState::load directly to avoid the lock
we still need to create the file on syncjob creation though, so
that we have the correct time for the schedule
to do this we add a new create_state_file that overwrites it on creation
of a syncjob
for safety, we subtract 30 seconds from the in-memory state in case
the statefile is missing
since we call create_state_file from proxmox-backup-api,
we have to chown the lock file after creating to the backup user,
else the sync job scheduling cannot aquire the lock
also we remove the lock file on statefile removal
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 06:38:02 +02:00
713b66b6ed
cleanup: replace id from do_sync_job with info from job
...
we already have it inside the job itself
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 06:36:43 +02:00
77bd2a469c
cleanup: merge endtime into TaskState
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-14 06:36:19 +02:00
02543a5c7f
api2/pull: extend do_sync_job to also handle schedule and jobstate
...
so that we can log if triggered by a schedule, and writing to a jobstatefile
also correctly polls now the abort_future of the worker, so that
users can stop a sync
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:49:28 +02:00
42b68f72e6
api/{pull, sync}: refactor to do_sync_job
...
and move the pull parameters into the worker, so that the task log
contains the error if there is one
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:40:52 +02:00
664d8a2765
api2/admin/sync: use JobState for faster access to state info
...
and delete the statefile again on syncjob removal
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:40:00 +02:00
ae197dda23
server/worker_task: let upid_read_status also return the endtime
...
the endtime should be the timestamp of the last log line
or if there is no log at all, the starttime
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:35:44 +02:00
4c116bafb8
server: change status of a task from a string to an enum
...
representing a state via an enum makes more sense in this case
we also implement FromStr and Display to make it easy to convet from/to
a string
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-08-13 11:35:19 +02:00
72dc68323c
replace and remove old ticket functions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-12 14:28:21 +02:00
81f293513e
backup: lock base snapshot and ensure existance on finish
...
To prevent forgetting the base snapshot of a running backup, and catch
the case when it still happens (e.g. via manual rm) to at least error
out instead of storing a potentially invalid backup.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-11 11:04:47 +02:00
8b5f72b176
Revert "backup: ensure base snapshots are still available after backup"
...
This reverts commit d53fbe2474
2020-08-11 11:03:53 +02:00
f23f75433f
backup: flock snapshot on backup start
...
An flock on the snapshot dir itself is used in addition to the group dir
lock. The lock is used to avoid races with forget and prune, while
having more granularity than the group lock (i.e. the group lock is
necessary to prevent more than one backup per group, but the snapshot
lock still allows backups unrelated to the currently running to be
forgotten/pruned).
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-11 11:02:21 +02:00
882c082369
mark signed manifests as such
...
for less-confusing display in the web interface
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:56:53 +02:00
14f6c9cb8b
chunk readers: ensure chunk/index CryptMode matches
...
an encrypted Index should never reference a plain-text chunk, and an
unencrypted Index should never reference an encrypted chunk.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:54:22 +02:00
2d55beeca0
datastore api: verify blob/index csum from manifest
...
when dowloading decoded files.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:52:45 +02:00
9238cdf50d
datastore api: only decode unencrypted indices
...
these checks were already in place for regular downloading of backed up
files, also do them when attempting to decode a catalog, or when
downloading decoded files referenced by a pxar index.
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-08-11 09:51:20 +02:00
5d30f03826
impl PartialEq between Realm and RealmRef
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:23:36 +02:00
14263ef989
assert that Username does not impl PartialEq
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:21:12 +02:00
e7cb4dc50d
introduce Username, Realm and Userid api types
...
and begin splitting up types.rs as it has grown quite large
already
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 12:05:01 +02:00
866c556faf
move types.rs to types/mod.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-10 10:32:31 +02:00
4dbe129284
backup: only allow finished backups as base snapshot
...
If the datastore holds broken backups for some reason, do not attempt to
base following snapshots on those. This would lead to an error on
/previous, leaving the client no choice but to upload all chunks, even
though there might be potential for incremental savings.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-08-07 07:32:56 +02:00
d74edc3d89
finish_backup: mark backup as finished only after checks have passed
...
Commit 9fa55e09
2020-08-06 06:39:34 +02:00
98c259b4c1
remove timer and lock functions, fix building with proxmox 0.3.2
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-04 11:33:02 +02:00
70030b43d0
list_snapshots: Returns new "comment" property (fisrt line from notes)
2020-07-31 11:34:42 +02:00
ff86ef00a7
cleanup: manifest is always CryptMode::None
2020-07-31 10:25:30 +02:00
912b3f5bc9
src/api2/admin/datastore.rs: add API to get/set Notes for backus
2020-07-31 10:17:35 +02:00
d7ee07d838
src/api2/backup/environment.rs: remove debug code
2020-07-31 07:48:53 +02:00
53705acece
src/api2/backup/environment.rs: remove debug code
2020-07-31 07:47:08 +02:00
c8fff67d88
finish_backup: add chunk_upload_stats to manifest
2020-07-31 07:45:47 +02:00
9fa55e09a7
finish_backup: test/verify manifest at server side
...
We want to make sure that the client uploaded a readable manifest.
2020-07-31 07:45:47 +02:00
f39a900722
api2/node/termproxy: fix user in worker task
...
'username' here is without realm, but we really want to use user@realm
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-30 11:57:43 +02:00
1fc82c41f2
src/api2/backup.rs: aquire backup lock earlier in create_locked_backup_group()
2020-07-30 11:03:05 +02:00
d2b0c78e23
api2/node/termproxy: fix zombies on worker abort
...
tokios kill_on_drop sometimes leaves zombies around, especially
when there is not another tokio::process::Command spawned after
so instead of relying on the 'kill_on_drop' feature, we explicitly
kill the child on a worker abort. to be able to do this
we have to use 'tokio::select' instead of 'futures::select' since
the latter requires the future to be fused, which consumes the
child handle, leaving us no possibility to kill it after fusing.
(tokio::select does not need the futures to be fused, so we
can reuse the child future after the select again)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-30 10:38:14 +02:00
adfdc36936
verify: keep track and log which dirs failed the verification
...
so that we can print a list at the end of the worker which backups
are corrupt.
this is useful if there are many snapshots and some in between had an
error. Before this patch, the task log simply says to 'look in the logs'
but if the log is very long it makes it hard to see what exactly failed.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-30 09:39:37 +02:00
d8594d87f1
verify: keep also track of corrupt chunks
...
so that we do not have to verify a corrupt one multiple times
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-30 09:39:37 +02:00
d53fbe2474
backup: ensure base snapshots are still available after backup
...
This should never trigger if everything else works correctly, but it is
still a very cheap check to avoid wrongly marking a backup as "OK" when
in fact some chunks might be missing.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 08:28:54 +02:00
95bda2f25d
backup: use flock on backup group to forbid multiple backups at once
...
Multiple backups within one backup group don't really make sense, but
break all sorts of guarantees (e.g. a second backup started after a
first would use a "known-chunks" list from the previous unfinished one,
which would be empty - but using the list from the last finished one is
not a fix either, as that one could be deleted or pruned once the first
simultaneous backup is finished).
Fix it by only allowing one backup per backup group at one time. This is
done via a flock on the backup group directory, thus remaining intact
even after a reload.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 08:26:26 +02:00
c9756b40d1
datastore: prevent deletion of snaps in use as "previous backup"
...
To prevent a race with a background GC operation, do not allow deletion
of backups who's index might currently be referenced as the "known chunk
list" for successive backups. Otherwise the GC could delete chunks it
thinks are no longer referenced, while at the same time telling the
client that it doesn't need to upload said chunks because they already
exist.
Additionally, prevent deletion of whole backup groups, if there are
snapshots contained that appear to be currently in-progress. This is
currently unlikely to trigger, as that function is only used for sync
jobs, but it's a useful safeguard either way.
Deleting a single snapshot has a 'force' parameter, which is necessary
to allow deleting incomplete snapshots on an aborted backup. Pruning
also sets force=true to avoid the check, since it calculates which
snapshots to keep on its own.
To avoid code duplication, the is_finished method is factored out.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-30 08:26:01 +02:00
2aaae9705e
src/backup/verify.rs: try to verify chunks only once
...
We use a HashSet (per BackupGroup) to track already verified chunks.
2020-07-29 13:29:13 +02:00
8aa67ee758
bump proxmox to 0.3, cleanup http_err macro usage
...
Also swap the order of a couple of `.map_err().await` to
`.await.map_err()` since that's generally more efficient.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 09:38:36 +02:00
3865e27e96
src/api2/node.rs: 'mod' statement cleanup
...
split them into groups: `pub`, `pub(crate)` and non-pub
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 09:19:57 +02:00
7fde1a71ca
upload_chunk: allow upload of empty blobs
...
a blob can be empty (e.g. an empty pct fw conf), so we
have to set the minimum size to the header size
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-28 11:03:36 +02:00
39f18b30b6
src/backup/data_blob.rs: new load_from_reader(), which verifies the CRC
...
And make verify_crc private for now. We always call load_from_reader() to
verify the CRC.
Also add load_chunk() to datastore.rs (from chunk_store::read_chunk())
2020-07-28 10:23:16 +02:00
e6b599aa6c
services: make reload safer and default to it in gui
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-25 20:23:12 +02:00
ee15af6bb8
api: service command: fix test for essential service
...
makes no sense to disallow reload or start (even if start cannot
really happen)
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-24 19:35:19 +02:00
33a88dafb9
server/state: add spawn_internal_task and use it for websockets
...
is a helper to spawn an internal tokio task without it showing up
in the task list
it is still tracked for reload and notifies the last_worker_listeners
this enables the console to survive a reload of proxmox-backup-proxy
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-24 11:17:33 +02:00
224c65f8de
termproxy: let users stop the termproxy task
...
for that we have to do a select on the workers abort_future
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-24 11:17:33 +02:00
3d3670d786
termproxy: cmd: support upgrade
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 13:12:18 +02:00
e744de0eb0
api: termproxy: fix ACL as /nodes is /system
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 12:57:11 +02:00
98b1733760
api: apt: use schema default const for quiet param
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 12:25:28 +02:00
1c2f842a98
api2/nodes: add termproxy and vncwebsocket api calls
...
Even though it has nothing to do with vnc, we keep the name of the api
call for compatibility with our xtermjs client.
termproxy:
verifies that the user is allowed to open a console and starts
termproxy with the correct parameters
starts a TcpListener on "localhost:0" so that the kernel decides the
port (instead of trying to rerserving like in pve). Then it
leaves the fd open for termproxy and gives the number as port
and tells it via '--port-as-fd' that it should interpret this
as an open fd
the vncwebsocket api call checks the 'vncticket' (name for compatibility)
and connects the remote side (after an Upgrade) with a local TcpStream
connecting to the port given via WebSocket from the proxmox crate
to make sure that only the client can connect that called termproxy and
no one can connect to an arbitrary port on the host we have to include
the port in the ticket data
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-23 12:06:38 +02:00
a4d1675513
api2/access: implement term ticket
...
modeled after pves/pmgs vncticket (i substituted the vnc with term)
by putting the path and username as secret data in the ticket
when sending the ticket to /access/ticket it only verifies it,
checks the privs on the path and does not generate a new ticket
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-23 11:55:00 +02:00
27fde64794
api: apt update must run protected
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 11:45:52 +02:00
fa3f0584bb
api: apt: support refreshing package index
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-23 11:21:54 +02:00
a4e86972a4
add .../apt/update API call
...
Depends on patched apt-pkg-native-rs. Changelog-URL detection is
inspired by PVE perl code for now, though marked with fixme to use 'apt
changelog' later on, if/when our repos have APT-compatible changelogs
set up.
list_installed_apt_packages iterates all packages and creates an
APTUpdateInfo with detailed information for every package matched by the
given filter Fn.
Sadly, libapt-pkg has some questionable design choices regarding their
use of 'iterators', which means quite a bit of nesting...
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-23 10:41:14 +02:00
91f5594c08
api: translate ENOTFOUND to 404 for downloads
...
and percolate the HttpError back up on the client side
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-07-22 09:19:29 +02:00
113c9b5981
move subscription API path to /nodes
...
This aligns it with PVE and allows the widget toolkit's update window
"refresh" to work without modifications once POST /apt/update is
implemented.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-07-21 19:33:52 +02:00
a26c27c8e6
api2/status: fix estimation bug
...
when a datastore has enough data to calculate the estimated full date,
but always has exactly the same usage, the factor b of the regression
is '0'
return 0 for that case so that the gui can show 'never' instead of
'not enough data'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-21 13:02:08 +02:00
bfcef26a99
api2/node/status: add fingerprint
...
and rename get_usage to get_status (since its not usage only anymore)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-10 11:08:53 +02:00
660a34892d
update proxmox crate to 0.2.0
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-10 11:08:27 +02:00
7957fabff2
api: add ZPOOL_NAME_SCHEMA and regex
...
poolnames can containe spaces and some other special characters
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-07-09 13:37:31 +02:00
521a0acb2e
DataStore::load_manifest: also return CryptMode
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:19:53 +02:00
f28d9088ed
introduce a CryptMode enum
...
This also replaces the recently introduced --encryption
parameter on the client with a --crypt-mode parameter.
This can be "none", "encrypt" or "sign-only".
Note that this introduces various changes in the API types
which previously did not take the above distinction into
account properly:
Both `BackupContent` and the manifest's `FileInfo`:
lose `encryption: Option<bool>`
gain `crypt_mode: Option<CryptMode>`
Within the backup manifest itself, the "crypt-mode" property
will always be set.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-07 15:24:19 +02:00
08cb2038bd
api: disks: indentation fixup
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-07-03 17:59:30 +02:00
2162e2c15d
src/api2/admin/datastore.rs: avoid slash in UPID strings
2020-06-30 13:11:22 +02:00
fe3e65c3ea
src/api2/backup.rs: call register_chunk in previous download api
2020-06-26 08:22:46 +02:00
b957aa81bd
update backup api for incremental backup
...
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-26 07:17:08 +02:00
8ea00f6e49
allow to abort verify jobs
...
And improve job description rendering on gui.
2020-06-25 12:56:36 +02:00
e22f4882e7
extract create_download_response API helper
...
and put it into a new "api2::helpers" module.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-25 11:57:37 +02:00
c2009e5309
src/api2/admin/datastore.rs: add verify api
2020-06-24 13:35:21 +02:00
facd9801cf
add incremental backup support
...
To support incremental backups (where not all chunks are sent to the
server), a new parameter "reuse-csum" is introduced on the
"create_fixed_index" API call. When set and equal to last backups'
checksum, the backup writer clones the data from the last index of this
archive file, and only updates chunks it actually receives.
In incremental mode some checks usually done on closing an index cannot
be made, since they would be inaccurate.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 10:01:25 +02:00
21302088de
remove debug println
2020-06-24 09:15:13 +02:00
8268c9d161
fix overflow panic during upload
...
if *only* data chunks are registered (high chance during incremental
backup), then chunk_count might be one lower then upload_stat.count
because of the zero chunk being unconditionally uploaded but not used.
Thus when subtracting the two, an overflow would occur.
In general, don't let the client make the server panic, instead just set
duplicates to 0.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-06-24 09:07:22 +02:00
b91b7d9ffd
api2/node/disks/zfs: check if default zfs mount path exists
...
and if it does bail, because otherwise we would get an
error on mounting and have a zpool that is not imported
and disks that are used
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:22:39 +02:00
d33d8f4e6a
api2/admin/datastore: add pxar-file-download api call
...
streams a file from a pxar file of an unencrypted backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:21:15 +02:00
5b1cfa01f1
api2/admin/datastore: add 'catalog' api call
...
returns the dir listing of the given filepath of the backup snapshot
the filepath has to be base64 encoded or 'root'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-24 07:16:12 +02:00
60f9a6ea8f
src/backup/datastore.rs: add new helpers to load blobs and verify chunks
2020-06-24 06:58:14 +02:00
cbef49bf4f
remove absolute paths when executing binaries
...
we set the paths manually, so this is ok
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 07:09:06 +02:00
f386f512d0
add AsyncReaderStream
...
and replace AsyncIndexReader's stream implementation with that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-23 06:33:31 +02:00
42d19fdf69
src/api2/node/disks/zfs.rs: always set pool name
2020-06-20 07:15:32 +02:00
4e37d9ce67
add general indented_list_to_tree implementation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-19 14:37:40 +02:00
6ef9bb59eb
api2/admin/datastore: add download-decoded endpoint
...
similar to 'download', but streams the decoded file
when it is not encrypted
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:39:15 +02:00
abdb976340
add Display trait to BackupDir
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 08:28:35 +02:00
1c090810f5
api2/admin/datastore/snapshos: show encrypted and size info per file
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:39:56 +02:00
e181d2f6da
add encrypted info to Manifest
...
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 07:35:39 +02:00
ba694720fc
api2/admin/datastore: log stream error during file download
...
the client cannot get an error during an chunked http transfer, so at
least log it server side
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-19 06:58:55 +02:00
026dc1d11f
src/api2/node/disks/zfs.rs: add zpool_details api
2020-06-18 15:04:46 +02:00
3f851d1321
src/api2/node/disks/directory.rs: add early check if disk is unused
2020-06-17 08:31:11 +02:00
1aef491e24
src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks
2020-06-17 08:07:54 +02:00
66af7f51bc
src/tools/disks/zfs.rs: make zfs list parser private
2020-06-17 07:00:54 +02:00
fb5a066500
src/api2/node/disks.rs: expose directory api
2020-06-16 13:36:32 +02:00
929a13b357
src/api2/node/disks/zfs.rs: add zpool api
2020-06-16 13:25:53 +02:00
fbbcd85839
src/api2/node/disks/directory.rs: implement add-datastore feature
2020-06-15 10:01:50 +02:00
0196b9bf5b
remove unnecessary .into
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:57:58 +02:00
20b3094bcb
api2/status: add task list endpoint
...
for now mostly copy/paste from nodes/nodename/tasks
(without the parameters)
but we should replace the 'read_task_list' with a method
that gives us the tasks since some timestamp
so that we can get a longer list of tasks than for the node
(we could of course embed this then in the nodes/node/task api call and
remove this again as long as the api is not stable)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:17 +02:00
df528ee6fa
implement From<TaskListInfo> for TaskListItem
...
and use it where its convenient
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 14:50:12 +02:00
476b4acadc
BackupEnvironment: do not set finished flag prematurely
...
we check if all dynamic_writers are closed and if the backup contains
any valid files, we can only mark the backup finished after those
checks, else the backup task gets marked as OK, even though it
is not finished and no cleanups run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:52 +02:00
ec8f042459
api2/status: use new rrd::extract_cached_data
...
and drop the now unused extract_lists function
this also fixes a bug, where we did not add the datastore to the list at
all when there was no rrd data
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:16 +02:00
431cc7b185
rrd: move creation of serde value into api
...
there is now a 'extract_cached_data' which just returns
the data of the specified field, and an api function that converts
a list of fields to the correct serde value
this way we do not have to create a serde value in rrd/cache.rs
(makes for a better interface)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 13:31:14 +02:00
d4f2397d4c
add api to format disks and create datastores
2020-06-10 11:03:36 +02:00
bda48e04da
api2: add status/datastore-usages api call
...
returns a list of the datastores and their usages, a list of usages of
the past month (for the gui) and an estimation of when its full
(using the linear regression)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:20:25 +02:00
33070956af
let disk_usage return StorageStatus and use it for datastores/nodes
...
disk_usage returned the same values as defined in StorageStatus,
so simply use that
with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-09 12:05:39 +02:00
c443f58b09
switch to external pxar and fuse crates
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-08 13:56:58 +02:00
2b388026f8
src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node
2020-06-08 07:20:59 +02:00
707974fdb3
src/api2/node/disks.rs: implement initgpt API
2020-06-07 10:30:34 +02:00
9069debcd8
src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA
2020-06-07 07:20:25 +02:00
ce8e3de401
move disks api to /node/<node>/disks
2020-06-06 14:43:36 +02:00
7fa2779559
src/api2/disks.rs: implement smart api
2020-06-06 12:23:11 +02:00
ff30caeaf8
src/api2/disks.rs - list-disks: add usage-type filter
2020-06-06 11:48:58 +02:00
553cd12ba6
src/api2/disks.rs: start disks api
2020-06-06 11:38:47 +02:00
8d78589969
improve display of 'next run' for sync jobs
...
if the last sync job is too far in the past (or there was none at all
for now) we run it at the next iteration, so we want to show that
we now calculate the next_run by using either the real last endtime
as time or 0
then in the frontend, we check if the next_run is < now and show 'pending'
(we do it this way also for replication on pve)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-06-04 07:03:54 +02:00
add5861e8d
typo fixes all over the place
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-30 16:39:08 +02:00
96d65fbcd0
cleanup: define/use const for predefined blob file names.
2020-05-30 14:04:15 +02:00
04b0ca8b59
add owner to group and snapshot listings
...
while touching it, make columns and tbar in DataStoreContent.js
declarative members and remove the (now) unnecessary initComponent
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-30 07:24:12 +02:00
d43f86f3f3
api2: add admin/sync endpoint
...
this returns the list of syncjobs with status, as opposed to
config/sync (which is just the config)
also adds an api call where users can run the job manually under
/admin/sync/$ID/run
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:31:32 +02:00
268687ddf0
api2/pull: refactor priv checking and creating pull parameters
...
we want to reuse those in the api call for manually running a sync job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:27:43 +02:00
426c1e353b
api2/config/sync: fix id parameter
...
'name' is not the correct parameter for get/post
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:54 +02:00
2888b27f4c
create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 11:24:25 +02:00
9857472211
fix removing of remotes
...
we have to save the remote config after removing the section
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-29 10:48:26 +02:00
090decbe76
BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names
...
The repo URL consists of
* optional userid
* optional host
* datastore name
All three have defined regex or format, but none of that is used, so
for example not all valid datastore names are accepted.
Move definition of the regex over to api2::types where we can access
all required regexes easily.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:29:23 +02:00
c32186595e
api2::types: factor out USER_ID regex
...
allows for better reuse in a next patch
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-05-29 06:27:38 +02:00
c94e1f655e
rrd stats: improve io delay stats
2020-05-28 19:12:13 +02:00
91e5bb49f5
src/bin/proxmox-backup-proxy.rs: simplify code
...
and gather all stats for the root disk
2020-05-28 12:30:54 +02:00
de4db62c57
remotes: save passwords as base64
...
to avoid having arbitrary characters in the config (e.g. newlines)
note that this breaks existings configs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 12:38:06 +02:00
1a0d3d11d2
src/api2/admin/datastore.rs: add rrd api
2020-05-26 12:26:14 +02:00
83fd4b3b1b
remote: try to use Struct for api
...
with a catch: password is in the struct but we do not want it to return
via the api, so we only 'serialize' it when the string is not empty
(this can only happen when the format is not checked by us, iow.
when its returned from the api) and setting it manually to ""
when we return remotes from the api
this way we can still use the type but do not return the password
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:55:07 +02:00
5dcdcea293
api2/config/remote: remove password from read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:49:12 +02:00
99f443c6ae
api2/config/remote: lock and use digest for removal
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:45 +02:00
4f966d0592
api2/config/remote: use rpcenv for digest for read_remote
...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:28 +02:00
db0c228719
config/remote: add 'name' to Remote struct
...
and use it as section id, like with User
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-26 08:48:05 +02:00
d8dc281992
www/DataStoreStatus.js: display loadavg stats
2020-05-25 11:54:15 +02:00
3e930f2bdc
www/DataStoreStatus.js: display root disk stats
2020-05-25 11:34:24 +02:00
c1b24fbf0b
www/DataStoreStatus.js: display swap stats
2020-05-25 10:39:54 +02:00
a608806f65
www/DataStoreStatus.js: display netin/netout
2020-05-24 19:02:35 +02:00
803ab12ad4
rrd: simplify code
2020-05-23 15:37:17 +02:00
a4a3f7ca5e
rrd: pack multiple rrd values into th estat list
2020-05-23 14:03:44 +02:00
a2f862eed6
add experimental rrd api to get cpu stats
2020-05-23 11:50:53 +02:00
07ad6470ca
src/client/pull.rs: split out pull related code
2020-05-22 08:04:20 +02:00
183125d576
src/api2/pull.rs: aquire try_shared_chunk_store_lock inside pull_store
2020-05-22 07:24:17 +02:00
6f652b1b3a
rename 'job' to 'sync'
2020-05-21 10:29:25 +02:00
b1d4edc769
src/api2/config/job.rs: add job api
2020-05-21 10:16:35 +02:00
b4900286ce
src/config/jobs.rs: use SectionConfig for jobs
2020-05-21 10:16:35 +02:00
2882c881e9
api2/access/acl: add path and exact parameter to list_acl
...
so that we can get only a subset of the acls, filtered by the backed
also return the digest here
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:44:36 +02:00
12e3895399
api2/access/acl: make update_acl a protected api call
...
since we want to set the owner of the acl config to 'root'
which is only possible when using a protected api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:22:41 +02:00
11b6391c83
add 'exact' parameter to extract_acl_node_data
...
so that we can return acls for a single path
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:22:10 +02:00
b05672579e
api2/roles: change return field of role to roleid
...
to be compatible with the pve api
with this, we can reuse the ui parts (RoleSelector)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:47 +02:00
5160c0e986
api2/acl: add privs array to roles
...
so that an admin can see which roles have which privileges
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-20 13:21:37 +02:00
872062ee9f
src/config/datastore.rs_ change prune types from i64 to u64
2020-05-20 13:00:13 +02:00
67f7ffd0db
src/config/datastore.rs: add prune settings
2020-05-20 11:29:59 +02:00
0fafac2492
src/api2/access/user.rs: remove useless description
...
The description is not used at all if we refer to a type.
2020-05-20 11:27:58 +02:00
49ff10921c
src/api2/types.rs: define PRUNE_SCHEMA_KEEP_*
2020-05-20 10:13:38 +02:00
42fdbe5112
src/config/datastore.rs: add gc-schedule property
2020-05-20 08:38:10 +02:00
7d4e362993
depend on proxmox 0.1.32, src/api2/access/user.rs: simplify code
2020-05-19 12:58:46 +02:00
522c0da0a0
use new 'id_property' for user::User and use it in api calls
...
this allows us to return a user::User (or Vec<> of it)
instead of a generic serde value
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-05-19 09:33:56 +02:00
e8d1da6a15
depend on proxmox 0.1.31 - use Value to store result metadata
2020-05-18 09:57:35 +02:00
30f577248b
src/api2/node/time.rs: avoid custom unsafe readlink implementations
2020-05-15 06:50:07 +02:00
f486e9e50e
add systemd configuration file parser/writer, start job configuration
2020-05-12 13:07:49 +02:00
3dd27a3bf8
src/api2/node/network.rs: add protected flag to revert
2020-05-08 09:30:25 +02:00
3aedb73816
src/api2/node/network.rs: pass bridge_ports and slaves a property strings
...
To make it compatible with pve.
2020-05-08 08:49:17 +02:00
bab5d18c3d
src/config/network.rs: implement bond_mode
...
and rename bond_slaves to slaves to make it compatible with pve.
2020-05-07 14:07:45 +02:00
c2ffc68554
src/api2/node/network.rs: cleanup - factor out check_duplicate_gateway
2020-05-07 11:26:30 +02:00
9651833130
src/api2/node/network.rs: allow to create bridge and bond
2020-05-07 11:09:12 +02:00
7b22acd0c2
src/config/network.rs: make it compatible with pve
...
and depend on proxmox 0.1.26
2020-05-07 09:28:25 +02:00
5751e49566
src/server/worker_task.rs: implement and use status command
2020-05-07 09:27:33 +02:00
99641a6bbb
garbage_collect: call fail_on_abort to abort GV when requested.
2020-05-05 09:06:34 +02:00
db1e061dcb
src/bin/proxmox-backup-client.rs: correctly format prune result list.
2020-05-05 06:45:37 +02:00
ed751dc2ab
src/api2/node/status.rs: rework api, implement reboot and shutdown
2020-04-30 11:52:40 +02:00
ca9dfe5fa4
src/api2/node/tasks.rs: use api macro features for default values
2020-04-30 11:51:56 +02:00
720af9f69b
src/api2/node/tasks.rs: allow users to list/access there own tasks
2020-04-30 10:05:50 +02:00
f1490da82a
use resonable acl paths (fixup)
2020-04-30 09:32:13 +02:00
74c08a5782
use reasonable acl paths
2020-04-30 09:30:00 +02:00
bc0d03885c
use proxmox 0.1.25, use new EnumEntry feature
2020-04-29 13:01:24 +02:00
30fb602578
src/api2/admin/datastore.rs - get_datastore_list: only return name and comment
...
We dont want to leak the full configuration to users with limited access permission.
Please use the api2::config::datastore api to get the full configuration.
2020-04-29 09:21:34 +02:00
0a00f6e01c
src/api2/config/datastore.rs_ add delete property to update method
2020-04-29 09:09:59 +02:00
30003baaa4
src/api2/config/remote.rs: fix white space
2020-04-29 09:09:39 +02:00
5211705ff1
src/api2/config/remote.rs: add delete parameter to update method
2020-04-29 09:04:17 +02:00
ec67af9af3
src/api2/pull.rs: require Datastore.Prune if delete flag is set.
2020-04-29 07:19:32 +02:00
8247db5b39
src/config/acl.rs: introduice privileges and roles for remotes
2020-04-29 07:03:44 +02:00
409f44247b
fix api2::types::ACL_ROLE_SCHEMA
...
make sure we list all roles ...
2020-04-28 13:25:02 +02:00
54552dda59
implemnt backup ownership, improve datastore access permissions
2020-04-28 10:22:25 +02:00
d00e1a216f
src/config/acl.rs: introduce more/better datastore privileges
2020-04-27 07:13:50 +02:00
9c7fe29dfc
src/config/acl.rs: rtename PRTIV_DATASTORE_ALLOCATE to PRIV_DATASTORE_MODIFY
2020-04-27 06:50:35 +02:00
14627d671a
src/bin/proxmox-backup-manager.rs: add dns sub command
...
Also improved the DNS api, added a --delete option.
2020-04-26 08:23:23 +02:00
26d9aebc28
move src/api2/config/network.rs to src/api2/node/network.rs
...
So that we have the same api path for network config as pve.
2020-04-25 17:00:38 +02:00
fd7c0979b4
src/bin/proxmox-backup-manager.rs: implement netwerk revert
2020-04-24 10:45:49 +02:00
2eefd9aee1
src/config/network.rs: implement network reload, set "changes" attribute
2020-04-24 09:55:46 +02:00
8a6b86b8a7
src/config/network.rs: use a simple String for comments
2020-04-24 07:46:08 +02:00
10a9be45bd
src/api2/config/network.rs: implement update/delete comments
2020-04-23 16:08:35 +02:00
5f60a58fd5
src/config/network.rs; support interface comments, cleanups
2020-04-23 15:54:30 +02:00
5e4e88e83f
src/api2/config/network.rs: implement update/delete for bridge_ports and bond_slaves
2020-04-23 11:21:27 +02:00
339965d720
src/api2/config/network.rs: only allow one default gateway
2020-04-23 10:37:40 +02:00
42fbe91a34
src/config/network.rs: parse bond-slaves
2020-04-23 09:31:10 +02:00
1d9a68c2fc
src/config/network.rs: parse bridge-ports
2020-04-23 09:24:17 +02:00
02269f3dba
src/config/network.rs: introduce NetworkInterfaceType
2020-04-23 08:45:03 +02:00
2c18efd902
src/config/network.rs: use a single mtu setting (instead of mtu_v4 and mtu_v6)
2020-04-23 07:07:14 +02:00
b1564af25a
src/bin/proxmox-backup-manager.rs: format datastore list output
2020-04-22 17:37:20 +02:00
f1026a5aa9
src/api2/config/network.rs: allow to update 'auto' flag
2020-04-22 16:46:46 +02:00
3fce3bc36e
src/config/network/parser.rs: parse MTU settings
2020-04-22 13:44:51 +02:00
c016482c7a
src/api2/config/network.rs: implement delete property
2020-04-22 12:19:31 +02:00
27f2c23049
src/api2/config/network.rs: allow to update configuration method
2020-04-22 11:32:36 +02:00
df6bb03d0e
src/api2/config/network.rs: improve network api
2020-04-22 10:54:07 +02:00
76cf5208cf
src/api2/types.rs: add schemas for IP/CIDR
2020-04-22 10:28:53 +02:00
2ea7bf1b3d
src/api2/config/datastore.rs_ fix method docs
2020-04-22 08:53:16 +02:00
8b57cd4441
src/config/network.rs: remove netmask support
...
rely on cidr instead.
2020-04-22 08:45:13 +02:00
68da20bf62
src/api2/types.rs: define NETWORK_INTERFACE_NAME_SCHEMA
2020-04-21 17:54:52 +02:00
c357260d09
src/config/network.rs: move type definitions to src/api2/types.rs
2020-04-21 17:25:05 +02:00
ca0e534796
src/api2/config/network.rs: start network configuration api
2020-04-21 14:28:26 +02:00
365f0f720c
fix permission tests using non-uri parameters
...
We nood to do those tests inside the fuction body instead...
2020-04-18 08:23:04 +02:00
bb072ba49c
src/api2/access.rs: cleanup
2020-04-18 07:28:25 +02:00
ff329f970b
src/api2/types.rs: use anyhow::Error in test cases
2020-04-18 07:05:31 +02:00
f7d4e4b506
switch from failure to anyhow
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-04-17 18:43:30 +02:00
404d78c41e
src/api2/pull.rs: add access permission
2020-04-17 15:27:04 +02:00
1bfc1efa50
src/api2/subscription.rs: add access permissions
2020-04-17 15:14:28 +02:00
73ce1d1146
src/api2/reader.rs: add access permissions
2020-04-17 15:01:56 +02:00
70e5f2461d
src/api2/config/remote.rs: add access permissions
2020-04-17 14:57:26 +02:00
c0ef209aeb
src/api2/config/datastore.rs: impl digest check for delete, add access permissions
2020-04-17 14:51:29 +02:00
9f9f7eefa3
src/api2/backup.rs: add access permissions
2020-04-17 14:40:20 +02:00
bb34b58910
src/api2/admin/datastore.rs: add access permissions - first try
...
We need to refine this later (introduce backup owner concept?)
2020-04-17 14:36:27 +02:00
5972def5ec
acl: change path "storage" to "datastore"
2020-04-17 14:15:44 +02:00
aa90ced3bf
src/api2/access/role.rs: use schema ACL_ROLE_SCHEMA
2020-04-17 14:14:06 +02:00
ca257c8097
move type defs from src/api2/access/acl.rs to src/api2/types.rs
2020-04-17 14:13:15 +02:00
3fff55b293
src/api2/access/role.rs: new api to list roles
2020-04-17 14:03:24 +02:00
4f66423fcc
src/api2/access/user.rs: add access permissions
2020-04-17 11:04:36 +02:00
d4f020f4c5
src/api2/access/user.rs: add access permissions
2020-04-17 10:08:45 +02:00
d28ddb8e04
src/api2/access/acl.rs: add access permissions
2020-04-17 10:03:09 +02:00
83b6a7cf71
src/api2/node/tasks.rs: use api macro, implement access permissions
2020-04-16 17:47:21 +02:00
e4681f9f71
src/api2/node/syslog.rs: add access permissions
2020-04-16 17:08:19 +02:00
b5037fa8ed
src/api2/node/status.rs: add access permissions
2020-04-16 17:05:09 +02:00
1cf7bbf412
src/api2/node/services.rs: add access permissions
2020-04-16 12:47:16 +02:00
68ed0c629d
src/api2/node/journal.rs: add access permissions
2020-04-16 12:47:16 +02:00
4b40148caa
start impl. access permissions
2020-04-16 12:47:16 +02:00
109d7817cd
src/config/user.rs - cached_config: do not store/return digest
2020-04-15 11:35:57 +02:00
bd098a7f77
src/api2/node/dns.rs: use api macro (cleanup)
2020-04-15 10:09:18 +02:00
9c06f6c292
fix previous commit - use result.
2020-04-14 17:48:10 +02:00
9f4e47dd93
acl update: check path
2020-04-14 17:23:48 +02:00
d83175dd69
acl update: check if user exist.
2020-04-14 13:46:27 +02:00
9765092ede
acl api: implement update
2020-04-14 10:16:49 +02:00
ed3e60ae69
start ACL api
2020-04-13 11:09:44 +02:00
b88f9c5b1e
PASSWORD_SCHEMA: set max_length to 1024 (for tickets)
2020-04-09 13:35:58 +02:00
879546aff6
api: add default property to domain list
2020-04-09 13:35:08 +02:00
73b40e9b46
api: correctly sort access subdirmap
2020-04-09 13:34:07 +02:00
708db4b3ae
api: add list_domains
2020-04-09 11:36:45 +02:00
685e13347e
api: move config/user to access/users, implement change_password
...
To make it similar to the pve api
2020-04-09 10:21:24 +02:00
7d817b0358
implement auth framework
2020-04-08 14:06:15 +02:00
579728c641
add user configiguration
2020-04-08 14:06:15 +02:00
cf459b1982
gc: log pending removals
2020-04-06 09:50:40 +02:00
dda7015497
prune api: return a usable result (we run synchronous anyways)
2020-04-01 12:24:28 +02:00
6650a242fb
rewrite future select in upgrade_to_backup_protocol using select macro
...
and handle all ok/err cases with at least logging
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-03-18 11:33:59 +01:00
f24fc1166b
src/bin/proxmox-backup-client.rs - list_snapshots: use format_and_print_result_full()
...
Depend on proxmox v0.1.14.
2020-02-27 11:28:44 +01:00
ea5f547fdc
src/bin/proxmox-backup-client.rs - list_snapshot_files: use format_and_print_result_full()
2020-02-26 13:49:47 +01:00
8528fce874
src/api2/node/tasks.rs; make list_tasks public
2020-01-30 13:31:20 +01:00
70fa1aa3fc
fixup
2020-01-30 13:31:20 +01:00
5eeea607ae
src/section_config.rs - convert_to_array: add option to skip properties
2020-01-30 13:26:46 +01:00
99384f7933
src/api2/types.rs: define and use struct TaskListItem
2020-01-28 11:23:08 +01:00
2c4b303c62
src/api2/node/tasks.rs: use api macro for list_tasks
2020-01-28 11:23:08 +01:00
d59dbeca1b
src/client/http_client.rs: new struct HttpClientOptions
2020-01-25 12:18:00 +01:00
6afbe1d846
src/config/remote.rs: add fingerprint
2020-01-25 09:49:45 +01:00
dcb8db66d9
src/api2/types.rs: add CERT_FINGERPRINT_SHA256_SCHEMA
2020-01-25 09:49:45 +01:00
4191018c79
src/api2/admin/datastore.rs - download_file: use BACKUP_ARCHIVE_NAME_SCHEMA for file-name
2020-01-23 13:49:30 +01:00
a92830dc39
src/api2/types.rs: define and use api type GarbageCollectionStatus
2020-01-23 13:40:12 +01:00
dfc58d47ad
src/api2/admin/datastore.rs: use api macro for start_garbage_collection
2020-01-23 13:01:21 +01:00
1dc117bbba
src/api2/types.rs: define and use struct StorageStatus
2020-01-23 12:51:30 +01:00
09b1f7b202
src/api2/types.rs: define struct BackupContent, and use it with list_snapshot_files
2020-01-23 11:17:38 +01:00
68a6a0ee03
src/api2/admin/datastore.rs: use api macro for delete_snapshot
2020-01-23 10:16:45 +01:00
c425bdc9b2
src/api2/pull.rs: delete fanished snapshots
2020-01-23 09:58:57 +01:00
18cc66ee85
src/api2/pull.rs: use new Datastore::last_successful_backup helper
...
Do not fail if local group does not exists.
2020-01-22 15:06:28 +01:00
1ca037d81c
followup: commit all changes
...
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-22 14:14:52 +01:00
44b65b4ec7
api/datastore: do not allow updating path for now
...
It's a bit dangerous as it points to all the saved backups, so they
would be seemingly lost after updating the path.
Follow our logic from other products, e.g. in PVE we do not allow to
update the backing path/location of a storage either for similar
reasons.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-22 13:55:37 +01:00
f1d99e3f6a
src/bin/proxmox-backup-client.rs: use a std channel to write the catalog
2020-01-22 12:49:08 +01:00
9ea4bce444
bump proxmox crate to 0.1.7
...
The -sys, -tools and -api crate have now been merged into
the proxmx crate directly. Only macro crates are separate
(but still reexported by the proxmox crate in their
designated locations).
When we need to depend on "parts" of the crate later on
we'll just have to use features.
The reason is mostly that these modules had
inter-dependencies which really make them not independent
enough to be their own crates.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 13:48:37 +01:00
4b4eba0b9e
src/api2/pull.rs: implement delete flag for vanished groups
2020-01-17 11:24:55 +01:00
b31c8019d7
src/api2/types.rs: define and use GroupListItem
2020-01-17 10:17:18 +01:00
f357390c15
renamed: src/config/remotes.rs -> src/config/remote.rs
...
And use 'remote' instead of 'remotes' everywhere.
2020-01-16 14:32:06 +01:00
94609e2380
src/api2/pull.rs: use reference to remote configuration
2020-01-16 13:55:19 +01:00
002a191abf
src/api2/config: check digest
2020-01-15 12:27:05 +01:00
347834df25
src/api2/config: correctly lock files
2020-01-15 11:57:12 +01:00
1c79e1017f
services: fix indentation
...
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-01-15 10:55:04 +01:00
f3ec5dae7f
src/api2/config: improve api schema by adding return type
2020-01-14 14:55:58 +01:00
c5799e4043
src/api2/config/datastore.rs: improve api, implement update and read
2020-01-14 14:45:56 +01:00
08195ac823
src/api2/config/remotes.rs: improve api, implement update and read
2020-01-14 14:20:16 +01:00
d0187a51a9
src/section_config.rs - convert_to_array: optionally add digest
...
datastore::config() -> also return digest
remotes::config() -> also return digest
2020-01-14 12:57:03 +01:00
7e7b781a18
src/api2/types.rs: also define PASSWORD_FORMAT and use it correctly
2020-01-14 11:32:02 +01:00
da4a15a351
src/api2/types.rs: define and use PASSWORD_REGEX
2020-01-14 11:22:42 +01:00
ff620a3d1a
src/api2/types.rs: add regression test for PROXMOX_USER_ID_SCHEMA
2020-01-13 17:26:43 +01:00
ae62c4fe59
src/api2/types.rs: fix regex patterns
2020-01-13 14:38:31 +01:00
163dc16c0b
src/api2/types.rs: define PROXMOX_AUTH_REALM_SCHEMA and PROXMOX_USER_ID_SCHEMA
...
And try to use nbew schemas with config api...
2020-01-13 14:18:19 +01:00
b25f313d66
src/api2/types.rs: define HOSTNAME_FORMAT, DNS_NAME_FORMAT and DNS_NAME_OR_IP_FORMAT
2020-01-13 13:10:52 +01:00
454c13edce
src/api2/types.rs: define SINGLE_LINE_COMMENT_SCHEMA
2020-01-13 12:02:13 +01:00
167971ed49
src/api2/types.rs: define REMOTE_ID_SCHEMA here
2020-01-13 11:47:07 +01:00
50af953e1b
src/section_config.rs: make set_data generic
2020-01-11 11:09:27 +01:00
688fbe07a1
cleanup config api, add remotes config cli interface
2020-01-11 10:42:09 +01:00
ca44172404
src/api2/config/remotes.rs: cleanup - avoid compiler warnings
2020-01-10 19:25:26 +01:00
141304d64e
src/api2/config/remotes.rs: new API to configure remotes
2020-01-10 13:28:15 +01:00
eb506c830c
rename src/api2/sync.rs -> src/api2/pull.rs
...
Use "pull" instead of "sync", because this also indicates a
direction (like "sync from").
2020-01-10 11:15:16 +01:00
71da3d6a1b
src/api2/types.rs: fix SnapshotListItem schema definition
2020-01-09 10:36:19 +01:00
de8ec04123
src/api2/sync.rs: implement remote sync
2020-01-08 14:03:52 +01:00
fc189b198c
src/api2/types.rs: define and use new api type SnapshotListItem
2020-01-07 12:52:52 +01:00
550e0d8870
src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate
...
Without calling external openssl binary.
2019-12-22 17:35:06 +01:00
5e62d19cfb
fix create_datastore parameters:
...
comment was declared as optional but unwrap()ed, causing
panics when not using it
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-20 12:30:25 +01:00
645995634a
src/api2/config/datastore.rs - create: pass uid and gid instead of User
2019-12-20 09:23:58 +01:00
e67770d496
src/backup/chunk_store.rs - create: pass User instead of CreateOptions
2019-12-20 09:11:40 +01:00
bca9093520
api/ui: datastore: allow to set simple comment
...
for now forbid all control characters[0] in the comment value, the
section config writer cannot cope with newlines in the value, it
writes them out literally, allowing "injection" or breaking the whole
config.
In the webinterface use also a textfield, not a textarea.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-19 17:58:01 +01:00
afdcfb5bc9
let ChunkStore::create take CreateOptions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-19 13:14:49 +01:00
2337df7b3d
api2: add very basic 'status/usage' endpoint for nodes
...
For returning the nodes basic "usage status", for now one gets memory
and CPU utilization.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-18 17:24:32 +01:00
feaa1ad35f
replace file_set_contents with replace_file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-18 11:16:04 +01:00
a66e7920dc
src/api2/config/datastore.rs: set protected flags for create/delete API
2019-12-18 10:52:45 +01:00
f0db500808
src/api2/config/datastore.rs: fix api docu
2019-12-18 10:48:19 +01:00
81cc71c0d7
use proxmox-mini-journalreader to display syslog
2019-12-17 14:06:48 +01:00
20197f7cf5
src/api2/node/syslog.rs: use api macro
2019-12-17 13:26:49 +01:00
bb084b9c91
api2: update for latest proxmox-api changes
...
- rename ApiFuture into ApiResponseFuture
- impl. ApiHandler::Async
2019-12-16 10:01:51 +01:00
236a396aa1
src/api2/admin/datastore.rs - prune: log retention options
2019-12-14 16:32:16 +01:00
503995c767
src/api2/admin/datastore.rs: fix prune worker id
2019-12-14 16:05:21 +01:00
db0cb9ce0b
update a chunk of stuff to the hyper release
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
1ae5677de0
src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT
2019-12-12 14:15:40 +01:00
d0adf270fb
src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA
2019-12-12 12:37:11 +01:00
66c49c21c3
src/api2/types.rs: factor out DATASTORE_SCHEMA
2019-12-11 13:45:23 +01:00
567d3e00fb
src/api2/node/tasks.rs: new filter "store" to filter tasks for one store
2019-12-11 12:53:34 +01:00
6d04612955
src/api2/node/tasks.rs: add "running" filter to list only running tasks
2019-12-11 10:52:27 +01:00
8560fe3e5f
src/api2/node/tasks.rs - extract_upid: avoid duplicate error message
2019-12-10 13:45:56 +01:00
5830c20560
src/bin/proxmox-backup-client.rs: add simple task management cli
2019-12-10 13:43:53 +01:00
3b03abfe14
src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune
...
And use display_task_log on the client side.
2019-12-08 10:59:47 +01:00
163e9bbe91
src/bin/proxmox-backup-client.rs: new display task log helper
...
New helper to display worker task logs. Use it for prune (experimental).
2019-12-07 16:11:26 +01:00
6b508dd563
src/server/formatter.rs: impl. new result attribute "active"
2019-12-07 15:29:42 +01:00
102d8d4136
src/backup/prune.rs: implement --keep-hourly
2019-12-07 11:23:33 +01:00
1f604f1a5a
src/api2/admin/datastore.rs: add new api test_prune()
2019-12-06 13:06:56 +01:00
0ab08ac980
src/api2/admin/datastore.rs: cleanup - move schema defininition in front of rust method
2019-12-06 12:36:41 +01:00
1a0678b601
src/api2/admin/datastore.rs: remove prune parameters from status call
...
Not idea why they are added there.
2019-12-06 12:32:53 +01:00
9e3f008804
src/backup/prune.rs: add new helper keeps_something()
2019-12-06 12:28:31 +01:00
9b78352188
src/backup/prune.rs: define new struct PruneOptions
2019-12-06 08:56:27 +01:00
92acbd69f7
src/backup/backup_info.rs: remove old prune code
2019-12-06 08:29:27 +01:00
8f0b4c1f90
src/api2/admin/datastore.rs - prune: log info about all snapshots
2019-12-05 15:35:11 +01:00
9ce42759ec
src/backup/backup_info.rs - compute_prune_list: remove unfinished backups
2019-12-05 11:18:10 +01:00
aeeac29bb1
src/backup/backup_info.rs: new compute_prune_list helper
...
So that we can write regression tests for this.
2019-12-04 15:49:11 +01:00
3d229a4a01
Revert "test another api macro use"
...
This reverts commit a9aa52e6a8
2019-12-04 14:18:11 +01:00
4f1e40a29e
src/api2/admin/datastore.rs - read_backup_index: add manifest with size
...
So that the client can display the size of the index file.
2019-12-04 10:03:52 +01:00
a9aa52e6a8
test another api macro use
...
The api macro now supports hyphens in parameter names and
referencing externally defined `Schema`s, so here's an
example.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-03 13:41:40 +01:00
41874331ed
whitespace fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:52:15 +01:00
2905f2b5e6
update api macro example usage
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:51:27 +01:00
7b6c41078b
update api macro invocation to new style
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 14:19:46 +01:00
6486cb853f
first api macro usage test/example
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 10:05:37 +01:00
329d40b50b
rename ApiHandler::Async into ApiHandler::AsyncHttp
2019-11-23 09:03:21 +01:00
ad51d02aa9
src/server/rest.rs: cleanup async code
2019-11-22 13:02:05 +01:00
cad540e969
api/compat: remove remaining api_schema references
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:53:15 +01:00
a2479cfa1a
api/compat: drop more compat imports from api_schema.rs
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:36:28 +01:00
2194824f77
api: BoxFut -> ApiFuture
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:16:37 +01:00
3d482025b3
move api schema into proxmox::api crate
...
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-21 14:14:54 +01:00
552c225948
sort all property lookup tables
...
Required, because we use binary sreach to find items.
2019-11-21 13:32:09 +01:00
255f378a1b
use const api definitions
2019-11-21 13:32:09 +01:00
0997967d65
do no restrict index names too much
2019-11-08 10:28:14 +01:00
62ee2eb405
avoid some clippy warnings
2019-10-26 11:42:05 +02:00
11377a47bb
avoid some clippy warnings
2019-10-25 18:04:37 +02:00
2c2097ee75
fix typo in error message
2019-10-07 12:48:56 +02:00
f7aa6f15d3
src/api2/reader.rs: use tokio::fs::read() to read file async
2019-10-07 12:24:06 +02:00
4ee8f53d07
remove DataChunk file format - use DataBlob instead
2019-10-06 10:31:06 +02:00
c0b1b14c05
src/api2/reader.rs - download_chunk: use blocking io
...
Turns out to be twice as fast as tokio async io...
2019-10-05 11:41:19 +02:00
fb6026b66b
backup writer api: verify checksum inside close
2019-09-23 10:56:53 +02:00
4a7de56e2f
avoid injecting ENV vars from Makefile
...
So that we can run "cargo build" without setting vars manually.
2019-09-09 10:51:08 +02:00
ffb6434485
src/api2/reader.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
7622005574
src/api2/backup/upload_chunk.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
59b2baa0f6
src/api2/backup.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
6be147b78c
src/api2/admin/datastore.rs: switch to async
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-09-02 15:17:58 +02:00
f3a8d1d7e0
api2/node/time: replace mem::uninitialized
...
and fixup use statements
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-08-22 11:14:41 +02:00
4af0ee055a
api: always use complete file names (including add exctensions)
2019-08-07 10:10:14 +02:00
d7c243977d
src/api2/admin/datastore.rs - list_snapshot_files: list all files
...
Add files that are not mentioned in index.json.
2019-08-07 09:16:14 +02:00
34066e2eeb
src/api2/backup/environment.rs - add_blob: always verify blob crc
2019-08-07 09:15:13 +02:00
f889b15867
src/backup/data_blob.rs: always compute crc
2019-08-06 11:42:14 +02:00
a17a0e7a9f
src/api2/admin/datastore.rs: extract backup size from index.json
2019-08-06 10:56:21 +02:00
8c70e3eb18
src/api2/admin/datastore.rs - files: return data from index.json
2019-08-05 13:22:19 +02:00
f69adc81a6
use new proxmox::tools::nodename
2019-08-03 17:06:23 +02:00
7f66c29e5c
remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead
2019-08-03 16:44:31 +02:00
e18a6c9ee5
update to nix 0.14, use code from proxmox:tools
2019-08-03 13:05:38 +02:00
9fdc3ef46f
src/api2/admin/datastore.rs: pass backup type/id (group) to prune
...
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
2019-07-27 08:49:14 +02:00
bbf9e7e951
src/api2/backup.rs: cleanup schema definitions
2019-07-26 09:07:29 +02:00
ca5d0b61ca
src/api2/backup.rs: new required backup-time parameter
...
The client should pass the time as parameter.
2019-07-25 13:44:01 +02:00
e128d4e84f
src/api2/admin/datastore.rs: backup logs may not be written twice
2019-07-25 12:29:04 +02:00
ef3254c086
src/api2/admin/datastore.rs: use correct .blob extension for log
2019-07-25 12:22:50 +02:00
07ee223590
src/api2/admin/datastore.rs: add api to upload backup client log file
2019-07-25 12:15:30 +02:00
15c847f136
src/api2/admin/datastore.rs: allow to list all snapshots
2019-07-18 09:11:43 +02:00
0eecf38fbf
src/api2/admin/datastore.rs: add status api call
2019-07-16 13:34:38 +02:00
b46c3fad03
src/backup/fixed_index.rs: use correct size
...
We need to consider that the last chunk may have smaller size.
2019-07-04 15:13:22 +02:00
5e04ec707d
src/api2/backup/environment.rs: improve chunk alignment/size checks
2019-07-04 13:40:43 +02:00
17243003b3
src/bin/download-speed.rs: h2 download speedtest for our server
...
I get about 30MB/s (much too slow)
2019-06-29 13:43:10 +02:00
09d7dc5024
src/api2/reader.rs: add download chunk api
2019-06-28 16:27:01 +02:00
fcfb84fedf
file download: avoid unnecessary copy
2019-06-28 07:07:52 +02:00
dd066d28e2
src/api2/reader.rs: implement backup reader protocol
2019-06-27 09:01:41 +02:00
42a87f7b96
src/server/h2service.rs: implement generic h2 service
2019-06-26 17:38:33 +02:00
f14a8c9a85
src/api2/admin/datastore.rs: use correct path for download
2019-06-25 12:59:36 +02:00
9e47c0a56e
src/api2/admin/datastore.rs: implement generic download method
...
We simply allow to download raw binary data.
2019-06-25 10:16:59 +02:00
da7d67213e
src/api2/admin/datastore/pxar.rs: remove pxar upload api
...
Upload needs to be done using the backup protocol.
2019-06-25 09:45:35 +02:00
dd8e744f9a
src/api2/admin/datastore.rs: use a worker log log prune actions
2019-06-25 08:12:25 +02:00
46bd880041
src/api2/backup/environment.rs: new helper add_blob()
2019-06-24 09:35:37 +02:00
cb08ac3efe
src/api2/backup.rs: replace upload_config with upload_blob
2019-06-23 11:44:30 +02:00
6f083b7a92
src/api2/backup/upload_chunk.rs: compute crc on upload
2019-06-23 09:04:42 +02:00
bffd40d6b7
src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools
2019-06-14 11:40:04 +02:00
00388226e6
src/api2/backup/environment.rs: log index checksums
2019-06-14 10:36:20 +02:00
36075475a6
src/api2/backup/environment.rs: log duplicate chunks
2019-06-14 07:12:30 +02:00
fa148dbd11
src/api2/backup/upload_chunk.rs: verify chunk data if possible
2019-06-14 06:23:46 +02:00
f98ac774ee
backup: Add support for client side encryption
...
first try ...
2019-06-13 11:47:23 +02:00
6762db70d6
src/api2/types.rs: define CHUNK_DIGEST_SCHEMA
2019-06-13 07:30:42 +02:00
dd5495d6dc
tree-wide: use 'dyn' for all trait objects
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
4800df7ba6
src/api2/admin/datastore.rs: remove test-upload api
2019-06-07 07:52:11 +02:00
113898d7e9
src/api2/admin/datastore/upload.rs: remove unused upload api
2019-06-07 07:39:41 +02:00
986bef16be
src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
...
So that we can include it in static doc strings.
2019-06-05 08:41:20 +02:00
c9ec0956cf
src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1
2019-06-05 08:12:13 +02:00
7773ccc11f
src/api2.rs: move backup api to /backup
2019-06-05 07:23:21 +02:00
339ddfcbfa
src/cli/command.rs: move doc generator code to src/api_schema/format.rs
2019-06-04 12:32:22 +02:00
39d6846e1e
src/api2/admin/datastore/backup.rs: implement config file upload
2019-06-03 09:45:14 +02:00
36898ffce6
src/backup/chunk_stream.rs: add optional chunk_size parameter
2019-05-30 13:28:24 +02:00
96482891ae
src/api2/admin/datastore/backup/environment.rs: log upload statistics
2019-05-30 09:21:19 +02:00
642322b433
src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload
...
So that we can provide better statistics (patches follows).
2019-05-30 09:21:19 +02:00
e6389f4e75
src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files)
2019-05-29 10:38:57 +02:00
39e60bd6fa
src/api2/admin/datastore/backup.rs: improve debug logging
2019-05-29 10:17:38 +02:00
a42d1f55c3
src/api2/admin/datastore/backup.rs: add debug flag
2019-05-29 09:35:21 +02:00
976595e1a9
src/backup/dynamic_index.rs: split class DynamicIndexWriter
2019-05-29 08:49:57 +02:00
60e589a111
src/api2/admin/datastore/backup.rs: log sucessful finish call
2019-05-28 12:55:22 +02:00
006f3ff407
src/api2/admin/datastore/backup/environment.rs: add more consistency checks
2019-05-28 09:21:31 +02:00
3dc5b2a203
src/client/http_client.rs: use start of chunk as offset
2019-05-28 09:21:31 +02:00
e3d525feb7
src/api2/admin/datastore/backup.rs: fix typo
2019-05-28 07:13:19 +02:00
a42fa400ee
src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api
2019-05-28 06:18:55 +02:00
a1e7cff3ea
src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk
2019-05-27 07:57:43 +02:00
771953f9eb
src/api2/admin/datastore/backup.rs: use smaller window sizes
2019-05-27 07:53:40 +02:00
05cba08c9c
src/client/http_client.rs: append chunks in correct order
...
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
2019-05-26 09:57:34 +02:00
417cb0731e
src/api2/admin/datastore/backup.rs: verify chunk offset
...
This clearly shows that the current approach does not work - seems we get
chunks out of order.
2019-05-26 08:49:29 +02:00
cc84a830c5
RFC: schema: make enums static
...
I don't see a reason to allow these to be dynamically
modifiable.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 12:38:51 +02:00
aa1b2e04fe
src/client/merge_known_chunks.rs: merge known chunks
...
To decrease the number of api calls required...
2019-05-23 12:31:09 +02:00
8bea85b42e
src/api2/admin/datastore/backup.rs: verify file size and chunk count on close
2019-05-23 08:50:36 +02:00
907f5bb0d8
src/api2/admin/datastore/backup/service.rs: improve error logging
2019-05-23 08:05:39 +02:00
a55fcd740f
src/api2/admin/datastore/backup.rs: suse POST for finish() api
2019-05-22 13:04:58 +02:00
5a0b521809
src/api2/admin/datastore/backup.rs: remove test callbacks
2019-05-22 10:55:27 +02:00
d0afeaa831
src/api2/admin/datastore/backup.rs: improve error message
2019-05-22 07:43:54 +02:00
7f3d2ffab9
src/backup/index.rs: rename ChunkListReader into DigestListEncoder
2019-05-21 12:28:44 +02:00
a95849321c
src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks
2019-05-21 09:54:11 +02:00
a09c0e38d8
src/api2/admin/datastore/backup/environment.rs: register/lookup known chunks
2019-05-20 18:05:10 +02:00
82ab72304e
src/client/http_client.rs: implement upload_stream
2019-05-20 14:19:24 +02:00
adec8ea263
src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed
...
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
2019-05-16 10:24:23 +02:00
372724afea
src/api2/admin/datastore/backup.rs: implement finish_backup
2019-05-15 12:58:55 +02:00
bb105f9dae
src/api2/admin/datastore/backup.rs: improve logging
2019-05-15 10:09:46 +02:00
a20772525f
src/api2/admin/datastore/backup.rs: implement close for dynamic writers
2019-05-15 07:58:05 +02:00
4e93f8c164
src/api2/types.rs: add schema/format for file names
2019-05-15 07:14:08 +02:00
fbb798f6d2
src/api2/admin/datastore/backup.rs: ignore errors from last_backup
2019-05-13 07:06:56 +02:00
d36113665b
src/api2/admin/datastore/backup.rs: implement chunk index download from previous backup
2019-05-11 16:05:50 +02:00
af1d68185c
src/api2/admin/datastore/backup.rs: add missing 'archive-name' parameter to schema
2019-05-11 12:25:15 +02:00
6b95c7df07
src/api2/admin/datastore/backup/environment.rs: do not store path
2019-05-11 12:13:44 +02:00
51a4f63ff7
src/backup/datastore.rs: move last_backup to BackupInfo
2019-05-11 12:07:09 +02:00
b02a52e3a5
src/backup/datastore.rs: impl last_backup helper
2019-05-11 11:36:46 +02:00
c09775011a
src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups
2019-05-11 10:19:34 +02:00
f9578f3c79
backup-api: allow to create DynamicIndexWriter and add chunks to to
2019-05-10 10:25:40 +02:00
0506d1ed04
upload_chunk.rs: add comment about possible unnecessary code
2019-05-10 06:15:25 +02:00
5f42ef009a
cleanup: remove unused code
2019-05-09 18:11:39 +02:00
b4b63e520d
handle_async_api_request: put rpcenv into a Box
...
So that we can pass rpcenv into futures.
2019-05-09 18:01:24 +02:00
2c41096a4b
src/api2/admin/datastore/backup/upload_chunk.rs: fix maximal chunk size
2019-05-09 13:29:50 +02:00
78b5191550
src/server/formatter.rs: further cleanups and renaming ...
2019-05-09 13:28:26 +02:00
21ee7912fa
src/api2/admin/datastore/backup.rs: implement upload chunk
2019-05-09 13:06:09 +02:00
4ebf0eabb0
src/api2/types.rs: use separate file for schema definitions
2019-05-09 07:44:09 +02:00
bd1507c4fa
src/api2/admin/datastore/backup/service.rs: move service code into extra file
2019-05-08 17:36:19 +02:00
0aadd40b4f
src/api2/admin/datastore/backup.rs: add required parameters, cleanups
2019-05-08 13:17:23 +02:00
d95ced64ad
src/api2/admin/datastore/backup/environment.rs: use separate file
2019-05-08 12:41:58 +02:00
ca60c371e2
api: rename h2upload to backup
2019-05-08 12:35:15 +02:00
d906b1124f
src/api2/admin/datastore/h2upload.rs: code cleanups
2019-05-08 11:36:43 +02:00
58c8d7d91f
rc/api2/admin/datastore/h2upload.rs: implement BackupEnvironment
...
To pass arbitrary information/state to api methods.
2019-05-08 11:26:54 +02:00
090ac9f79a
src/api2/admin/datastore/h2upload.rs: avoid unwrap
2019-05-08 08:21:18 +02:00
4e3da4b366
src/api2/admin/datastore/h2upload.rs: do not log client address
2019-05-08 08:15:55 +02:00
e3a44552ee
src/api2/admin/datastore/h2upload.rs: code cleanups
2019-05-08 08:12:49 +02:00
a66ab8ae5e
src/api2/admin/datastore/h2upload.rs: make worker abortable
2019-05-08 07:14:40 +02:00
72375ce6c6
src/api2/admin/datastore/h2upload.rs: log to worker task
2019-05-07 14:07:00 +02:00
d9bd06eae8
src/api2/admin/datastore/h2upload.rs: use WorkerTask
2019-05-07 13:42:00 +02:00
52cf506e48
src/api2/admin/datastore/h2upload.rs: implement async test api call
2019-05-07 12:26:55 +02:00
92ac375aff
src/api2/admin/datastore/h2upload.rs: improve http2 service
2019-05-07 11:24:44 +02:00
96e95fc179
src/api2/admin/datastore/h2upload.rs: test if version < 2
...
We can only upgrade http 1.x connections.
2019-05-06 10:29:34 +02:00
cf639a47de
rc/client/http_client.rs: add experimental h2 upgrade code
2019-04-29 11:57:58 +02:00
152764ec15
src/api2/admin/datastore/h2upload.rs: add test code for http2
2019-04-27 08:57:35 +02:00
6358b68236
cleanup: avoid compiler warnings
2019-04-23 12:17:24 +02:00
13f1cc17ea
src/api_schema/router.rs: implement list_subdirs() helper
2019-04-16 12:07:02 +02:00
062d4916ff
api_schema: allow generic api handler functions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-16 11:22:23 +02:00
1a6c9415ae
src/api2/node/services.rs: add proxmox-backup-proxy service
2019-04-15 09:48:02 +02:00
75bc49bed7
src/server/worker_task.rs: implement abort channel
2019-04-15 09:38:05 +02:00
912524752e
src/api2/admin/datastore/pxar.rs: create worker task for file upload
2019-04-15 08:38:14 +02:00
f2b99c34f7
src/api2/admin/datastore.rs: implement API to return last GC status
2019-04-11 12:04:25 +02:00
a665dea128
src/api2/node/tasks.rs: implement stop_task api
2019-04-10 13:55:05 +02:00
c360bd73d3
src/api2/node/tasks.rs: fix task status api
2019-04-09 15:12:20 +02:00
d8d40dd0c2
src/api2/node.rs: add node parameter
2019-04-09 14:43:30 +02:00
634132fe10
src/server/upid.rs: moved code into separate file
2019-04-08 18:43:26 +02:00
5a12c0e2fb
src/api2/node/tasks.rs: start upid API
2019-04-07 14:36:57 +02:00
d2a2e02bb7
src/api2/node/tasks.rs: implement userfilter
2019-04-07 13:17:19 +02:00
3c3bee2e32
src/api2/node/tasks.rs: correctly handle start parameter
2019-04-07 12:41:24 +02:00
063ca5be77
src/api2/node/tasks.rs: implement list_tasks
2019-04-07 12:18:58 +02:00
d4b59ae0b8
src/backup/datastore.rs: pass WorkerTask to GC
...
So that we can log to the task log file.
2019-04-06 17:57:38 +02:00
0f778e0652
src/api2/admin/datastore.rs: use new WorkerTask::new_thread()
2019-04-06 11:27:23 +02:00
3e6a7deeaa
src/api2/admin/datastore.rs: start GC in a separate thread
2019-04-01 12:02:46 +02:00
ad281d1bd9
www/DataStoreConfig.js: add button to start GC
2019-04-01 08:08:34 +02:00
8968258b66
rename catar into pxar
...
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
d5c34d98c0
src/catar/decoder.rs: simplify public restore API
2019-03-11 14:31:01 +01:00
01a1342364
src/api2/admin/datastore.rs: impl list_snapshot_files
2019-03-06 11:02:08 +01:00
dd79265a8a
src/api2/admin/datastore/catar.rs: use archive_name directly
2019-03-06 11:02:08 +01:00
e2d007f76e
api2/admin/datastore: add a backup protocol test api path
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
f8f94534d4
src/api2/access.rs: authenticate_user() - add ticket login
2019-03-05 12:56:21 +01:00
391d310741
backup_info: BackupDir shortcut
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
b3483782d1
src/backup/backup_info.rs: move code into separate file
...
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
2b01a22507
src/backup/datastore.rs: add helper to sort backup lists
2019-03-04 18:20:57 +01:00
875fb1c01a
src/backup/datastore.rs: use unix epoch to create DateTime
...
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
9b492eb256
src/backup/datastore.rs: protect BackupDir fields, impl new()
2019-03-04 13:51:36 +01:00
1e9a94e579
src/backup/datastore.rs: protect BackupGroup fields, impl new()
2019-03-04 13:38:23 +01:00
6f62c9240a
src/api2/admin/datastore.rs: imp delete_snapshot
2019-03-03 11:29:00 +01:00
e77a02ed87
catar api: cleanup parameter names
2019-03-03 09:48:32 +01:00
49dc074007
parameter name cleanups (use '-' instead of '_')
2019-03-02 16:34:10 +01:00
ad20d19877
src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups
2019-03-02 16:28:36 +01:00
184f17afad
src/api2/admin/datastore.rs: impl list_snapshots
2019-03-02 16:20:50 +01:00
812c6f8716
src/api2/admin/datastore.rs: impl list_backup_groups
2019-03-02 11:29:05 +01:00
e909522f06
src/backup/datastore.rs: use timestamp with zone offset
2019-03-01 17:36:40 +01:00
dc4c09faae
src/backup/datastore.rs: split BackupDir
2019-03-01 16:26:35 +01:00
38b0dfa511
src/backup/datastore.rs: split BackupInfo
2019-03-01 16:26:35 +01:00
541b908ea4
src/backup/datastore.rs: use DateTime<Utc> instead of unix epoch
2019-02-28 17:03:48 +01:00
8f57971714
src/api2/admin/datastore.rs: implement prune
2019-02-28 12:51:27 +01:00
83b7db0209
src/api2/admin/datastore.rs: start prune api
...
just a dummy for now.
2019-02-27 16:53:17 +01:00
8731e40a7f
backup/datastore: create_backup_dir: show if path existed
...
To enable asserting the creation of a new backup rather than
adding to an existing one.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-27 10:15:36 +01:00
8c75372b79
src/backup/datastore.rs: list_backups() - include list of index files
2019-02-20 13:37:44 +01:00
247cdbce72
src/api2/admin/datastore/catar.rs: allow to configure chunk-size
2019-02-19 15:19:12 +01:00
1c7a88aeff
UploadCaTar: currectly write all data using write_all()
2019-02-18 18:31:02 +01:00
ef2f2efbcc
improve api_schema module structure
2019-02-17 10:16:33 +01:00
dc9a007b11
rename src/api to src/api_schema
2019-02-17 09:59:20 +01:00
1d77b6cf6b
update to pam 0.7 (renamed from pam-auth)
...
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-15 10:34:40 +01:00
f02e6fc478
api2/admin/datastore/catar.rs: simplify/fix debug message
2019-02-15 10:04:37 +01:00
541a302224
api2/admin/datastore/catar.rs: use nested file extensions
...
i.e. archive.catar.didx
2019-02-13 10:37:00 +01:00
e5064ba607
simplify backup lib structure (pub use xxx:*), improve doc
2019-02-12 14:13:31 +01:00
93d5d77952
rename ArchiveIndex to DynamicIndex
...
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00
ace9e3531a
access: use proxmox-backup-auth for pam
...
allows customization via /etc/pam.d/proxmox-backup-auth
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-07 13:40:22 +01:00
13f8310cac
api2: time: use /etc/localtime as fallback
...
in case there is no /etc/timezone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-05 14:14:56 +01:00
4b2cdeb9a6
implement relead_timezone flag
2019-02-01 10:04:46 +01:00
c82bc1a1f9
depend on pam-auth, use pam for root login
...
Please use username "root@pam" for login via GUI.
2019-02-01 09:30:50 +01:00
a154a8e8a4
delay unauthorized request (rate limit)
2019-01-31 14:34:21 +01:00
b9903d6331
server/rest.rs: verify auth cookie
2019-01-31 12:22:00 +01:00
9f49fe1d5d
avoid compiler warnings
2019-01-30 18:25:37 +01:00
34f956bc25
api2/access.rs: add ticket api
2019-01-30 15:16:10 +01:00
a859fa95a9
add protected flag for some api methods
2019-01-28 18:16:47 +01:00
f120483353
server/rest.rs: add proxy_sync_api_request() dummy
2019-01-28 17:30:39 +01:00
48849593ee
api2/node/services.rs: implement service commands
2019-01-27 12:40:31 +01:00
e82dad9700
also pass rpcenv to async handlers
2019-01-27 10:18:52 +01:00
d2ab5f19e2
api2/node/services.rs: start services api
2019-01-26 17:54:18 +01:00
6049b71f41
api: pass RpcEnvirnment to api handlers
2019-01-26 14:50:37 +01:00
d96d82736d
use crate log and syslog
2019-01-26 10:56:11 +01:00
e182ab4a4a
api2/node/syslog.rs: improve error handling
2019-01-26 10:23:52 +01:00
0441886842
api2/node/syslog.rs: implement dump journal
2019-01-25 18:20:51 +01:00
4f9a726833
api2/node/syslog.rs: add syslog api schema
2019-01-25 17:17:30 +01:00
af2fddea4f
api2/node/dns.rs: implement concurrent update protection
2019-01-25 11:38:59 +01:00
de6b0721fa
pi2/node/dns.rs: return file digest
2019-01-25 11:03:41 +01:00
46b79b9ee5
api2/node/dns.rs: improve dns api
2019-01-25 10:15:32 +01:00
8f973f8181
api2/node/dns.rs: implement resolv.conf parser
2019-01-24 14:59:40 +01:00
e6ffeb918c
api2/node/time.rs: implement set timezone
2019-01-24 12:05:06 +01:00
cd8b91f78c
api2/node/time.rs: add return schema
2019-01-24 11:07:11 +01:00
447787ab7c
tools.rs: implement file_read_firstline
2019-01-24 10:43:30 +01:00
0463602a79
api2/node/time.rs: implement read api
2019-01-23 15:14:14 +01:00
b2b3485d5f
src/api2/node: start node configuration api
2019-01-23 13:09:17 +01:00
7e13b2d67f
api2/subscription.rs: add subscription api class
2019-01-22 12:50:19 +01:00
576e3bf252
rename api3 back to api2
...
There is no real need to change the path, so using api2 we can reuse
all helpers (like tools from proxmox widget toolkit).
2019-01-22 12:10:38 +01:00
Stefan Reiter
Dominik Csapak
Fabian Grünbichler
Dietmar Maurer
Thomas Lamprecht
Hannes Laimer
Wolfgang Bumiller
Oguz Bektas