tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,633 Commits 2 Branches 54 Tags 14 MiB
cef5c72682
Commit Graph

1191 Commits

Author SHA1 Message Date
Dominik Csapak
005a5b9677 api2/node/tasks: move userfilter to function signature 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-28 07:18:13 +02:00
Dominik Csapak
d7a122a026 use jobstate mechanism for verify/garbage_collection schedules 
also changes:
* correct comment about reset (replace 'sync' with 'action')
* check schedule change correctly (only when it is actually changed)

with this changes, we can drop the 'lookup_last_worker' method

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-24 17:06:12 +02:00
Dietmar Maurer
a83e2ffeab src/api2/reader.rs: use std::fs::read instead of tokio::fs::read 
Because it is about 10%& faster this way.
 2020-09-22 13:27:23 +02:00
Dominik Csapak
9866de5e3d datastore/prune schedules: use JobState for tracking of schedules 
like the sync jobs, so that if an admin configures a schedule it
really starts the next time that time is reached not immediately

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-19 06:24:37 +02:00
Fabian Grünbichler
be3bd0f90b fix #3015: allow user self-service 
listing, updating or deleting a user is now possible for the user
itself, in addition to higher-privileged users that have appropriate
privileges on '/access/users'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-18 15:45:11 +02:00
Fabian Grünbichler
3c053adbb5 role api: fix description 
wrongly copy-pasted at some point

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-18 14:55:00 +02:00
Hannes Laimer
2b67de2e3f api2: make verify_schedule deletable 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:12:29 +02:00
Hannes Laimer
477859662a api2: add optional verify-schdule field to create/update datastore endpoint 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:12:16 +02:00
Hannes Laimer
f37ef25bdd api2: add VERIFY_SCHEDULE_SCHEMA 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:11:39 +02:00
Fabian Grünbichler
b93bbab454 fix #3014: allow DataStoreAdmins to list DS config 
filtered by those they are privileged enough to read individually. this
allows such users to configure prune/GC schedules via the GUI (the API
already allowed it previously).

permission-wise, a user with this privilege can already:
- list all stores they have access to (returns just name/comment)
- read the config of each store they have access to individually
(returns full config of that datastore + digest of whole config)

but combines them to
- read configs of all datastores they have access to (returns full
config of those datastores + digest of whole config)

user that have AUDIT on just /datastore without propagate can now no
longer read all configurations (but this could be added it back, it just
seems to make little sense to me).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-18 12:09:13 +02:00
Dominik Csapak
8f2f3dd710 fix #2942: implement lacp bond mode and bond_xmit_hash_policy 
this was not yet implemented, should be compatible with pve and the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-17 08:36:25 +02:00
Dominik Csapak
85959a99ea api2/network: add bond-primary parameter 
needed for 'active-backup' bond mode

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-17 08:36:14 +02:00
Dominik Csapak
36700a0a87 api2/pull: make pull worker abortable 
by selecting between the pull_future and the abort future

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-17 06:11:33 +02:00
Fabian Grünbichler
9626c28619 always allow retrieving (censored) subscription info 
like we do for PVE. this is visible on the dashboard, and caused 403 on
each update which bothers me when looking at the dev console.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-17 06:03:25 +02:00
Fabian Grünbichler
463c03462a fix #2957: allow Sys.Audit access to node RRD 
this is the same privilege needed to query the node status.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-17 06:03:25 +02:00
Stefan Reiter
d10332a15d SnapshotVerifyState: use enum for state 
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-09-15 13:06:04 +02:00
Stefan Reiter
43772efc6e backup: check all referenced chunks actually exist 
A client can omit uploading chunks in the "known_chunks" list, those
then also won't be written on the server side. Check all those chunks
mentioned in the index but not uploaded for existance and report an
error if they don't exist instead of marking a potentially broken backup
as "successful".

This is only important if the base snapshot references corrupted chunks,
but has not been negatively verified. Also, it is important to only
verify this at the end, *after* all index writers are closed, since only
then can it be guaranteed that no GC will sweep referenced chunks away.

If a chunk is found missing, also mark the previous backup with a
verification failure, since we know the missing chunk has to referenced
in it (only way it could have been inserted to known_chunks with
checked=false). This has the benefit of automatically doing a
full-upload backup if the user attempts to retry after seeing the new
error, instead of requiring a manual verify or forget.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-09-15 10:00:05 +02:00
Stefan Reiter
0af2da0437 backup: check verify state of previous backup before allowing reuse 
Do not allow clients to reuse chunks from the previous backup if it has
a failed validation result. This would result in a new "successful"
backup that potentially references broken chunks.

If the previous backup has not been verified, assume it is fine and
continue on.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-09-15 09:59:29 +02:00
Dietmar Maurer
d09db6c2e9 rename BackupDir::new_with_group to BackupDir::with_group 2020-09-15 09:40:03 +02:00
Dietmar Maurer
6a7be83efe avoid chrono dependency, depend on proxmox 0.3.8 
- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
 2020-09-15 07:12:57 +02:00
Fabian Grünbichler
e0e5b4426a BackupDir: make constructor fallible 
since converting from i64 epoch timestamp to DateTime is not always
possible. previously, passing invalid backup-time from client to server
(or vice-versa) panicked the corresponding tokio task. now we get proper
error messages including the invalid timestamp.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-11 15:49:35 +02:00
Fabian Grünbichler
cf9ea3c4c7 server: set http2 max frame size 
else we get the default of 16k, which is quite low for our use case.
this improves the TLS upload benchmark speed by about 30-40% for me.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-10 12:43:51 +02:00
Hannes Laimer
61d7b5013c add benchmark flag to backup creation for proper cleanup when running a benchmark 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2020-09-10 08:25:24 +02:00
Stefan Reiter
3cddfb29be backup: ensure no fixed index writers are left over either 
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-09-10 06:29:38 +02:00
Stefan Reiter
a9767cf7de gc: remove .bad files on garbage collect 
The iterator of get_chunk_iterator is extended with a third parameter
indicating whether the current file is a chunk (false) or a .bad file
(true).

Count their sizes to the total of removed bytes, since it also frees
disk space.

.bad files are only deleted if the corresponding chunk exists, i.e. has
been rewritten. Otherwise we might delete data only marked bad because
of transient errors.

While at it, also clean up and use nix::unistd::unlinkat instead of
unsafe libc calls.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-09-08 12:43:13 +02:00
Dominik Csapak
15ec790a40 tools/systemd/time: convert the resulting timestamp into an option 
we want to use dates for the calendarspec, and with that there are some
impossible combinations that cannot be detected during parsing
(e.g. some datetimes do not exist in some timezones, and the timezone
can change after setting the schedule)

so finding no timestamp is not an error anymore but a valid result

we omit logging in that case (since it is not an error anymore)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-04 15:28:05 +02:00
Fabian Grünbichler
ecd55041a2 fix #2978: allow non-root to view datastore usage 
for datastores where the requesting user has read or write permissions,
since the API method itself filters by that already. this is the same
permission setting and filtering that the datastore list API endpoint
does.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-04 06:18:20 +02:00
Dietmar Maurer
63d9aca96f verify: log progress 2020-09-02 07:43:28 +02:00
Dietmar Maurer
4f09d31085 src/backup/verify.rs: use global hashes (instead of per group) 
This makes verify more predictable.
 2020-09-01 13:33:04 +02:00
Dietmar Maurer
6b809ff59b src/backup/verify.rs: use separate thread to load data 2020-09-01 12:56:25 +02:00
Thomas Lamprecht
3b2046d263 save last verify result in snapshot manifest 
Save the state ("ok" or "failed") and the UPID of the respective
verify task. With this we can easily allow to open the relevant task
log and show when the last verify happened.

As we already load the manifest when listing the snapshots, just add
it there directly.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-08-26 07:35:13 +02:00
Thomas Lamprecht
1ffe030123 various typo fixes 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-08-25 18:52:31 +02:00
Thomas Lamprecht
5255e641fa SnapshotListItem: add comment field also to schema 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-08-25 16:24:36 +02:00
Thomas Lamprecht
5a718dce17 api datastore: fix typo in error message 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-08-25 14:16:40 +02:00
Dominik Csapak
a1e9c05738 api2/node/services: turn service api calls into workers 
to be in line with pve/pmg and be able to show the progress in the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-14 12:37:17 +02:00
Dietmar Maurer
98161fddb5 cleanup last patch 2020-08-14 07:30:05 +02:00
Hannes Laimer
be614c625f api2/node/../disks/directory: added DELETE endpoint for removal of mount-units 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-08-14 07:06:10 +02:00
Dominik Csapak
93bb51fe7e config/jobstate: replace Job:load with create_state_file 
it really is not necessary, since the only time we are interested in
loading the state from the file is when we list it, and there
we use JobState::load directly to avoid the lock

we still need to create the file on syncjob creation though, so
that we have the correct time for the schedule

to do this we add a new create_state_file that overwrites it on creation
of a syncjob

for safety, we subtract 30 seconds from the in-memory state in case
the statefile is missing

since we call create_state_file from  proxmox-backup-api,
we have to chown the lock file after creating to the backup user,
else the sync job scheduling cannot aquire the lock

also we remove the lock file on statefile removal

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-14 06:38:02 +02:00
Dominik Csapak
713b66b6ed cleanup: replace id from do_sync_job with info from job 
we already have it inside the job itself

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-14 06:36:43 +02:00
Dominik Csapak
77bd2a469c cleanup: merge endtime into TaskState 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-14 06:36:19 +02:00
Dominik Csapak
02543a5c7f api2/pull: extend do_sync_job to also handle schedule and jobstate 
so that we can log if triggered by a schedule, and writing to a jobstatefile
also correctly polls now the abort_future of the worker, so that
users can stop a sync

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-13 11:49:28 +02:00
Dominik Csapak
42b68f72e6 api/{pull, sync}: refactor to do_sync_job 
and move the pull parameters into the worker, so that the task log
contains the error if there is one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-13 11:40:52 +02:00
Dominik Csapak
664d8a2765 api2/admin/sync: use JobState for faster access to state info 
and delete the statefile again on syncjob removal

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-13 11:40:00 +02:00
Dominik Csapak
ae197dda23 server/worker_task: let upid_read_status also return the endtime 
the endtime should be the timestamp of the last log line
or if there is no log at all, the starttime

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-13 11:35:44 +02:00
Dominik Csapak
4c116bafb8 server: change status of a task from a string to an enum 
representing a state via an enum makes more sense in this case
we also implement FromStr and Display to make it easy to convet from/to
a string

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-08-13 11:35:19 +02:00
Wolfgang Bumiller
72dc68323c replace and remove old ticket functions 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-08-12 14:28:21 +02:00
Stefan Reiter
81f293513e backup: lock base snapshot and ensure existance on finish 
To prevent forgetting the base snapshot of a running backup, and catch
the case when it still happens (e.g. via manual rm) to at least error
out instead of storing a potentially invalid backup.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-08-11 11:04:47 +02:00
Stefan Reiter
8b5f72b176 Revert "backup: ensure base snapshots are still available after backup" 
This reverts commit d53fbe2474.

The HashSet and "register" function are unnecessary, as we already know
which backup is the one we need to check: the last one, stored as
'last_backup'.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-08-11 11:03:53 +02:00
Stefan Reiter
f23f75433f backup: flock snapshot on backup start 
An flock on the snapshot dir itself is used in addition to the group dir
lock. The lock is used to avoid races with forget and prune, while
having more granularity than the group lock (i.e. the group lock is
necessary to prevent more than one backup per group, but the snapshot
lock still allows backups unrelated to the currently running to be
forgotten/pruned).

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-08-11 11:02:21 +02:00
Fabian Grünbichler
882c082369 mark signed manifests as such 
for less-confusing display in the web interface

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-08-11 09:56:53 +02:00
Fabian Grünbichler
14f6c9cb8b chunk readers: ensure chunk/index CryptMode matches 
an encrypted Index should never reference a plain-text chunk, and an
unencrypted Index should never reference an encrypted chunk.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-08-11 09:54:22 +02:00
Fabian Grünbichler
2d55beeca0 datastore api: verify blob/index csum from manifest 
when dowloading decoded files.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-08-11 09:52:45 +02:00
Fabian Grünbichler
9238cdf50d datastore api: only decode unencrypted indices 
these checks were already in place for regular downloading of backed up
files, also do them when attempting to decode a catalog, or when
downloading decoded files referenced by a pxar index.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-08-11 09:51:20 +02:00
Wolfgang Bumiller
5d30f03826 impl PartialEq between Realm and RealmRef 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-08-10 12:23:36 +02:00
Wolfgang Bumiller
14263ef989 assert that Username does not impl PartialEq 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-08-10 12:21:12 +02:00
Wolfgang Bumiller
e7cb4dc50d introduce Username, Realm and Userid api types 
and begin splitting up types.rs as it has grown quite large
already

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-08-10 12:05:01 +02:00
Wolfgang Bumiller
866c556faf move types.rs to types/mod.rs 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-08-10 10:32:31 +02:00
Stefan Reiter
4dbe129284 backup: only allow finished backups as base snapshot 
If the datastore holds broken backups for some reason, do not attempt to
base following snapshots on those. This would lead to an error on
/previous, leaving the client no choice but to upload all chunks, even
though there might be potential for incremental savings.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-08-07 07:32:56 +02:00
Stefan Reiter
d74edc3d89 finish_backup: mark backup as finished only after checks have passed 
Commit 9fa55e09 "finish_backup: test/verify manifest at server side"
moved the finished-marking above some checks, which means if those fail
the backup would still be marked as successful on the server.

Revert that part and comment the line for the future.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-08-06 06:39:34 +02:00
Wolfgang Bumiller
98c259b4c1 remove timer and lock functions, fix building with proxmox 0.3.2 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-08-04 11:33:02 +02:00
Dietmar Maurer
70030b43d0 list_snapshots: Returns new "comment" property (fisrt line from notes) 2020-07-31 11:34:42 +02:00
Dietmar Maurer
ff86ef00a7 cleanup: manifest is always CryptMode::None 2020-07-31 10:25:30 +02:00
Dietmar Maurer
912b3f5bc9 src/api2/admin/datastore.rs: add API to get/set Notes for backus 2020-07-31 10:17:35 +02:00
Dietmar Maurer
d7ee07d838 src/api2/backup/environment.rs: remove debug code 2020-07-31 07:48:53 +02:00
Dietmar Maurer
53705acece src/api2/backup/environment.rs: remove debug code 2020-07-31 07:47:08 +02:00
Dietmar Maurer
c8fff67d88 finish_backup: add chunk_upload_stats to manifest 2020-07-31 07:45:47 +02:00
Dietmar Maurer
9fa55e09a7 finish_backup: test/verify manifest at server side 
We want to make sure that the client uploaded a readable manifest.
 2020-07-31 07:45:47 +02:00
Dominik Csapak
f39a900722 api2/node/termproxy: fix user in worker task 
'username' here is without realm, but we really want to use user@realm

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-30 11:57:43 +02:00
Dietmar Maurer
1fc82c41f2 src/api2/backup.rs: aquire backup lock earlier in create_locked_backup_group() 2020-07-30 11:03:05 +02:00
Dominik Csapak
d2b0c78e23 api2/node/termproxy: fix zombies on worker abort 
tokios kill_on_drop sometimes leaves zombies around, especially
when there is not another tokio::process::Command spawned after

so instead of relying on the 'kill_on_drop' feature, we explicitly
kill the child on a worker abort. to be able to do this
we have to use 'tokio::select' instead of 'futures::select' since
the latter requires the future to be fused, which consumes the
child handle, leaving us no possibility to kill it after fusing.
(tokio::select does not need the futures to be fused, so we
can reuse the child future after the select again)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-30 10:38:14 +02:00
Dominik Csapak
adfdc36936 verify: keep track and log which dirs failed the verification 
so that we can print a list at the end of the worker which backups
are corrupt.

this is useful if there are many snapshots and some in between had an
error. Before this patch, the task log simply says to 'look in the logs'
but if the log is very long it makes it hard to see what exactly failed.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-30 09:39:37 +02:00
Dominik Csapak
d8594d87f1 verify: keep also track of corrupt chunks 
so that we do not have to verify a corrupt one multiple times

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-30 09:39:37 +02:00
Stefan Reiter
d53fbe2474 backup: ensure base snapshots are still available after backup 
This should never trigger if everything else works correctly, but it is
still a very cheap check to avoid wrongly marking a backup as "OK" when
in fact some chunks might be missing.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-07-30 08:28:54 +02:00
Stefan Reiter
95bda2f25d backup: use flock on backup group to forbid multiple backups at once 
Multiple backups within one backup group don't really make sense, but
break all sorts of guarantees (e.g. a second backup started after a
first would use a "known-chunks" list from the previous unfinished one,
which would be empty - but using the list from the last finished one is
not a fix either, as that one could be deleted or pruned once the first
simultaneous backup is finished).

Fix it by only allowing one backup per backup group at one time. This is
done via a flock on the backup group directory, thus remaining intact
even after a reload.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-07-30 08:26:26 +02:00
Stefan Reiter
c9756b40d1 datastore: prevent deletion of snaps in use as "previous backup" 
To prevent a race with a background GC operation, do not allow deletion
of backups who's index might currently be referenced as the "known chunk
list" for successive backups. Otherwise the GC could delete chunks it
thinks are no longer referenced, while at the same time telling the
client that it doesn't need to upload said chunks because they already
exist.

Additionally, prevent deletion of whole backup groups, if there are
snapshots contained that appear to be currently in-progress. This is
currently unlikely to trigger, as that function is only used for sync
jobs, but it's a useful safeguard either way.

Deleting a single snapshot has a 'force' parameter, which is necessary
to allow deleting incomplete snapshots on an aborted backup. Pruning
also sets force=true to avoid the check, since it calculates which
snapshots to keep on its own.

To avoid code duplication, the is_finished method is factored out.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-07-30 08:26:01 +02:00
Dietmar Maurer
2aaae9705e src/backup/verify.rs: try to verify chunks only once 
We use a HashSet (per BackupGroup) to track already verified chunks.
 2020-07-29 13:29:13 +02:00
Wolfgang Bumiller
8aa67ee758 bump proxmox to 0.3, cleanup http_err macro usage 
Also swap the order of a couple of `.map_err().await` to
`.await.map_err()` since that's generally more efficient.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-29 09:38:36 +02:00
Wolfgang Bumiller
3865e27e96 src/api2/node.rs: 'mod' statement cleanup 
split them into groups: `pub`, `pub(crate)` and non-pub

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-29 09:19:57 +02:00
Dominik Csapak
7fde1a71ca upload_chunk: allow upload of empty blobs 
a blob can be empty (e.g. an empty pct fw conf), so we
have to set the minimum size to the header size

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-28 11:03:36 +02:00
Dietmar Maurer
39f18b30b6 src/backup/data_blob.rs: new load_from_reader(), which verifies the CRC 
And make verify_crc private for now. We always call load_from_reader() to
verify the CRC.

Also add load_chunk() to datastore.rs (from chunk_store::read_chunk())
 2020-07-28 10:23:16 +02:00
Thomas Lamprecht
e6b599aa6c services: make reload safer and default to it in gui 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-25 20:23:12 +02:00
Thomas Lamprecht
ee15af6bb8 api: service command: fix test for essential service 
makes no sense to disallow reload or start (even if start cannot
really happen)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-24 19:35:19 +02:00
Dominik Csapak
33a88dafb9 server/state: add spawn_internal_task and use it for websockets 
is a helper to spawn an internal tokio task without it showing up
in the task list

it is still tracked for reload and notifies the last_worker_listeners

this enables the console to survive a reload of proxmox-backup-proxy

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-24 11:17:33 +02:00
Dominik Csapak
224c65f8de termproxy: let users stop the termproxy task 
for that we have to do a select on the workers abort_future

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-24 11:17:33 +02:00
Thomas Lamprecht
3d3670d786 termproxy: cmd: support upgrade 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-23 13:12:18 +02:00
Thomas Lamprecht
e744de0eb0 api: termproxy: fix ACL as /nodes is /system 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-23 12:57:11 +02:00
Thomas Lamprecht
98b1733760 api: apt: use schema default const for quiet param 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-23 12:25:28 +02:00
Dominik Csapak
1c2f842a98 api2/nodes: add termproxy and vncwebsocket api calls 
Even though it has nothing to do with vnc, we keep the name of the api
call for compatibility with our xtermjs client.

termproxy:
verifies that the user is allowed to open a console and starts
termproxy with the correct parameters

starts a TcpListener on "localhost:0" so that the kernel decides the
port (instead of trying to rerserving like in pve). Then it
leaves the fd open for termproxy and gives the number as port
and tells it via '--port-as-fd' that it should interpret this
as an open fd

the vncwebsocket api call checks the 'vncticket' (name for compatibility)
and connects the remote side (after an Upgrade) with a local TcpStream
connecting to the port given via WebSocket from the proxmox crate

to make sure that only the client can connect that called termproxy and
no one can connect to an arbitrary port on the host we have to include
the port in the ticket data

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-23 12:06:38 +02:00
Dominik Csapak
a4d1675513 api2/access: implement term ticket 
modeled after pves/pmgs vncticket (i substituted the vnc with term)
by putting the path and username as secret data in the ticket

when sending the ticket to /access/ticket it only verifies it,
checks the privs on the path and does not generate a new ticket

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-23 11:55:00 +02:00
Thomas Lamprecht
27fde64794 api: apt update must run protected 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-23 11:45:52 +02:00
Thomas Lamprecht
fa3f0584bb api: apt: support refreshing package index 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-23 11:21:54 +02:00
Stefan Reiter
a4e86972a4 add .../apt/update API call 
Depends on patched apt-pkg-native-rs. Changelog-URL detection is
inspired by PVE perl code for now, though marked with fixme to use 'apt
changelog' later on, if/when our repos have APT-compatible changelogs
set up.

list_installed_apt_packages iterates all packages and creates an
APTUpdateInfo with detailed information for every package matched by the
given filter Fn.

Sadly, libapt-pkg has some questionable design choices regarding their
use of 'iterators', which means quite a bit of nesting...

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-07-23 10:41:14 +02:00
Fabian Grünbichler
91f5594c08 api: translate ENOTFOUND to 404 for downloads 
and percolate the HttpError back up on the client side

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-07-22 09:19:29 +02:00
Stefan Reiter
113c9b5981 move subscription API path to /nodes 
This aligns it with PVE and allows the widget toolkit's update window
"refresh" to work without modifications once POST /apt/update is
implemented.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-07-21 19:33:52 +02:00
Dominik Csapak
a26c27c8e6 api2/status: fix estimation bug 
when a datastore has enough data to calculate the estimated full date,
but always has exactly the same usage, the factor b of the regression
is '0'

return 0 for that case so that the gui can show 'never' instead of
'not enough data'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-21 13:02:08 +02:00
Dominik Csapak
bfcef26a99 api2/node/status: add fingerprint 
and rename get_usage to get_status (since its not usage only anymore)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-10 11:08:53 +02:00
Wolfgang Bumiller
660a34892d update proxmox crate to 0.2.0 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-10 11:08:27 +02:00
Dominik Csapak
7957fabff2 api: add ZPOOL_NAME_SCHEMA and regex 
poolnames can containe spaces and some other special characters

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-07-09 13:37:31 +02:00
Wolfgang Bumiller
521a0acb2e DataStore::load_manifest: also return CryptMode 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-08 09:19:53 +02:00
Wolfgang Bumiller
f28d9088ed introduce a CryptMode enum 
This also replaces the recently introduced --encryption
parameter on the client with a --crypt-mode parameter.

This can be "none", "encrypt" or "sign-only".

Note that this introduces various changes in the API types
which previously did not take the above distinction into
account properly:

Both `BackupContent` and the manifest's `FileInfo`:
    lose `encryption: Option<bool>`
    gain `crypt_mode: Option<CryptMode>`

Within the backup manifest itself, the "crypt-mode" property
will always be set.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-07 15:24:19 +02:00
Thomas Lamprecht
08cb2038bd api: disks: indentation fixup 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-03 17:59:30 +02:00
Dietmar Maurer
2162e2c15d src/api2/admin/datastore.rs: avoid slash in UPID strings 2020-06-30 13:11:22 +02:00
Dietmar Maurer
fe3e65c3ea src/api2/backup.rs: call register_chunk in previous download api 2020-06-26 08:22:46 +02:00
Dietmar Maurer
b957aa81bd update backup api for incremental backup 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-06-26 07:17:08 +02:00
Dietmar Maurer
8ea00f6e49 allow to abort verify jobs 
And improve job description rendering on gui.
 2020-06-25 12:56:36 +02:00
Stefan Reiter
e22f4882e7 extract create_download_response API helper 
and put it into a new "api2::helpers" module.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-06-25 11:57:37 +02:00
Dietmar Maurer
c2009e5309 src/api2/admin/datastore.rs: add verify api 2020-06-24 13:35:21 +02:00
Stefan Reiter
facd9801cf add incremental backup support 
To support incremental backups (where not all chunks are sent to the
server), a new parameter "reuse-csum" is introduced on the
"create_fixed_index" API call. When set and equal to last backups'
checksum, the backup writer clones the data from the last index of this
archive file, and only updates chunks it actually receives.

In incremental mode some checks usually done on closing an index cannot
be made, since they would be inaccurate.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-06-24 10:01:25 +02:00
Dietmar Maurer
21302088de remove debug println 2020-06-24 09:15:13 +02:00
Stefan Reiter
8268c9d161 fix overflow panic during upload 
if *only* data chunks are registered (high chance during incremental
backup), then chunk_count might be one lower then upload_stat.count
because of the zero chunk being unconditionally uploaded but not used.
Thus when subtracting the two, an overflow would occur.

In general, don't let the client make the server panic, instead just set
duplicates to 0.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-06-24 09:07:22 +02:00
Dominik Csapak
b91b7d9ffd api2/node/disks/zfs: check if default zfs mount path exists 
and if it does bail, because otherwise we would get an
error on mounting and have a zpool that is not imported
and disks that are used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-24 07:22:39 +02:00
Dominik Csapak
d33d8f4e6a api2/admin/datastore: add pxar-file-download api call 
streams a file from a pxar file of an unencrypted backup

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-24 07:21:15 +02:00
Dominik Csapak
5b1cfa01f1 api2/admin/datastore: add 'catalog' api call 
returns the dir listing of the given filepath of the backup snapshot
the filepath has to be base64 encoded or 'root'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-24 07:16:12 +02:00
Dietmar Maurer
60f9a6ea8f src/backup/datastore.rs: add new helpers to load blobs and verify chunks 2020-06-24 06:58:14 +02:00
Dominik Csapak
cbef49bf4f remove absolute paths when executing binaries 
we set the paths manually, so this is ok

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-23 07:09:06 +02:00
Dominik Csapak
f386f512d0 add AsyncReaderStream 
and replace AsyncIndexReader's stream implementation with that

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-23 06:33:31 +02:00
Dietmar Maurer
42d19fdf69 src/api2/node/disks/zfs.rs: always set pool name 2020-06-20 07:15:32 +02:00
Wolfgang Bumiller
4e37d9ce67 add general indented_list_to_tree implementation 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-19 14:37:40 +02:00
Dominik Csapak
6ef9bb59eb api2/admin/datastore: add download-decoded endpoint 
similar to 'download', but streams the decoded file
when it is not encrypted

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-19 08:39:15 +02:00
Dominik Csapak
abdb976340 add Display trait to BackupDir 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-19 08:28:35 +02:00
Dominik Csapak
1c090810f5 api2/admin/datastore/snapshos: show encrypted and size info per file 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-19 07:39:56 +02:00
Dominik Csapak
e181d2f6da add encrypted info to Manifest 
we want to save if a file of a backup is encrypted, so that we can
* show that info on the gui
* can later decide if we need to decrypt the backup

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-19 07:35:39 +02:00
Dominik Csapak
ba694720fc api2/admin/datastore: log stream error during file download 
the client cannot get an error during an chunked http transfer, so at
least log it server side

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-19 06:58:55 +02:00
Dietmar Maurer
026dc1d11f src/api2/node/disks/zfs.rs: add zpool_details api 2020-06-18 15:04:46 +02:00
Dietmar Maurer
3f851d1321 src/api2/node/disks/directory.rs: add early check if disk is unused 2020-06-17 08:31:11 +02:00
Dietmar Maurer
1aef491e24 src/bin/proxmox_backup_manager/disk.rs: add cli to create mounted disks 2020-06-17 08:07:54 +02:00
Dietmar Maurer
66af7f51bc src/tools/disks/zfs.rs: make zfs list parser private 2020-06-17 07:00:54 +02:00
Dietmar Maurer
fb5a066500 src/api2/node/disks.rs: expose directory api 2020-06-16 13:36:32 +02:00
Dietmar Maurer
929a13b357 src/api2/node/disks/zfs.rs: add zpool api 2020-06-16 13:25:53 +02:00
Dietmar Maurer
fbbcd85839 src/api2/node/disks/directory.rs: implement add-datastore feature 2020-06-15 10:01:50 +02:00
Wolfgang Bumiller
0196b9bf5b remove unnecessary .into 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-12 14:57:58 +02:00
Dominik Csapak
20b3094bcb api2/status: add task list endpoint 
for now mostly copy/paste from nodes/nodename/tasks
(without the parameters)
but we should replace the 'read_task_list' with a method
that gives us the tasks since some timestamp

so that we can get a longer list of tasks than for the node
(we could of course embed this then in the nodes/node/task api call and
remove this again as long as the api is not stable)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-12 14:50:17 +02:00
Dominik Csapak
df528ee6fa implement From<TaskListInfo> for TaskListItem 
and use it where its convenient

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-12 14:50:12 +02:00
Dominik Csapak
476b4acadc BackupEnvironment: do not set finished flag prematurely 
we check if all dynamic_writers are closed and if the backup contains
any valid files, we can only mark the backup finished after those
checks, else the backup task gets marked as OK, even though it
is not finished and no cleanups run

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-12 10:37:52 +02:00
Dominik Csapak
ec8f042459 api2/status: use new rrd::extract_cached_data 
and drop the now unused extract_lists function

this also fixes a bug, where we did not add the datastore to the list at
all when there was no rrd data

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-10 13:31:16 +02:00
Dominik Csapak
431cc7b185 rrd: move creation of serde value into api 
there is now a 'extract_cached_data' which just returns
the data of the specified field, and an api function that converts
a list of fields to the correct serde value

this way we do not have to create a serde value in rrd/cache.rs
(makes for a better interface)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-10 13:31:14 +02:00
Dietmar Maurer
d4f2397d4c add api to format disks and create datastores 2020-06-10 11:03:36 +02:00
Dominik Csapak
bda48e04da api2: add status/datastore-usages api call 
returns a list of the datastores and their usages, a list of usages of
the past month (for the gui) and an estimation of when its full
(using the linear regression)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-09 12:20:25 +02:00
Dominik Csapak
33070956af let disk_usage return StorageStatus and use it for datastores/nodes 
disk_usage returned the same values as defined in StorageStatus,
so simply use that

with that we can replace the logic of the datastore status with that
function and also use it for root disk usage of the nodes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-09 12:05:39 +02:00
Wolfgang Bumiller
c443f58b09 switch to external pxar and fuse crates 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-06-08 13:56:58 +02:00
Dietmar Maurer
2b388026f8 src/api2/node/disks.rs: correctly use disk_by_name insteadf of disk_by_node 2020-06-08 07:20:59 +02:00
Dietmar Maurer
707974fdb3 src/api2/node/disks.rs: implement initgpt API 2020-06-07 10:30:34 +02:00
Dietmar Maurer
9069debcd8 src/api2/types.rs: define BLOCKDEVICE_NAME_SCHEMA 2020-06-07 07:20:25 +02:00
Dietmar Maurer
ce8e3de401 move disks api to /node/<node>/disks 2020-06-06 14:43:36 +02:00
Dietmar Maurer
7fa2779559 src/api2/disks.rs: implement smart api 2020-06-06 12:23:11 +02:00
Dietmar Maurer
ff30caeaf8 src/api2/disks.rs - list-disks: add usage-type filter 2020-06-06 11:48:58 +02:00
Dietmar Maurer
553cd12ba6 src/api2/disks.rs: start disks api 2020-06-06 11:38:47 +02:00
Dominik Csapak
8d78589969 improve display of 'next run' for sync jobs 
if the last sync job is too far in the past (or there was none at all
for now) we run it at the next iteration, so we want to show that

we now calculate the next_run by using either the real last endtime
as time or 0

then in the frontend, we check if the next_run is < now and show 'pending'
(we do it this way also for replication on pve)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-06-04 07:03:54 +02:00
Thomas Lamprecht
add5861e8d typo fixes all over the place 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-05-30 16:39:08 +02:00
Dietmar Maurer
96d65fbcd0 cleanup: define/use const for predefined blob file names. 2020-05-30 14:04:15 +02:00
Dominik Csapak
04b0ca8b59 add owner to group and snapshot listings 
while touching it, make columns and tbar in DataStoreContent.js
declarative members and remove the (now) unnecessary initComponent

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-30 07:24:12 +02:00
Dominik Csapak
d43f86f3f3 api2: add admin/sync endpoint 
this returns the list of syncjobs with status, as opposed to
config/sync (which is just the config)

also adds an api call where users can run the job manually under
/admin/sync/$ID/run

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-29 11:31:32 +02:00
Dominik Csapak
268687ddf0 api2/pull: refactor priv checking and creating pull parameters 
we want to reuse those in the api call for manually running a sync job

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-29 11:27:43 +02:00
Dominik Csapak
426c1e353b api2/config/sync: fix id parameter 
'name' is not the correct parameter for get/post

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-29 11:24:54 +02:00
Dominik Csapak
2888b27f4c create SYNC_SCHEDULE_SCHEMA to adapt description for sync jobs 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-29 11:24:25 +02:00
Dominik Csapak
9857472211 fix removing of remotes 
we have to save the remote config after removing the section

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-29 10:48:26 +02:00
Thomas Lamprecht
090decbe76 BACKUP_REPO_URL_REGEX: move to api2::types and allow all valid data store names 
The repo URL consists of
* optional userid
* optional host
* datastore name

All three have defined regex or format, but none of that is used, so
for example not all valid datastore names are accepted.

Move definition of the regex over to api2::types where we can access
all required regexes easily.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-05-29 06:29:23 +02:00
Thomas Lamprecht
c32186595e api2::types: factor out USER_ID regex 
allows for better reuse in a next patch

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-05-29 06:27:38 +02:00
Dietmar Maurer
c94e1f655e rrd stats: improve io delay stats 2020-05-28 19:12:13 +02:00
Dietmar Maurer
91e5bb49f5 src/bin/proxmox-backup-proxy.rs: simplify code 
and gather all stats for the root disk
 2020-05-28 12:30:54 +02:00
Dominik Csapak
de4db62c57 remotes: save passwords as base64 
to avoid having arbitrary characters in the config (e.g. newlines)
note that this breaks existings configs

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-26 12:38:06 +02:00
Dietmar Maurer
1a0d3d11d2 src/api2/admin/datastore.rs: add rrd api 2020-05-26 12:26:14 +02:00
Dominik Csapak
83fd4b3b1b remote: try to use Struct for api 
with a catch: password is in the struct but we do not want it to return
via the api, so we only 'serialize' it when the string is not empty
(this can only happen when the format is not checked by us, iow.
when its returned from the api) and setting it manually to ""
when we return remotes from the api

this way we can still use the type but do not return the password

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-26 08:55:07 +02:00
Dominik Csapak
5dcdcea293 api2/config/remote: remove password from read_remote 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-26 08:49:12 +02:00
Dominik Csapak
99f443c6ae api2/config/remote: lock and use digest for removal 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-26 08:48:45 +02:00
Dominik Csapak
4f966d0592 api2/config/remote: use rpcenv for digest for read_remote 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-26 08:48:28 +02:00
Dominik Csapak
db0c228719 config/remote: add 'name' to Remote struct 
and use it as section id, like with User

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-26 08:48:05 +02:00
Dietmar Maurer
d8dc281992 www/DataStoreStatus.js: display loadavg stats 2020-05-25 11:54:15 +02:00
Dietmar Maurer
3e930f2bdc www/DataStoreStatus.js: display root disk stats 2020-05-25 11:34:24 +02:00
Dietmar Maurer
c1b24fbf0b www/DataStoreStatus.js: display swap stats 2020-05-25 10:39:54 +02:00
Dietmar Maurer
a608806f65 www/DataStoreStatus.js: display netin/netout 2020-05-24 19:02:35 +02:00
Dietmar Maurer
803ab12ad4 rrd: simplify code 2020-05-23 15:37:17 +02:00
Dietmar Maurer
a4a3f7ca5e rrd: pack multiple rrd values into th estat list 2020-05-23 14:03:44 +02:00
Dietmar Maurer
a2f862eed6 add experimental rrd api to get cpu stats 2020-05-23 11:50:53 +02:00
Dietmar Maurer
07ad6470ca src/client/pull.rs: split out pull related code 2020-05-22 08:04:20 +02:00
Dietmar Maurer
183125d576 src/api2/pull.rs: aquire try_shared_chunk_store_lock inside pull_store 2020-05-22 07:24:17 +02:00
Dietmar Maurer
6f652b1b3a rename 'job' to 'sync' 2020-05-21 10:29:25 +02:00
Dietmar Maurer
b1d4edc769 src/api2/config/job.rs: add job api 2020-05-21 10:16:35 +02:00
Dietmar Maurer
b4900286ce src/config/jobs.rs: use SectionConfig for jobs 2020-05-21 10:16:35 +02:00
Dominik Csapak
2882c881e9 api2/access/acl: add path and exact parameter to list_acl 
so that we can get only a subset of the acls, filtered by the backed
also return the digest here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-20 13:44:36 +02:00
Dominik Csapak
12e3895399 api2/access/acl: make update_acl a protected api call 
since we want to set the owner of the acl config to 'root'
which is only possible when using a protected api call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-20 13:22:41 +02:00
Dominik Csapak
11b6391c83 add 'exact' parameter to extract_acl_node_data 
so that we can return acls for a single path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-20 13:22:10 +02:00
Dominik Csapak
b05672579e api2/roles: change return field of role to roleid 
to be compatible with the pve api
with this, we can reuse the ui parts (RoleSelector)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-20 13:21:47 +02:00
Dominik Csapak
5160c0e986 api2/acl: add privs array to roles 
so that an admin can see which roles have which privileges

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-20 13:21:37 +02:00
Dietmar Maurer
872062ee9f src/config/datastore.rs_ change prune types from i64 to u64 2020-05-20 13:00:13 +02:00
Dietmar Maurer
67f7ffd0db src/config/datastore.rs: add prune settings 2020-05-20 11:29:59 +02:00
Dietmar Maurer
0fafac2492 src/api2/access/user.rs: remove useless description 
The description is not used at all if we refer to a type.
 2020-05-20 11:27:58 +02:00
Dietmar Maurer
49ff10921c src/api2/types.rs: define PRUNE_SCHEMA_KEEP_* 2020-05-20 10:13:38 +02:00
Dietmar Maurer
42fdbe5112 src/config/datastore.rs: add gc-schedule property 2020-05-20 08:38:10 +02:00
Dietmar Maurer
7d4e362993 depend on proxmox 0.1.32, src/api2/access/user.rs: simplify code 2020-05-19 12:58:46 +02:00
Dominik Csapak
522c0da0a0 use new 'id_property' for user::User and use it in api calls 
this allows us to return a user::User (or Vec<> of it)
instead of a generic serde value

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-05-19 09:33:56 +02:00
Dietmar Maurer
e8d1da6a15 depend on proxmox 0.1.31 - use Value to store result metadata 2020-05-18 09:57:35 +02:00
Dietmar Maurer
30f577248b src/api2/node/time.rs: avoid custom unsafe readlink implementations 2020-05-15 06:50:07 +02:00
Dietmar Maurer
f486e9e50e add systemd configuration file parser/writer, start job configuration 2020-05-12 13:07:49 +02:00
Dietmar Maurer
3dd27a3bf8 src/api2/node/network.rs: add protected flag to revert 2020-05-08 09:30:25 +02:00
Dietmar Maurer
3aedb73816 src/api2/node/network.rs: pass bridge_ports and slaves a property strings 
To make it compatible with pve.
 2020-05-08 08:49:17 +02:00
Dietmar Maurer
bab5d18c3d src/config/network.rs: implement bond_mode 
and rename bond_slaves to slaves to make it compatible with pve.
 2020-05-07 14:07:45 +02:00
Dietmar Maurer
c2ffc68554 src/api2/node/network.rs: cleanup - factor out check_duplicate_gateway 2020-05-07 11:26:30 +02:00
Dietmar Maurer
9651833130 src/api2/node/network.rs: allow to create bridge and bond 2020-05-07 11:09:12 +02:00
Dietmar Maurer
7b22acd0c2 src/config/network.rs: make it compatible with pve 
and depend on proxmox 0.1.26
 2020-05-07 09:28:25 +02:00
Dietmar Maurer
5751e49566 src/server/worker_task.rs: implement and use status command 2020-05-07 09:27:33 +02:00
Dietmar Maurer
99641a6bbb garbage_collect: call fail_on_abort to abort GV when requested. 2020-05-05 09:06:34 +02:00
Dietmar Maurer
db1e061dcb src/bin/proxmox-backup-client.rs: correctly format prune result list. 2020-05-05 06:45:37 +02:00
Dietmar Maurer
ed751dc2ab src/api2/node/status.rs: rework api, implement reboot and shutdown 2020-04-30 11:52:40 +02:00
Dietmar Maurer
ca9dfe5fa4 src/api2/node/tasks.rs: use api macro features for default values 2020-04-30 11:51:56 +02:00
Dietmar Maurer
720af9f69b src/api2/node/tasks.rs: allow users to list/access there own tasks 2020-04-30 10:05:50 +02:00
Dietmar Maurer
f1490da82a use resonable acl paths (fixup) 2020-04-30 09:32:13 +02:00
Dietmar Maurer
74c08a5782 use reasonable acl paths 2020-04-30 09:30:00 +02:00
Dietmar Maurer
bc0d03885c use proxmox 0.1.25, use new EnumEntry feature 2020-04-29 13:01:24 +02:00
Dietmar Maurer
30fb602578 src/api2/admin/datastore.rs - get_datastore_list: only return name and comment 
We dont want to leak the full configuration to users with limited access permission.
Please use the api2::config::datastore api to get the full configuration.
 2020-04-29 09:21:34 +02:00
Dietmar Maurer
0a00f6e01c src/api2/config/datastore.rs_ add delete property to update method 2020-04-29 09:09:59 +02:00
Dietmar Maurer
30003baaa4 src/api2/config/remote.rs: fix white space 2020-04-29 09:09:39 +02:00
Dietmar Maurer
5211705ff1 src/api2/config/remote.rs: add delete parameter to update method 2020-04-29 09:04:17 +02:00
Dietmar Maurer
ec67af9af3 src/api2/pull.rs: require Datastore.Prune if delete flag is set. 2020-04-29 07:19:32 +02:00
Dietmar Maurer
8247db5b39 src/config/acl.rs: introduice privileges and roles for remotes 2020-04-29 07:03:44 +02:00
Dietmar Maurer
409f44247b fix api2::types::ACL_ROLE_SCHEMA 
make sure we list all roles ...
 2020-04-28 13:25:02 +02:00
Dietmar Maurer
54552dda59 implemnt backup ownership, improve datastore access permissions 2020-04-28 10:22:25 +02:00
Dietmar Maurer
d00e1a216f src/config/acl.rs: introduce more/better datastore privileges 2020-04-27 07:13:50 +02:00
Dietmar Maurer
9c7fe29dfc src/config/acl.rs: rtename PRTIV_DATASTORE_ALLOCATE to PRIV_DATASTORE_MODIFY 2020-04-27 06:50:35 +02:00
Dietmar Maurer
14627d671a src/bin/proxmox-backup-manager.rs: add dns sub command 
Also improved the DNS api, added a --delete option.
 2020-04-26 08:23:23 +02:00
Dietmar Maurer
26d9aebc28 move src/api2/config/network.rs to src/api2/node/network.rs 
So that we have the same api path for network config as pve.
 2020-04-25 17:00:38 +02:00
Dietmar Maurer
fd7c0979b4 src/bin/proxmox-backup-manager.rs: implement netwerk revert 2020-04-24 10:45:49 +02:00
Dietmar Maurer
2eefd9aee1 src/config/network.rs: implement network reload, set "changes" attribute 2020-04-24 09:55:46 +02:00
Dietmar Maurer
8a6b86b8a7 src/config/network.rs: use a simple String for comments 2020-04-24 07:46:08 +02:00
Dietmar Maurer
10a9be45bd src/api2/config/network.rs: implement update/delete comments 2020-04-23 16:08:35 +02:00
Dietmar Maurer
5f60a58fd5 src/config/network.rs; support interface comments, cleanups 2020-04-23 15:54:30 +02:00
Dietmar Maurer
5e4e88e83f src/api2/config/network.rs: implement update/delete for bridge_ports and bond_slaves 2020-04-23 11:21:27 +02:00
Dietmar Maurer
339965d720 src/api2/config/network.rs: only allow one default gateway 2020-04-23 10:37:40 +02:00
Dietmar Maurer
42fbe91a34 src/config/network.rs: parse bond-slaves 2020-04-23 09:31:10 +02:00
Dietmar Maurer
1d9a68c2fc src/config/network.rs: parse bridge-ports 2020-04-23 09:24:17 +02:00
Dietmar Maurer
02269f3dba src/config/network.rs: introduce NetworkInterfaceType 2020-04-23 08:45:03 +02:00
Dietmar Maurer
2c18efd902 src/config/network.rs: use a single mtu setting (instead of mtu_v4 and mtu_v6) 2020-04-23 07:07:14 +02:00
Dietmar Maurer
b1564af25a src/bin/proxmox-backup-manager.rs: format datastore list output 2020-04-22 17:37:20 +02:00
Dietmar Maurer
f1026a5aa9 src/api2/config/network.rs: allow to update 'auto' flag 2020-04-22 16:46:46 +02:00
Dietmar Maurer
3fce3bc36e src/config/network/parser.rs: parse MTU settings 2020-04-22 13:44:51 +02:00
Dietmar Maurer
c016482c7a src/api2/config/network.rs: implement delete property 2020-04-22 12:19:31 +02:00
Dietmar Maurer
27f2c23049 src/api2/config/network.rs: allow to update configuration method 2020-04-22 11:32:36 +02:00
Dietmar Maurer
df6bb03d0e src/api2/config/network.rs: improve network api 2020-04-22 10:54:07 +02:00
Dietmar Maurer
76cf5208cf src/api2/types.rs: add schemas for IP/CIDR 2020-04-22 10:28:53 +02:00
Dietmar Maurer
2ea7bf1b3d src/api2/config/datastore.rs_ fix method docs 2020-04-22 08:53:16 +02:00
Dietmar Maurer
8b57cd4441 src/config/network.rs: remove netmask support 
rely on cidr instead.
 2020-04-22 08:45:13 +02:00
Dietmar Maurer
68da20bf62 src/api2/types.rs: define NETWORK_INTERFACE_NAME_SCHEMA 2020-04-21 17:54:52 +02:00
Dietmar Maurer
c357260d09 src/config/network.rs: move type definitions to src/api2/types.rs 2020-04-21 17:25:05 +02:00
Dietmar Maurer
ca0e534796 src/api2/config/network.rs: start network configuration api 2020-04-21 14:28:26 +02:00
Dietmar Maurer
365f0f720c fix permission tests using non-uri parameters 
We nood to do those tests inside the fuction body instead...
 2020-04-18 08:23:04 +02:00
Dietmar Maurer
bb072ba49c src/api2/access.rs: cleanup 2020-04-18 07:28:25 +02:00
Dietmar Maurer
ff329f970b src/api2/types.rs: use anyhow::Error in test cases 2020-04-18 07:05:31 +02:00
Wolfgang Bumiller
f7d4e4b506 switch from failure to anyhow 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-04-17 18:43:30 +02:00
Dietmar Maurer
404d78c41e src/api2/pull.rs: add access permission 2020-04-17 15:27:04 +02:00
Dietmar Maurer
1bfc1efa50 src/api2/subscription.rs: add access permissions 2020-04-17 15:14:28 +02:00
Dietmar Maurer
73ce1d1146 src/api2/reader.rs: add access permissions 2020-04-17 15:01:56 +02:00
Dietmar Maurer
70e5f2461d src/api2/config/remote.rs: add access permissions 2020-04-17 14:57:26 +02:00
Dietmar Maurer
c0ef209aeb src/api2/config/datastore.rs: impl digest check for delete, add access permissions 2020-04-17 14:51:29 +02:00
Dietmar Maurer
9f9f7eefa3 src/api2/backup.rs: add access permissions 2020-04-17 14:40:20 +02:00
Dietmar Maurer
bb34b58910 src/api2/admin/datastore.rs: add access permissions - first try 
We need to refine this later (introduce backup owner concept?)
 2020-04-17 14:36:27 +02:00
Dietmar Maurer
5972def5ec acl: change path "storage" to "datastore" 2020-04-17 14:15:44 +02:00
Dietmar Maurer
aa90ced3bf src/api2/access/role.rs: use schema ACL_ROLE_SCHEMA 2020-04-17 14:14:06 +02:00
Dietmar Maurer
ca257c8097 move type defs from src/api2/access/acl.rs to src/api2/types.rs 2020-04-17 14:13:15 +02:00
Dietmar Maurer
3fff55b293 src/api2/access/role.rs: new api to list roles 2020-04-17 14:03:24 +02:00
Dietmar Maurer
4f66423fcc src/api2/access/user.rs: add access permissions 2020-04-17 11:04:36 +02:00
Dietmar Maurer
d4f020f4c5 src/api2/access/user.rs: add access permissions 2020-04-17 10:08:45 +02:00
Dietmar Maurer
d28ddb8e04 src/api2/access/acl.rs: add access permissions 2020-04-17 10:03:09 +02:00
Dietmar Maurer
83b6a7cf71 src/api2/node/tasks.rs: use api macro, implement access permissions 2020-04-16 17:47:21 +02:00
Dietmar Maurer
e4681f9f71 src/api2/node/syslog.rs: add access permissions 2020-04-16 17:08:19 +02:00
Dietmar Maurer
b5037fa8ed src/api2/node/status.rs: add access permissions 2020-04-16 17:05:09 +02:00
Dietmar Maurer
1cf7bbf412 src/api2/node/services.rs: add access permissions 2020-04-16 12:47:16 +02:00
Dietmar Maurer
68ed0c629d src/api2/node/journal.rs: add access permissions 2020-04-16 12:47:16 +02:00
Dietmar Maurer
4b40148caa start impl. access permissions 2020-04-16 12:47:16 +02:00
Dietmar Maurer
109d7817cd src/config/user.rs - cached_config: do not store/return digest 2020-04-15 11:35:57 +02:00
Dietmar Maurer
bd098a7f77 src/api2/node/dns.rs: use api macro (cleanup) 2020-04-15 10:09:18 +02:00
Dietmar Maurer
9c06f6c292 fix previous commit - use result. 2020-04-14 17:48:10 +02:00
Dietmar Maurer
9f4e47dd93 acl update: check path 2020-04-14 17:23:48 +02:00
Dietmar Maurer
d83175dd69 acl update: check if user exist. 2020-04-14 13:46:27 +02:00
Dietmar Maurer
9765092ede acl api: implement update 2020-04-14 10:16:49 +02:00
Dietmar Maurer
ed3e60ae69 start ACL api 2020-04-13 11:09:44 +02:00
Dietmar Maurer
b88f9c5b1e PASSWORD_SCHEMA: set max_length to 1024 (for tickets) 2020-04-09 13:35:58 +02:00
Dietmar Maurer
879546aff6 api: add default property to domain list 2020-04-09 13:35:08 +02:00
Dietmar Maurer
73b40e9b46 api: correctly sort access subdirmap 2020-04-09 13:34:07 +02:00
Dietmar Maurer
708db4b3ae api: add list_domains 2020-04-09 11:36:45 +02:00
Dietmar Maurer
685e13347e api: move config/user to access/users, implement change_password 
To make it similar to the pve api
 2020-04-09 10:21:24 +02:00
Dietmar Maurer
7d817b0358 implement auth framework 2020-04-08 14:06:15 +02:00
Dietmar Maurer
579728c641 add user configiguration 2020-04-08 14:06:15 +02:00
Dietmar Maurer
cf459b1982 gc: log pending removals 2020-04-06 09:50:40 +02:00
Dietmar Maurer
dda7015497 prune api: return a usable result (we run synchronous anyways) 2020-04-01 12:24:28 +02:00
Dominik Csapak
6650a242fb rewrite future select in upgrade_to_backup_protocol using select macro 
and handle all ok/err cases with at least logging

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-03-18 11:33:59 +01:00
Dietmar Maurer
f24fc1166b src/bin/proxmox-backup-client.rs - list_snapshots: use format_and_print_result_full() 
Depend on proxmox v0.1.14.
 2020-02-27 11:28:44 +01:00
Dietmar Maurer
ea5f547fdc src/bin/proxmox-backup-client.rs - list_snapshot_files: use format_and_print_result_full() 2020-02-26 13:49:47 +01:00
Dietmar Maurer
8528fce874 src/api2/node/tasks.rs; make list_tasks public 2020-01-30 13:31:20 +01:00
Dietmar Maurer
70fa1aa3fc fixup 2020-01-30 13:31:20 +01:00
Dietmar Maurer
5eeea607ae src/section_config.rs - convert_to_array: add option to skip properties 2020-01-30 13:26:46 +01:00
Dietmar Maurer
99384f7933 src/api2/types.rs: define and use struct TaskListItem 2020-01-28 11:23:08 +01:00
Dietmar Maurer
2c4b303c62 src/api2/node/tasks.rs: use api macro for list_tasks 2020-01-28 11:23:08 +01:00
Dietmar Maurer
d59dbeca1b src/client/http_client.rs: new struct HttpClientOptions 2020-01-25 12:18:00 +01:00
Dietmar Maurer
6afbe1d846 src/config/remote.rs: add fingerprint 2020-01-25 09:49:45 +01:00
Dietmar Maurer
dcb8db66d9 src/api2/types.rs: add CERT_FINGERPRINT_SHA256_SCHEMA 2020-01-25 09:49:45 +01:00
Dietmar Maurer
4191018c79 src/api2/admin/datastore.rs - download_file: use BACKUP_ARCHIVE_NAME_SCHEMA for file-name 2020-01-23 13:49:30 +01:00
Dietmar Maurer
a92830dc39 src/api2/types.rs: define and use api type GarbageCollectionStatus 2020-01-23 13:40:12 +01:00
Dietmar Maurer
dfc58d47ad src/api2/admin/datastore.rs: use api macro for start_garbage_collection 2020-01-23 13:01:21 +01:00
Dietmar Maurer
1dc117bbba src/api2/types.rs: define and use struct StorageStatus 2020-01-23 12:51:30 +01:00
Dietmar Maurer
09b1f7b202 src/api2/types.rs: define struct BackupContent, and use it with list_snapshot_files 2020-01-23 11:17:38 +01:00
Dietmar Maurer
68a6a0ee03 src/api2/admin/datastore.rs: use api macro for delete_snapshot 2020-01-23 10:16:45 +01:00
Dietmar Maurer
c425bdc9b2 src/api2/pull.rs: delete fanished snapshots 2020-01-23 09:58:57 +01:00
Dietmar Maurer
18cc66ee85 src/api2/pull.rs: use new Datastore::last_successful_backup helper 
Do not fail if local group does not exists.
 2020-01-22 15:06:28 +01:00
Thomas Lamprecht
1ca037d81c followup: commit all changes 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-01-22 14:14:52 +01:00
Thomas Lamprecht
44b65b4ec7 api/datastore: do not allow updating path for now 
It's a bit dangerous as it points to all the saved backups, so they
would be seemingly lost after updating the path.
Follow our logic from other products, e.g. in PVE we do not allow to
update the backing path/location of a storage either for similar
reasons.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-01-22 13:55:37 +01:00
Dietmar Maurer
f1d99e3f6a src/bin/proxmox-backup-client.rs: use a std channel to write the catalog 2020-01-22 12:49:08 +01:00
Wolfgang Bumiller
9ea4bce444 bump proxmox crate to 0.1.7 
The -sys, -tools and -api crate have now been merged into
the proxmx crate directly. Only macro crates are separate
(but still reexported by the proxmox crate in their
designated locations).

When we need to depend on "parts" of the crate later on
we'll just have to use features.

The reason is mostly that these modules had
inter-dependencies which really make them not independent
enough to be their own crates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-01-21 13:48:37 +01:00
Dietmar Maurer
4b4eba0b9e src/api2/pull.rs: implement delete flag for vanished groups 2020-01-17 11:24:55 +01:00
Dietmar Maurer
b31c8019d7 src/api2/types.rs: define and use GroupListItem 2020-01-17 10:17:18 +01:00
Dietmar Maurer
f357390c15 renamed: src/config/remotes.rs -> src/config/remote.rs 
And use 'remote' instead of 'remotes' everywhere.
 2020-01-16 14:32:06 +01:00
Dietmar Maurer
94609e2380 src/api2/pull.rs: use reference to remote configuration 2020-01-16 13:55:19 +01:00
Dietmar Maurer
002a191abf src/api2/config: check digest 2020-01-15 12:27:05 +01:00
Dietmar Maurer
347834df25 src/api2/config: correctly lock files 2020-01-15 11:57:12 +01:00
Oguz Bektas
1c79e1017f services: fix indentation 
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
 2020-01-15 10:55:04 +01:00
Dietmar Maurer
f3ec5dae7f src/api2/config: improve api schema by adding return type 2020-01-14 14:55:58 +01:00
Dietmar Maurer
c5799e4043 src/api2/config/datastore.rs: improve api, implement update and read 2020-01-14 14:45:56 +01:00
Dietmar Maurer
08195ac823 src/api2/config/remotes.rs: improve api, implement update and read 2020-01-14 14:20:16 +01:00
Dietmar Maurer
d0187a51a9 src/section_config.rs - convert_to_array: optionally add digest 
datastore::config() -> also return digest
remotes::config() -> also return digest
 2020-01-14 12:57:03 +01:00
Dietmar Maurer
7e7b781a18 src/api2/types.rs: also define PASSWORD_FORMAT and use it correctly 2020-01-14 11:32:02 +01:00
Dietmar Maurer
da4a15a351 src/api2/types.rs: define and use PASSWORD_REGEX 2020-01-14 11:22:42 +01:00
Dietmar Maurer
ff620a3d1a src/api2/types.rs: add regression test for PROXMOX_USER_ID_SCHEMA 2020-01-13 17:26:43 +01:00
Dietmar Maurer
ae62c4fe59 src/api2/types.rs: fix regex patterns 2020-01-13 14:38:31 +01:00
Dietmar Maurer
163dc16c0b src/api2/types.rs: define PROXMOX_AUTH_REALM_SCHEMA and PROXMOX_USER_ID_SCHEMA 
And try to use nbew schemas with config api...
 2020-01-13 14:18:19 +01:00
Dietmar Maurer
b25f313d66 src/api2/types.rs: define HOSTNAME_FORMAT, DNS_NAME_FORMAT and DNS_NAME_OR_IP_FORMAT 2020-01-13 13:10:52 +01:00
Dietmar Maurer
454c13edce src/api2/types.rs: define SINGLE_LINE_COMMENT_SCHEMA 2020-01-13 12:02:13 +01:00
Dietmar Maurer
167971ed49 src/api2/types.rs: define REMOTE_ID_SCHEMA here 2020-01-13 11:47:07 +01:00
Dietmar Maurer
50af953e1b src/section_config.rs: make set_data generic 2020-01-11 11:09:27 +01:00
Dietmar Maurer
688fbe07a1 cleanup config api, add remotes config cli interface 2020-01-11 10:42:09 +01:00
Dietmar Maurer
ca44172404 src/api2/config/remotes.rs: cleanup - avoid compiler warnings 2020-01-10 19:25:26 +01:00
Dietmar Maurer
141304d64e src/api2/config/remotes.rs: new API to configure remotes 2020-01-10 13:28:15 +01:00
Dietmar Maurer
eb506c830c rename src/api2/sync.rs -> src/api2/pull.rs 
Use "pull" instead of "sync", because this also indicates a
direction (like "sync from").
 2020-01-10 11:15:16 +01:00
Dietmar Maurer
71da3d6a1b src/api2/types.rs: fix SnapshotListItem schema definition 2020-01-09 10:36:19 +01:00
Dietmar Maurer
de8ec04123 src/api2/sync.rs: implement remote sync 2020-01-08 14:03:52 +01:00
Dietmar Maurer
fc189b198c src/api2/types.rs: define and use new api type SnapshotListItem 2020-01-07 12:52:52 +01:00
Dietmar Maurer
550e0d8870 src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate 
Without calling external openssl binary.
 2019-12-22 17:35:06 +01:00
Wolfgang Bumiller
5e62d19cfb fix create_datastore parameters: 
comment was declared as optional but unwrap()ed, causing
panics when not using it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-20 12:30:25 +01:00
Dietmar Maurer
645995634a src/api2/config/datastore.rs - create: pass uid and gid instead of User 2019-12-20 09:23:58 +01:00
Dietmar Maurer
e67770d496 src/backup/chunk_store.rs - create: pass User instead of CreateOptions 2019-12-20 09:11:40 +01:00
Thomas Lamprecht
bca9093520 api/ui: datastore: allow to set simple comment 
for now forbid all control characters[0] in the comment value, the
section config writer cannot cope with newlines in the value, it
writes them out literally, allowing "injection" or breaking the whole
config.

In the webinterface use also a textfield, not a textarea.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-19 17:58:01 +01:00
Wolfgang Bumiller
afdcfb5bc9 let ChunkStore::create take CreateOptions 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-19 13:14:49 +01:00
Thomas Lamprecht
2337df7b3d api2: add very basic 'status/usage' endpoint for nodes 
For returning the nodes basic "usage status", for now one gets memory
and CPU utilization.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-18 17:24:32 +01:00
Wolfgang Bumiller
feaa1ad35f replace file_set_contents with replace_file 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-18 11:16:04 +01:00
Dietmar Maurer
a66e7920dc src/api2/config/datastore.rs: set protected flags for create/delete API 2019-12-18 10:52:45 +01:00
Dietmar Maurer
f0db500808 src/api2/config/datastore.rs: fix api docu 2019-12-18 10:48:19 +01:00
Dietmar Maurer
81cc71c0d7 use proxmox-mini-journalreader to display syslog 2019-12-17 14:06:48 +01:00
Dietmar Maurer
20197f7cf5 src/api2/node/syslog.rs: use api macro 2019-12-17 13:26:49 +01:00
Dietmar Maurer
bb084b9c91 api2: update for latest proxmox-api changes 
- rename ApiFuture into ApiResponseFuture
- impl. ApiHandler::Async
 2019-12-16 10:01:51 +01:00
Dietmar Maurer
236a396aa1 src/api2/admin/datastore.rs - prune: log retention options 2019-12-14 16:32:16 +01:00
Dietmar Maurer
503995c767 src/api2/admin/datastore.rs: fix prune worker id 2019-12-14 16:05:21 +01:00
Wolfgang Bumiller
db0cb9ce0b update a chunk of stuff to the hyper release 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-13 11:24:41 +01:00
Dietmar Maurer
1ae5677de0 src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT 2019-12-12 14:15:40 +01:00
Dietmar Maurer
d0adf270fb src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA 2019-12-12 12:37:11 +01:00
Dietmar Maurer
66c49c21c3 src/api2/types.rs: factor out DATASTORE_SCHEMA 2019-12-11 13:45:23 +01:00
Dietmar Maurer
567d3e00fb src/api2/node/tasks.rs: new filter "store" to filter tasks for one store 2019-12-11 12:53:34 +01:00
Dietmar Maurer
6d04612955 src/api2/node/tasks.rs: add "running" filter to list only running tasks 2019-12-11 10:52:27 +01:00
Dietmar Maurer
8560fe3e5f src/api2/node/tasks.rs - extract_upid: avoid duplicate error message 2019-12-10 13:45:56 +01:00
Dietmar Maurer
5830c20560 src/bin/proxmox-backup-client.rs: add simple task management cli 2019-12-10 13:43:53 +01:00
Dietmar Maurer
3b03abfe14 src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune 
And use display_task_log on the client side.
 2019-12-08 10:59:47 +01:00
Dietmar Maurer
163e9bbe91 src/bin/proxmox-backup-client.rs: new display task log helper 
New helper to display worker task logs. Use it for prune (experimental).
 2019-12-07 16:11:26 +01:00
Dietmar Maurer
6b508dd563 src/server/formatter.rs: impl. new result attribute "active" 2019-12-07 15:29:42 +01:00
Dietmar Maurer
102d8d4136 src/backup/prune.rs: implement --keep-hourly 2019-12-07 11:23:33 +01:00
Dietmar Maurer
1f604f1a5a src/api2/admin/datastore.rs: add new api test_prune() 2019-12-06 13:06:56 +01:00
Dietmar Maurer
0ab08ac980 src/api2/admin/datastore.rs: cleanup - move schema defininition in front of rust method 2019-12-06 12:36:41 +01:00
Dietmar Maurer
1a0678b601 src/api2/admin/datastore.rs: remove prune parameters from status call 
Not idea why they are added there.
 2019-12-06 12:32:53 +01:00
Dietmar Maurer
9e3f008804 src/backup/prune.rs: add new helper keeps_something() 2019-12-06 12:28:31 +01:00
Dietmar Maurer
9b78352188 src/backup/prune.rs: define new struct PruneOptions 2019-12-06 08:56:27 +01:00
Dietmar Maurer
92acbd69f7 src/backup/backup_info.rs: remove old prune code 2019-12-06 08:29:27 +01:00
Dietmar Maurer
8f0b4c1f90 src/api2/admin/datastore.rs - prune: log info about all snapshots 2019-12-05 15:35:11 +01:00
Dietmar Maurer
9ce42759ec src/backup/backup_info.rs - compute_prune_list: remove unfinished backups 2019-12-05 11:18:10 +01:00
Dietmar Maurer
aeeac29bb1 src/backup/backup_info.rs: new compute_prune_list helper 
So that we can write regression tests for this.
 2019-12-04 15:49:11 +01:00
Dietmar Maurer
3d229a4a01 Revert "test another api macro use" 
This reverts commit a9aa52e6a8.

Because we do not want to use macros for the backup protocol for now.

And because it crashes backup tasks for some unknown reason.
 2019-12-04 14:18:11 +01:00
Dietmar Maurer
4f1e40a29e src/api2/admin/datastore.rs - read_backup_index: add manifest with size 
So that the client can display the size of the index file.
 2019-12-04 10:03:52 +01:00
Wolfgang Bumiller
a9aa52e6a8 test another api macro use 
The api macro now supports hyphens in parameter names and
referencing externally defined `Schema`s, so here's an
example.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-03 13:41:40 +01:00
Wolfgang Bumiller
41874331ed whitespace fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-29 09:52:15 +01:00
Wolfgang Bumiller
2905f2b5e6 update api macro example usage 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-29 09:51:27 +01:00
Wolfgang Bumiller
7b6c41078b update api macro invocation to new style 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-27 14:19:46 +01:00
Wolfgang Bumiller
6486cb853f first api macro usage test/example 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-27 10:05:37 +01:00
Dietmar Maurer
329d40b50b rename ApiHandler::Async into ApiHandler::AsyncHttp 2019-11-23 09:03:21 +01:00
Dietmar Maurer
ad51d02aa9 src/server/rest.rs: cleanup async code 2019-11-22 13:02:05 +01:00
Wolfgang Bumiller
cad540e969 api/compat: remove remaining api_schema references 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:53:15 +01:00
Wolfgang Bumiller
a2479cfa1a api/compat: drop more compat imports from api_schema.rs 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:36:28 +01:00
Wolfgang Bumiller
2194824f77 api: BoxFut -> ApiFuture 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:16:37 +01:00
Wolfgang Bumiller
3d482025b3 move api schema into proxmox::api crate 
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:14:54 +01:00
Dietmar Maurer
552c225948 sort all property lookup tables 
Required, because we use binary sreach to find items.
 2019-11-21 13:32:09 +01:00
Dietmar Maurer
255f378a1b use const api definitions 2019-11-21 13:32:09 +01:00
Dietmar Maurer
0997967d65 do no restrict index names too much 2019-11-08 10:28:14 +01:00
Dietmar Maurer
62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer
11377a47bb avoid some clippy warnings 2019-10-25 18:04:37 +02:00
Dietmar Maurer
2c2097ee75 fix typo in error message 2019-10-07 12:48:56 +02:00
Dietmar Maurer
f7aa6f15d3 src/api2/reader.rs: use tokio::fs::read() to read file async 2019-10-07 12:24:06 +02:00
Dietmar Maurer
4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
Dietmar Maurer
c0b1b14c05 src/api2/reader.rs - download_chunk: use blocking io 
Turns out to be twice as fast as tokio async io...
 2019-10-05 11:41:19 +02:00
Dietmar Maurer
fb6026b66b backup writer api: verify checksum inside close 2019-09-23 10:56:53 +02:00
Dietmar Maurer
4a7de56e2f avoid injecting ENV vars from Makefile 
So that we can run "cargo build" without setting vars manually.
 2019-09-09 10:51:08 +02:00
Wolfgang Bumiller
ffb6434485 src/api2/reader.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
7622005574 src/api2/backup/upload_chunk.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
59b2baa0f6 src/api2/backup.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
6be147b78c src/api2/admin/datastore.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
f3a8d1d7e0 api2/node/time: replace mem::uninitialized 
and fixup use statements

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:14:41 +02:00
Dietmar Maurer
4af0ee055a api: always use complete file names (including add exctensions) 2019-08-07 10:10:14 +02:00
Dietmar Maurer
d7c243977d src/api2/admin/datastore.rs - list_snapshot_files: list all files 
Add files that are not mentioned in index.json.
 2019-08-07 09:16:14 +02:00
Dietmar Maurer
34066e2eeb src/api2/backup/environment.rs - add_blob: always verify blob crc 2019-08-07 09:15:13 +02:00
Dietmar Maurer
f889b15867 src/backup/data_blob.rs: always compute crc 2019-08-06 11:42:14 +02:00
Dietmar Maurer
a17a0e7a9f src/api2/admin/datastore.rs: extract backup size from index.json 2019-08-06 10:56:21 +02:00
Dietmar Maurer
8c70e3eb18 src/api2/admin/datastore.rs - files: return data from index.json 2019-08-05 13:22:19 +02:00
Dietmar Maurer
f69adc81a6 use new proxmox::tools::nodename 2019-08-03 17:06:23 +02:00
Dietmar Maurer
7f66c29e5c remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead 2019-08-03 16:44:31 +02:00
Dietmar Maurer
e18a6c9ee5 update to nix 0.14, use code from proxmox:tools 2019-08-03 13:05:38 +02:00
Dietmar Maurer
9fdc3ef46f src/api2/admin/datastore.rs: pass backup type/id (group) to prune 
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
 2019-07-27 08:49:14 +02:00
Dietmar Maurer
bbf9e7e951 src/api2/backup.rs: cleanup schema definitions 2019-07-26 09:07:29 +02:00
Dietmar Maurer
ca5d0b61ca src/api2/backup.rs: new required backup-time parameter 
The client should pass the time as parameter.
 2019-07-25 13:44:01 +02:00
Dietmar Maurer
e128d4e84f src/api2/admin/datastore.rs: backup logs may not be written twice 2019-07-25 12:29:04 +02:00
Dietmar Maurer
ef3254c086 src/api2/admin/datastore.rs: use correct .blob extension for log 2019-07-25 12:22:50 +02:00
Dietmar Maurer
07ee223590 src/api2/admin/datastore.rs: add api to upload backup client log file 2019-07-25 12:15:30 +02:00
Dietmar Maurer
15c847f136 src/api2/admin/datastore.rs: allow to list all snapshots 2019-07-18 09:11:43 +02:00
Dietmar Maurer
0eecf38fbf src/api2/admin/datastore.rs: add status api call 2019-07-16 13:34:38 +02:00
Dietmar Maurer
b46c3fad03 src/backup/fixed_index.rs: use correct size 
We need to consider that the last chunk may have smaller size.
 2019-07-04 15:13:22 +02:00
Dietmar Maurer
5e04ec707d src/api2/backup/environment.rs: improve chunk alignment/size checks 2019-07-04 13:40:43 +02:00
Dietmar Maurer
17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server 
I get about 30MB/s (much too slow)
 2019-06-29 13:43:10 +02:00
Dietmar Maurer
09d7dc5024 src/api2/reader.rs: add download chunk api 2019-06-28 16:27:01 +02:00
Dietmar Maurer
fcfb84fedf file download: avoid unnecessary copy 2019-06-28 07:07:52 +02:00
Dietmar Maurer
dd066d28e2 src/api2/reader.rs: implement backup reader protocol 2019-06-27 09:01:41 +02:00
Dietmar Maurer
42a87f7b96 src/server/h2service.rs: implement generic h2 service 2019-06-26 17:38:33 +02:00
Dietmar Maurer
f14a8c9a85 src/api2/admin/datastore.rs: use correct path for download 2019-06-25 12:59:36 +02:00
Dietmar Maurer
9e47c0a56e src/api2/admin/datastore.rs: implement generic download method 
We simply allow to download raw binary data.
 2019-06-25 10:16:59 +02:00
Dietmar Maurer
da7d67213e src/api2/admin/datastore/pxar.rs: remove pxar upload api 
Upload needs to be done using the backup protocol.
 2019-06-25 09:45:35 +02:00
Dietmar Maurer
dd8e744f9a src/api2/admin/datastore.rs: use a worker log log prune actions 2019-06-25 08:12:25 +02:00
Dietmar Maurer
46bd880041 src/api2/backup/environment.rs: new helper add_blob() 2019-06-24 09:35:37 +02:00
Dietmar Maurer
cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer
6f083b7a92 src/api2/backup/upload_chunk.rs: compute crc on upload 2019-06-23 09:04:42 +02:00
Dietmar Maurer
bffd40d6b7 src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools 2019-06-14 11:40:04 +02:00
Dietmar Maurer
00388226e6 src/api2/backup/environment.rs: log index checksums 2019-06-14 10:36:20 +02:00
Dietmar Maurer
36075475a6 src/api2/backup/environment.rs: log duplicate chunks 2019-06-14 07:12:30 +02:00
Dietmar Maurer
fa148dbd11 src/api2/backup/upload_chunk.rs: verify chunk data if possible 2019-06-14 06:23:46 +02:00
Dietmar Maurer
f98ac774ee backup: Add support for client side encryption 
first try ...
 2019-06-13 11:47:23 +02:00
Dietmar Maurer
6762db70d6 src/api2/types.rs: define CHUNK_DIGEST_SCHEMA 2019-06-13 07:30:42 +02:00
Wolfgang Bumiller
dd5495d6dc tree-wide: use 'dyn' for all trait objects 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-06-07 13:13:48 +02:00
Dietmar Maurer
4800df7ba6 src/api2/admin/datastore.rs: remove test-upload api 2019-06-07 07:52:11 +02:00
Dietmar Maurer
113898d7e9 src/api2/admin/datastore/upload.rs: remove unused upload api 2019-06-07 07:39:41 +02:00
Dietmar Maurer
986bef16be src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1 
So that we can include it in static doc strings.
 2019-06-05 08:41:20 +02:00
Dietmar Maurer
c9ec0956cf src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1 2019-06-05 08:12:13 +02:00
Dietmar Maurer
7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Dietmar Maurer
339ddfcbfa src/cli/command.rs: move doc generator code to src/api_schema/format.rs 2019-06-04 12:32:22 +02:00
Dietmar Maurer
39d6846e1e src/api2/admin/datastore/backup.rs: implement config file upload 2019-06-03 09:45:14 +02:00
Dietmar Maurer
36898ffce6 src/backup/chunk_stream.rs: add optional chunk_size parameter 2019-05-30 13:28:24 +02:00
Dietmar Maurer
96482891ae src/api2/admin/datastore/backup/environment.rs: log upload statistics 2019-05-30 09:21:19 +02:00
Dietmar Maurer
642322b433 src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload 
So that we can provide better statistics (patches follows).
 2019-05-30 09:21:19 +02:00
Dietmar Maurer
e6389f4e75 src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files) 2019-05-29 10:38:57 +02:00
Dietmar Maurer
39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Dietmar Maurer
a42d1f55c3 src/api2/admin/datastore/backup.rs: add debug flag 2019-05-29 09:35:21 +02:00
Dietmar Maurer
976595e1a9 src/backup/dynamic_index.rs: split class DynamicIndexWriter 2019-05-29 08:49:57 +02:00
Dietmar Maurer
60e589a111 src/api2/admin/datastore/backup.rs: log sucessful finish call 2019-05-28 12:55:22 +02:00
Dietmar Maurer
006f3ff407 src/api2/admin/datastore/backup/environment.rs: add more consistency checks 2019-05-28 09:21:31 +02:00
Dietmar Maurer
3dc5b2a203 src/client/http_client.rs: use start of chunk as offset 2019-05-28 09:21:31 +02:00
Dietmar Maurer
e3d525feb7 src/api2/admin/datastore/backup.rs: fix typo 2019-05-28 07:13:19 +02:00
Dietmar Maurer
a42fa400ee src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api 2019-05-28 06:18:55 +02:00
Dietmar Maurer
a1e7cff3ea src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk 2019-05-27 07:57:43 +02:00
Dietmar Maurer
771953f9eb src/api2/admin/datastore/backup.rs: use smaller window sizes 2019-05-27 07:53:40 +02:00
Dietmar Maurer
05cba08c9c src/client/http_client.rs: append chunks in correct order 
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
 2019-05-26 09:57:34 +02:00
Dietmar Maurer
417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset 
This clearly shows that the current approach does not work - seems we get
chunks out of order.
 2019-05-26 08:49:29 +02:00
Wolfgang Bumiller
cc84a830c5 RFC: schema: make enums static 
I don't see a reason to allow these to be dynamically
modifiable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-05-23 12:38:51 +02:00
Dietmar Maurer
aa1b2e04fe src/client/merge_known_chunks.rs: merge known chunks 
To decrease the number of api calls required...
 2019-05-23 12:31:09 +02:00
Dietmar Maurer
8bea85b42e src/api2/admin/datastore/backup.rs: verify file size and chunk count on close 2019-05-23 08:50:36 +02:00
Dietmar Maurer
907f5bb0d8 src/api2/admin/datastore/backup/service.rs: improve error logging 2019-05-23 08:05:39 +02:00
Dietmar Maurer
a55fcd740f src/api2/admin/datastore/backup.rs: suse POST for finish() api 2019-05-22 13:04:58 +02:00
Dietmar Maurer
5a0b521809 src/api2/admin/datastore/backup.rs: remove test callbacks 2019-05-22 10:55:27 +02:00
Dietmar Maurer
d0afeaa831 src/api2/admin/datastore/backup.rs: improve error message 2019-05-22 07:43:54 +02:00
Dietmar Maurer
7f3d2ffab9 src/backup/index.rs: rename ChunkListReader into DigestListEncoder 2019-05-21 12:28:44 +02:00
Dietmar Maurer
a95849321c src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks 2019-05-21 09:54:11 +02:00
Dietmar Maurer
a09c0e38d8 src/api2/admin/datastore/backup/environment.rs: register/lookup known chunks 2019-05-20 18:05:10 +02:00
Dietmar Maurer
82ab72304e src/client/http_client.rs: implement upload_stream 2019-05-20 14:19:24 +02:00
Dietmar Maurer
adec8ea263 src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed 
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
 2019-05-16 10:24:23 +02:00
Dietmar Maurer
372724afea src/api2/admin/datastore/backup.rs: implement finish_backup 2019-05-15 12:58:55 +02:00
Dietmar Maurer
bb105f9dae src/api2/admin/datastore/backup.rs: improve logging 2019-05-15 10:09:46 +02:00
Dietmar Maurer
a20772525f src/api2/admin/datastore/backup.rs: implement close for dynamic writers 2019-05-15 07:58:05 +02:00
Dietmar Maurer
4e93f8c164 src/api2/types.rs: add schema/format for file names 2019-05-15 07:14:08 +02:00
Dietmar Maurer
fbb798f6d2 src/api2/admin/datastore/backup.rs: ignore errors from last_backup 2019-05-13 07:06:56 +02:00
Dietmar Maurer
d36113665b src/api2/admin/datastore/backup.rs: implement chunk index download from previous backup 2019-05-11 16:05:50 +02:00
Dietmar Maurer
af1d68185c src/api2/admin/datastore/backup.rs: add missing 'archive-name' parameter to schema 2019-05-11 12:25:15 +02:00
Dietmar Maurer
6b95c7df07 src/api2/admin/datastore/backup/environment.rs: do not store path 2019-05-11 12:13:44 +02:00
Dietmar Maurer
51a4f63ff7 src/backup/datastore.rs: move last_backup to BackupInfo 2019-05-11 12:07:09 +02:00
Dietmar Maurer
b02a52e3a5 src/backup/datastore.rs: impl last_backup helper 2019-05-11 11:36:46 +02:00
Dietmar Maurer
c09775011a src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups 2019-05-11 10:19:34 +02:00
Dietmar Maurer
f9578f3c79 backup-api: allow to create DynamicIndexWriter and add chunks to to 2019-05-10 10:25:40 +02:00
Dietmar Maurer
0506d1ed04 upload_chunk.rs: add comment about possible unnecessary code 2019-05-10 06:15:25 +02:00
Dietmar Maurer
5f42ef009a cleanup: remove unused code 2019-05-09 18:11:39 +02:00
Dietmar Maurer
b4b63e520d handle_async_api_request: put rpcenv into a Box 
So that we can pass rpcenv into futures.
 2019-05-09 18:01:24 +02:00
Dietmar Maurer
2c41096a4b src/api2/admin/datastore/backup/upload_chunk.rs: fix maximal chunk size 2019-05-09 13:29:50 +02:00
Dietmar Maurer
78b5191550 src/server/formatter.rs: further cleanups and renaming ... 2019-05-09 13:28:26 +02:00
Dietmar Maurer
21ee7912fa src/api2/admin/datastore/backup.rs: implement upload chunk 2019-05-09 13:06:09 +02:00
Dietmar Maurer
4ebf0eabb0 src/api2/types.rs: use separate file for schema definitions 2019-05-09 07:44:09 +02:00
Dietmar Maurer
bd1507c4fa src/api2/admin/datastore/backup/service.rs: move service code into extra file 2019-05-08 17:36:19 +02:00
Dietmar Maurer
0aadd40b4f src/api2/admin/datastore/backup.rs: add required parameters, cleanups 2019-05-08 13:17:23 +02:00
Dietmar Maurer
d95ced64ad src/api2/admin/datastore/backup/environment.rs: use separate file 2019-05-08 12:41:58 +02:00
Dietmar Maurer
ca60c371e2 api: rename h2upload to backup 2019-05-08 12:35:15 +02:00
Dietmar Maurer
d906b1124f src/api2/admin/datastore/h2upload.rs: code cleanups 2019-05-08 11:36:43 +02:00
Dietmar Maurer
58c8d7d91f rc/api2/admin/datastore/h2upload.rs: implement BackupEnvironment 
To pass arbitrary information/state to api methods.
 2019-05-08 11:26:54 +02:00
Dietmar Maurer
090ac9f79a src/api2/admin/datastore/h2upload.rs: avoid unwrap 2019-05-08 08:21:18 +02:00
Dietmar Maurer
4e3da4b366 src/api2/admin/datastore/h2upload.rs: do not log client address 2019-05-08 08:15:55 +02:00
Dietmar Maurer
e3a44552ee src/api2/admin/datastore/h2upload.rs: code cleanups 2019-05-08 08:12:49 +02:00
Dietmar Maurer
a66ab8ae5e src/api2/admin/datastore/h2upload.rs: make worker abortable 2019-05-08 07:14:40 +02:00
Dietmar Maurer
72375ce6c6 src/api2/admin/datastore/h2upload.rs: log to worker task 2019-05-07 14:07:00 +02:00
Dietmar Maurer
d9bd06eae8 src/api2/admin/datastore/h2upload.rs: use WorkerTask 2019-05-07 13:42:00 +02:00
Dietmar Maurer
52cf506e48 src/api2/admin/datastore/h2upload.rs: implement async test api call 2019-05-07 12:26:55 +02:00
Dietmar Maurer
92ac375aff src/api2/admin/datastore/h2upload.rs: improve http2 service 2019-05-07 11:24:44 +02:00
Dietmar Maurer
96e95fc179 src/api2/admin/datastore/h2upload.rs: test if version < 2 
We can only upgrade http 1.x connections.
 2019-05-06 10:29:34 +02:00
Dietmar Maurer
cf639a47de rc/client/http_client.rs: add experimental h2 upgrade code 2019-04-29 11:57:58 +02:00
Dietmar Maurer
152764ec15 src/api2/admin/datastore/h2upload.rs: add test code for http2 2019-04-27 08:57:35 +02:00
Dietmar Maurer
6358b68236 cleanup: avoid compiler warnings 2019-04-23 12:17:24 +02:00
Dietmar Maurer
13f1cc17ea src/api_schema/router.rs: implement list_subdirs() helper 2019-04-16 12:07:02 +02:00
Wolfgang Bumiller
062d4916ff api_schema: allow generic api handler functions 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-04-16 11:22:23 +02:00
Dietmar Maurer
1a6c9415ae src/api2/node/services.rs: add proxmox-backup-proxy service 2019-04-15 09:48:02 +02:00
Dietmar Maurer
75bc49bed7 src/server/worker_task.rs: implement abort channel 2019-04-15 09:38:05 +02:00
Dietmar Maurer
912524752e src/api2/admin/datastore/pxar.rs: create worker task for file upload 2019-04-15 08:38:14 +02:00
Dietmar Maurer
f2b99c34f7 src/api2/admin/datastore.rs: implement API to return last GC status 2019-04-11 12:04:25 +02:00
Dietmar Maurer
a665dea128 src/api2/node/tasks.rs: implement stop_task api 2019-04-10 13:55:05 +02:00
Dietmar Maurer
c360bd73d3 src/api2/node/tasks.rs: fix task status api 2019-04-09 15:12:20 +02:00
Dietmar Maurer
d8d40dd0c2 src/api2/node.rs: add node parameter 2019-04-09 14:43:30 +02:00
Dietmar Maurer
634132fe10 src/server/upid.rs: moved code into separate file 2019-04-08 18:43:26 +02:00
Dietmar Maurer
5a12c0e2fb src/api2/node/tasks.rs: start upid API 2019-04-07 14:36:57 +02:00
Dietmar Maurer
d2a2e02bb7 src/api2/node/tasks.rs: implement userfilter 2019-04-07 13:17:19 +02:00
Dietmar Maurer
3c3bee2e32 src/api2/node/tasks.rs: correctly handle start parameter 2019-04-07 12:41:24 +02:00
Dietmar Maurer
063ca5be77 src/api2/node/tasks.rs: implement list_tasks 2019-04-07 12:18:58 +02:00
Dietmar Maurer
d4b59ae0b8 src/backup/datastore.rs: pass WorkerTask to GC 
So that we can log to the task log file.
 2019-04-06 17:57:38 +02:00
Dietmar Maurer
0f778e0652 src/api2/admin/datastore.rs: use new WorkerTask::new_thread() 2019-04-06 11:27:23 +02:00
Dietmar Maurer
3e6a7deeaa src/api2/admin/datastore.rs: start GC in a separate thread 2019-04-01 12:02:46 +02:00
Dietmar Maurer
ad281d1bd9 www/DataStoreConfig.js: add button to start GC 2019-04-01 08:08:34 +02:00
Dietmar Maurer
8968258b66 rename catar into pxar 
To avoid confusion with the casync implementation.
 2019-03-14 10:54:09 +01:00
Dietmar Maurer
d5c34d98c0 src/catar/decoder.rs: simplify public restore API 2019-03-11 14:31:01 +01:00
Dietmar Maurer
01a1342364 src/api2/admin/datastore.rs: impl list_snapshot_files 2019-03-06 11:02:08 +01:00
Dietmar Maurer
dd79265a8a src/api2/admin/datastore/catar.rs: use archive_name directly 2019-03-06 11:02:08 +01:00
Wolfgang Bumiller
e2d007f76e api2/admin/datastore: add a backup protocol test api path 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-03-06 10:26:12 +01:00
Dietmar Maurer
f8f94534d4 src/api2/access.rs: authenticate_user() - add ticket login 2019-03-05 12:56:21 +01:00
Wolfgang Bumiller
391d310741 backup_info: BackupDir shortcut 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-03-05 10:15:06 +01:00
Dietmar Maurer
b3483782d1 src/backup/backup_info.rs: move code into separate file 
Also changed create_backup_dir() parameters - uses &BackupDir now.
 2019-03-05 07:18:12 +01:00
Dietmar Maurer
2b01a22507 src/backup/datastore.rs: add helper to sort backup lists 2019-03-04 18:20:57 +01:00
Dietmar Maurer
875fb1c01a src/backup/datastore.rs: use unix epoch to create DateTime 
To make sure that we have a timestamp without nanosecond.
 2019-03-04 17:58:22 +01:00
Dietmar Maurer
9b492eb256 src/backup/datastore.rs: protect BackupDir fields, impl new() 2019-03-04 13:51:36 +01:00
Dietmar Maurer
1e9a94e579 src/backup/datastore.rs: protect BackupGroup fields, impl new() 2019-03-04 13:38:23 +01:00
Dietmar Maurer
6f62c9240a src/api2/admin/datastore.rs: imp delete_snapshot 2019-03-03 11:29:00 +01:00
Dietmar Maurer
e77a02ed87 catar api: cleanup parameter names 2019-03-03 09:48:32 +01:00
Dietmar Maurer
49dc074007 parameter name cleanups (use '-' instead of '_') 2019-03-02 16:34:10 +01:00
Dietmar Maurer
ad20d19877 src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups 2019-03-02 16:28:36 +01:00
Dietmar Maurer
184f17afad src/api2/admin/datastore.rs: impl list_snapshots 2019-03-02 16:20:50 +01:00
Dietmar Maurer
812c6f8716 src/api2/admin/datastore.rs: impl list_backup_groups 2019-03-02 11:29:05 +01:00
Dietmar Maurer
e909522f06 src/backup/datastore.rs: use timestamp with zone offset 2019-03-01 17:36:40 +01:00
Dietmar Maurer
dc4c09faae src/backup/datastore.rs: split BackupDir 2019-03-01 16:26:35 +01:00
Dietmar Maurer
38b0dfa511 src/backup/datastore.rs: split BackupInfo 2019-03-01 16:26:35 +01:00
Dietmar Maurer
541b908ea4 src/backup/datastore.rs: use DateTime<Utc> instead of unix epoch 2019-02-28 17:03:48 +01:00
Dietmar Maurer
8f57971714 src/api2/admin/datastore.rs: implement prune 2019-02-28 12:51:27 +01:00
Dietmar Maurer
83b7db0209 src/api2/admin/datastore.rs: start prune api 
just a dummy for now.
 2019-02-27 16:53:17 +01:00
Wolfgang Bumiller
8731e40a7f backup/datastore: create_backup_dir: show if path existed 
To enable asserting the creation of a new backup rather than
adding to an existing one.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-27 10:15:36 +01:00
Dietmar Maurer
8c75372b79 src/backup/datastore.rs: list_backups() - include list of index files 2019-02-20 13:37:44 +01:00
Dietmar Maurer
247cdbce72 src/api2/admin/datastore/catar.rs: allow to configure chunk-size 2019-02-19 15:19:12 +01:00
Dietmar Maurer
1c7a88aeff UploadCaTar: currectly write all data using write_all() 2019-02-18 18:31:02 +01:00
Dietmar Maurer
ef2f2efbcc improve api_schema module structure 2019-02-17 10:16:33 +01:00
Dietmar Maurer
dc9a007b11 rename src/api to src/api_schema 2019-02-17 09:59:20 +01:00
Wolfgang Bumiller
1d77b6cf6b update to pam 0.7 (renamed from pam-auth) 
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-15 10:34:40 +01:00
Dietmar Maurer
f02e6fc478 api2/admin/datastore/catar.rs: simplify/fix debug message 2019-02-15 10:04:37 +01:00
Dietmar Maurer
541a302224 api2/admin/datastore/catar.rs: use nested file extensions 
i.e. archive.catar.didx
 2019-02-13 10:37:00 +01:00
Dietmar Maurer
e5064ba607 simplify backup lib structure (pub use xxx:*), improve doc 2019-02-12 14:13:31 +01:00
Dietmar Maurer
93d5d77952 rename ArchiveIndex to DynamicIndex 
also changed the file extension from .aidx to .didx
 2019-02-12 12:05:33 +01:00
Wolfgang Bumiller
ace9e3531a access: use proxmox-backup-auth for pam 
allows customization via /etc/pam.d/proxmox-backup-auth

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-07 13:40:22 +01:00
Wolfgang Bumiller
13f8310cac api2: time: use /etc/localtime as fallback 
in case there is no /etc/timezone

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-05 14:14:56 +01:00
Dietmar Maurer
4b2cdeb9a6 implement relead_timezone flag 2019-02-01 10:04:46 +01:00
Dietmar Maurer
c82bc1a1f9 depend on pam-auth, use pam for root login 
Please use username "root@pam" for login via GUI.
 2019-02-01 09:30:50 +01:00
Dietmar Maurer
a154a8e8a4 delay unauthorized request (rate limit) 2019-01-31 14:34:21 +01:00
Dietmar Maurer
b9903d6331 server/rest.rs: verify auth cookie 2019-01-31 12:22:00 +01:00
Dietmar Maurer
9f49fe1d5d avoid compiler warnings 2019-01-30 18:25:37 +01:00
Dietmar Maurer
34f956bc25 api2/access.rs: add ticket api 2019-01-30 15:16:10 +01:00
Dietmar Maurer
a859fa95a9 add protected flag for some api methods 2019-01-28 18:16:47 +01:00
Dietmar Maurer
f120483353 server/rest.rs: add proxy_sync_api_request() dummy 2019-01-28 17:30:39 +01:00
Dietmar Maurer
48849593ee api2/node/services.rs: implement service commands 2019-01-27 12:40:31 +01:00
Dietmar Maurer
e82dad9700 also pass rpcenv to async handlers 2019-01-27 10:18:52 +01:00
Dietmar Maurer
d2ab5f19e2 api2/node/services.rs: start services api 2019-01-26 17:54:18 +01:00
Dietmar Maurer
6049b71f41 api: pass RpcEnvirnment to api handlers 2019-01-26 14:50:37 +01:00
Dietmar Maurer
d96d82736d use crate log and syslog 2019-01-26 10:56:11 +01:00
Dietmar Maurer
e182ab4a4a api2/node/syslog.rs: improve error handling 2019-01-26 10:23:52 +01:00
Dietmar Maurer
0441886842 api2/node/syslog.rs: implement dump journal 2019-01-25 18:20:51 +01:00
Dietmar Maurer
4f9a726833 api2/node/syslog.rs: add syslog api schema 2019-01-25 17:17:30 +01:00
Dietmar Maurer
af2fddea4f api2/node/dns.rs: implement concurrent update protection 2019-01-25 11:38:59 +01:00
Dietmar Maurer
de6b0721fa pi2/node/dns.rs: return file digest 2019-01-25 11:03:41 +01:00
Dietmar Maurer
46b79b9ee5 api2/node/dns.rs: improve dns api 2019-01-25 10:15:32 +01:00
Dietmar Maurer
8f973f8181 api2/node/dns.rs: implement resolv.conf parser 2019-01-24 14:59:40 +01:00
Dietmar Maurer
e6ffeb918c api2/node/time.rs: implement set timezone 2019-01-24 12:05:06 +01:00
Dietmar Maurer
cd8b91f78c api2/node/time.rs: add return schema 2019-01-24 11:07:11 +01:00
Dietmar Maurer
447787ab7c tools.rs: implement file_read_firstline 2019-01-24 10:43:30 +01:00
Dietmar Maurer
0463602a79 api2/node/time.rs: implement read api 2019-01-23 15:14:14 +01:00
Dietmar Maurer
b2b3485d5f src/api2/node: start node configuration api 2019-01-23 13:09:17 +01:00
Dietmar Maurer
7e13b2d67f api2/subscription.rs: add subscription api class 2019-01-22 12:50:19 +01:00
Dietmar Maurer
576e3bf252 rename api3 back to api2 
There is no real need to change the path, so using api2 we can reuse
all helpers (like tools from proxmox widget toolkit).
 2019-01-22 12:10:38 +01:00