Commit Graph

4030 Commits

Author SHA1 Message Date
Wolfgang Bumiller d1993187b6 introduce TaskState trait
Used to not require access to the WorkerTask struct outside
the `server` and `api2` module, so it'll be easier to
separate those backup/server/client parts into separate
crates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-12 14:11:57 +02:00
Thomas Lamprecht adfcfb6788 server: get index: make content-type non mutable
feels more idiomatic

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 13:36:45 +02:00
Thomas Lamprecht 07995a3ca3 server/rest: code cleanup: use async
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 13:36:45 +02:00
Thomas Lamprecht dd76eba73e readme: allow to directly copy+paste+execute commands
plus fix s/ssh:/git:/ for protocol

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-12 13:36:34 +02:00
Fabian Grünbichler b13da548b9 fix #3070: replace internal with public URLs
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-12 13:33:16 +02:00
Thomas Lamprecht fe0efb25e8 backup: index readers: drop useless shared lock
This is only acquired in those two methods, both as shared. So it has
no use.

It seems, that it was planned in the past that the index deletion
should take the exclusive, while read and write takes the shared
flock on the index, as one can guess from the lock comments in commit
0465218953

But then later, in commit c8ec450e37)
the documented semantics where changed to use a temp file and do an
atomic rename instead for atomicity.

The reader shared flock on the index file was done inbetween,
probably as preparatory step, but was not removed again when strategy
was changed to using the file rename instead.

Do so now, to avoid confusion of readers and a useless flock.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-09 12:59:31 +02:00
Hannes Laimer b0b00c4a47 add "Build" section to README.rst
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
2020-10-09 12:56:11 +02:00
Fabian Grünbichler 19ca962b15 reader: actually allow users to downlod their own backups
via HTTP2/backup reader protocol. they already could do so via the plain
HTTP download-file/.. API calls that the GUI uses, but the reader
environment required READ permission on the whole datastore instead of
just BACKUP on the backup group itself.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-09 12:54:30 +02:00
Fabian Grünbichler d479f0c810 reader: track index chunks and limit access
a reader connection should not be allowed to read arbitrary chunks in
the datastore, but only those that were previously registered by opening
the corresponding index files.

this mechanism is needed to allow unprivileged users (that don't have
full READ permissions on the whole datastore) access to their own
backups via a reader environment.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-09 12:54:09 +02:00
Fabian Grünbichler 1d5dac1b1d REST: don't print CSRF token
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-08 15:57:22 +02:00
Fabian Grünbichler 96c3d98256 Userid: fix borrow/deref recursion
not triggered by any current code, but this would lead to a stack
exhaustion since borrow would call deref which would call borrow again..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-08 15:57:10 +02:00
Fabian Grünbichler 0b3dc8ed8c Userid: simplify comparison with str
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-08 15:56:57 +02:00
Fabian Grünbichler 9a75eb11cb depend on proxmox 0.4.3
needed for constnamedbitmap macro

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-08 13:09:52 +02:00
Thomas Lamprecht 92dd02aaf6 api: datastore: require allocate privilege for deletion
makes only sense if we allow addition of a datastore also just with
that privilege

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-08 09:44:46 +02:00
Thomas Lamprecht 41bfd24919 server: add Datastore.Allocate privilege
Previously only Datastore.Modify was required for creating a new
datastore.

But, that endpoint allows one to pass an arbitrary path, of which all
parent directories will be created, this can allow any user with the
"Datastore Admin" role on "/datastores" to do some damage to the
system. Further, it is effectively a side channel for revealing the
systems directory structure through educated guessing and error
handling.

Add a new privilege "Datastore.Allocate" which, for now, is used
specifically for the create datastore API endpoint.

Add it only to the "Admin" role.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-08 09:12:08 +02:00
Thomas Lamprecht fddc8aa410 acl: use modified constnamedbitmap macro
avoiding the need for reshuffling all bits when a new privilege is
added at the start or in the middle of this definition.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-08 09:09:39 +02:00
Stefan Reiter 735ee5206a fuse_loop: handle unmap on crashed instance
If a fuse_loop instance dies suddenly (e.g. SIGKILL), the FUSE mount and
loop device assignment are left behind. We can determine this scenario
on specific unmap, when the PID file is either missing or contains a PID
of a non-running process, but the backing file and potentially loop
device are still there.

If that's the case, do an "emergency cleanup", by unassigning the
loopdev, calling 'fusermount -u' and then cleaning any leftover files
manually.

With this in place, pretty much any situation is now recoverable via
only the 'proxmox-backup-client' binary, by either calling 'unmap' with
or without parameters.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:38:25 +02:00
Stefan Reiter a86bf52390 fuse_loop: wait for instance to close after killing
On unmap, only report success if the instance we are killing actually
terminates. This is especially important so that cleanup routines can be
assured that /run files are actually cleaned up after calling
cleanup_unused_run_files.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:37:47 +02:00
Stefan Reiter 2deee0e01f fuse_loop: add automatic cleanup of run files and dangling instances
A 'map' call will only clean up what it needs, that is only leftover
files or dangling instances of it's own name.

For a full cleanup the user can call 'unmap' without any arguments.

The 'cleanup on error' behaviour of map_loop is removed. It is no longer
needed (since the next call will clean up anyway), and in fact fixes a
bug where trying to map an image twice would result in an error, but
also cleanup the .pid file of the running instance, causing 'unmap' to
fail afterwards.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:37:05 +02:00
Stefan Reiter 2d7d6e61be mount/map: use names for map/unmap for easier use
So user doesn't need to remember which loop devices he has mapped to
what.

systemd unit encoding is used to transform a unique identifier for the
mapped image into a suitable name. The files created in /run/pbs-loopdev
will be named accordingly.

The encoding all happens outside fuse_loop.rs, so the fuse_loop module
does not need to care about encodings - it can always assume a name is a
valid filename.

'unmap' without parameter displays all current mappings. It's
autocompletion handler will list the names of all currently mapped
images for easy selection. Unmap by /dev/loopX or loopdev number is
maintained, as those can be distinguished from mapping names.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:35:52 +02:00
Stefan Reiter 4ec17f7eb5 loopdev: add module doc
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:34:39 +02:00
Stefan Reiter fcad02e1de fuse_loop: add documentation
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:33:34 +02:00
Stefan Reiter 708fab3082 format: fix typo in function name
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-08 08:32:21 +02:00
Oguz Bektas 3bbb70b3d3 docs: typo fixups
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-10-07 14:09:41 +02:00
Oguz Bektas 0c80f4fa87 ui: network: remove create VLAN option
for now this isn't needed and would take quite a bit of effort to
match the API schema with PVE.

if there are a lot of requests at some point we can add it in.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-10-07 14:07:46 +02:00
Thomas Lamprecht 21486225c8 ui: dashboard: code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-06 15:16:00 +02:00
Thomas Lamprecht a2920c3757 ui: dashboard: add tooltip to gears edit tool
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-06 15:15:43 +02:00
Thomas Lamprecht 6e0f58e7a9 ui: fixup: s/Hours/Days/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-06 15:15:18 +02:00
Dominik Csapak dee74aa440 ui: Dashboard/TaskSummary: show task overlay when clicking on a count
when clicking on a count in the summary, a small task overlay now pops
up that shows those tasks. this way, the user has an easy way
of seeing which tasks failed exactly

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-06 12:58:52 +02:00
Dominik Csapak 4acd7229d3 ui: Dashboard/TaskSummary: add Verifies to the Summary
and count every type that starts with 'verify' (e.g. verifyjob)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-06 12:58:52 +02:00
Dominik Csapak 9608ac3486 ui: Dashboard/TaskSummary: refactor types and title
by moving the definition into the controller and dynamically use them
in the updateTasks function

we will reuse/extend this later

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-06 12:58:52 +02:00
Dominik Csapak ad9d1625a6 ui: implment task history limit and make it configurable
we showed 'last month' even if we did not limit the api call
implement that and make the number of days configurable
(we have most of the code already available for that, since
the base dashboard got copied from pmg and never cleaned up)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-06 12:58:52 +02:00
Dominik Csapak 1a558edd0b api2/status: add type- and statusfilter to tasks api call
we will use this for the pbs dashboard

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-06 12:58:52 +02:00
Dominik Csapak 5976c392ad api2/types: add TaskStateType struct
the same as the regular TaskState, but without its fields, so that
we can use the api macro and use it as api call parameter

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-06 12:58:52 +02:00
Fabian Grünbichler a92b2d6a00 d/control: add ',' after qrencode dependency
'${misc:Depends}' is empty at the moment, otherwise this would have
already generated invalid packages..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-10-06 12:48:12 +02:00
Thomas Lamprecht 7d4bf881f7 docs: installation: add system requirements section
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-06 12:27:47 +02:00
Thomas Lamprecht 05be0984b4 acl: document Admin and NoAccess a bit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-06 12:23:22 +02:00
Thomas Lamprecht cdbc18fc4e docs: typo fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-06 10:42:43 +02:00
Dietmar Maurer 2995aedf1d src/bin/proxmox_backup_client/mount.rs: fix img name completion 2020-10-06 09:40:00 +02:00
Stefan Reiter 45f9b32e0f client: implement map/unmap commands for .img backups
Allows mapping fixed-index .img files (usually from VM backups) to be
mapped to a local loopback device.

The architecture uses a FUSE-backed temp file mapped to a loopdev:

  /dev/loopX -> FUSE /run/pbs-loopdev/xxx -> backup client -> PBS

Since unmapping requires some cleanup (unmap the loopdev, stop FUSE,
remove the temp files) a special 'unmap' command is added, which uses a
PID file to send SIGINT to the backup-client instance started with
'map', which will handle the cleanup itself.

The polling with select! in mount.rs needs to be split in two, since we
have a chicken and egg problem between running FUSE and setting up the
loop device - so we need to do them concurrently, until the loopdev is
assigned, at which point we can report success and daemonize, and then
continue polling the FUSE loop future.

A loopdev module is added to tools containing all required functions for
mapping a loop device to the FUSE file, with the ioctls moved into an
inline module to avoid exposing them directly.

The client code is placed in the 'mount' module, which, while
admittedly a loose fit, allows reuse of the daemonizing code.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-06 09:08:14 +02:00
Stefan Reiter 1d0b662b42 mount: handle SIGTERM as well
instead of only SIGINT

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-10-06 09:08:14 +02:00
Thomas Lamprecht 38f5cb5b71 fix comma dangle
...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-05 17:15:32 +02:00
Oguz Bektas 476328b302 docs: fix broken reference to backup_remote
while restructuring the docs, explicit title wasn't included in the
correct file

fixes commit 04e24b14f0

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-10-05 17:14:16 +02:00
Oguz Bektas 4c3efb532d gui: add onlineHelp for 'Prune Options'
also renamed the 'pruning' ref to 'backup-pruning' for clarity.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-10-05 17:12:18 +02:00
Oguz Bektas dafe3197ab scanrefs: match all instances of 'onlineHelp' in js files
previously it looked for the first instance. this behavior
became an issue while trying to add multiple onlineHelp buttons

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-10-05 17:00:49 +02:00
Dominik Csapak 90d7425afe ui: refactor render_icon code
we will reuse this later

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-05 16:48:01 +02:00
Dominik Csapak 2d81f7b0c0 task archive rotation: better handle non-existing archive
if the archive file does not exist yet, we cannot rotate it, but it's not
actually an error, so just return Ok(false) to indicate no rotation took
place

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-05 10:04:16 +02:00
Dylan Whyte 04e24b14f0 Restructure docs (more first level headings)
This removes the "Backup Management" first level heading in the docs,
and either uses the sub headings contained within it as first level
headings, or groups previous sections logically under new headings.

The administration-guide.rst file is also removed. Its contents are
instead separated into various files, that relate to their respective
first level heading.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-10-05 09:23:45 +02:00
Dylan Whyte a2bf852818 Add section "GUI"
Section provides a brief overview of the web interface

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-10-05 09:23:45 +02:00
Thomas Lamprecht 0ac612476a REST server: avoid hard coding world readable API endpoints
while we probably do not add much more to them, it still looks ugly.

If this was made so that adding a World readable API call is "hard"
and not done by accident, it rather should be done as a test on build
time. But, IMO, the API permission schema definitions are easy to
review, and not often changed/added - so any wrong World readable API
call will normally still caught.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-10-05 08:29:43 +02:00