when executing this code as non-root, we use sg-tape-cmd (a setuid binary)
to execute various ioctls on the tape device
we give the command the open tape device fd as stdin, but did not
dup it, so the std::process:Stdio handle closed it on drop,
which let subsequent operation on that file fail (since it was closed)
fix it by dup'ing it before giving it to the command, and also refactor
the calling code, so that we do not forget to do this
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
we did this for 'mtx', but missed it for the sg_pt_changer code
refactor it into the MtxStatus strut, and call it from both
code paths
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
mostly typical CRUD interface for managing drives, with an
additional actioncolumn containing some useful actions, e.g.
* reading the label
* show volume-statistics
* show the status
* label the inserted tape
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this lets the users manage changers and lets them view the status of one
by having an overview of:
* slots for tapes
* import/export slots
* drives
lets the user:
* barcode-label all the tapes in the library
* move tapes between slots, into/out of drives
* show some basic info when a tape is loaded into a drive
* show the status of a drive
* clean a drive
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
shows all tapes with the relevant info
* which pool it belongs to
* what backups are on it
* which media-set
* location
* etc.
This is very rough, and maybe not the best way to display this information.
It may make sense to reverse the tree, i.e. having pools at top-level,
then media-sets, then tapes, then snapshots..
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
includes edit windows for
* Drives
* Changers
* Media Pools
* Labeling Media
* Making new Tape Backups
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this check is not perfect since there are often multiple device
nodes per drive/changer, but from the scan api we should return always
the same, so for an api user this should be enough
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
so that an api user can get the drives belonging to a changer
without having to parse the config listing themselves
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
similar to the changers, create a listing at /tape/drive and put
the specific api calls below that
move the scan api call up one level
remove the status info from the config listing
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
so that an api user can see which drive belongs to which drivenum of a changer
for ones with multiple drives
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
add a changer listing here (copied from api2/config/changer)
and put the status and transfer api calls below that
puts the changer scan into the top level tape api
and removes the (now redundant) info from the config api path
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
improves upid_read_status with:
* ignore multiple newlines at the end
* remove all code that could panic (array index access)
the one place where we access with '[pos+1..]' is ok since
we explicitely test the len of the vector, this is done to
let rust optimize away the range checks, so it cannot panic
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
the password field should not be indented differently than the rest of
the fields, and we never have a border on the panels
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
we can reuse the edit window from widget toolkit for the most part
this solves some spacing and layout issues and is less code
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
an empty schedule means 'none', so do not fill it with the default
in case we edit an existing job (like we do already for sync jobs)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
it seems that sometimes, the child process signal gets handled
before the parent process signal. Systemd then ignores the
childs signal (finished reloading) and only after going into
reloading state because of the parent. this will never finish.
Instead, wait for the state to change to 'reloading' after sending
that signal in the parent, an only fork afterwards. This way
we ensure that systemd knows about the reloading before actually trying
to do it.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
of ProcessLockSharedGuard.
We use a counter to determine if we can unlock the file again, but
we never actually decremented the writer count, so we held the
lock forever.
This fixes the issue that we could not start a garbage collect after
a reload, as long as the old process is still running, even when that
process has no active backup anymore but another long running task
(e.g. file download, terminal, etc.).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this fixes the issue that on some filesystems, you cannot recursively
remove a directory when you hold a lock on a file inside (e.g. nfs/cifs)
it is not really backwards compatible (so during an upgrade, there
could be two daemons have the lock), but since the locking was
broken before (see previous patch) it should not really matter
(also it seems very unlikely that someone will trigger this)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
'lock_manifest' returns a Result<File, Error> so we always got the result,
even when we did not get the lock, but we acted like we had.
bubble the locking error up
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
if no groups were found, the task log was very confusing as it
contained no real information why nothing was synced, e.g.:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
this patch simply logs how many groups were found and are about to be synced:
Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
Sync datastore 'local-datastore' from 'remote/datastore'
found 0 groups to sync
sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
TASK OK
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
findRecord does not match exactly, but only at the beginning and
case insensitive, by default. Change all calls to be case sensitive
and an exactmatch (we never want the default behaviour afaics).
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
since we systemd-encode parts of the upid string, and those can contain
characters that are invalid in urls (e.g. '\'), we have to percent encode
those
add a 'percent_encode_component' helper, so that we can maybe change
the AsciiSet for all uses at the same time
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
when we could not load the config (e.g. missing permissions)
show the comment from the global datastore-list
also show a messagebox for a load error instead of setting
the text of the comment box
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
for now this only does the 'postfix' -> 'postfix@-' conversion,
fixes the issue that we only showed the 'postfix' service syslog
(which is rather empty in a default setup) instead of the instance one
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
sd_notify is not synchronous, iow. it only waits until the message
reaches the queue not until it is processed by systemd
when the process that sent such a message exits before systemd could
process it, it cannot be associated to the correct pid
so in case of reloading, we send a message with 'MAINPID=<newpid>'
to signal that it will change. if now the old process exits before
systemd knows this, it will not accept the 'READY=1' message from the
child, since it rejects the MAINPID change
since there is no (AFAICS) library interface to check the unit status,
we use 'systemctl is-active <SERVICE_NAME>' to check the state until
it is not 'reloading' anymore.
on newer systemd versions, there is 'sd_notify_barrier' which would
allow us to wait for systemd to have all messages from the current
pid to be processed before acknowledging to the child, but on buster
the systemd version is to old...
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
its rather hacky, but our cbind mixin does not support columns (yet).
if it does sometime in the future, we could use that instead
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
changes the layout to look i little bit more like the statistics panel
we have for ceph in pve, while changing to the UsageChart and adding
some more datastore infos (from last garbage collect)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
heavily inspired by pveRunningChart, without the dynamically adding
of data and specific for the usage of datastores
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
a panel for a single datastore that gets updated from an external caller
shows the usage, estimated full date, history and task summary grid
a panel that dynamically generates the panel above for each datastore
and a tabpanel that includes the panel above, as well as a global
syncview, verifiyview and aclview
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this makes it a little easier to provide good data, without
hardcoding all types in the source object
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
it seems that under certain circumstances, extjs does not initialize
or remove the content from objects in controllers
move it to the view, were they always exist
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
we want to use this panel again for a 'global' overview, without
any datastore preselected, so we have to handle that, and
adding a datastore selector in the editwindow
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
to parse the datastore out of a worker_id
for this we need some regexes that are the same as in the backend
for now we only parse out the datastore, but we can extend this
in the future to parse relevant info (e.g. remote for syncs,
id/type for backups)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
to easily check the store of a worker_id
this fixes the issue that one could not filter by type 'syncjob' and
datastore simultaneously
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
the basedir is already /usr/share/javascript/proxmox-backup/
so adding a subdir of that as alias is not needed
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
instead of await'ing the result of 'create_service' directly,
poll it together with the shutdown_future
if we reached that, fork_restart the new daemon, and await
the open future from 'create_service'
this way the old process still handles open connections until they finish,
while we already start a new process that handles new incoming connections
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
when a file shrunk during backup, we endlessly looped, reading/copying 0 bytes
we already have code that handles shrunk files, but we forgot to
break from the read loop
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
this is a panel that is heavily inspired from widget-toolkits
node/Tasks panel, but is adapted to use the extended api calls of
pbs (e.g. since/until filter)
has 'filter' panel (like pmgs log tracker gui), but it is collapsible
if we extend the api calls of the other projects, we can merge this
again into the widget-toolkit one and use that
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
and change all users of the /status/tasks api call to this
with this change we can now delete the /status/tasks api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
instead of returning 0 elements (which does not really make sense anyway),
change it so that there is no limit anymore (besides usize::MAX)
this is technically a breaking change for the api, but i guess
no one is using limit=0 for anything sensible anyway
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
in the api we use PROXMOX_SAFE_ID_REGEX for backup ids, but here
(where we use it to list them) we use a local regex
since the first is a superset of the one used here, simply extend
the local one
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
in case the garbage_collection errors out, we never set the in-memory
state, so if it failed, the last 'good' starttime was considered
for the schedule
this could lead to the job running every minute instead of the
correct schedule
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
instead of manually, this has the advantage that we now set
the jobstate correctly and can return with an error if it is
currently running (instead of failing in the task)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
index files that were smaller than their respective header size,
would fail with
"failed to fill whole buffer"
instead now check explicitely for the size and fail with
"index too small (size)"
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
instead of prerotating 1000 tasks
(which resulted in 2 writes each time an active worker was finished)
simply append finished tasks to the archive (which will be rotated)
page cache should be good enough so that we can get the task logs fast
since existing installations might have an 'index' file, we
still have to read tasks from there, but only if it exists
this simplifies the TaskListInfoIterator a good amount
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
by moving the properties of the storage status out again to the top
level object
also introduce proper structs for the types used, to get type-safety
and better documentation for the api calls
this changes the backup counts from an array of [groups,snapshots] to
an object/struct with { groups, snapshots } and include 'other' types
(though we do not have any at this moment)
this way it is better documented
this also adapts the ui code to cope with the api changes
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
saves files mtime as i64 instead of u64 which enables backup of
files with negative mtime
the catalog_decode_i64 is compatible to encoded u64 values (if < 2^63)
but not reverse, so all "old" catalogs can be read with the new
decoder, but catalogs that contain negative mtimes will decode wrongly
on older clients
also remove the arbitrary maximum value of 2^63 - 1 for
encode_u64 (we just use up to 10 bytes now) and correctly
decode them and update the comments accordingly
adds also test for i64 encode/decode and for compatibility between
u64 encode and i64 decode
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
we never actually compressed any files, since we only looked at
the extension:
* if it was 'zst' (which was always true for newly rotated files), we
would not compress it
* even if it was not 'zst', we compressed it inplace, never adding '.zst'
(possibly compressing them multiple times as zstd)
now we add new rotated files simply as '.X' and add a 'target' to the
compress fn, where we rename it to (but now we have to unlink the source
path)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
so that the last selected tab for datastores will get selected
the next time any datastore is selected, even across browser
reloads
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this fixes some bugs related to selection handling in the treelist:
* datastores were not selected after a reload
* reloading when in a tabpanel on any tab but the first, would
not select a treenode
* changing between datastores on any tab but the first would
not select the same tab on the new datastore
fixed those by mostly rewriting the changePath handling for
datastores and tabpanels in general
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
and make 'Datastore' unclickable
since we have all options and information on the relevant datastore panels,
we do not need a datastore config anymore (besides the creation,
which we add here)
this also fixes the sorted insertion and removal of new/old datastores
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
add the datastore as parameter for the store, remove
the datastore selector for the edit windows and give the datastore
to it instead
also remove the autostart from the rstore, since we only want to start
it when we change to the relevant tab
and add icons for all other datastore tabs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
this adds a 'Summary' panel to the datastores, similar to what we have
for PVE's nodes/guests/storages
contains an info panel with useful information, a comment field, and
the charts from the statistics panel (which can be deleted since it is
not necessary any more)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
a simple objectgrid to display datastore gc/prune options
needs the prune inputpanel to be refactored in its own class
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
and load it again when opening it
this way we can persist the status of the last garbage collect across
daemon reloads and reboots
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
by using the new ZipEncoder and recursively add files to it
the zip only contains directories, normal files and hardlinks (by simply
copying the content), no symlinks, etc.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
similar to StdChannelWriter, but implements AsyncWrite and sends
to a tokio::sync::mpsc::Sender
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
