Commit Graph

6201 Commits

Author SHA1 Message Date
Dietmar Maurer 7116a2d9da tape: lock media_catalog file to to get a consistent view with load_catalog 2021-07-29 13:34:25 +02:00
Dietmar Maurer 0d5e990a62 cleanup: factor out tape catalog path helpers 2021-07-29 13:34:18 +02:00
Dominik Csapak 4f57f4ad84 tape: changer: add tests for decode_element_status_page
a test for a valid status_page, one with excess data
(in the descriptor as well in the page as a whole)
and a test with too little data

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-28 12:23:21 +02:00
Dominik Csapak 13e13d836f tape: changer: handle libraries that sends wrong amount of data
if the library sends more data than advertised, simply cut it off,
but if it sends less data, bail out (depending on how much data is
missing, trying to parse it could lead to a panic, so bail out early)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-28 12:22:48 +02:00
Dominik Csapak 3ab2432ab6 tape: changer: remove unnecesary inquiry parameter
this is never used, so remove it.
Ok, since they are only non public functions.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-28 12:17:07 +02:00
Dominik Csapak 76e8565076 api2: tape/restore: commit temporary catalog at the end
in 'restore_archive', we reach that 'catalog.commit()' for
* every skipped snapshot (we already call 'commit_if_large' then before)
* every skipped chunk archive (no change in catalog since we do not read
  the chunk archive in that case)
* after reading a catalog (no change in catalog)

in all other cases, we call 'commit_if_large' and return early,
meaning that the 'commit' there was executed too often and
unnecessary, so move it after the loop over the files, before
finishing the temporary database.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-28 11:28:03 +02:00
Dominik Csapak a5f30a562b docs: tape: add instructions on how to restore the catalog
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-26 13:41:38 +02:00
Dominik Csapak a2ef36d445 tape: media_catalog: improve chunk_archive interface
instead of having a public start/end_chunk_archive and register_chunks,
simply expose a 'register_chunk_archive' method since we always have
a list of chunks anywhere we want to add them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-26 10:18:13 +02:00
Dominik Csapak 9a1ecae0b7 ui: tape/ChangerStatus: improve layout for large libraries
instead of having the grid be as tall as possible and the containing
panel scroll. limit the grids height to the panel size and scroll the
grid.

this has two advantages:
* if a user has many slots, it is now possible to to navigate the other
  grids to the position wanted
* having the grids scroll, means it can use extjs' buffered renderer,
  which makes the view much more responsive (in case of hundreds of
  slots)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-26 10:12:03 +02:00
Dominik Csapak 42b010174e tape: changer: handle invalid descriptor data from library in status page
We get the descriptor length from the library and use that in
'chunks_exact', which panics on length 0. Catch that case
and bail out, since that makes no sense here anyway.

This could prevent a panic, in case a library sends wrong data.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-26 10:05:37 +02:00
Dietmar Maurer 68e77657e6 datastore config: cleanup code (use flatten attribute) 2021-07-23 12:43:33 +02:00
Thomas Lamprecht 1b2f851e42 bump version to 2.0.7-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 08:44:41 +02:00
Thomas Lamprecht cc99866ea3 restore daemon: rust format
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 08:26:10 +02:00
Thomas Lamprecht 1ea3f23f7e file restore: improve some comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 08:25:34 +02:00
Thomas Lamprecht 3f780ddf73 restore daemon: log about doing basic system env setup
debugging history showed that its surely nice to have more logs at
when stuff happens (and thus fails)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 08:24:30 +02:00
Thomas Lamprecht 9edf96e6b6 restore daemon: setup backup system user and group
now required as we always enforce lock files to be owned by the
backup user, and the restore code uses such code indirectly as the
REST server module is reused from proxmox-backup-server. Once that is
refactored out we may do away such things, but until then we need to
have a somewhat complete system env.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 08:19:38 +02:00
Thomas Lamprecht 73e1ba65ca restore daemon: add setup_system_env helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-23 08:10:55 +02:00
Dominik Csapak 02631056b8 tape: changer: handle missing dvcid information
the dvcid information is not always available, so skip it if is missing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-22 12:00:30 +02:00
Dominik Csapak 131d0f10c2 tape: changer: improve error message on wrong counts
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-22 11:37:14 +02:00
Dominik Csapak f9aa980c7d tape: changer: correctly consume data in decode_element_status_page
instead of 'blindly' trusting the changer to deliver the fields written
in the specification, trust the length data it returns in the header.

we slice the descriptor data into equal sized chunks of the correct
size, then we do not have care bout the len and empty checks anymore

this also makes the code to read the rest of the page obsolete,
since the next descriptor is on the correct offset anyway

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-22 11:37:14 +02:00
Wolfgang Bumiller ad0364c558 tools: xattr: don't test things beyond our control
whether the kernel allows super-long names or weird
namespace prefixes is not our concern...

also the latter fails under fakeroot

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-22 11:34:40 +02:00
Thomas Lamprecht 76486eb3d1 bump version to 2.0.6-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-22 10:22:33 +02:00
Thomas Lamprecht 65ab4ca976 docs: simplify list of ENV var alternative
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-22 10:21:40 +02:00
Dietmar Maurer 99a73fad15 doc: Document new environment variabless to specify secret values
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-22 10:10:40 +02:00
Dietmar Maurer 16a01c19dd support new ENV vars to get secret values through a file or a command
We want to allow passing a secret not only directly through the
environment value, but also indirectly through a file path, an open
file descriptor or a command that can write it to standard out.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-22 10:09:53 +02:00
Fabian Ebner 86b8ba448c ui: server administration: repos: add online help
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-22 10:09:53 +02:00
Thomas Lamprecht 9b8e8012a7 cargo: update proxmox to 0.12.1
For the FS compat improvement in the atomic create file helper

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-22 10:09:53 +02:00
Fabian Grünbichler b29292a87b build: unbreak 'nocheck'
to skip test cases for faster builds or in case your local system does
not support running (all) tests..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-21 17:22:18 +02:00
Dominik Csapak c1feb447e8 tape: changer: sg_pt: fix typo
ok, since its a private struct

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-21 17:02:16 +02:00
Dominik Csapak 62a0e190cb tape: changer: sg_pt: add SCSI_VOLUME_TAG_LEN const
so that we do have less 'magic' constants without description

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-21 17:01:37 +02:00
Dominik Csapak 5890143920 api: types: CHANGER_DRIVENUM_SCHEMA: increase maximum drives per changer
to 255. 8 drives per changer was a rather arbitrary limitation and could
well be reached in practice with big libraries.

Altough 255 is still a arbirtrary limitation, this is much less likely
to be reached in practice.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-21 16:59:02 +02:00
Wolfgang Bumiller ef4df211ab move CachedChunkReader to pbs-datastore
this was actually still missing from the previous commit

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-21 14:20:03 +02:00
Wolfgang Bumiller eb5e0ae65a move remaining client tools to pbs-tools/datastore
pbs-datastore now ended up depending on tokio after all, but
that's fine for now

for the fuse code I added pbs-fuse-loop (has the old
fuse_loop and its 'loopdev' module)
ultimately only binaries should depend on this to avoid the
library link

the only thins remaining to move out the client binary are
the api method return types, those will need to be moved to
pbs-api-types...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-21 14:12:24 +02:00
Wolfgang Bumiller bbc71e3b02 client: fix panic message
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-21 13:28:55 +02:00
Dietmar Maurer ac81ed17b9 fix regression test file permission problems
By simply using the current user/group instead of backup:backup
2021-07-21 09:30:22 +02:00
Thomas Lamprecht 89145cde34 bump version to 2.0.5-2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-21 09:12:46 +02:00
Thomas Lamprecht ef4b2c2470 buildcfg: fix version
now set here, but we really need to automate this soon, just to easy
to forget.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-21 09:11:57 +02:00
Thomas Lamprecht 7190cbf2ac buildsys: run test before compile to avoid clobbering the openid build binaries
dh_auto_test also checks for the build flags used, including any
`--cfg`, so it rebuilds and overwrites our carefully assembled daemon
binaries with openid support as it is run after build and before
install.
So manually ensure the order of first test then build (argh, hackes
of hackes >.<)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 20:27:40 +02:00
Thomas Lamprecht f726e1e0ea buildsys: cargo build target: one binary per line
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 20:27:40 +02:00
Thomas Lamprecht 6d81e65986 bump version to 2.0.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 19:34:49 +02:00
Thomas Lamprecht ba5f5083c3 d/control: record fonts-font-awesome dependency for docs
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 19:34:49 +02:00
Dominik Csapak 314db4072c docs: add missing font-awesome link for lto-barcode generator
else it cannot load the icons and does not show them in the action column

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-07-20 19:34:49 +02:00
Thomas Lamprecht baff2324f3 pbs-tools: fix doctest reference to moved cache modules
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 19:15:28 +02:00
Thomas Lamprecht 02eae829f7 tests: move pxar test to its crate
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 18:54:23 +02:00
Thomas Lamprecht bb77143108 d/control: update build dependencies
needs to be done manually for now..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 18:54:23 +02:00
Thomas Lamprecht 02cb5b5f80 cargo: bump proxmox-http to 0.3.0
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 18:54:23 +02:00
Dietmar Maurer a301c362e3 add helpers to write configuration files 2021-07-20 18:54:23 +02:00
Dietmar Maurer 7526d86419 use new atomic_open_or_create_file
Factor out open_backup_lockfile() method to acquire locks owned by
user backup with permission 0660.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-07-20 18:54:23 +02:00
Wolfgang Bumiller a00888e93f fixup examples
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-20 15:26:25 +02:00
Wolfgang Bumiller fc5870be53 move channel/stream helpers to pbs-tools
pbs_tools
  ::blocking: std/async wrapping with block_in_place
  ::stream: stream <-> AsyncRead/AsyncWrite wrapping

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-07-20 11:27:40 +02:00