Compare commits

...

1132 Commits

Author SHA1 Message Date
c4430a937d bump version to 1.0.11-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-18 12:36:28 +01:00
237314ad0d tape: improve catalog consistency checks
Try to check if we read the correct catalog by verifying uuid, media_set_uuid
and seq_nr.

Note: this changes the catalog format again.
2021-03-18 08:43:55 +01:00
caf76ec592 tools/subscription: ignore ENOENT for apt auth config removal
deleting a nonexistant file is hardly an error worth mentioning

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-17 20:12:58 +01:00
0af8c26b74 ui: tape/BackupOverview: insert a datastore level
since we can now backup multiple datastores in the same media-set,
we show the datastores as first level below that

the final tree structucture looks like this:

tapepool A
- media set 1
 - datastore I
  - tape x
   - ct/100
    - ct/100/2020-01-01T00:00:00Z

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-17 13:37:49 +01:00
825dfe7e0d ui: tape/DriveStatus: fix updating pointer+click handler on info widget
we can only do this after it is rendered, the element does not exist
before

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-17 13:37:39 +01:00
30a0809553 ui: tape/DriveStatus: add erase button
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-17 13:37:17 +01:00
6ee3035523 tape: define magic number for catalog archives 2021-03-17 13:35:23 +01:00
b627ebbf40 tape: improve catalog parser 2021-03-17 11:29:23 +01:00
ef4bdf6b8b tape: proxmox-tape media content - add 'store' attribute 2021-03-17 11:17:54 +01:00
54722acada tape: store datastore name in tape archives and media catalog
So that we can store multiple datastores on a single media set.
Deduplication is now per datastore (not per media set).
2021-03-17 11:08:51 +01:00
0e2bf3aa1d SnapshotReader: add self.datastore_name() helper 2021-03-17 10:16:34 +01:00
365126efa9 tape: PoolWriter - remove unnecessary move_to_eom 2021-03-17 10:16:34 +01:00
03d4c9217d update OnlineHelpInfo.js 2021-03-17 10:16:34 +01:00
8498290848 docs: technically not everything is in rust/js
I mean the whole distro uses quite some C and the like as base, so
avoid being overly strict here.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 20:37:16 +01:00
654db565cb docs: features: mention that there are no client/data limits
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 20:37:16 +01:00
51f83548ed docs: drop uncommon spelled out GCM
It does not help users if that is spelled out, and its not a common
use of GCM, and especially in the AES 256 context its clear what is
meant. The link to Wikipedia stays, so interested people can still
read up on it and others get a better overview due to the text being
more concise.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 20:37:16 +01:00
5847a6bdb5 docs: fix linking, avoid over long text in main feature list
The main feature list should provide a short overview of the, well,
main features. While enterprise support *is* a main and important
feature, it's not the place here to describe things like personal
volume/ngo/... offers and the like.

Move parts of it to getting help, which lacked mentioning the
enterprise support too and is a good place to describe the customer
portal.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-16 20:37:16 +01:00
313e5e2047 docs: mention support subscription plans
and change enterprise repository section to present tense.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-03-16 19:24:23 +01:00
7914e62b10 tools/zip: only add zip64 field when necessary
if neither offset nor size exceeds 32bit, do not add the
zip64 extension field

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 09:13:39 +01:00
84d3284609 ui: tape/DriveStatus: open task window on click on state
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 09:00:07 +01:00
70fab5b46e ui: tape: convert slot selection on transfer to combogrid
this is much handier than number field, and the user can instantly
see which one is an import/export slot

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 08:57:48 +01:00
e36135031d ui: tape/Restore: let the user choose an owner
so that the tape backup can be restored as any user, given
the current logged in user has the correct permission.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 08:55:42 +01:00
5a5ee0326e proxmox-tape: add missing notify-user to 'proxmox-tape restore'
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-16 08:54:38 +01:00
776dabfb2e tape: use MB/s for backup speed (to match drive speed specification) 2021-03-16 08:51:49 +01:00
5c4755ad08 tape: speedup backup by doing read/write in parallel 2021-03-16 08:51:49 +01:00
7c1666289d tools/zip: add missing start_disk field for zip64 extension
it is not optional, even though we give the size explicitely

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-15 12:36:40 +01:00
cded320e92 backup info: run rustfmt
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-14 19:18:35 +01:00
b31cdec225 update to pxar 0.10
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:48:09 +01:00
591b120d35 fix feature flag logic in pxar create
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-03-12 10:17:51 +01:00
e8913fea12 tape: write_chunk_archive - do not consume partially written chunk at EOT
So that it is re-written to the next tape.
2021-03-12 07:14:50 +01:00
355a41a763 bump version to 1.0.10-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-11 13:40:22 +01:00
5bd4825432 d/postinst: fixup tape permissions if existing with wrong permissions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-11 13:40:22 +01:00
8f7e5b028a d/postinst: only check for broken task index on upgrade
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-11 13:40:22 +01:00
2a29d9a1ee d/postinst: tell user that we restart when updating from older version 2021-03-11 13:40:00 +01:00
e056966bc7 d/postinst: restart when updating from older version
Else one has quite a terrible UX when installing from 1.0 ISO and
then upgrading to latest release..

commit 0ec79339f7 for the fix and some other details

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-11 09:56:12 +01:00
ef0ea4ba05 server/worker_task: improve endtime for unknown tasks
instead of always using the starttime, use the last timestamp from the log
this way, one can see when the task was aborted without having to read
the log

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-11 09:56:12 +01:00
2892624783 tape/send_load_media_email: move to server/email_notifications
and reuse 'send_job_status_mail' there so that we get consistent
formatted mails from pbs (e.g. html part and author)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-11 09:56:12 +01:00
2c10410b0d tape: improve backup task log 2021-03-11 08:43:13 +01:00
d1d74c4367 typo fixes all over the place
found and semi-manually replaced by using:
 codespell -L mut -L crate -i 3 -w

Mostly in comments, but also email notification and two occurrences
of misspelled  'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 16:39:57 +01:00
8b7f3b8f1d ui: fix typo in comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 15:24:39 +01:00
3f6c2efb8d ui: fix typo in options
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-03-10 15:23:17 +01:00
227f36497a d/postinst: fix typo in comment
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-10 15:22:52 +01:00
5ef4c7bcd3 tape: fix scsi volume_statistics and cartridge_memory for quantum drives 2021-03-10 14:13:48 +01:00
70d00e0149 tape: documentation language fixup
Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-03-10 11:05:30 +01:00
dcf155dac9 ui: tape: increase tapestore interval
from 2 to 60 seconds. To retain the response time of the gui
when adding/editing/removing, trigger a manual reload on these actions

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-10 11:00:10 +01:00
3c5b523631 ui: NavigationTree: do not modify list while iterating
iterating over a nodeinterfaces children while removing them
will lead to 'child' being undefined

instead collect the children to remove in a separate list
and iterate over them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-10 10:59:59 +01:00
6396bace3d tape: improve backup task log (show percentage) 2021-03-10 10:59:13 +01:00
713a128adf tape: improve backup task log format 2021-03-10 09:54:51 +01:00
affc224aca tape: read_tape_mam - pass correct allocation len 2021-03-10 09:24:38 +01:00
6f82d32977 tape: cleanup - remove wrong inline comment 2021-03-10 08:11:51 +01:00
2a06e08618 api2/tape/backup: continue on vanishing snapshots
when we do a prune during a tape backup, do not cancel the tape backup,
but continue with a warning

the task still fails and prompts the user to check the log

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-09 10:20:54 +01:00
1057b1f5a5 tape: lock artificial "__UNASSIGNED__" pool to avoid races 2021-03-09 10:00:26 +01:00
af76234112 tape: improve MediaPool allocation by sorting tapes by ctime and label_text 2021-03-09 08:33:21 +01:00
1825c1a9b7 bump version to 1.0.9-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 17:06:57 +01:00
9a8bf2cac9 ui: navigation: adapt tape icons also here
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 17:06:57 +01:00
cc5ef79bec ui: tape drive icon: invert
Less anti-aliasing to do, so it looks better when small.

Note, I use a mask as else we could not cut out something
transparent, but only over paint it filled with white, which is not
the same and not how font awesome does it.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 15:51:49 +01:00
3725d95c65 ui: tape panel: add/use icons
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 14:35:36 +01:00
4fb068019e ui: tape: code style: xtype coms first
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 14:35:36 +01:00
6446a078a0 ui: tape/window/Tape{Backup, Restore}: allow empty notify user field
we have to set skipEmptyText on comboGrids

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:36 +01:00
1d7fcbece8 tape/linux_tape: remove debug prints
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:19 +01:00
8703a68a31 tape/backup: add success/error notifications
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 14:29:06 +01:00
9bcdade85f ui: remove cartridge memory panel (show window after butten press instead) 2021-03-08 12:52:06 +01:00
b0156179b9 ui: fix date rendering in drive status 2021-03-08 11:30:12 +01:00
d0a0bad9d6 ui: tape/Drive{Config, Status} move actions to status panel
so that the 'DriveConfig' panel is purely for configuration

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 11:18:25 +01:00
a4003d9078 ui: tape/DriveStatus: add cartridge memory grid
that the user can load when a tape is inserted

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-08 11:18:01 +01:00
3f4a62de2f docs: fix note about global media pool namespace 2021-03-08 10:56:28 +01:00
bf23f63aa5 docs: alabaster theme: better contrast for background color of topics
The old value just was way to heavy, and notes/warnings/...
admonitions did not stick out anymore.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 10:19:22 +01:00
fd641b99c3 ui: add online help to drive/changer status 2021-03-08 09:47:44 +01:00
225affc9ca tape: add docu link targets for online help 2021-03-08 09:37:40 +01:00
9ce2481a69 tape: clean - fix if cleaning tape is already loaded 2021-03-08 09:08:55 +01:00
d95c74c6e7 tape: test tape alert flags after drive clean 2021-03-08 09:08:55 +01:00
218ee3269f ui: images: simplify tape icon
1. Use same height as width
2. drop the top/bottom "frame" lines
3. instead of two "bearings" side by side, like VHS or music-compact
   tapes have, use a single bearing and an outer circle to denote the
   magnetic tape edge

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-08 09:02:32 +01:00
5ca5f8daf3 pmt: add tape-alert-flags command 2021-03-08 07:37:05 +01:00
98cdee781a docs: use code-block inside tape-backup.rst 2021-03-06 12:20:26 +01:00
9cf4504909 docs: document tape backup jobs 2021-03-06 12:10:39 +01:00
5f846a3fc1 add proxmox-tape bash-completion 2021-03-06 10:22:08 +01:00
c9793d47f9 api2/tape: add notify_user to backup(-jobs) and restore api calls
so that a user can be given that will be notified for
manual intervention (e.g. inserting a tape)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-05 17:20:37 +01:00
be8adca115 ui: tape: add notify-user fields
to backup jobs, backups and restores

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-05 12:53:41 +01:00
9152a0077f config/tape_job: add notify-user config to tape jobs
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2021-03-05 12:51:25 +01:00
0b90c67fb4 tape: avoid Updatable for now 2021-03-05 12:51:25 +01:00
b4975d3102 tape: finish api permission checks 2021-03-05 11:49:07 +01:00
ee33795b72 acl: use /tape/device instead of /tape/drive and /tape/changer 2021-03-05 10:06:19 +01:00
90e16be3ae tape: improve PoolWriter logging
Log reason why we allocate a new media set.
2021-03-05 09:59:38 +01:00
cf90a369e2 cleanup: rename token_user into auth_id_filter 2021-03-05 08:36:18 +01:00
6b303323be ui: use shorter datetime format for encryption key creation time 2021-03-04 17:57:40 +01:00
1576c7a0d9 ui: do not show MediaSet Uuid in backup content grid 2021-03-04 17:09:57 +01:00
cd5d6103ea tape: fix media_expire_time 2021-03-04 16:58:24 +01:00
207f763d1a tfa docs: language fixup
Simplifies the introduction a bit and makes it more readable.
Also some other minor language fixes throughout the section.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-03-04 15:22:09 +01:00
1bed3aedc8 tape: fix wrong media set expire time 2021-03-04 12:38:28 +01:00
ab77d660cc tape: improve media status in list_media 2021-03-04 10:17:10 +01:00
b74a1daae9 ui: add acl paths for tape permission 2021-03-04 08:33:26 +01:00
bec357e2cb ui: tape: add missing monStoreErrors calls for the grids
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
78593b5b5c tape/drive: improve error and email handling for requesting a tape load
Try once first to load the correct tape before sending an email to
insert the correct one. This way, the admin does not get a mail
if the correct tape is already inserted.

Also include the error we got that prompted the email to insert the
tape. This means that if the admin gets prompted to insert e.g.
"FOO" but inserts "BAR", he'll get an email that the wrong
tape is inserted.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
7d6f03a7fe ui: tape: only add tapestore and nav element once
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
f46573f8c3 add proxmox-tape zsh-completions
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
b83e136fb6 ui: tape/DriveStatus: fix handler name for eject
We call 'ejectMedia' not 'eject', so rename the function.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 16:09:39 +01:00
5c4203b20c ui: webauthn registration: code cleanup long variable name
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
7f9eef1d47 ui: webauthn registration: adapt error messages
There are variuos hints that 'InvalidStateError' can be assumed to
come from an already registered token[0][1].

So indicate that more strongly, also omit the extra line and warning
triangle, there's already an error icon present.

[0]: step 2: https://w3c.github.io/webauthn/#sctn-registering-a-new-credential
[1]: step 3: https://w3c.github.io/webauthn/#sctn-op-make-cred

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
a8a0132766 ui: LoginView: show webauthn errors in window
instead of silently discarding the error, else the user might be
confused because nothing happened

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 14:05:05 +01:00
831c43c91b config/tfa: webauthn: disallow registering a token twice
by adding the existing credential id to the 'excludeCredentials' list

this prevents the browser from registering a token twice, which
lets authentication fail on some browser/token combinations
(e.g. onlykey/solokey+chromium)
while is seems this is currently a bug in chromium, in a future spec
update the underlying behaviour should be better defined, making this
an authenticator bug

also explicitly catch registering errors and show appropriate error messages

0: https://bugs.chromium.org/p/chromium/issues/detail?id=1087642

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 14:05:05 +01:00
b452e2df74 Revert "ui: window/Settings / WebAuthn: add browser setting for userVerificationo"
even if the options *could* be set in the frontend, the backend
actually has to do validation of those settings, thus we should not
make that a browser setting

additionally, having the value 'preferred' does not actually make sense,
since it does not add any security (the backend skips the
userverification check then)

This reverts commit aca4c2b5a9.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 14:05:05 +01:00
7f37cacfac config/tfa: set UserVerificationPolicy to Discouraged
the current default is 'Preferred', which is not really useful, as the
(web) client can simply change this to discouraged, since the
webauthn_rs crate does not verify the 'user_verified' bit of the
response in that case

setting this to 'Required' is not really useful either at the moment,
since a user can have a mix of different authenticators that may or
may not support user verification

there is ongoing discussion in the crate how to handle that[0]

we could probably expose this setting(discouraged/required) to the user/admin
and save it to the credential and allow only registering credentials
of the same type or filter them out on login (i.e. if there is an
authenticator that can handle userVerification, require it)

in any case, the current default is not helpful for security, but
makes loggin in harder, since the key will by default want to verify
the user

0: https://github.com/kanidm/webauthn-rs/pull/49

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-03 14:05:05 +01:00
3bb7e62e88 zpool parsers: small code cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
3b060167f6 zpool parsers: fixup test value
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-03 14:05:05 +01:00
8a76e71129 tape: impl access permission for media content list 2021-03-03 13:16:15 +01:00
396fd747a6 tape: impl access permissions for tape jobs 2021-03-03 12:48:06 +01:00
16bd08b297 tape: add acl path /tape/job/{id} 2021-03-03 12:48:06 +01:00
ccdf327ac8 tape: impl access permissions for encryption key config 2021-03-03 12:21:55 +01:00
8cd63df0dc tape: impl api permissions for drive/changer/pool configuration 2021-03-03 12:10:00 +01:00
b90cb34fd6 complete_acl_path: add more paths 2021-03-03 11:09:54 +01:00
d6c1e12c06 add roles to implement tape backup permissions 2021-03-03 11:09:02 +01:00
d33d1c880b docs: mark tape backup as technology preview 2021-03-03 10:04:11 +01:00
985e84e369 Fix 3321: fix interactive restore command explanation
The previous description in the docs was false. The restore command
with the pattern parameter will search the entire backup archive,
regardless of pwd.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-03-03 09:34:19 +01:00
cc2c5c7762 install proxmox-tape binary 2021-03-03 09:02:02 +01:00
40bf636b47 ui: enable experimental tape UI if tape.cfg exists 2021-03-03 09:02:02 +01:00
347cde827b dashboard: fix datastore full estimate for not changing usage
If the usage of a datastore did not change, we did not
return an estimate. The ui interpreted this as 'not enough data', but
it should actually be 'never'.

Fixing this by always setting the estimate first to 0 and overwriting
if we successfully calculated one, and checking for 'undefined' in the ui.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 17:51:13 +01:00
ac4a1fb35c ui: render medium-wearout as percentage 2021-03-02 13:08:01 +01:00
6f3714b9aa ui: tape/DriveStatus: use the drive state to determine a status-load
since the status api call blocks the drive, it only makes sense to do
when the drive is idle.

so do not reload on activate, but when the drive changes the first
time from busy to idle

also disable the reload button when the drive is busy

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:52:39 +01:00
d810014eeb ui: tape/DriveStatus: add state to Info panel
and optimize the updateData function a bit

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:52:26 +01:00
e0f6892625 ui: tape/DriveStatus: add more status lines
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:51:50 +01:00
9d5b426a6d ui: tape: refactor renderDriveState to Utils
we will use this later again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:51:38 +01:00
8bf5769382 api2/tape/drive: add state to DriveListEntry
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-02 12:51:19 +01:00
2970cd3d6d ui: use TaskProgress for clean drive 2021-03-02 12:16:30 +01:00
d41114c5a8 ui: fix changer panel reload after clean 2021-03-02 12:09:00 +01:00
6c92449702 ui: improve media pool config grid - add comment field 2021-03-02 11:43:25 +01:00
db04d10d14 tape: add comment to media pool config 2021-03-02 11:42:58 +01:00
5a4233f07b ui: use fa-navicon for tape changer 2021-03-02 10:07:42 +01:00
3c715edd07 ui: correctly enable/disable changer drive buttons 2021-03-02 09:54:33 +01:00
bbe05d7fe9 ui: add 'Clean' button to changer drive (remove clean action) 2021-03-02 08:59:00 +01:00
2af8b8ef91 ui: add 'Catalog' button to changer drive 2021-03-02 08:53:18 +01:00
d4bfdfe749 ui: add 'Label Media' button to changer drive 2021-03-02 08:44:29 +01:00
1d14c31658 Revert "tape: also abort backup/restore on server shutdown"
This reverts commit 9bd81bb384.

Turns out this is not really a good idea.
2021-03-02 08:00:10 +01:00
9bd81bb384 tape: also abort backup/restore on server shutdown 2021-03-02 07:19:31 +01:00
d64226efee disks/zfs: fix test input and enforce check for config key
we test for the config key in the API so it makes sense to have as
test here too. Actually it would be better if we'd have a expect
Value defined here and enforce that it matches, but better than
nothing.

Fix the input for test 1, where tabs got replaced by spaces, as else
it fails

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-01 17:36:16 +01:00
2440eaa2df disks/zfs: pull common test code out in helper
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-03-01 14:48:35 +01:00
e8bf4f31f2 ui: tape: fix eslint warnings
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 14:18:44 +01:00
6682461d88 ui: tape/DriveConfig: remove label/catalog/eject buttons
they live now in the drive status panel

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 14:18:23 +01:00
41f1132e0e ui: tape/DriveStatus: add label/eject/catalog buttons
and enable them only if the status says that a tape is inserted.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 14:18:10 +01:00
d938c9337a ui: tape/BackupOverview: add sublevel for groups
For tapes with many snapshots/group, a simple list of them is too
big. Instead, add a level for just the groups, this makes searching
for a specific backup much easier.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 14:17:53 +01:00
9896a75caf ui: tape/ChangerStatus: handle vanishing view during reload
since reload is an async function, the view can be destroyed during
any 'await' point. Subsequent accesses to the view will fail, and we
will land in the catch. Check there if the view is destroyed, and
do not raise an error with the user then

also cancel any outstanding timer on 'deactivate' and 'destroy'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 14:01:31 +01:00
7eefd0c3d7 ui: tape: ChangerStatus: remove changerselector combobox
because we now select it directly in the left-hand tree
so we have to adapt the changer to the one set by the router
and not by the drop down field

and remove it from the TapeManagement tabpanel

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 12:36:09 +01:00
2e268e311c ui: NavigationTree: add entries for changers/drives
and only check TapeManagement once in the init function

we now have 2 updatestores that update individually
(one for datastores, one for drives/changers)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 12:35:43 +01:00
3e182fd828 ui: MainView: adapt router to add changer/drive entries
by generalizing the isDataStorePath logic to a 'parseRouterPath'.

We still have to keep the isDataStore logic for tabpanel handling,
If we add tabs to changer-/drivestatus panels, we have to adapt
that too.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 12:34:37 +01:00
7b60850334 ui: tape: add DriveStatus panel
not used yet

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-03-01 12:33:53 +01:00
1552d9699c tape: add 'Restore Key' button 2021-03-01 11:11:38 +01:00
7507b19cd2 OnlineHelpInfo.js: update 2021-03-01 11:11:07 +01:00
16f9ea6708 tape: restore - do not check if pool is configured 2021-03-01 09:43:23 +01:00
d984a9acf0 tape: add Inventory.media_list() helper 2021-03-01 09:30:45 +01:00
955f4aefcd tape: show media inventory/content, even if there is no pool configuration 2021-03-01 09:21:11 +01:00
858bbfbbd1 doc: barcode gen: some eslint/code-style fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-28 19:35:43 +01:00
c1570b373f doc: barcode gen: allow label heights from 15 mm
Americans seem to sometimes use Avery 6577 which has 5/8" x 3"
labels, equaling 15.875 mm x 76.2 mm, so do not set the lower bound
to 17mm (which even breaks our used Avery 3240 sheets which have an
label height of 16.9 mm

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-28 19:32:43 +01:00
d336363771 docs: barcode gen: fix acery3420 dimensions
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-28 19:31:35 +01:00
e57aa36d3e docs: add file extensions to file formats 2021-02-28 11:25:22 +01:00
b488f850aa docs: document fixed and dynamic index formats 2021-02-28 11:12:31 +01:00
ec07a280ba docs: document data blob format 2021-02-28 10:32:56 +01:00
5006632550 docs: improve backup protocol docs 2021-02-28 09:07:13 +01:00
7eb9f48485 doc: improve backup protocol docs 2021-02-27 12:44:03 +01:00
31cba7098d ui: rename 'Changer Slot' to 'Drive Number' 2021-02-27 09:51:13 +01:00
f4571b0b50 ui: use grouping feature for drive configuration grid 2021-02-27 09:37:47 +01:00
3832911d50 docs: remove backup protocol api dumps
We now have the api-viewer for that.
2021-02-26 19:00:08 +01:00
28c86760da docs: use mathjax library from debian package instead of CDN
By default, sphinx embeds the cloudflare CDN version of mathjax. This
is bad for privacy, webistes enforcing cross-site origin protection
and in environments with no WAN access.

Luckily there's a Debian package we can use instead.

The config is the default sphinx config used.

Reported-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-26 18:05:36 +01:00
c4604ca468 docs: adapt TOC styling to something a bit more light weight
marketing noted that it looked quite heavy weight, to the point where
it was hard to read to the full black squares, bold monospace text
and bottom borders with rather distinctive darkness.

Address those by:
* change color for list points from black to mid-dark grey
* use empty circles for second heading level
* ensure monospaced text has a normal font weight in the TOC headings
* some lighter color for the bottom border

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-26 18:01:39 +01:00
464c409aa3 ui: tape inventory - allow to set/clear media status 2021-02-26 11:06:11 +01:00
08ec39be0c tape: add api to set media status 2021-02-26 10:44:07 +01:00
25350f3370 tape: set media status if we detect damaged medium at start of backup 2021-02-26 09:00:50 +01:00
0023cfa385 tape: fix 'proxmox-tape backup' cli command 2021-02-26 08:51:24 +01:00
ed24142767 ui: TapeInventory - show catalog status
Clearly mark tapes without valid catalog.
2021-02-26 07:48:13 +01:00
917230e4f8 verify-api: support nested AllOf schemas 2021-02-25 13:44:17 +01:00
05228f17f5 ui: use button for tape Catalog (instead of action) 2021-02-25 13:04:50 +01:00
e8653b96be ui: add tape drive Eject button 2021-02-25 12:52:04 +01:00
1cf191c597 ui: fix sumbit url in LabelMedia.js 2021-02-25 12:25:32 +01:00
3d3e31b7f8 ui: factor out common code 2021-02-25 11:52:42 +01:00
8730cfcc3e ui: use butten for 'Label Media' (instead of action icon) 2021-02-25 11:00:24 +01:00
5830e5620d tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig 2021-02-25 09:44:31 +01:00
46d53e3e90 bump proxmox dep to 0.11.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-25 09:23:30 +01:00
3554fe6480 update to schema changes in proxmox
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-24 14:50:08 +01:00
0dadf66dc7 tape: implement backup using latest-only flag 2021-02-24 13:51:53 +01:00
a941bbd0c9 client: raise HTTP_TIMEOUT to 120s
As 20s is really not that high, especially for loaded setups one is
connected to through a spotty network (looking at you ÖBB railnet)
and gets latency spikes of 5 - 10s for some minutes at a time..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-24 11:57:21 +01:00
21e3ed3449 tape: add 'latest-only' property to backup job config 2021-02-24 11:19:12 +01:00
81678129fb ui: improve tape job edit (add eject-media and export-media-set)
And do not auto-generate job id.
2021-02-24 08:42:58 +01:00
52d8db7925 api2/config/tape_backup_job: fix duplicate id parameter
since the PUT api call is using the 'Updater', the 'id' parameter is
already encoded in there, tripping up the api verify tests with
'Duplicate keys found in AllOf schema: id'

"fixing" it by removing the explicit id from the api call and
taking it from the Updater (and failing if it does not exists there;
even though that should never happen)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 17:23:47 +01:00
875d375d7a api2/types: ArchiveEntry: fix inverse 'leaf' value
a 'leaf' node is every file *except* directories, so we have
to reverse the logtic here

this fixes the pxar.didx browser in the web ui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 17:23:47 +01:00
cba167b874 pxar/extract: if possible create files sparesly
instead of filling them with zeroes

this fixes an issue where we could not restore a container with large
sparse files in the backup (e.g. a 10GiB sparse file in a container
with a 8GiB disk)

if the last operation of the copy was a seek, we need to truncate
the file to the correct size (seek beyond filesize does not change it)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 14:44:59 +01:00
e68c0e68bd bump proxmox dep to 0.10.7
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-23 14:44:59 +01:00
ff2bc2d21f ui: tape/BackupJobs: add CRUD functions
similar to the other jobs grids (add/edit/remove etc.)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 13:18:10 +01:00
4961404c7c api2/config/tape_backup_job: enable update api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 13:17:49 +01:00
3fbf2311e4 tape: improve backup task logging 2021-02-23 12:58:44 +01:00
41685061f7 tape: volume-statistics - use format_size to display byte counts 2021-02-23 11:38:41 +01:00
35a7ab5778 ui: use grid to display tape drivbe status 2021-02-23 11:29:43 +01:00
e1beaae4a2 ui: fix tape volume statistics 2021-02-23 11:20:27 +01:00
965bd58693 ui: use grid to display tape label information 2021-02-23 11:13:48 +01:00
00fdaaf12b api-viewer: do not show permission for backup/restore protocol
Permission check is done before the H2 upgrade.
2021-02-23 10:04:16 +01:00
60473d234a ui: tape/ChangerStatus: do not show progress on drive clean
since we have the state in the grid

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 09:16:10 +01:00
4f688e09a4 ui: tape/ChangerStatus: do not show progressbar for (un)load
since we already show the state of the drive in the grid, this is
not necessary

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 09:10:21 +01:00
24e84128e4 ui: tape/ChangerStatus: add missing model fields
the DiffStore only updates the fiels given in the model, so we have to
list all the fields we depend on, else we have incomplete data,
e.g. for rendering

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 09:09:41 +01:00
e63457b6b2 ui: tape/ChangerStatus: disable drive buttons when it's blocked
if the drive state is not empty, some action blocks it, so we cannot
do any task anyway. Since we now refresh the state every 5 seconds,
we can disable/enable them 'live'.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 08:37:16 +01:00
a83cedc2ac ui: tape/ChangerStatus: convert grid stores to DiffStores
to not deselect items when the store is updated

this makes the ui a bit better to use

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 08:36:51 +01:00
076afa6197 ui: tape/ChangerStatus: reload changer status (cached) every 5000ms
so that we can have an updated drive status without having to
press 'reload'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 08:35:42 +01:00
423e3cbd18 ui: tape/ChangerStatus: only show loading on 'force refresh'
so that we can update in the background without interfering with the
users workflow

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-23 08:34:54 +01:00
0263396187 docs: fix some typos/grammar
Rund `codespell` tool, but it picked up not as much as I hoped.
Rest was found with vim + (hun)spell

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-22 18:08:59 +01:00
043018cfbe doc: fix wrong api method description 2021-02-22 12:10:34 +01:00
2037d9af03 api-viewer: show permissions 2021-02-22 12:01:24 +01:00
7f07991035 docs: improve MEDIA_SET_NAMING_TEMPLATE_SCHEMA description 2021-02-22 09:50:16 +01:00
18ce01caff tape: use correct schema for changer path (SCSI_CHANGER_PATH_SCHEMA) 2021-02-22 09:43:13 +01:00
5bc8e80a99 cleanup: cleanup use clause (avoid *) 2021-02-22 09:33:29 +01:00
6252df4c18 docs: fix EXPORT_SLOT_LIST_SCHEMA description 2021-02-22 09:05:07 +01:00
451856d21d api-viewer: fix path for upgraded HTTP/2 proptocls 2021-02-22 08:38:27 +01:00
aa30663ca5 docgen: improve regex format 2021-02-21 16:14:11 +01:00
8616a4afe5 docgen: dump string format (property strings) 2021-02-21 15:54:14 +01:00
bc2358319b docgen: improve api schema dump 2021-02-21 12:00:06 +01:00
0bf4b81370 docs: api-viewer - add backup and restore APIs 2021-02-21 10:38:33 +01:00
c9dd5a2452 .gitignore: do not ingnor .html files - we have some of them in the repository 2021-02-21 10:04:52 +01:00
cf95f616c5 add missing file docs/api-viewer/index.html 2021-02-21 10:04:23 +01:00
1adbc7c13c docs: install API viewer 2021-02-21 09:06:58 +01:00
9d28974c27 adopt PVEAPI.js for PBS, rename to PBSAPI.js 2021-02-21 09:06:58 +01:00
3dbc35b5c1 copy api-viewer from pve-docs 2021-02-21 09:06:58 +01:00
fee0fe5422 docgen: implement api schema generation (for api-viewer)
Just a start - not complete jet.
2021-02-21 09:06:58 +01:00
86d9f4e733 tape: extend MediaChange trait to return MtxStatus 2021-02-20 10:23:16 +01:00
3f16f1b006 tape: update changer status inside ScsiMediaChange implementation 2021-02-20 09:56:27 +01:00
cbd9899389 tape: update changer status cache after load/unload 2021-02-20 09:06:17 +01:00
cd44fb8d84 tape: create tmp dirs early at server startup 2021-02-19 18:10:02 +01:00
aca4c2b5a9 ui: window/Settings / WebAuthn: add browser setting for userVerificationo
some fido2/webauthn keys can have a pin, and the client can request
a mode for the user verification.

'default' (no value set), lets the browser/device decide if the user has to
enter the pin of the device
'discouraged' requests that the user should not need to enter the pin
'preferred' requests that the user should need to enter the pin (if possible)

since we use webauthn only as a 2nd factor, having the user enter
the device pin on login may seem too much hassle for some users, so
give them the option

since this is a client option anyway, do not save it in the backend, but
in the browser local storage

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 16:53:06 +01:00
85eedfb78b ui: add browser settings window
mostly copied from pve (for now; will refactor when i add it to
pmg too (soon)) without the pve specific features like dashboard
storages

contains some eslint fixes comparing to pves window

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 16:52:50 +01:00
f26276bc4e ui: tape/ChangerStatus: fixup for conditional
the statement !a === "somestring" cannot be true since
!a is either true or false and thus not a string

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 16:52:26 +01:00
6d62e69f9a ui: tape: fix eslint warnings (trailing comma)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 16:52:12 +01:00
4188fd59a0 tape: cache changer state 2021-02-19 16:48:19 +01:00
5b9f575648 update gitignore
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-19 12:42:14 +01:00
0d890ec414 ui: tape/ChangerStatus: show the state of the drive
an optimize the columns for smaller layouts (1280 width)
we show either:
* Idle
* spinner + status (if no upid)
* spinner + rendered UPID (clickable, opens task viewer)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:21:40 +01:00
926d05ef0b api2/tape/{backup, restore}, proxmox-tape: set device state
set the drive device state everywhere we lock it, so that we
know what it currently does

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:16:30 +01:00
8be48ddfc7 api2/tape/changer: add drive state to changer status output
if we can find the drive in the config and it has a state

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:15:15 +01:00
41e66bfaf6 api2/tape/drive: wrap some api calls in run_drive_blocking_task
those calls could also block, so we have to run them in a blocking
tokio task, as to not block the current thread

nice side effect is that we now also update the state for that
drive in those instances

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:14:08 +01:00
47a7241410 api2/tape/drive: use run_drive_blocking_task where possible
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:13:36 +01:00
54c77b3d62 api2/tape/drive: add wrapper for tokio::task::spawn_blocking
similar to the worker wrapper, lock, write status, run code, unset status

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:13:21 +01:00
a1c5575308 api2/tape/drive: use 'run_drive_worker' where possible
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:11:04 +01:00
a44c934b5d api2/tape/drive: add run_drive_worker wrapper
a wrapper for locking, starting the worker and writing the state

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:08:27 +01:00
546d2653ee tape/drive: add get/set status functions
simply writes into/reads from a file in /run, we will use this
for writing the upid (or potential other states) per drive

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:05:09 +01:00
33c06b3388 tape/drive: add test_device_path_lock
this makes it possible to detect if the drive was locked in a
non-blocking manner

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-19 10:03:30 +01:00
1917ea3ce1 cleanup: use serde(flatten) for VerificationJobStatus, improve code reuse 2021-02-19 09:50:25 +01:00
70842b9ef2 cleanup: use serde(flatten) for SyncJobStatus, improve code reuse 2021-02-19 09:36:39 +01:00
e6122a657e ui: cleanup - reuse existing code 2021-02-19 09:14:31 +01:00
9e860ac01a ui: tape/BackupJobs.js - render task scheduling status 2021-02-19 09:08:00 +01:00
7690a8e7bd api: list tape backup jobs with scheduling status 2021-02-19 09:02:13 +01:00
1860208560 add helper to compute job scheduling state 2021-02-19 08:58:50 +01:00
1689296d46 api: add new type JobScheduleStatus 2021-02-19 08:23:41 +01:00
7aa4851b77 ui: start tape backup job GUI 2021-02-18 12:48:54 +01:00
6ef8e2902f ui: tape/ChangerStatus.js - avoid update status in list_media
We do not use the media location, so there is no need to update
the online status in the list_media call. Besides, we already update
the online status when we query the changer/status.
2021-02-18 12:06:49 +01:00
aa16b7b284 ui: tape/ChangerStatus.js - only update status for selected changer
To reduce load/wait time.
2021-02-18 11:08:13 +01:00
9bbd83b1f2 tape: media_list API - allow to update online status for a single changer 2021-02-18 10:59:33 +01:00
65535670f9 ui: tape/TapeInventory.js - avoid update changer status
We do not update changer status by default - only when pressing "Reload" button.
2021-02-18 10:10:51 +01:00
9d42fe4d3b ui: tape/TapeInventory: set defaultValue for pool to ''
otherwise the grouping feature of the grid gets confused as it encounters
undefined values

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-18 09:35:05 +01:00
918a367258 ui: tape/ChangerStatus: adapt load/unload to WorkerTask
those are now workertasks (behind a 'POST' request), so show the
progress with the upid

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-18 09:34:53 +01:00
970a70b41e ui: tape/ChangerStatus: rename 'slot' column to 'id'
slot is already in the title of the grid, and hide it by default for
drives (the user does not need it there)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-18 09:34:40 +01:00
4094fe5a31 ui: tape/ChangerStatus: add Inventory column to drives/import-export
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-18 09:34:04 +01:00
dea8e2cb54 ui: improve task description for loat/unload media 2021-02-18 09:24:56 +01:00
0514a4308f update d/control
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-18 09:13:38 +01:00
d0647e5a02 tape: use worker tasks for media load/unload 2021-02-18 09:04:51 +01:00
bbe06f97be ui: tape/TapeInventory.js - use grid grouping feature - group by pool 2021-02-18 07:30:37 +01:00
f1a83e9759 client: extract common functions to proxmox_client_tools module
...including common schemata, connect(), extract_*() and completion
functions.

For later use with proxmox-file-restore binary.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 10:13:10 +01:00
38a81c6b46 bump pxar to 0.9.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:56:56 +01:00
6afb60abf5 asyncify pxar create_archive
...to take advantage of the aio::Encoder from the pxar create.

Rather straightforward conversion, but does require getting rid of
references in the Archiver struct, and thus has to be given the Mutex
for the catalog directly. The callback is boxed.

archive_dir_contents can call itself recursively, and thus needs to
return a boxed future.

Users are adjusted, namely PxarBackupStream is converted to use an
Abortable future instead of a thread so it supports async in its handler
function, and the pxar bin create_archive is converted to an async API
function. One test case is made to just use 'block_on'.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-17 09:24:20 +01:00
a42212fc1e pxar/extract: add extract_sub_dir
to extract some subdirectory of a pxar into a given target
this will be used in the client

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
2e21948156 api2/admin/datastore: refactor create_zip into pxar/extract
we will reuse that code in the client, so we need to move it to
where we can access it from the client

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

[clippy fixes]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
5279ee745f api2/admin/datastore: accept "/" as path for root
makes more sense than sending "root'"

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
227501c063 api2/admin/datastore: refactor list_dir_content in catalog_reader
we will reuse that later in the client, so we need it somewhere
we can use from there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>

[add strongly typed ArchiveEntry and put api code into helpers.rs]
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:32:41 +01:00
89d25b1931 client: add VsockClient to connect to virtio-vsock VMs
Currently useful only for single file restore, but kept generic enough
to use any compatible API endpoint over a virtio-vsock[0,1] interface.

VsockClient is adapted and slimmed down from HttpClient.

A tower-compatible VsockConnector is implemented, using a wrapped
UnixStream as transfer. The UnixStream has to be wrapped in a custom
struct to implement 'Connection', Async{Read,Write} are simply forwarded
directly to the underlying stream.

[0] https://www.man7.org/linux/man-pages/man7/vsock.7.html
[1] https://wiki.qemu.org/Features/VirtioVsock

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 08:15:43 +01:00
b57c0dbe30 rest: implement tower service for UnixStream
This allows anything that can be represented as a UnixStream to be used
as transport for an API server (e.g. virtio sockets).

A tower service expects an IP address as it's peer, which we can't
reliably provide for unix socket based transports, so just fake one.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2021-02-17 07:50:35 +01:00
8b910bb6bc tape: add regression test for compute_media_state 2021-02-17 06:57:29 +01:00
dfde34e612 ui: tape/BackupOverview: insert the tapes between media-set and snapshot
but auto-expand them, so no additional click is necessary

this shows the user which tapes are involved for the media sets

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
2530811e22 ui: css: remove vertical align for pbs-tape-icon
and combine the unnecessary duplicate definition

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
85205bc253 ui: tape: improve tape-erase and add it to library view
refactor erase window, limit drives to changer if given

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
3cdd1a3424 tape: erase_media - automatically load media (with tape libraries). 2021-02-17 06:55:53 +01:00
002865405c ui: tape/BackupOverview: show mediaset loading error in msg box instead
if a catalog is missing (or the loading otherwise throws an error), show
the error message in a msg box instead of a mask. this way a user can
still navigate the tree

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
8a73ef897a ui: tape/TapeInventory: add erase button
to erase the selected tape

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
be61c56c21 api2/tape/drive: add optional label-text to erase
if given, erases the tape only iff the inserted tape contains that label
used to safeguard tape erasing from ui for standalone drives

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-17 06:55:53 +01:00
dbaef7d04d tape: add regression test for alloc_writable_media 2021-02-17 06:55:49 +01:00
2048073355 ui: tape/TapeManagement: change Text of Backup/Tape Inventory
to 'Content' and 'Inventory' respectively

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-16 08:34:18 +01:00
a585e1f696 ui: tape/TapeInventory: add 'Add Tape' button
by adapting and reusing the 'LabelMedia' window

shows a short notice about inserting the correct tape in the drive
and now allows to select a drive

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-16 08:33:59 +01:00
415737b2b8 ui: tape/TapeInventory: change to regular store
listing the media can potentially hang (changer status), so do not
reload in the background, only on activate

also increase the timeout to 5 minutes and add a reload button

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-16 08:33:23 +01:00
54f7007cc5 ui: tape/ChangerStatus: use whole drives grid for masking on unload
instead of just the tableview

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-15 12:22:22 +01:00
b0338178d7 ui: tape/ChangerStatus: increase timeout for api calls
since tape commands can take a while and we do not want to change
all of those to worker tasks, increase the timeout to 5 minutes

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-15 12:22:06 +01:00
159100b944 tape: ui - avoid changer status query in backup content panel 2021-02-15 12:17:36 +01:00
41a8db3576 tape: implement API to run tape backup jobs 2021-02-15 11:09:30 +01:00
fe291ab794 cleanup: tape_backup_job.rs - do not use #[allow(non_camel_case_types)] 2021-02-15 10:41:04 +01:00
adb65b9889 tape: fix proxmox-tape pool config command 2021-02-15 08:58:05 +01:00
8513626b9f proxmox-backup-proxy: schedule tape backup jobs 2021-02-15 08:30:42 +01:00
7ca0ba4515 tape: add tape-job.cfg manual page 2021-02-15 07:05:39 +01:00
42200c405a fix api sort order 2021-02-15 06:35:45 +01:00
be327dbccd tape: add tape backup job configuration 2021-02-14 10:35:51 +01:00
c724dc3892 docs: update generated OnlineHelpInfo.js 2021-02-14 10:35:51 +01:00
70dc2ff3ab docs/barcode: eslint auto-fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-13 14:49:16 +01:00
81f5d03e8d docs: update benchmark results 2021-02-13 10:52:19 +01:00
e50c6b94c1 fix build: install proxmox-tape man page
Although we do not install the binary.
2021-02-13 10:33:16 +01:00
28eaff20bd docs: fix target refs in user-management.rst 2021-02-13 10:23:02 +01:00
8d1a1b2976 docs: conv.py - add man pages to exclude_patterns
To avoid sphinx warnings.
2021-02-13 10:16:41 +01:00
92eaec53db docs: add proxmox-backup service description 2021-02-13 09:48:21 +01:00
b3c2c57897 docs: add services.rst to section sysadmin 2021-02-13 09:40:36 +01:00
f458e97fda docs: split out output-format.rst 2021-02-13 08:54:23 +01:00
80bf9ae99b docs: add short descriptions for main binaries. 2021-02-13 08:43:54 +01:00
bebd4a7ca4 tape: generate proxmox-tape.1 manual page 2021-02-12 18:00:01 +01:00
9468e94412 tape: add pmt and pmtx command to tape-backup.rst 2021-02-12 17:45:55 +01:00
6b66c8507f lto-barcode: fix page offset calibration 2021-02-12 13:34:19 +01:00
167e5406c3 Makefile: fix typo (s/madia-pool/media-pool/) 2021-02-12 12:18:25 +01:00
c111c9a931 tape: add manual page for tape.cfg 2021-02-12 12:09:08 +01:00
bb71e3a023 tape: add manual page for media-pool.cfg 2021-02-12 11:50:56 +01:00
7b1bf4c098 tape: erase_media - automatically erase catalog and inventory 2021-02-12 11:30:34 +01:00
32b88d928a ui: tape/BackupOverview: skip tapes without media-set
there is no data on them (yet), even if they belong to a pool

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-12 08:53:16 +01:00
f8e1932337 ui: tape/TapeInventory: add 'Set Tape Location' button
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:24:10 +01:00
7c9fb570cc api2/types/tape/media_location: fix deserialization of MediaLocation::Vault
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:23:54 +01:00
56d22c66c0 api2/tape/media: add 'move' api call
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:23:27 +01:00
85cdc4f371 proxmox-tape: change lookup_drive_name to extract_drive_name
in most uses, we want to remove the drive from the param afterwards
where we don't, we already overwrite it with the result of this function

this fixes some commands (like 'proxmox-tape read-label --drive foo')
that failed with:

parameter 'drive': duplicate parameter.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-11 14:22:50 +01:00
96bcfb9b1f docs: add hint how to edit/modify the datastore.cfg file 2021-02-11 14:20:16 +01:00
4a874665eb proxmox-backup-manger verify-job CLI
Add missing command line interface to manage verification jobs.
2021-02-11 14:06:44 +01:00
6f6b69946e docs: use type_text for config::acl::Role
The list of roles is simply too long, so we use type_text to generate
reasonable docs.
2021-02-11 13:47:01 +01:00
5b7f44555e docs: add manual page for verification.cfg 2021-02-11 13:40:11 +01:00
2ca396c015 docs: add manual page for acl.cfg 2021-02-11 13:13:34 +01:00
d8dae16035 docs: add manual page fro sysc.cfg 2021-02-11 11:52:37 +01:00
8f02db04f9 docs: use new type_text option for calendar events 2021-02-11 11:51:22 +01:00
9f35e44681 fix proxmox_backup_manager remote list: s/userid/auth-id/ 2021-02-11 11:31:20 +01:00
6279b8f5a5 docs: add manukal page for remote.cfg 2021-02-11 11:25:15 +01:00
3084232cb5 depend on proxmox 0.10.5 2021-02-11 10:49:16 +01:00
67cc79ec52 docs: add user.cfg.5 manual page 2021-02-10 16:51:05 +01:00
b9a09a9501 docs: add "Configuration Files" appendix 2021-02-10 11:28:56 +01:00
4a0d3a3e3f docs: add datastore.cfg.5 man page 2021-02-10 11:05:02 +01:00
2322a980d0 docs: add utility binary to generate docs 2021-02-10 11:05:02 +01:00
c19f5b85a3 update debian/control 2021-02-10 11:05:02 +01:00
7f9d8438ab tfa: derive WebauthnConfigUpdater via api macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-10 10:47:47 +01:00
51c80c5a52 depend on proxmox 0.10.4 2021-02-10 10:24:36 +01:00
6477ebcf6f copyright: adopt year 2021-02-10 09:56:33 +01:00
bc02c2789c ui: tape/ChangerStatus: show more inventory info for tapes in slots
do not only show if labeled, but the whole status (full/writeable/etc.)
and to which pool the tape belongs (if any)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-10 06:21:49 +01:00
c0b3d09236 ui: tape/TapeInventory: show expired status
this is saved in a separate property, but show in status like
'proxmox-tape media list'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-10 06:21:39 +01:00
3ddbab6193 ui: tape/BackupOverview: do not iterate over empty tapes
tapes not belonging to a pool cannot contain data, so skip them

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-10 06:21:20 +01:00
befd95a90a pxar: improve error messages
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-09 13:09:00 +01:00
ab6cd4229b pxar: skip write_attr_fd if they are equal
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-09 13:09:00 +01:00
9213744ecb fix #3026: pxar: metadata: apply flags _after_ updating mtime
otherwise we'll fail to update the mtime of immutable files

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-09 13:08:49 +01:00
41c0333814 ui: tape/window/LabelMedia: change to POST
this changed in the backend

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-09 08:46:14 +01:00
afcf8b3ed6 api2/tape/drive: add missing locks for some drive actions
else a backup/restore can easily be disrupted

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-09 08:46:03 +01:00
69ebbec40b ui: tape/BackupOverview: rework BackupOverview (again)
instead of showing the snapshots directly under the pool and then the
media-sets, list the media-sets under the pool and only after the
snapshots

this has several advantages:
* we only have to read one set of tape catalog data on expand and not all of
  them everytime (which does not scale)
* we can show media-sets without snapshots, this can happen when we
  inventoried a set of tapes from another pbs instance, or lost the
  catalog data somehow

the disadvantage is that one has to go look for the media set where the
snapshot is included, but we can solve this by implementing a search
function in the future (in the backend)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-09 08:42:08 +01:00
b22a9c14a4 docs: disable smartquotes 2021-02-09 08:36:09 +01:00
54067d8225 ui: loginview: fix typo in view-model variable name
s/availabel/available/g

no semantic change at all

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-08 12:57:33 +01:00
d64c4eeab0 ui: dashboard/TaskSummary: map type filter back to original type
we include all tasks whose type start with 'verif' in the type 'verify'
but if we want to actually show them in the pop-up, we have to reverse
map this back to 'verif', else there are tasks missing from there

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-08 12:55:52 +01:00
15d2c7786e client: move test output files to target/testout
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-08 09:52:16 +01:00
73a1da5ed6 pmt: document driver options 2021-02-08 08:34:31 +01:00
fbf8779388 doc: improve pmt/pmtx manual pages 2021-02-06 16:50:56 +01:00
3231c35fb8 doc: fix problems in pxar manual page
- Avoid doubled title (Description/Description)
- Avoid error with sphinx :ref: role
2021-02-06 16:21:36 +01:00
ced7838de4 tape: fix path for test files 2021-02-06 09:32:50 +01:00
2f26b8668a client: track key source, print when used
to avoid confusing messages about using encryption keys when restoring
plaintext backups, or about loading master keys when they are not
actually used for the current operation.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:07:31 +01:00
9432838914 client: refactor crypto_parameter handling
pull out the crypt-mode to logically group arms and make the whole mess
a bit more "human-parsable".

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:06:28 +01:00
1a89a7794e client: extend tests for master key handling
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:05:01 +01:00
c0a87c12fb client: allow passing specific master key
it's needed for PVE's LXC integration, and might be interesting for
other more special usage scenarios as well.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:04:42 +01:00
c6a7ea0a2f client: refactor keyfile_parameters
no semantic changes intended

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 09:03:03 +01:00
5bb057e5a2 client: add test for keyfile_parameters
this will get more complex soon, so add test to document current
behaviour.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:58:15 +01:00
2924b37d6d key: rustfmt module
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:56:19 +01:00
42c0f784e2 key: add show-master-pubkey command
and print public key when generating/importing..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:55:21 +01:00
05f17d1ec4 key: make 'default' master key explicit
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-06 08:52:11 +01:00
777690a121 docs/scanrefs: fix handling if ref is same as headline
If the ref is named the same as the headline (once normalized), sphinx
will return a 'idX' value in node['ids'][1] which we use for the label
ID. The headline is always present at index 0.

Checking for that and using index 0 in case we do get a 'idX' helps us
to avoid using the 'idX' as keys in our OnlineHelpInfo.js and actually
use the intended key.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-02-06 07:49:21 +01:00
a98e228766 docs: fix references to changed refs
With commit ec1ae7e631 some refs were
changed by getting prefixes and such. We need to adapt the places that
reference them as well

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2021-02-06 07:47:56 +01:00
4c9174ce26 ui: tape/DriveConfig: add missing tooltips
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:13:04 +01:00
1d70e3812c ui: tape/DriveConfig: add Catalog button
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:12:49 +01:00
e2225aa882 ui: tape/ChangerStatus: add inventory button
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:12:18 +01:00
99dd709f3e ui: tape/ChangerStatus: remove unnecessary buttons
they exist in the changer config pane, and are unnecessary here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:12:18 +01:00
f197c286d5 tape/inventory: fix missing parameter in error message
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-05 16:12:18 +01:00
b121711baa update debian/control 2021-02-05 16:12:18 +01:00
085655b21b Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup 2021-02-05 12:53:43 +01:00
4c209d6b10 install pmt binary 2021-02-05 12:42:20 +01:00
8dc45e291a depend on proxmox 0.10.2 2021-02-05 12:42:20 +01:00
ec1ae7e631 docs/online-help: prefix some refs with their chapter name
and fix some issues from referenced named the same as their heading
they anchor too.

This should be fixed for real in our python plugin to scan for such
references, its probably a bug there, but as most of the problematic
ones where wrong (missing chapter prefix) anyway changing them is OK
too.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-05 11:42:06 +01:00
25aa55b5f5 tape: correctly lock the tape device 2021-02-05 10:50:21 +01:00
b5c6088130 ui: autofocus after rendering text fields for 2fa
Co-Authored-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Tested-by: Oguz Bektas <o.bektas@proxmox.com>
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-02-05 06:55:56 +01:00
a65eb0ec29 ui: wrap regexText in gettext() for translations
also fix a small typo in recovery keys text

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-02-05 06:55:56 +01:00
42eef1451c ui: tape/ChangerStatus: add 'is labeled' column for tapes
so that a user can see if a tape in a library is not yet labeled

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-04 17:36:29 +01:00
11ecf058e4 ui: tape/PoolConfig: add flex to columns
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-04 17:36:18 +01:00
5f1f7ef564 ui: tape/PoolEdit: add selector for encryption keys
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-04 17:36:06 +01:00
2e4e698633 ui: tape: add EncryptionPanel to add/remove encryption keys
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-04 17:35:46 +01:00
02dce8cad0 docs: index: update copyright years
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-04 16:06:38 +01:00
8aa4842fa8 bump version to 1.0.8-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-04 12:39:52 +01:00
efc09f63cc docs: tech overview: avoid 'we' and other small style fixes/additions
"we" should be avoided, it's never quite clear who is "we" in the
context here and it leads to some technical wrong meanings, e.g., we
(here assumed to be "we developers") do not read any backup data, the
Proxmox Backup client does.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-04 12:27:14 +01:00
3253d8a2e4 docs: tech overfiew: fix line length
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-04 12:05:27 +01:00
1531185dd0 docs: explain some technical details about datastores/chunks
adds explanations for:
* what datastores are
* their relation with snapshots/chunks
* basic information about chunk directory structures
* fixed-/dynamically-sized chunks
* special handling of encrypted chunks
* hash collision probability
* limitation of file-based backups

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-04 11:30:42 +01:00
baf9c3704e ui: task summary: add verification jobs to count
fixes a bug in which verification jobs were being excluded from the
verify task summary.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-02-04 11:07:22 +01:00
cdf39e62b3 tape: MediaPool - replace use_offline_media with changer_name
This way, we can improve location_is_available, because we only
consider media from that changer as available.
2021-02-04 10:15:18 +01:00
b81e37f6ab tape: improve code reuse 2021-02-04 09:39:16 +01:00
ddebbb52fd tape: fix tests for BlockedReader 2021-02-04 08:54:54 +01:00
983e929e25 tape: add multi volume reader/writer implementations
We currently do not use it. Added anaways, to show the possibility.
2021-02-04 08:36:35 +01:00
f47e035721 tape: cleanup - move tape file readers/writers into src/tape/file_formats folder 2021-02-04 07:59:37 +01:00
a80d72f999 tape: allow to abort restore tasks 2021-02-04 07:05:43 +01:00
8de9a9917f cleanup: use task_log macro 2021-02-04 06:55:18 +01:00
fa016c1697 HttpsConnector: use hostname instead of URL again
fixes connecting to hosts with valid certificates without a
pinned fingerprint
this was accidentally changed in the tokio-1.0 updates
apparently

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
Fixes: 0f860f712f ("tokio 1.0: update to new tokio-openssl interface")
2021-02-03 15:18:18 +01:00
7d2c156eb1 tape: BlockedReader - always consume EOF 2021-02-03 13:25:59 +01:00
04cec92e8d update copyright years
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-03 12:04:27 +01:00
64394b0de8 bump version to 1.0.7-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-03 10:36:18 +01:00
2f617a4548 docs: tfa: add screenshots
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-03 10:36:18 +01:00
2ba64bed18 ui: tfa: fix emptyText for password
One needs to enter their password, not the one from the user one
adds/deletes TFA.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-03 10:36:18 +01:00
cafccb5991 d/control: update
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-03 10:36:18 +01:00
b22e8c3632 tape: add media pool regression tests 2021-02-03 10:23:04 +01:00
7929292618 tape: add regresion test for media state 2021-02-03 09:34:31 +01:00
0d4e4cae7f tape: improve pmt command line completion 2021-02-03 08:54:12 +01:00
f4ba2e3155 depend on proxmox 0.10.1 2021-02-03 08:53:34 +01:00
7101ed6e27 ui: tape: add TapeInventory panel
since we do not show the tapes anymore in the BackupOverview, add
another panel where we can list the available tapes in the inventory

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:47:35 +01:00
85ac35aa9a ui: tape: add Restore Window
in the BackupOverview, when a media-set is selected

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:47:21 +01:00
40590561fe ui: tape: TapeBackupWindow: add missing DriveSelector
and make it a bit wider

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:47:05 +01:00
631e550920 ui: tape: rework BackupOverview
instead of grouping by tape (which is rarely interesting),
group by pool -> group -> id -> mediaset

this way a user looking for a backup of specific vm can do just that

we may want to have an additional view here were we list all snapshots
included in the selected media-set ?

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:46:43 +01:00
f806c0effa ui: refactor get_type_icon_cls
we need this later again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:46:15 +01:00
50a4797fb1 api2/types/tape/media: add media_set_ctime to MediaContentEntry
to be able to better sort in the ui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-02 14:45:54 +01:00
cc2a0b12f8 test: define tape tests as submodule 2021-02-02 14:38:15 +01:00
988e8de122 tape: set correct ownership on lock file 2021-02-02 14:18:57 +01:00
2f8809c6bc test: src/tape/inventory.rs - avoid chown when running tests 2021-02-02 13:43:16 +01:00
92b7775fa1 fix debian/control 2021-02-02 12:33:00 +01:00
f4d231e70a test: add regression tests for tape inventory 2021-02-02 12:19:28 +01:00
b419050aa7 bump pxar to 0.8
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-02 11:02:08 +01:00
8937c65951 tape: add pmt stoptions/stsethoptions/stclearoptions 2021-02-02 08:58:02 +01:00
6c6ad82d90 tape: add pmt setblk 2021-02-02 07:19:54 +01:00
d0f11b66f7 thape: add read_tapedev_options, display driver options with status command 2021-02-02 06:40:40 +01:00
f9fcac51a5 docs: add initial TFA documentation
better than nothing..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-01 19:46:24 +01:00
ca953d831f cleanup: remove MT_ST_ prefix from SetDrvBufferOptions 2021-02-01 17:54:53 +01:00
01c023d50f paperkey: rustfmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 17:05:40 +01:00
c2113a405e paperkey: simplify block generation
the chunk-iterator already does exactly what we want here..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 17:05:32 +01:00
5dae81d199 paperkey: allow RSA keys without passphrase
some users might want to store the plain version of their master key for
long-term storage and rely on physical security instead of a passphrase
to protect the paper key.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 17:05:22 +01:00
bd768c3320 ui: tfa: adapt low recovery key hint, drop unused other hint
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-01 15:39:56 +01:00
572fc035a2 ui: webauthn: add notes/warnings for better UX
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-01 15:37:47 +01:00
99b2f045af ui: tfa: add auto-fill button for webAuthn setup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-01 15:37:47 +01:00
6248e51797 change half-ticket time range from -120..240 to -60..600
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-02-01 15:13:11 +01:00
19e4a36c70 tape: do not use drive.open() within pmt
Do not fail if no media is loaded. Inportant for load command.
2021-02-01 12:39:50 +01:00
90769e5694 tape: add pmt lock/unlock 2021-02-01 12:18:55 +01:00
b8cbe5d65b tape: fix tape alert flag decoding 2021-02-01 12:18:55 +01:00
35c95ca653 bump apt-pkg-native dependency
our patches got applied upstream, and a release was cut, so we no longer
need to depend on a manually patched version here.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 11:53:25 +01:00
2dbc1a9a55 ui: tfa: improve button text for webAuthn
So users now what to press for starting off a webauthn challenge.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-02-01 11:48:43 +01:00
dceecb0bbf debcargo: fix maintainer directive"
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-02-01 11:21:21 +01:00
d690d14568 tape: add pmt bsr/fsr 2021-02-01 10:39:04 +01:00
85ef624440 tape: add pmt asf 2021-02-01 10:32:21 +01:00
e995996290 tape: pmt - fix count parameter schema 2021-02-01 10:21:25 +01:00
8e6ad4301d tape: add pmt fsfm/bsfm, pass count as arg_param 2021-02-01 10:18:18 +01:00
86740dfc89 tape: ui - remove drive from pool config 2021-02-01 10:01:06 +01:00
1399c592d1 garbage_collection: only ignore 'missing chunk' errors
with the fix for #2909 (improving handling missing chunks), we
changed from bailing to warning during a garbage collection when
updating the atime of a chunk.

but, updating the atime can not only fail when the chunk is missing,
but also on other occasions, e.g. no permissions or more importantly,
no space left on the device. in that case, the atime of a valid and used
chunk cannot be updated, and the second sweep of the gc will remove that chunk.
[0] is a real world example of that happening.

instead, only warn on really missin chunks, and bail on all other
errors.

0: https://forum.proxmox.com/threads/pbs-server-full-two-days-later-almost-empty.83274/

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-02-01 09:18:59 +01:00
9883b54cba tape: remove drive from pool config 2021-02-01 09:14:28 +01:00
83b8949a98 tape: add pmt weof 2021-01-31 17:33:07 +01:00
28f60e5291 cleanup: avoid compiler warnings 2021-01-31 17:02:55 +01:00
1f31d06f48 tape: add pmt bsf 2021-01-31 17:00:15 +01:00
2f2e83c890 tape: add pmt fsf 2021-01-31 16:54:16 +01:00
b22c618734 tape: add pmt erase 2021-01-31 16:34:10 +01:00
1e041082bb tape: add pmt command line tool
Experimental, not installed by now.
2021-01-31 16:19:53 +01:00
a57ce270ac postinst: add user backup to group tape
So that it is possible to access tape and changer devcies.
2021-01-30 11:48:49 +01:00
b5b99a52cd tape: API type cleanup, use serde flatten to derive types 2021-01-30 09:36:54 +01:00
9586ce2f46 tape: move scan_drives API code to correct file 2021-01-30 08:03:17 +01:00
b8d526f18d ui: tape/ChangerStatus - use POST for barcode-label-media 2021-01-29 17:06:53 +01:00
d2edc68ead ui: tape/ChangerStatus: add missing tooltips
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 16:54:37 +01:00
4d651378e2 ui: tape: change wrong window title
this is the 'status' msgbox not the label information

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 16:54:19 +01:00
58791864d7 ui: tape/ChangerStatus: add import action for import/export slots
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 16:54:03 +01:00
1a41e9af4f ui: tape: add Changer config grid
analogous to the drive grid

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 16:53:33 +01:00
c297835b01 tape: proxmox-tape - use API instead of direct functions calls 2021-01-29 11:49:11 +01:00
e68269fcaf tape: proxmox-tape inventory: call API 2021-01-29 11:21:57 +01:00
5243df4712 tape: proxmox-tape - use API instead of direct functions calls 2021-01-29 10:50:11 +01:00
4470eba551 cleanup: factor out common client code to view task log/result 2021-01-29 10:10:04 +01:00
1f2c4713ef tape: improve backup task abort behaviour 2021-01-29 09:23:39 +01:00
a6c16894ff worker_task: log something when we receive an abort request 2021-01-29 09:22:37 +01:00
271764deb9 tape: make it possible to abort tape backup tasks (check_abort)
Also use task_log makro instead of worker.log.
2021-01-29 09:07:55 +01:00
52f7a73009 display_task_log: make it possible to abort tasks with CTRL-C 2021-01-29 09:06:15 +01:00
bdb6e6b83f api2/reader: asyncify the reader worker task
this way, the code is much more readable

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-29 06:59:25 +01:00
41dacd5d3d tape: use worker task for eject-media api 2021-01-28 16:49:08 +01:00
eb1dfb02b5 tape: proxmox-tape - use api for erase-media and rewind 2021-01-28 16:36:10 +01:00
1a0eb86344 tape: gui: s/encryption/encrypt/ in media pool config panel 2021-01-28 15:50:01 +01:00
bdb62b20a3 tape: media_pool config api - set protected flags where required 2021-01-28 15:42:32 +01:00
f2ca03d7d0 cleanup: avoid compiler warning 2021-01-28 15:32:21 +01:00
00ac86c31b tape/drive/linux_tape: fix and refactor usage of sg-tape-cmd
when executing this code as non-root, we use sg-tape-cmd (a setuid binary)
to execute various ioctls on the tape device

we give the command the open tape device fd as stdin, but did not
dup it, so the std::process:Stdio handle closed it on drop,
which let subsequent operation on that file fail (since it was closed)

fix it by dup'ing it before giving it to the command, and also refactor
the calling code, so that we do not forget to do this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:24:32 +01:00
627d000098 tape: change changer-drive-id to changer-drivenum
because it changed in the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:11:22 +01:00
4be4736603 tape/changer: refactor marking of import/export slots from config
we did this for 'mtx', but missed it for the sg_pt_changer code
refactor it into the MtxStatus strut, and call it from both
code paths

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:10:55 +01:00
2da7aca8e8 tape/changer: add vendor/model to DriveStatus
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:10:31 +01:00
8306b8b1a5 ui: tape: use panels in tape interface
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:08:56 +01:00
605cfd4ab1 ui: tape: move TapeManagement.js to tape dir
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:08:31 +01:00
dec3147501 ui: tape: add PoolConfig
CRUD interface to manage media pools

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:08:21 +01:00
c642aec128 ui: tape: add DriveConfig panel
mostly typical CRUD interface for managing drives, with an
additional actioncolumn containing some useful actions, e.g.
* reading the label
* show volume-statistics
* show the status
* label the inserted tape

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:08:08 +01:00
fd9aa8dfa2 ui: tape: add ChangerStatus panel
this lets the users manage changers and lets them view the status of one
by having an overview of:
* slots for tapes
* import/export slots
* drives

lets the user:
* barcode-label all the tapes in the library
* move tapes between slots, into/out of drives
* show some basic info when a tape is loaded into a drive
* show the status of a drive
* clean a drive

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:07:57 +01:00
07d6c0967d ui: tape: add BackupOverview Panel
shows all tapes with the relevant info
* which pool it belongs to
* what backups are on it
* which media-set
* location
* etc.

This is very rough, and maybe not the best way to display this information.
It may make sense to reverse the tree, i.e. having pools at top-level,
then media-sets, then tapes, then snapshots..

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:07:44 +01:00
80a3749088 ui: tape: add Edit Windows
includes edit windows for
* Drives
* Changers
* Media Pools
* Labeling Media
* Making new Tape Backups

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:07:29 +01:00
c72fdb53ae ui: tape: add form fields
this includes selectors for
* Allocation Policy
* Retention Policy
* Drives
* Changers
* Tape Device Paths
* Pools

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:07:09 +01:00
b03ec281bf api2/config/{drive, changer}: prevent adding same device multiple times
this check is not perfect since there are often multiple device
nodes per drive/changer, but from the scan api we should return always
the same, so for an api user this should be enough

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:03:56 +01:00
cef4654ff4 api2/tape/drive: change methods of some api calls from put to get
makes more sense to have retrieving api calls as get instead of put

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:02:52 +01:00
f45dceeb73 api2/tape/drive: add load_media as api call
code was already there, just add it as api call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:02:13 +01:00
18262a88c9 api2/tape/changer: add changer filter to list_drives api call
so that an api user can get the drives belonging to a changer
without having to parse the config listing themselves

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 15:01:41 +01:00
87f4be7998 tape: use api to run proxmox-tape backup 2021-01-28 14:56:42 +01:00
d737adc6be tape: rename changer_drive_id to changer_drivenum 2021-01-28 11:29:59 +01:00
5fdaecf6f4 api2/tape/drive: reorganize drive api
similar to the changers, create a listing at /tape/drive and put
the specific api calls below that

move the scan api call up one level

remove the status info from the config listing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 11:15:17 +01:00
d8792b88ef api2/types/tape/drive: add changer_drivenum
so that an api user can see which drive belongs to which drivenum of a changer
for ones with multiple drives

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-28 11:14:28 +01:00
8b1174f50a ui: tfa: drop useless extjs state save handling
was replaced with our own, not much more code and actually works.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 20:20:35 +01:00
8c8f7b5a09 ui: tfa: disable confirm during handling of challenge
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 20:20:35 +01:00
44915932d5 ui: tfa: webautn: move spinning icon down to waiting message
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 20:20:35 +01:00
e90fdf5bed ui: tfa: make webAuthn abortable and restartable
Fix two things:
* do not reject the login promise when we get the abort DOMException
  error
* safely save the original challenge string as we work on a reference
  here and avoid to convert to a UInt8 array twice to avoid an
  exception.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 20:20:35 +01:00
a11c8ab485 ui: tfa: only immediately trigger webAuthn when its the initial tab
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 19:38:40 +01:00
74a50158ca ui: tfa: drop bogus console.error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 19:38:08 +01:00
6ee85d57be ui: tfa: save last used TFA method and prefer it next time
simple heuristic for those people who always prefer a specific TFA
method and have the others only as backup.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 18:45:36 +01:00
b2fc6f9228 fix build: commit missing file 2021-01-27 18:13:58 +01:00
f91481eded ui: rework TFA prompt on login
Improve UX by avoiding the need to click some buttons twice, or
calling TOTP and Recovery codes both "OTP" codes and showing multiple
buttons, with all having the same goal "submit a TFA token" at the
same time.

Instead use a tab panel with a single submit button.

WebAuthn can and should be still improved, but that can be OK as
followup.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-27 13:21:25 +01:00
651a61f559 pmtx: implement scan command 2021-01-27 12:40:51 +01:00
b06edeca02 remove generated file synopsis.rst (no need to track in git) 2021-01-27 12:38:02 +01:00
89ccb125d1 tape: use 36 byte Inquiry (recommended size) 2021-01-27 12:35:28 +01:00
c972704477 install pmtx binary 2021-01-27 11:36:15 +01:00
887f1cb90c cleanup: move scan changers API implementation 2021-01-27 09:58:16 +01:00
16b4d78400 tape: rename retry_command to execute_scsi_command, make retry a flag 2021-01-27 09:34:24 +01:00
ec8d9c6b80 tape: repeat changer scsi command until successful 2021-01-27 08:59:10 +01:00
49c2d1dcad sgutils2: use sg_get_asc_ascq_str to produce error messages 2021-01-27 06:56:11 +01:00
d0f51651f9 sgutils2: add ASC codes from tandeberg docs 2021-01-26 18:54:08 +01:00
481ccf16a5 sgutils2: further improve error messages 2021-01-26 15:19:43 +01:00
a223458753 sgutils2: support RequestSense Descriptor format 2021-01-26 13:38:16 +01:00
e1740f3f01 tape/changer/mtx: add mtx parser test
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:51:26 +01:00
740dc9d1d4 api2/tape/changer: reorganize api
add a changer listing here (copied from api2/config/changer)
and put the status and transfer api calls below that

puts the changer scan into the top level tape api
and removes the (now redundant) info from the config api path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:47:34 +01:00
bbf01b644c tape: fix typos
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:39:54 +01:00
66d066964c docs/tape: fix some typos and improve wording
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 12:39:06 +01:00
c81c46c336 sgutils2: improve error messages 2021-01-26 12:24:58 +01:00
c3747b93c8 tape: add new command line tool "pmtx"
Also improve sgutil2 error reporting
2021-01-26 11:57:15 +01:00
d43265b7f1 ui: add missing uri encoding in user edit and view
userid parameter needs to be properly encoded when shown on the browser

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 10:53:30 +01:00
6864fd0149 server/worker_task: improve newline handling in upid_read_status
improves upid_read_status with:
* ignore multiple newlines at the end
* remove all code that could panic (array index access)
  the one place where we access with '[pos+1..]' is ok since
  we explicitely test the len of the vector, this is done to
  let rust optimize away the range checks, so it cannot panic

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-26 10:48:15 +01:00
340c0bf9e3 pxar: don't clone patterns unnecessarily
The options struct has no Drop handler and is passed by-move
so we can partially move out of it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-26 10:24:18 +01:00
4d104cd4d8 clippy: more misc fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:55 +01:00
367c0ff7c6 clippy: allow api functions with many arguments
some of those can be reduced/cleaned up when we have updater support in
the api macro.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:52 +01:00
9c26a3d61a verify: factor out common parameters
all the verify methods pass along the following:
- task worker
- datastore
- corrupt and verified chunks

might as well pull that out into a common type, with the added bonus of
now having a single point for construction instead of copying the
default capacaties in three different modules..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:49 +01:00
93e3581ce7 derive/impl and use Default for some structs
and revamp HttpClientOptions with two constructors for the common use
cases

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:45 +01:00
f4e52bb27d authid: make Tokenname(Ref) derive Eq
it's needed to derive Hash, and we always compare Authids or their
Userid components, never just the Tokenname part anyway..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:40 +01:00
72064fd0df pxar: extract PxarExtractOptions
same as PxarCreateOptions, but for extraction/restore rather than
create.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:36 +01:00
77486a608e pxar: factor out PxarCreateOptions
containing the CLI parameters that are mostly passed-through from the
client to our pxar archive creation wrapper in pxar::create

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:32 +01:00
e97025ab02 pxar: typedef on_error as ErrorHandler
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:26 +01:00
e43b9175c0 client: factor out UploadOptions
to reduce function signature complexity.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:18 +01:00
9cc1415ef5 systemd/time: extract Time/DateSpec structs
could be pulled up into CalendarEvent if desired..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:13 +01:00
bd215dc0e4 async index reader: typedef ReadFuture
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:58 +01:00
12e874cef0 allow complex Futures in tower_service impl
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:55 +01:00
6d233161b0 client: refactor catalog upload spawning
by pulling out Result type into separate struct

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:51 +01:00
905a570489 broadcast_future: refactor broadcast/future binding
into its own, private struct.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:48 +01:00
432fe44187 report: type-alias function call tuple
to make clippy happy.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:53:43 +01:00
51b938496d tools::sgutils2: name fixup
it's not a box anymore

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 15:05:52 +01:00
b7f9b25e4d tools::sgutils2: use NonNull
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:56:10 +01:00
fe61280b6b tools::sgutils2: extern 'C' and import ordering
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:54:25 +01:00
68c087d578 tools::sgutils2: don't transmute to a Box
Otherwise we run the drop handler for the scsi pt object AND
the box itself, which shouldn't even work as it should be
doing a double-free (unless the library does some kind of
reference counting in which case this should simply crash
later on?)

anyway, let's make a wrapper simply called `SgPt` containing
the pointer from `construct_scsi_pt_obj()`

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:48:27 +01:00
d6bf87cab7 tools::sgutils2: const correctness
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 14:33:45 +01:00
2b96a43879 tape: cleanup - use ScsiMediaChange trait instead of mtx_status() 2021-01-25 13:25:22 +01:00
697c41c584 tape: add/use rust scsi changer implementation using libsgutil2 2021-01-25 13:14:07 +01:00
a2379996e6 sgutils2: add scsi_inquiry command 2021-01-25 13:14:07 +01:00
29077d95db http-client: further clippy cleanups
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:54 +01:00
dbd00a57b0 http-client: fix typoed ticket cache condition
which was even copy-pasted once without noticing.

found with clippy.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:51 +01:00
d08cff51a4 rework GC traversal error handling
the error message don't make sense with an empty default

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:48 +01:00
3e461dec1c apt: let api handle optional bool with default
one less FIXME :)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:46 +01:00
4d08e25913 clippy: rewrite ifs with identical return values
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:43 +01:00
43313c2ee7 clippy: rewrite comparison chains
chunk_stream one can be collapsed, since split == split_to with at set
to buffer.len() anyway.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:39 +01:00
81b2a87232 clippy: fix Mutex with unused value
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:36 +01:00
3d8cd0ced7 clippy: add is_empty() when len() is implemented
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-25 11:41:32 +01:00
7c78d54231 sgutils: allow command which does not transfer any data 2021-01-24 15:19:43 +01:00
f9d71e8b17 sgutils2: allow to set custom timeouts 2021-01-24 14:54:30 +01:00
0107fd323c cleanup: avoid compiler warnings 2021-01-23 17:34:26 +01:00
8ba47929a0 tape: add docu about paperkey 2021-01-23 15:34:28 +01:00
794b0fe9ce tape: document hardware encryption 2021-01-23 15:19:28 +01:00
979dccc7ec tape: avoid error when clearing encryption key
Simply ignore clear request when sg_spin_data_encryption_caps fails.
Assume those are tapes without hardware encryption support.
2021-01-23 10:20:43 +01:00
44a5f38bc4 docs: clarify that client-server communication is secure
This clarifies the fact that all communication between client and server
uses TLS for secure communication.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2021-01-22 16:07:44 +01:00
bf78f70885 improve code docs in api2
Note: API methos should be declared pub, so that they show up in the generated docu.
2021-01-22 15:57:42 +01:00
545706cbee d/control: bump B-D on pve-eslint
the old one does not understand www/config/TfaView.js and fails the
build..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-22 14:47:39 +01:00
0d916ac531 tape: add media pool config code docs 2021-01-22 12:01:46 +01:00
d4ab407045 tape: add drive config code docs 2021-01-22 11:51:36 +01:00
45212a8c78 fix mtx parser bug: s/strip_suffix/strip_prefix/ 2021-01-22 11:00:56 +01:00
64b83c3d70 tape: implement paperkey command for tape encryption keys 2021-01-22 09:56:14 +01:00
639a6782bd paperkey: move code to src/tools/paperkey.rs 2021-01-22 09:42:59 +01:00
5f34d69bcc tape: add volume-statistics api/command 2021-01-22 08:45:35 +01:00
337ff5a3cc tape: add estimated medium wearout to status 2021-01-22 08:06:25 +01:00
8e6459a818 tape: set encryption key on restore 2021-01-22 07:26:42 +01:00
aff3e16194 tape: add code docs to src/config/tape_encryption_keys.rs 2021-01-21 18:23:07 +01:00
9372c0787d renamed src/tape/sgutils2.rs -> src/tools/sgutils2.rs 2021-01-21 17:57:17 +01:00
83fb2da53e tape: move MediaCatalog magic number into struct (doc cleanup) 2021-01-21 17:48:07 +01:00
645a044bf6 tape: further hierarchy improvements 2021-01-21 17:25:32 +01:00
37796ff73f tape: change code hierarchy to improve docs 2021-01-21 17:12:01 +01:00
e1fdcb1678 tape: do not export/doc low level libsgutils2 bindings 2021-01-21 16:38:24 +01:00
aab9a26409 ui: cleanup order of declraing properties
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-21 15:09:22 +01:00
958055a789 ui: fix on-parse use of global Proxmox.UserName
This is wrong most of the time, when not loading the web interface
with valid credentials, and thus some checks or defaults did not
evaluated correctly when the underlying value was only set later.

Needs to be set on component creation only, this can be done through
initComponent, even listeners, view controllers or cbind closures.

Use the latter, as all affected components already use cbind.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-21 15:08:46 +01:00
edda5039d4 tape: improve code docs 2021-01-21 13:19:07 +01:00
1c86893d95 cleanup: always compute fingerprint in KeyConfig constructors 2021-01-21 11:56:54 +01:00
d543587d34 Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup 2021-01-21 10:56:52 +01:00
780bc4cad2 tape: try to set encryption key with read-label command 2021-01-21 10:31:49 +01:00
18bd6ba13d tape: restore_key - always update key, even if there is already an entry 2021-01-21 10:31:49 +01:00
4dafc513cc tape: fix file permissions for tape encryptiuon keys 2021-01-21 10:31:49 +01:00
7acd5c5659 cleanup: remove missleading wording from code docs 2021-01-21 10:31:49 +01:00
8428063d9e cleanup: KeyConfig::decrypt - show password hint on error 2021-01-21 10:31:49 +01:00
f490dda05a tape: use type Uuid instead of String 2021-01-21 10:31:49 +01:00
2b191385ea tape: use specialized encryption key per media-set 2021-01-21 10:31:49 +01:00
bc228e5eaf api: add types for UUIDs 2021-01-20 17:16:46 +01:00
8be65e34de clippy: replace transmute with &*
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:41:02 +01:00
d967d8f1a7 clippy: remove drop(&..)
it does nothing.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:41:02 +01:00
50deb0d3f8 clippy: use is_null to check for null pointers
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:41:02 +01:00
1d928b25fe clippy: remove some unnecessary reference taking
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
f2f81791d1 clippy: fix for_kv_map
and allow it in the one case where the entry loop is intended, but the
code is not yet implemented fully.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
382f10a0cc clippy: fix/allow needless_range_loop
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
0d2133db98 clippy: use while let loops
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
09faa9ee95 clippy: pass &str/&[..] instead of &String/&Vec
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
ccec086e25 clippy: remove unnecessary &mut
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
05725ac9a4 clippy: remove unnecessary let binding
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
96b7483138 clippy: remove/replace needless explicit lifetimes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
81281d04a4 clippy: fix/allow identity_op
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
e062ebbc29 clippy: us *_or_else with function calls
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
b92cad0938 clippy: convert single match to if let
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
ea368a06cd clippy: misc. fixes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3f48cdb380 clippy: don't pass along unit value
make it explicit. this whole section should probably be re-written with
select!

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
17c7b46a69 clippy: use unwrap_or_default
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a375df6f4c clippy: use copied/cloned instead of map
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a3775bb4e8 clippy: shorten assignments
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
1e0c6194b5 clippy: fix option_as_ref_deref
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
a6bd669854 clippy: use matches!
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
6334bdc1c5 clippy: collapse nested ifs
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3b82f3eea5 clippy: avoid useless format!
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
38556bf60d clippy: remove explicit returns
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
d8d8af9826 clippy: use chars / byte string literals
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
3984a5fd77 clippy: is_some/none/ok/err/empty
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
397356096a clippy: remove needless bool literals
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:52 +01:00
365915da9a clippy: use strip_prefix instead of manual stripping
it's less error-prone (off-by-one!)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
87152fbac6 clippy: drop redundant 'static lifetime
those declarations are already const/static..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
22a9189ee0 clippy: remove unnecessary closures
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
4428818412 clippy: remove unnecessary clones
and from::<T>(T)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
47ea98e0e3 clippy: collapse/rework nested ifs
no semantic changes (intended).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
6dd0513546 tape: allocate new media set when pool encryption key changes 2021-01-20 15:43:39 +01:00
8abe51b71d improve code docs 2021-01-20 15:43:19 +01:00
69b8bc3bfa tape: implemenmt show key
Moved API types Kdf and KeyInfo to src/api2/types/mod.rs.
2021-01-20 15:43:19 +01:00
301b8aa0a5 tape: implement change-passphrase for tape encryption keys 2021-01-20 15:43:19 +01:00
e5b6c93323 tape: add --kdf parameter to create key api 2021-01-20 15:43:19 +01:00
9a045790ed cleanup KeyConfig 2021-01-20 15:43:19 +01:00
82a103c8f9 add "password hint" to KeyConfig 2021-01-20 15:43:19 +01:00
0123039271 ui: tfa: rework removal confirmation dialog
present all relevant information about the TFA token to be removed,
so that a user can make a better decision.

Rework layout to match our commonly used style.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-19 19:46:10 +01:00
9a0e115a37 ui: tfa view: add userid to TFA data model
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-19 19:46:10 +01:00
867bfc4378 ui: login view: fix missing trailing comma
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-19 19:46:10 +01:00
feb1645f37 tape: generate random encryptions keys and store key_config on media 2021-01-19 11:20:07 +01:00
8ca37d6a65 cleanup: factor out decrypt_key_config 2021-01-19 11:20:07 +01:00
ac163a7c18 ui: tfa/totp: fix setting issuer in secret URL
it's recommended to set the issuer for both, the get parameter and
the initial issuer label prefix[0].

[0]: https://github.com/google/google-authenticator/wiki/Key-Uri-Format#label

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 16:27:02 +01:00
9b6bddb24c tfa: remove/empty description for recovery keys
While the user chosen description is not allowed to be
empty, we do leave it empty for recovery keys, as a "dummy
description" makes little sense...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 15:20:39 +01:00
f57ae48286 ui: tfa: fix ctime column width
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 14:31:15 +01:00
4cbd7eb7f9 gui: tfa: make description fill the remaining space
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 14:06:12 +01:00
310686726a gui: tfa: show when entries were created
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 14:06:12 +01:00
ad5cee1d22 tfa: add 'created' timestamp to entries
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 14:06:12 +01:00
bad6e32075 docs: fix typo in client manpage
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-18 13:52:11 +01:00
8ae6d28cd4 gui: enumerate recovery keys and list in 2nd factor window
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 13:51:23 +01:00
ca1060862e tfa: remember recovery indices
and tell the client which keys are still available rather
than just yes/no/low

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-18 13:51:23 +01:00
8a0046f519 tape: implement encrypted backup - simple version
This is just a proof of concept, only storing the encryption key fingerprint
inside the media-set label.
2021-01-18 13:38:22 +01:00
84cbdb35c4 implement FromStr for Fingerprint 2021-01-18 13:38:22 +01:00
1e93fbb5c1 tape: add encrypt property to media pool configuration 2021-01-18 13:38:22 +01:00
619554af2b tape: clear encryption key before writing labels
We always write labels unencrypted.
2021-01-18 13:38:22 +01:00
d5a48b5ce4 tape: add hardware encryption key managenent api 2021-01-18 13:38:22 +01:00
4e9cc3e97c ui: tfa: fix title for removal confirmation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 13:28:02 +01:00
492bc2ba63 ui: tfa/recovery: add print button to key info window
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 10:45:47 +01:00
995492100a ui: tfa/recovery: fix copy button text, add icon
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 10:45:28 +01:00
854319d88c ui: tfa/recovery: disallow to close key info window with ESC
to avoid accidental closing it

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 10:44:40 +01:00
3189d05134 ui: tfa: specify which confirmation password is required
Clarify that the password of the user one wants to add TFA too is
required, which is not necessarily the one of the current logged in
user. Use an empty text for that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 10:12:23 +01:00
b2a43b987c ui: tfa totp: whitespace and padding fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 10:10:16 +01:00
6676409f7f ui: access: stream line add/edit/.. button order and separators
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-18 09:33:29 +01:00
44de5bcc00 pull: add error context for initial group list call
otherwise the user is confronted with a generic error like "permission
check failed" with no indication that it refers to a request made to the
remote PBS instance..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-18 06:51:05 +01:00
e2956c605d pull: rustfmt
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-18 06:50:23 +01:00
b22b6c2299 tape: encryption scsi command cleanup 2021-01-16 18:24:04 +01:00
90950c9c20 tape: add scsi commands to control drive hardware encryption 2021-01-16 15:59:05 +01:00
0c5b9e7820 tape: sgutils2.rs - add do_out_command()
Make it possible to run commands that writes data.
2021-01-16 15:59:05 +01:00
a9ffa010c8 ui: webauthn config: set default values for unconfigured case
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-15 16:25:47 +01:00
a6a903293b ui: webauthn config: use ID instead of Id/id
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-15 16:25:26 +01:00
3fffcb5d77 gui: tfa configuration
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
a670b99db1 tfa: add webauthn configuration API entry points
Currently there's not yet a node config and the WA config is
somewhat "tightly coupled" to the user entries in that
changing it can lock them all out, so for now I opted for
fewer reorganization and just use a digest of the
canonicalized config here, and keep it all in the tfa.json
file.

Experimentally using the flatten feature on the methods with
an`Updater` struct similar to what the api macro is supposed
to be able to derive on its own in the future.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
aefd74197a bakckup::manifest: use tools::json for canonical representation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
9ff747ef50 add tools::json for canonical json generation
moving this from backup::manifest, no functional changes

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-15 15:19:52 +01:00
a08a198577 tape: do not abort backup if tape drive does not support tape-alert-flags 2021-01-15 11:43:17 +01:00
4cfb123448 tape: update restore docu 2021-01-15 09:44:46 +01:00
198ebc6c86 d/rules: patch out wrongly linked libraries from ELFs
this is a HACK!

It seems that due to lots of binaries getting compiled from a single
crate the compiler is confused when linking in dependencies to each
binaries ELF.

It picks up the combined set (union) of all dependencies and sets
those to every ELF. This results in the client, for example, linking
to libapt-pkg or libsystemd even if none of that symbols are used..

This could be possibly fixed by restructuring the source tree into
sub crates/workspaces or what not, not really tested and *lots* of
work.

So as stop gap measure use `ldd -u` to find out unused linkage and
remove them using `patchelf`.

While this works well, and seems to not interfere with any debug
symbol usage or other usage in general it still is a hack and should
be dropped once the restructuring of the source tree has shown to
bring similar effects.

This allows for much easier re-use of the generated client .deb
package on other Debian derivaties (e.g., Ubuntu) which got blocked
until now due to wrong libt-apt verison or the like.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-15 08:52:53 +01:00
a8abcd9b30 debian/control: set VCS urls
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-15 08:52:53 +01:00
b7469f5a9a d/control: sort and fix whitespace errors
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-15 08:52:53 +01:00
6bbe49aa14 access: restrict password changes on @pam realm to superuser
for behavior consistency with `update_user`

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-15 08:49:22 +01:00
5aa1019010 access: limit editing pam credentials to superuser
modifying @pam users credentials should be only possible for root@pam,
otherwise it can have unintended consequences.

also enforce the same limit on user creation (except self_service check,
since it makes no sense during user creation)

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2021-01-15 08:49:22 +01:00
29a59b380c proxmox 0.10: adapt to moved ParameterSchema
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0bfcea6a11 cleanup: remove unnecessary 'mut' and '.clone()'
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
19f5aa252f examples: unify h2 examples
update them to the new tokio-openssl API and remove socket buffer size
setting - it was removed from the TcpStream API, and is now only
available via TcpSocket (which can in turn be converted to a
TcpListener), but this is not needed for this example.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
89e9134a3f hyper: use new hyper::upgrade
the old Body::on_upgrade method is no more

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
b5a202acb6 tokio 1.0: update to new Signal interface
Signal does not yet re-implement Stream (and is not yet wrapped in
tokio-stream either).

see https://github.com/tokio-rs/tokio/pull/3383

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0f860f712f tokio 1.0: update to new tokio-openssl interface
connect/accept are now happening on pinned SslStreams

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
7c66701366 tokio 1.0: use ReceiverStream from tokio-stream
to wrap a Receiver in a Stream. this will likely move back into tokio
proper once we have a std Stream..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
585e90c0de tokio: adapt to 1.0 process:Child changes
Child itself is no longer a Future, but it has a new wait() async fn
that does the same thing

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
5c852d5b82 tokio: adapt to 1.0 runtime changes
enter() now returns a guard, and the builder got revamped to make the
choice between MT and current thread explicit.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
484172b5f8 tokio 1.0: AsyncRead/Seek with ReadBuf
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
d148958b67 proxmox 0.10: use tokio::time::timeout directly
TimeoutFutureExt is no more

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
0a8d773ad0 tokio 1.0: delay -> sleep
almost the same thing, new name(s), no longer Unpin

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
427d90e6c1 update to tokio 1.0
and various related crates

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-14 16:01:33 +01:00
9b2e4079d0 d/control: sort and fix whitespace errors
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-14 15:11:06 +01:00
1a0b410554 manager: user/token list: fix rendering 0 (never) expire date
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-14 13:59:08 +01:00
2d50a6192f tape: sg-tape-cmd - add more ways to specify devices 2021-01-14 13:05:26 +01:00
781da7f6f0 tape: add --inventorize flag to read-label API/CLI 2021-01-14 11:51:23 +01:00
646221cc29 ui: window/{AddWebauthn, TfaEdit}: fix spacing/border of the windows
the password field should not be indented differently than the rest of
the fields, and we never have a border on the panels

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-13 16:46:47 +01:00
b168a27f73 ui: window/AddTotp: fix spacing styling of form fields
by moving the lower fields into the form itself and dropping the padding

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-13 16:46:47 +01:00
a442bd9792 ui: window/AddTfaRecovery: fix style of TfaRecoveryShow window
to have a more similar layout/spacing to our other windows

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-13 16:46:47 +01:00
884fec7735 ui: window/AddTfaRecovery: rewrite to a Proxmox.window.Edit
we can reuse the edit window from widget toolkit for the most part
this solves some spacing and layout issues and is less code

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-13 16:46:47 +01:00
1cb89f302f ui: config/TfaView: disable Remove button by default
gets enabled when an item is clicked

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-13 16:46:47 +01:00
da36bbe756 ui: LoginView: remove not used viewModel
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-13 16:46:47 +01:00
25e464c5ce tape: MediaPool - allow to allocate free tapes 2021-01-13 14:25:51 +01:00
8446fbca85 tape: rename changer_id to label_text 2021-01-13 13:26:59 +01:00
9738dd545f tape: docu - explain manual backups and tape cleaning 2021-01-12 17:26:15 +01:00
0bce2118e7 tape: improve docu 2021-01-12 16:37:23 +01:00
6543214dde tape: MediaListEntry - add ctime 2021-01-12 12:01:21 +01:00
d91c6fd4e1 ui: tfa: drop bogus gettext of empty string
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2021-01-12 11:44:05 +01:00
711d1f6fc3 ui: notify options: Remove gettext for root@pam
Translating root@pam is not useful, especially as the empty text symbolises the
default value.

Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-01-12 11:41:24 +01:00
e422beec74 fix #3245: only use default schedule for new jobs
an empty schedule means 'none', so do not fill it with the default
in case we edit an existing job (like we do already for sync jobs)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2021-01-12 10:26:59 +01:00
a484c9cf96 tape: automatically reload tapes inside autoloader
We always automatically unload tapes to free library slots,
so it should not happen that an ejected tape resides inside the drive.

This is just a safe guard to handle the situation in case it happens ...

You can manually produce the situation by ejecting a tape without unloading:

 mt -f /dev/nst0 eject

Note: Our "proxmox-tape eject" does automatic unload
2021-01-12 09:49:05 +01:00
5654d8ceba tape: make eject/export more reliable, improve logging 2021-01-12 09:16:16 +01:00
31cf625af5 tape: improve backup logs 2021-01-11 13:23:12 +01:00
93be18ffd2 tape: fix tape alert flag values 2021-01-11 13:23:12 +01:00
e96464c795 d/control bump
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 12:09:19 +01:00
ad0ed40a59 api: return "invalid" as CSRF token for partial tickets
So that old clients don't `unwrap` a `None` value.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
63fd8e58b2 gui: masks for: adding recovery and removals
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
758a827c2d gui: add load mask during webauthn api calls
so that if we run into the 3s delay due to the wrong
password the window is properly masked

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
7ad33e8052 tfa: use UNAUTHORIZED http status in password check
to trigger our 3s delay in the rest handler

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
abfe0c0e70 tfa: fixup for challenge file split
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:13 +01:00
f22dfb5ece tfa: remove tfa user when a user is deleted
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:10 +01:00
4bda51688b tfa: improve user existence check
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
eab25e2f33 tfa: allow deletion of entries of non-existent users
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
94bd11bae2 typo fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
759af9f00c tfa api: return types and 'pub' structs/methods
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
f58e5132aa tfa: entry access/iteration cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
d831846706 tfa: r#type parameter name
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
1fc9ac0433 tfa: _entry api method name suffix consistency
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:23:03 +01:00
5c48d0af1f tfa gui: fix adding recovery keys as user
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
30fb19be35 tfa view: html-escape description text
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
fbeac4ea28 gui: tfa support
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
7f066a9b21 proxy: expose qrcodejs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
c5a767cd1d depend on libjs-qrcodejs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
027ef213aa api: tfa management and login
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
dc1fdd6267 config: add tfa configuration
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
96918252e5 buildcfg: add rundir helper macro
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
014dc5f9d7 tools: add create_run_dir helper
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
59e94227af add tools::serde_filter submodule
can be used to perform filtering at parse time

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-11 10:22:32 +01:00
e84b801c2e tape: improve retention period docu 2021-01-11 07:11:17 +01:00
6638c034d2 tape: remove unused eject_on_unload method 2021-01-10 16:20:18 +01:00
04df41cec1 tape: more MediaChange cleanups
Try to provide generic implementation for complex operations:

- unload_to_free_slot
- load_media
- export media
- clean drive
- online_media_changer_ids
2021-01-10 15:32:52 +01:00
483da89d03 tape: improve export media to directly export from drive, add CLI 2021-01-10 13:44:44 +01:00
c92e3832bf tape: cleanup: s/transfer/transfer_media/, avoid compiler warnings 2021-01-10 12:18:30 +01:00
edb90f6afa tape: backup - implement export-media-set option 2021-01-10 11:59:55 +01:00
0057f0e580 tape: MediaChange - add transfer, implement export 2021-01-10 11:51:09 +01:00
e6217b8b36 tape: renamed src/tape/changer/linux_tape.rs -> src/tape/changer/mtx.rs 2021-01-10 10:07:40 +01:00
6fe16039b9 tape: simplify media changer implementation - new struct MtxMediaChanger 2021-01-10 10:02:01 +01:00
42967bf185 tape: backup - implement --eject-media option 2021-01-09 15:17:03 +01:00
5843268c47 tape: abort backup when we detect critical tape alert flags 2021-01-09 12:34:00 +01:00
7273ba3de2 tape: change default media set naming template to "%c" 2021-01-09 10:51:51 +01:00
0bf1c314da tape: show catalog status in media list 2021-01-09 10:24:48 +01:00
c7926d8e8c tape: split MediaSet into extra file 2021-01-09 08:54:58 +01:00
44ce25e7ac tape: docu - improve Administration section 2021-01-08 19:17:31 +01:00
3a2cc5c66e tape: minor docu update in retention policy 2021-01-08 19:01:38 +01:00
3838ce3330 tape: add retention policy docu 2021-01-08 17:34:58 +01:00
59217472aa tape: improve media set docu 2021-01-08 16:53:46 +01:00
df69a4fc59 tape: implement drive clean 2021-01-08 11:32:56 +01:00
25d3965769 tape: correctly skip cleaning tapes (not regular tapes) 2021-01-08 09:16:42 +01:00
08d8b2a4fd tape: add some media pool docu 2021-01-08 08:46:25 +01:00
879569d73f tape: changer transfer - make name parameter optional 2021-01-07 17:09:47 +01:00
b63f833d36 tape: fix paramater name - s/slot/source-slot/ 2021-01-07 15:39:25 +01:00
482c6e33dd tape: changer status command: make changer name optional 2021-01-07 15:12:19 +01:00
46a1863f88 tape: improve MediaChange trait
We expose the whole MtxStatus, and we can load/store from/to
specified slot numbers.
2021-01-07 14:26:43 +01:00
632756b6fb tape: more docs 2021-01-06 16:13:58 +01:00
04eba29c55 tape: document tape drive configuration 2021-01-06 16:00:31 +01:00
0912878ecf tape: document new export-slots feature 2021-01-06 14:11:35 +01:00
d5035c5600 tape: mtx_status - consider new export-slots property 2021-01-06 11:53:33 +01:00
38ae42b11a tape: changer - add export-slot config 2021-01-06 11:06:50 +01:00
a174854a0d tape: improve tape changer docs 2021-01-06 09:45:36 +01:00
c4b2b9ab41 tape: only query volume stats if we can read MAM 2021-01-06 09:20:36 +01:00
ef942e04c2 tape: add function to classify tape-alert-flags 2021-01-05 17:23:30 +01:00
f54cd66924 ui: running tasks: Use gettext for column labels
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
2021-01-05 13:53:33 +01:00
b40ab10d38 tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus 2021-01-05 13:43:17 +01:00
f8ccbfdedd tape: implement read_volume_statistics 2021-01-05 12:58:18 +01:00
470f1c798a tape: status - show thape alert flags 2021-01-04 13:15:30 +01:00
5c012b392a tape: use LP 12h TapeAlert Response to query tape alert flags 2021-01-04 13:14:02 +01:00
165b641c1d tape: changer status - show full slots (for cartridge without barcode) 2021-01-04 12:06:05 +01:00
66e42bec05 tape: further PoolWriter cleanups 2021-01-03 12:08:40 +01:00
c503ea7045 tape: cleanup - rename 'info' to 'media_id'
Second try.
2021-01-03 11:38:00 +01:00
745ec187ce Revert "tape: cleanup - rename 'info' to 'media_id'"
This reverts commit f046313c0e.

media_id is already use as parameter, so this commit is totally buggy.
2021-01-03 11:14:58 +01:00
f046313c0e tape: cleanup - rename 'info' to 'media_id' 2021-01-03 10:37:42 +01:00
74595b8821 tape: sg-tape-cmd tape-alert-flags 2021-01-03 10:09:43 +01:00
c9fdd142a4 tape: commit missing file 2021-01-02 13:39:34 +01:00
abaa6d0ac9 tape: decode TapeAlertFlags in cartridge-memory command 2021-01-02 10:55:30 +01:00
cfae8f0656 tape: merge MediaStateDatabase into Inventory 2021-01-01 16:15:13 +01:00
54f4ecd46a tape: implement MediaPool flag to consider offline media
For standalone tape drives.
2021-01-01 10:03:59 +01:00
1835d86e9d gui: update tape job descriptions 2020-12-31 10:37:09 +01:00
b9b4b31284 tape: add basic restore api/command 2020-12-31 10:26:48 +01:00
b4772d1c43 tape: new inventory helper - lookup_media_set_pool 2020-12-31 10:03:17 +01:00
9933dc3133 update TODO 2020-12-31 08:38:22 +01:00
08ac90f920 api: allow tokens to list users
their owner, or all if they have the appropriate privileges.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:49 +01:00
13f5863561 api: improve error messages for restricted endpoints
the old variant attempted to parse a tokenid as userid and returned the
cryptic parsing error to the client, which is rather confusing.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-31 08:29:09 +01:00
81764111fe tape: media_change - log all errors 2020-12-30 19:17:18 +01:00
cb022525ff tape: only log to stdout in CLI environment 2020-12-30 19:01:39 +01:00
75656a78c6 tape: improve inline docu 2020-12-30 17:28:33 +01:00
284eb5daff tape: cleanup/simplify media_change code 2020-12-30 17:16:57 +01:00
ff58c51919 tape: improve media request/load 2020-12-30 13:09:28 +01:00
2fb1bdda20 verify-api: fix allOf duplicates check
it triggered with a wrongly-formatted message on schemas that did NOT
contain any duplicates..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-30 12:36:00 +01:00
12299b333b tape: set minimal media label length to 2 2020-12-30 10:15:02 +01:00
b017bbc441 tape: add restore code, implement catalog api/command 2020-12-30 09:48:18 +01:00
9e8c0d2e33 tape: cleanup - remove debug messages 2020-12-30 08:41:30 +01:00
250c29edd2 tape: correctly sort media api entries 2020-12-29 12:09:51 +01:00
c431659d05 cleanup: remove debug output 2020-12-29 11:59:57 +01:00
a33389c391 tape: implement media content list api 2020-12-29 11:58:26 +01:00
3460565414 tape: create the MediaCatalog when we label a tape 2020-12-29 10:55:20 +01:00
26b62138ee cleanup: disable debug message when we detect a stopped worker task 2020-12-29 10:53:16 +01:00
afb0220642 tape: cleanup LinuxDriveStatus - make density optional 2020-12-29 09:10:30 +01:00
0993923ed5 tape: factor out get_drive_and_media_status 2020-12-29 08:39:06 +01:00
e0362b0d0f tape: correctly parse mtx import/export slots 2020-12-28 13:32:56 +01:00
df3a74d7e0 debian: correctly install sg-tape-cmd setuid binary 2020-12-28 13:22:17 +01:00
d5d457e667 fix typo in Makefile 2020-12-28 11:41:10 +01:00
b27c32821c tape: install new sg-tape-cmd setuid binary 2020-12-28 11:10:25 +01:00
76b15a035f tape: MediaCatalog: write magic number before content 2020-12-26 11:05:25 +01:00
eb8feb1281 tape: add LTO1 to TapeDensity 2020-12-26 10:48:32 +01:00
fc6ce9835b tape: fix non-rewinding tape device check 2020-12-25 15:38:29 +01:00
8ae9f4efc2 tape: minor cleanups 2020-12-25 13:45:26 +01:00
c9d13b0fc4 tape: expose check_tape_is_linux_tape_device 2020-12-24 15:51:49 +01:00
bfacc1d8c3 tape: cleanup - factor out open_linux_tape_device 2020-12-24 11:24:45 +01:00
02d484370f fix build depends 2020-12-23 11:54:44 +01:00
5ae86dfaa1 tape: return media usage info with status command 2020-12-23 11:24:34 +01:00
dbe7e556b0 tape: implement binding for libsgutils2
So that we can read cartridge memory without calling "sg_raw". In future,
we may need further low level command to control the tape..
2020-12-23 09:44:53 +01:00
4799280ccd http_client: add timeouts for critical connects
Use timeout futures for sections that might hang in certain error
conditions. This is mostly intended to be used as a safeguard, not a
first line of defense - i.e. best-effort avoidance of total hangs.

Not every future used for the HttpClient/H2Client is changed, only those
where a quick response is to be expected. For example, the response
reading futures are left alone, so data transfer is never capped with
timeout, only the initial server connect.

It is also used for upgrading to H2 connections, as that can take a long
time on overloaded servers.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-12-22 13:31:10 +01:00
cb4865466e depend on proxmox 0.9.1 2020-12-22 13:30:41 +01:00
cb80d900b3 tape: add drive status api 2020-12-22 10:42:22 +01:00
ee01737e87 tape: rename 'mam' api to 'cartridge-memory' 2020-12-22 09:27:34 +01:00
2012825913 depend on proxmox 0.9.0 2020-12-22 08:52:24 +01:00
eb5e3420ae tests: verify-api: check AllOf schemas
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-22 07:31:38 +01:00
b2362a1207 adaptions for proxmox 0.9 and proxmox-api-macro 0.3
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-22 07:31:05 +01:00
54d968664a tape: update user docu 2020-12-21 12:13:35 +01:00
1e20f819d5 tape: add command to read cartridge memory (MAM)
Thsi add an additional dependency to sg3-utils (small).
2020-12-21 12:12:33 +01:00
8001c82e81 tape: update user docu - howto label tapes 2020-12-20 10:41:40 +01:00
baefbc444e tape: update user docu 2020-12-20 09:16:09 +01:00
4a227b54bf add LTO barcode generator App 2020-12-19 17:39:48 +01:00
8a192bedde tape: update user docu 2020-12-19 16:56:54 +01:00
d5efa18ae4 tape: update user docu 2020-12-19 15:13:38 +01:00
5f79dc2805 tape: start user documentation 2020-12-19 11:14:56 +01:00
9aa58f0143 cleanup: rename mtfsf into forward_space_count_files 2020-12-18 16:57:49 +01:00
8835664653 tape: add tape backup api 2020-12-18 15:32:12 +01:00
d37da6b7fc tape: add PoolWriter 2020-12-18 15:27:44 +01:00
b9ee86efe1 tape: use SnapshotReader to create snapshot archive 2020-12-18 12:11:29 +01:00
d108b610fd tape: fix write_media_set_label - move to correct position 2020-12-18 12:11:29 +01:00
0ec79339f7 tools/daemon: improve reload behaviour
it seems that sometimes, the child process signal gets handled
before the parent process signal. Systemd then ignores the
childs signal (finished reloading) and only after going into
reloading state because of the parent. this will never finish.

Instead, wait for the state to change to 'reloading' after sending
that signal in the parent, an only fork afterwards. This way
we ensure that systemd knows about the reloading before actually trying
to do it.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-By: Fabian Ebner <f.ebner@proxmox.com>
2020-12-18 10:30:37 +01:00
2afdc7f27d tape: MediaPool::with_config() - remove name parameter
Not required, because config already contains the pool name.
2020-12-18 08:14:24 +01:00
26aa9aca40 tape: return current_file_number as u64 2020-12-18 07:44:50 +01:00
3e2984bcb9 tools/process_locker: Decrement writer count in drop handler
of ProcessLockSharedGuard.

We use a counter to determine if we can unlock the file again, but
we never actually decremented the writer count, so we held the
lock forever.

This fixes the issue that we could not start a garbage collect after
a reload, as long as the old process is still running, even when that
process has no active backup anymore but another long running task
(e.g. file download, terminal, etc.).

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-18 07:15:08 +01:00
a7a5406c32 acl: rustfmt module
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:07:01 +01:00
4f727a783e acl: reformat privileges
for better readability, and tell rustfmt to leave those definitions
alone.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:45 +01:00
23dc68fdea acl: add docs and adapt visibility
document all public things, add some doc links and make some
previously-public things only available for test cases or within the
crate:

previously public, now private:
- AclTreeNode::extract_user_roles (we have extract_roles())
- AclTreeNode::extract_group_roles (same)
- AclTreeNode::delete_group_role (exists on AclTree)
- AclTreeNode::delete_user_role (same)
- AclTreeNode::insert_group_role (same)
- AclTreeNode::insert_user_role (same)
- AclTree::write_config (we have save_config())
- AclTree::load (we have config()/cached_config())

previously public, now crate-internal:
- AclTree::from_raw (only used by tests)
- split_acl_path (used by some test binaries)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-18 07:05:11 +01:00
b532dd00c4 tape: add helper to read snapshot contents
- lock the snapshot for reading
- use openat to open files
- provides an iterator over all chunks
2020-12-17 13:07:52 +01:00
c01742855a KeyConfig: bail on wrong fingerprint
instead of just logging the error. this should never happen in practice
unless someone is messing with the keyfile, in which case, it's better
to abort.

update tests accordingly (wrong fingerprint should fail, no fingerprint
should get the expected one).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 11:27:06 +01:00
9c953dd260 tape: add code to write backup snapshot files (without chunks) to tape 2020-12-17 08:28:47 +01:00
3fbf2d2fcd tape: cleanup MediaCatalog 2020-12-17 08:05:53 +01:00
e0af222ec3 KeyConfig: always calculate fingerprint
and warn if stored and calculated fingerprint don't match.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:52:55 +01:00
73b5011786 KeyConfig: add encrypt/decrypt test
the RSA key and the encryption key itself are hard-coded to avoid
stalling the test runs because of lack of entropy, they have no special
significance otherwise.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:47:45 +01:00
2ea5abcd65 docs: replace openssl command with client
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:46:59 +01:00
7137630d43 client: add 'import-with-master-key' command
to import an encrypted encryption key using a master key.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:46:24 +01:00
8acfd15d6e key: move RSA-encryption to KeyConfig
since that is what gets encrypted, and not a CryptConfig.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:43:34 +01:00
48fbbfeb7e fix #3197: skip fingerprint check when restoring key
when restoring an encrypted key, the original one is obviously not
available to check the fingerprint with.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:37:54 +01:00
9990af3042 master key: store blob name in constant
since we will use it in more than one place.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-17 06:36:06 +01:00
fe6c19383b tape: remove MediaLabelInfo, use MediaId instead
The additional content_uuid was quite useless...
2020-12-16 13:31:32 +01:00
42150d263b update pxar dependency to 0.6.2
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-16 13:13:31 +01:00
9839d3f778 tape: improve docu 2020-12-16 12:43:51 +01:00
dd59e3c2a1 tape: improve docu 2020-12-16 12:23:52 +01:00
0b7432ae09 tape: add chunk archive reader/writer 2020-12-16 12:08:34 +01:00
c1c2c8f635 tape: cleanup MediaLocation type for direct use with API 2020-12-16 10:49:01 +01:00
7680525eec docs: prune-sim: folluwp: add missing semicolon
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-16 10:08:11 +01:00
42298d5896 tape: add magic number to identify media catalog files 2020-12-16 09:00:14 +01:00
39478aa52c prune sim: correctly keep track of already included backups
This needs to happen in a separate loop, because some time intervals are not
subsets of others, i.e. weeks and months. Previously, with a daily backup
schedule, having:
* a backup on Sun, 06 Dec 2020 kept by keep-daily
* a backup on Sun, 29 Nov 2020 kept by keep-weekly
would lead to the backup on Mon, 30 Nov 2020 to be selected for keep-monthly,
because the iteration did not yet reach the backup on Sun, 29 Nov 2020 that
would mark November as being covered.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 14:03:18 +01:00
6a99b930c4 followup: use arrow function for sorting
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-15 13:45:51 +01:00
f6ce45b373 prune sim: fix #3192: by fixing usage of sort()
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-12-15 13:45:51 +01:00
205e187613 tape: add MediaCatalog implementation 2020-12-15 13:40:49 +01:00
a78348acbb tape: rename DriveLabel to MediaLabel 2020-12-14 17:37:16 +01:00
410611b4f2 tape: improve file format docu 2020-12-14 17:29:57 +01:00
af07ec8f29 tape: minor code cleanup 2020-12-14 16:56:26 +01:00
3f803af00b tape: scan - print more debug info 2020-12-14 13:16:18 +01:00
ac461bd651 tape: implement scan command (useful for debug) 2020-12-14 12:55:49 +01:00
ce955e1635 tape: implement eod cli command (debug tool) 2020-12-14 09:56:59 +01:00
e20d008c6a tape: rename cli 'media media-destroy' toö 'media destroy' 2020-12-14 09:30:32 +01:00
fb657d8ee5 tape: implement destroy_media 2020-12-14 08:58:40 +01:00
fba0b77469 tape: add media api 2020-12-14 07:55:57 +01:00
b5c1296eaa tape: make changer get_status async 2020-12-14 07:14:24 +01:00
065df12872 tape: split api type definitions for changers into extra file 2020-12-13 09:31:02 +01:00
7e1d4712b8 tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA 2020-12-13 09:22:08 +01:00
49c965a497 tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA 2020-12-13 09:18:16 +01:00
6fe9aedd0b tape: correctly call Async handler in proxmox-tape 2020-12-12 09:58:47 +01:00
42cb9bd6a5 tape: avoid executor blocking in changer api 2020-12-12 09:45:08 +01:00
66dbe5639e tape: avoid executor blocking in drive API
By using tokio::task::spawn_blocking().
2020-12-12 09:20:04 +01:00
2d87f2fb73 bump version to 1.0.6-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-11 14:19:28 +01:00
4c81273274 debian: just install whole images directory
fixes build for recently added tape icon (and includes it for real)

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-12-11 14:19:28 +01:00
73b8f6793e tape: add svg icon 2020-12-11 13:02:23 +01:00
663ef85992 tape: use WorkerTask for erase and rewind 2020-12-11 11:19:33 +01:00
e92c75815b tape: split inventory api
inventory: sync, list labels with uuids,
update_inventory: WorkerTask, updates database
2020-12-11 10:42:29 +01:00
6dbad5b4b5 tape: run label commands as WorkerTask (threads) 2020-12-11 09:10:22 +01:00
bff7e3f3e4 tape: implement barcode-label-mdedia 2020-12-11 07:50:19 +01:00
83abc7497d tape: implement inventory command 2020-12-11 07:39:28 +01:00
8bc5eebeb8 depend on package mt-st
We do not use the mt utility directly, but the package also provides
an udev helper to correctly initialize tape drives (stinit). Also,
the mt utility is helpful for debugging tap issues.
2020-12-11 06:38:45 +01:00
1433b96ba0 control.in: fix indentation
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-12-11 06:31:30 +01:00
be1a8c94ae fix build: add missing file 2020-12-10 13:40:20 +01:00
4606f34353 tape: implement read-label command 2020-12-10 13:20:39 +01:00
7bb720cb4d tape: implement label command 2020-12-10 12:30:27 +01:00
c4d8542ec1 tape: add media pool handling 2020-12-10 11:41:35 +01:00
9700d5374a tape: add media pool cli 2020-12-10 11:13:12 +01:00
05e90d6463 tape: add media pool config api 2020-12-10 10:52:27 +01:00
55118ca18e tape: correctly sort drive api subdir 2020-12-10 10:09:12 +01:00
f70d8091d3 tape: implement option changer-drive-id 2020-12-10 09:09:06 +01:00
a3c709ef21 tape: cli cleanup - avoid api redefinition 2020-12-10 08:35:11 +01:00
4917f1e2d4 tape: implement delete property for drive update command 2020-12-10 08:25:46 +01:00
93829fc680 tape: cleanup load-slot api 2020-12-10 08:04:55 +01:00
5605ca5619 tape: cli cleanup - rename scana-for-* into scan 2020-12-10 07:58:45 +01:00
e49f0c03d9 tape: implement load-media command 2020-12-10 07:52:56 +01:00
0098b712a5 tape: implement eject 2020-12-09 17:50:48 +01:00
5fb694e8c0 tape: implement rewind 2020-12-09 17:43:38 +01:00
583a68a446 tape: implement erase media 2020-12-09 17:35:31 +01:00
e6604cf391 tape: add command line interface proxmox-tape 2020-12-09 13:00:20 +01:00
43cfb3c35a tape: do not remove changer while still used 2020-12-09 12:55:54 +01:00
8a16c571d2 tape: add changer property to drive create api 2020-12-09 12:55:10 +01:00
314652a499 tape: set protected flag for configuration change api methods 2020-12-09 12:02:55 +01:00
6b68e5d597 client: move connect_to_localhost into client module 2020-12-09 11:59:50 +01:00
cafd51bf42 tape: add media state database 2020-12-09 11:21:56 +01:00
eaff09f483 update control file 2020-12-09 11:21:56 +01:00
9b93c62044 remove unused descriptions from api macros
these are now a hard error in the api macro

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-12-09 10:55:18 +01:00
5d90860688 tape: expose basic tape/changer functionality at api2/tape/ 2020-12-08 15:42:50 +01:00
5ba83ed099 tape: check digest on config update 2020-12-08 11:24:38 +01:00
50bf10ad56 tape: add changer configuration API 2020-12-08 09:04:56 +01:00
16d444c979 tape: add tape drive configuration API 2020-12-07 13:04:32 +01:00
fa9c9be737 tape: add tape device driver 2020-12-07 08:29:22 +01:00
2e7014e31d tape: add BlockeReader/BlockedWriter streams
This is the basic format used to write data to tapes.
2020-12-06 12:09:55 +01:00
a84050c1f0 tape: add BlockHeader impl 2020-12-06 10:26:24 +01:00
7c9835465e tape: add helpers to emulate tape read/write behavior 2020-12-06 09:41:16 +01:00
ec00200411 fix bug #3189: fix change_password permission checks, run protected 2020-12-05 16:20:29 +01:00
956e5fec1f depend on mtx (tape changer control)
A very small package with no additional dependencies.
2020-12-05 14:54:12 +01:00
b107fdb99a tape: add tape changer support using 'mtx' command 2020-12-05 14:54:12 +01:00
7320e9ff4b tape: add media invenotry 2020-12-05 12:54:15 +01:00
c4d2d54a6d tape: define useful constants 2020-12-05 12:20:46 +01:00
1142350e8d tape: add media pool config 2020-12-05 11:59:38 +01:00
d735b31345 tape: add tape read trait 2020-12-05 10:54:38 +01:00
e211fee562 tape: add tape write trait 2020-12-05 10:51:34 +01:00
8c15560b68 tape: add file format definitions 2020-12-05 10:45:08 +01:00
327e93711f commit missing file: tape api type definitions 2020-12-04 16:00:52 +01:00
a076571470 tape support: add drive configuration 2020-12-04 15:42:32 +01:00
ff50c07ebf start experimental tape management GUI
You need to set the environment TEST_TAPE_GUI=1 to enable this.
The current GUI is only a placeholder.
2020-12-04 12:50:08 +01:00
179145dc24 backup/datastore: move manifest locking to /run
this fixes the issue that on some filesystems, you cannot recursively
remove a directory when you hold a lock on a file inside (e.g. nfs/cifs)

it is not really backwards compatible (so during an upgrade, there
could be two daemons have the lock), but since the locking was
broken before (see previous patch) it should not really matter
(also it seems very unlikely that someone will trigger this)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-03 09:56:42 +01:00
6bd0a00c46 backup/datastore: really lock manifest on delete
'lock_manifest' returns a Result<File, Error> so we always got the result,
even when we did not get the lock, but we acted like we had.

bubble the locking error up

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 14:37:05 +01:00
f6e28f4e62 client/pull: log how many groups to pull were found
if no groups were found, the task log was very confusing as it
contained no real information why nothing was synced, e.g.:

 Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
 Sync datastore 'local-datastore' from 'remote/datastore'
 sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
 TASK OK

this patch simply logs how many groups were found and are about to be synced:

 Starting datastore sync job 'remote:datastore:local-datastore:s-79412799-e6ee'
 Sync datastore 'local-datastore' from 'remote/datastore'
 found 0 groups to sync
 sync job 'remote:datastore:local-datastore:s-79412799-e6ee' end
 TASK OK

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-12-02 07:22:50 +01:00
37f1b7dd8d docs: add more thoughts about chunk size 2020-12-01 10:28:06 +01:00
60e6ee46de doc: add some thoughts about large chunk sizes 2020-12-01 08:47:15 +01:00
2260f065d4 cleanup: use extra file for StoreProgress 2020-12-01 06:34:33 +01:00
6eff8dec4f cleanup: remove unnecessary StoreProgress clone() 2020-12-01 06:29:11 +01:00
7e25b9aaaa verify: use same progress as pull
percentage of verified groups, interpolating based on snapshot count
within the group. in most cases, this will also be closer to 'real'
progress since added snapshots (those which will be verified) in active
backup groups will be roughly evenly distributed, while number of total
snapshots per group will be heavily skewed towards those groups which
have existed the longest, even though most of those old snapshots will
only be re-verified very infrequently.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:55 +01:00
f867ef9c4a progress: add format variants
for iterating over a single group, or iterating just on the group level

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:22:12 +01:00
fc8920e35d pull: factor out interpolated progress
and add group/snapshot count info.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:13:11 +01:00
7f3b0f67e7 remove BackupGroup::list_groups
BackupInfo::list_backup_groups is identical code-wise, and makes more
sense as entry point for listing groups.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:09:44 +01:00
844660036b gc: don't limit index listing to same filesystem
WalkDir does not follow symlinks by default anyway, and this behaviour
is not documented anywhere. e.g., if a sysadmin mounts 'extra storage'
for some backup group or type (not knowing that only metadata is stored
in those directories), GC will ignore all the indices contained within
and happily garbage collect their chunks..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:07:09 +01:00
efcac39d34 gc: remove duplicate variable
list_images already returns absolute paths, we don't need to prepend
anything.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:51 +01:00
cb4b721cb0 gc: log index files found outside of expected scheme
for safety reason, GC finds and marks all index files below the
datastore base path. as a result of regular operations, only index files
within the expected scheme of <TYPE>/<ID>/<TIMESTAMP> should exist.

add a small check + warning if the index list contains index files out
side of this expected scheme, so that an admin with shell access can
investigate.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:06:17 +01:00
7956877f14 gc: shorten progress messages
we have messages starting the phases anyway, and limit the number of
progress updates so that context remains available at all times.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-12-01 06:04:13 +01:00
2241c6795f d/control bump
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:28:02 +01:00
43e60ceb41 file logger: remove test.log after test as well
and a doc formatting fixup

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:13:21 +01:00
b760d8a23f derive PartialEq for Userid
the manual implementation is equivalent

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:10:17 +01:00
2c1592263d tiny clippy hint
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 14:03:43 +01:00
616533823c don't enforce Vec and String in tools::join
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:56:59 +01:00
913dddea85 minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:56:21 +01:00
3530430365 tools avoid unnecessary copying of parameters/properties
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:53:49 +01:00
a4ba60be8f minor cleanups
whitespace, formatting and superfluous lifetime annotations

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 13:47:31 +01:00
99e98f605c network helpers: fix fd leak in get_network_interfaces
This one always leaked.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
935ee97b17 use fd_change_cloexec helper
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
6b9bfd7fe9 minor cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
dd519bbad1 pxar: stricter file descriptor guards
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
35fe981c7d client: use tools::pipe instead of nix
nix::unistd::pipe returns unguarded RawFds which should be
avoided

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
b6570abe79 changes for proxmox 0.8
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
54813c650e bump proxmox dep to 0.8.0
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
781106f8c5 ui: fix usage of findRecord
findRecord does not match exactly, but only at the beginning and
case insensitive, by default. Change all calls to be case sensitive
and an exactmatch (we never want the default behaviour afaics).

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-27 07:20:32 +01:00
96f35520a0 bump version to 1.0.5-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 15:30:06 +01:00
490560e0c6 restore: print to STDERR
else restoring to STDOUT is broken..

Reported-by: Dominic Jäger <d.jaeger@proxmox.com>

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 14:38:02 +01:00
52f53d8280 control: update versions 2020-11-25 10:35:51 +01:00
27b8a3f671 bump version to 1.0.4-1 2020-11-25 08:03:11 +01:00
abf9b6da42 docs: fix renamed commands 2020-11-25 08:03:11 +01:00
0c9209b04c cli: rename command "upload-log" to "snapshot upload-log" 2020-11-25 07:57:39 +01:00
edebd52374 cli: rename command "forget" to "snapshot forget" 2020-11-25 07:57:39 +01:00
61205f00fb cli: rename command "files" to "snapshot files" 2020-11-25 07:57:39 +01:00
a303e00289 fingerprint: add new() method 2020-11-25 07:57:39 +01:00
af9f72e9d8 fingerprint: add bytes() accessor
needed for libproxmox-backup-qemu0

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-25 06:34:34 +01:00
5176346b30 ui: fix broken gettext use
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-25 00:21:17 +01:00
731eeef25b cli: use new alias feature for "snapshots"
Now maps to "snapshot list".
2020-11-24 13:26:43 +01:00
a65e3e4bc0 client: add 'snapshot notes show/update' command
to show and update snapshot notes from the cli

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-24 11:44:19 +01:00
027eb2bbe6 bump version to 1.0.3-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 08:56:18 +01:00
6982a54701 gui: add snapshot/file fingerprint tooltip
display short key ID, like backend's Display trait.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
035c40e638 list_snapshots: return manifest fingerprint
for display in clients.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
79c535955d refactor BackupInfo -> SnapshotListItem helper
before adding more fields to the tuple, let's just create the struct
inside the match arms to improve readability.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
8b7f8d3f3d expose previous backup time in backup env
and use this information to add more information to client backup log
and guide the download manifest decision.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:44:55 +01:00
866c859a1e bump version to 1.0.2-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-24 08:33:20 +01:00
23e4e90540 verification: fix message in notification mail
the errors Vec can contain failed groups as well (e.g., if a group has
no or an invalid owner).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:33:20 +01:00
a4fa3fc241 verification job: log failed dirs
else users have to manually search through a potentially very long task
log to find the entries that are different.. this is the same summary
printed at the end of a manual verify task.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 08:33:20 +01:00
81d10c3b37 cleanup: remove dead code 2020-11-24 08:03:00 +01:00
f1e2904150 paperkey: refactor common code
from formatting functions to main function, and pass along the key data
lines instead of the full string.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 07:57:21 +01:00
23f9503a31 client: check fingerprint after downloading manifest
this is stricter than the check that happened on manifest load, as it
also fails if the manifest is signed but we don't have a key available.

add some additional output at the start of a backup to indicate whether
a previous manifest is available to base the backup on.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 07:55:12 +01:00
a0ef68b93c manifest: check fingerprint when loading with key
otherwise loading will run into the signature mismatch which is
technically true, but not the complete picture in this case.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 07:49:51 +01:00
6b127e6ea0 fix #3139: add key fingerprint to manifest
if the manifest is signed/the contained archives/blobs are encrypted.
stored in 'unprotected' area, since there is already a strong binding
between key and manifest via the signature, and this avoids breaking
backwards compatibility for a simple usability improvement.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-24 07:45:11 +01:00
5e17dbf2bb cli: cleanup 'key show' - use format_and_print_result_full
We now expose all key derivation functions on the cli, so users can
choose between scrypt or pbkdf2.
2020-11-24 07:32:34 +01:00
dfb04575ad client: add 'key show' command
for (pretty-)printing a keyfile.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-23 13:15:29 +01:00
6f2626ae19 client: print key fingerprint and master key
for operations where it makes sense.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-23 13:11:26 +01:00
37e60ddcde key: add fingerprint to key config
and set/generate it on
- key creation
- key passphrase change
- key decryption if not already set
- key encryption with master key

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-23 13:03:46 +01:00
05cdc05347 crypt config: add fingerprint mechanism
by computing the ID digest of a hash of a static string.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-23 13:03:16 +01:00
6364115b4b OnlineHelpInfo.js problems
Anybody known why I always get the following diff:
2020-11-23 12:57:41 +01:00
2133cd9103 update debian/control 2020-11-23 12:13:58 +01:00
01f84fcce1 ui: datastore content: use our keep field for group pruning
sets some defaults and provides the clear trigger, so less code and
slightly nicer UX.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-21 19:52:03 +01:00
08b3823025 bump dependency on proxmox to 0.7.1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-20 17:38:34 +01:00
968a0ab261 fix systemd-encoded upid strings in http client
since we systemd-encode parts of the upid string, and those can contain
characters that are invalid in urls (e.g. '\'), we have to percent encode
those

add a 'percent_encode_component' helper, so that we can maybe change
the AsciiSet for all uses at the same time

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-19 11:01:19 +01:00
21b552848a prune sim: make numberfields more similar to PBS's
by creating a new class that adds a clear trigger and also uses the
clear-trigger image. Code was taken from the one in PBS's prune window,
but we have default values here, so a bit of adapting was necessary. For
example, we don't want to reset to the original value (which might have
been one of the defaults) when clearing, but always to 'null'.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-19 09:47:51 +01:00
fd19256470 gc: treat .bad files like regular chunks
Simplify the phase 2 code by treating .bad files just like regular
chunks, with the exception of stat logging.

To facilitate, we need to touch .bad files in phase 1. We only do this
under the condition that 1) the original chunk is missing (as before),
and 2) the original chunk is still referenced somewhere (since the code
lives in the error handler for a failed chunk touch, it only gets called
for chunks we expect to be there, i.e. ones that are referenced).

Untouched they will then be cleaned up after 24 hours (or after the last
longer-running task finishes).

Reason 2) is also a fix for .bad files not being cleaned up at all if
the original is no longer referenced anywhere (e.g. a user deleting all
snapshots after seeing some corrupt chunks appear).

cond_touch_path is introduced to touch arbitrary paths in the chunk
store with the same logic as touching chunks.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-18 14:04:49 +01:00
1ed022576c api: include store in invalid owner errors
since a group might exist in plenty stores

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-18 11:11:24 +01:00
f6aa7b38bf drop now unused BackupInfo::list_backups
all global backup listing now happens via BackupGroup

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-18 11:11:21 +01:00
fdfcb74d67 api: filter snapshot counts
unprivileged users should only see the counts related to their part of
the datastore.

while we're at it, switch to a list groups, filter groups, count
snapshots approach (like list_snapshots) to speedup calls to this
endpoint when many unprivileged users share a datastore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-18 11:05:50 +01:00
98afc7b152 api: make expensive parts of datastore status opt-in
used in the PBS GUI, but also for PVE usage queries which don't need all
the extra expensive information..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-18 11:05:47 +01:00
0d08fceeb9 improve group/snapshot listing
by listing groups first, then filtering, then listing group snapshots.

this cuts down the number of openat/getdirents calls for users that just
have a partial view of the datastore.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-18 10:37:04 +01:00
3c945d73c2 client/http_client: add put method
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-16 16:59:14 +01:00
58fcbf5ab7 client: expose all-file-systems option
Useful to avoid the need for a long (and possibly changing) list of include-dev
options in certain situations, e.g. nested ZFS file systems. The option is
already implemented and seems to work as expected. The checks for virtual
filesystems are not affected by this option.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-16 16:59:14 +01:00
3a3f31c947 ui: datastores: hide "no datastore" box by default
avoids that it shows during store load, we do not know if there are
no datastores at that point and have already a loading mask.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-16 16:59:14 +01:00
8fc63287df ui: improve comment behaviour for datastore Summary
when we could not load the config (e.g. missing permissions)
show the comment from the global datastore-list

also show a messagebox for a load error instead of setting
the text of the comment box

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-16 10:39:34 +01:00
172473e4de ui: DataStoreList: show message when there are no datastores
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-16 10:39:34 +01:00
76f549debb ui: DataStoreList: remove datastores also from hash
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-16 10:39:34 +01:00
c9097ff801 pxar: avoid including archive root's exclude patterns in .pxarexclude-cli
The patterns from the archive root's .pxarexclude file are already present in
self.patterns when encode_pxarexclude_cli is called. Pass along the number of
CLI patterns and slice accordingly.

Suggested-By: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 13:05:09 +01:00
fb01fd3af6 visibility cleanups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:53:50 +01:00
fa4bcbcad0 pxar: only generate .pxarexclude-cli if there were CLI parameters
previously a .pxarexclude entry in the root of the archive caused the file to
be generated as well, because the patterns are read before calling
generate_directory_file_list and within the function it wasn't possible to
distinguish between a pattern coming from the CLI and a pattern coming from
archive/root/.pxarexclude

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:18:08 +01:00
189cdb7427 pxar: include .pxarexclude files in the archive
The documentation states:
.pxarexclude files are treated as regular files and will be included in the
backup archive.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:18:06 +01:00
874bd5454d pxar: fix anchored exclusion at archive root
There is no leading slash in an entry's full_path, causing an anchored
exclude at the root level to fail, e.g. having "/name" as the content of the
file archive/root/.pxarexclude didn't match the file archive/root/name

Fix this by prepending a leading slash before matching.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:18:04 +01:00
b649887e9a remove unused function
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:15:15 +01:00
8c62c15f56 follouwp: whitespace cleanup
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 11:02:45 +01:00
51ac17b56e api: apt/versions: fix running_kernel string for unknown package case
Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-11-12 11:02:20 +01:00
fc5a012068 manager: versions: non-verbose should actually print server pkg info
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 10:28:03 +01:00
5e293f1315 apt: use typed response for get_versions
...and cleanup get_versions for manager CLI.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-12 10:15:32 +01:00
87367decf2 ui: tell ESLint to be strict in check target
the .lint-incremental target, which is implicitly used by the install
target, is still more forgiving to allow faster "change, build, test"
iteration when developing.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 09:54:39 +01:00
f792220dd4 d/control: update for new pin-project dependency
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-12 09:54:39 +01:00
97030c9407 cleanup clippy leftovers
this used to contain a pointer cast, now it doesn't

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 09:43:38 +01:00
5d1d0f5d6c use pin-project to remove more unsafe blocks
we already have it in our dependency tree, so use it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 09:43:38 +01:00
294466ee61 manager: versions: unify printing
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 18:30:33 +01:00
c100fe9108 add versions command to proxmox-backup-manager
Add the versions command to proxmox-backup-manager with a similar output
to pveversion [-v]. It prints the packages line by line with only the
package name, followed by the version and, for proxmox-backup and
proxmox-backup-server, some additional information (running kernel,
running version).

In addition it supports the optional output-format parameter which can
be used to print the complete data in either json, json-pretty or text
format. If output-format is specified, the --verbose parameter is
ignored and the detailed list of packages is printed.

With the addition of the versions command, the report is extended as
well.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-11-11 18:30:33 +01:00
e754da3ac2 api: versions: add version also in server package unknown case
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 18:30:33 +01:00
bc1e52bc38 api: versions: rust fmt cleanups
line length limit is 100

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 18:30:33 +01:00
6f0073bbb5 api: apt update info: do not serialize extra info if none
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 18:30:33 +01:00
2decf85d6e add extra_info field to APTUpdateInfo
Add an optional string field to APTUpdateInfo which can be used for
extra information.

This is used for passing running kernel and running version information
in the versions API call together with proxmox-backup and
proxmox-backup-server.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
2020-11-11 16:39:11 +01:00
1d8f849457 api2/node/syslog: use 'real_service_name' here also
for now this only does the 'postfix' -> 'postfix@-' conversion,
fixes the issue that we only showed the 'postfix' service syslog
(which is rather empty in a default setup) instead of the instance one

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-11 16:36:42 +01:00
beb07279b6 log source of encryption key
This patch prints the source of the encryption key when running
operations with proxmox-backup-client.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-11-11 16:35:20 +01:00
8c6854c8fd inform user when using default encryption key
Currently if you generate a default encryption key:
`proxmox-backup-client key create --kdf none`

all backup operations which don't explicitly disable encryption will be
encrypted with this key.

I found it quite surprising, that my backups were all encrypted without
me explicitly specfying neither key nor encryption mode

This patch informs the user when the default key is used (and no
crypt-mode is provided explicitly)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
2020-11-11 16:35:20 +01:00
57f472d9bb report: use '$' instead of '#' for showing commands
since some files can contain '#' character for comments. (i.e.,
/etc/hosts)

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-11-11 16:19:37 +01:00
94ffca10a2 report: fix grammar error
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-11-11 16:19:33 +01:00
0a274ab0a0 ui: UserView: render name as 'Firstname Lastname'
instead of only the firstname

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-11 14:09:40 +01:00
c0026563b0 make user properties deletable
by using our usual pattern for the update call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-11 14:09:40 +01:00
e411924c7c rest: check for disabled token (user)
when authenticating a token, and not just when authenticating a
user/ticket.

Reported-By: Dominik Jäger <d.jaeger@proxmox.com>

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-11 12:21:29 +01:00
709c15abaa bump version to 1.0.1-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 10:21:30 +01:00
b404e4d930 d/control: check in new dependnecies to generated control
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 10:21:30 +01:00
f507580c3f docs: faq: fix first releases
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 10:14:01 +01:00
291b786076 docs: fix prune retention example
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 10:14:01 +01:00
06c9059dac daemon: rename method, endless loop, bail on exec error
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 10:14:01 +01:00
d7c6ad60dd daemon: add hack for sd_notify
sd_notify is not synchronous, iow. it only waits until the message
reaches the queue not until it is processed by systemd

when the process that sent such a message exits before systemd could
process it, it cannot be associated to the correct pid

so in case of reloading, we send a message with 'MAINPID=<newpid>'
to signal that it will change. if now the old process exits before
systemd knows this, it will not accept the 'READY=1' message from the
child, since it rejects the MAINPID change

since there is no (AFAICS) library interface to check the unit status,
we use 'systemctl is-active <SERVICE_NAME>' to check the state until
it is not 'reloading' anymore.

on newer systemd versions, there is 'sd_notify_barrier' which would
allow us to wait for systemd to have all messages from the current
pid to be processed before acknowledging to the child, but on buster
the systemd version is to old...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-11 09:43:00 +01:00
0a0ba0785b prune sim: avoid colon to separate keep desc from count
hack for space issues for monthly keeps and >9 counts

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 08:20:13 +01:00
6ed79592f2 prune sim: make backup schedule a form, bind update button to its validity
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 08:11:46 +01:00
4c75ee3471 prune sim: do not use unecesarry variable, declare in line
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 08:11:16 +01:00
6f997da8cd prune sim: set min-heigth for calendar day cells
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 08:10:43 +01:00
03e40aa4ee ui: datastore add: set default schedule
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 07:49:01 +01:00
be1d6cbcc6 ui: shorten automatic ID length a bit
Without hyphens, we had 20 hex digits, so ~80 bit which is probably overkill.
Use 12 (13 with hyphen), this is still 48 bit.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 07:40:23 +01:00
ffaca016ad ui: datastore summary: drop removed bytes display
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 07:27:21 +01:00
71f82a98d7 d/control: add missing dependencies for non ISO installations
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-11 07:26:05 +01:00
deef6fbc0c cargo: extend authors list
this was mostly selected by executing

and adding those with more than a hand full of commits, so no hard
feelings here, this was definitively also a team effort to get stuff
polished!

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 14:47:48 +01:00
4ac529141f bump version to 1.0.0-1
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 14:47:48 +01:00
a108a2e967 ui: drop debug beta code
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 14:47:48 +01:00
ff7a29104c postinst: fix version check for remote.cfg cleanup
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-10 14:35:37 +01:00
240b2ffb9b ui: improve activeTab selection from fragment and state
handle invalid fragments for tabs, as well as not rendered tabpanels

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-10 14:21:54 +01:00
a86e703661 tools::runtime: pin_mut instead of unsafe block
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-10 14:18:45 +01:00
1ecf4e6d20 async_io: require Unpin for EitherStream and HyperAccept
We use it with Unpin types and this way we get rid of a lot
of `unsafe` blocks.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-10 14:18:45 +01:00
9f9a661b1a verify: cleanup logging order/messages
otherwise we end up printing warnings before the start message..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-10 14:11:36 +01:00
1b1cab8321 verify: log/warn on invalid owner
in order to trigger a notification/make the problem more visible than
just in syslog.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-10 14:11:36 +01:00
f4f9a503de ui: add mising panel help buttons
add missing help buttons (question mark, top right) so that we are
consistent and each panel has it.

I chose the IMHO most fitting sections.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 13:53:21 +01:00
a4971d5f90 docs: add ref for sysadmin host admin section
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 13:53:21 +01:00
477ebe6b78 docs: user management: avoid some inconsistencies
The space between '--' and 'path' in two of the commands was wrong. The other
changes make the names of the store and token consistent with the rest of the
section and should improve readability.

Also add the Datastore.Verify permission in the output of the command:
proxmox-backup-manager user permissions john@pbs --path /datastore/store1
A DatastoreAdmin now has this permission and that's what john@pbs is in the
example.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 13:47:52 +01:00
38efbfc148 ui: app: fix fixme
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 13:38:30 +01:00
10052ea644 remote.cfg: rename userid to 'auth-id'
and fixup config file on upgrades accordingly

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-10 13:25:24 +01:00
b57619ea29 ui: datastores sync: future proof and move local store column in front
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 13:22:54 +01:00
445b0043b2 ui: show (local)datastore column only in global sync/verifyview
its rather hacky, but our cbind mixin does not support columns (yet).
if it does sometime in the future, we could use that instead

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-10 13:14:47 +01:00
8b62cbe752 docs: update package repositories
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 13:14:04 +01:00
81f99362d9 docs: installation: don't mention ext3 as an option anymore
Support for ext3 was removed by commit 0abf0d3683b74421eca24ba61d1d4e100d35211a
in pve-installer.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 13:13:44 +01:00
414c23facb fix #3060:: improve get_owner error handling
log invalid owners to system log, and continue with next group just as
if permission checks fail for the following operations:
- verify store with limited permissions
- list store groups
- list store snapshots

all other call sites either handle it correctly already (sync/pull), or
operate on a single group/snapshot and can bubble up the error.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-10 12:58:44 +01:00
c5608cf86c encryption: add best practice for storing master key
Further clarify that the paperkey should be a last resort
recovery option, after a password manager and usb drive.

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-11-10 12:51:30 +01:00
5d08c750ef HttpsConnector: include destination on connect errors
for more useful log output
old:
Nov 10 11:50:51 foo pvestatd[3378]: proxmox-backup-client failed: Error: error trying to connect: tcp connect error: No route to host (os error 113)
new:
Nov 10 11:55:21 foo pvestatd[3378]: proxmox-backup-client failed: Error: error trying to connect: error connecting to https://thebackuphost:8007/ - tcp connect error: No route to host (os error 113)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-10 11:58:19 +01:00
f3fde36beb client: error context when building HttpClient
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-10 11:58:19 +01:00
0c83e8891e ui: fix task description 2020-11-10 11:53:39 +01:00
133de2dd1f ui: add/fix help buttons
added a few more help buttons were appropriate:

* GC and Prune schedule windows
* Create Directory window
* API Tokens, link directly to token section
* verify jobs window

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 11:51:03 +01:00
c8219747f0 ui: add all online help refs found in docs
recommit the onlinehelp after the scanrefs script has been adapted and
the docs are up to date

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 11:50:56 +01:00
0247f794e9 docs: add network management reference
needed in order for the help button in the network edit window to work.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 11:50:17 +01:00
710f787c41 docs: add maintenance chapter prefix to verification ref
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 11:50:12 +01:00
d8916a326c scanrefs: only scan docs, not JS files
This is a temporary hack until we find a sensible way to scan the
proxmox-widget-toolkit JS files as well.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-10 11:50:09 +01:00
924d6d4072 prune sim: show count for rule
and rename 'all zero' to 'keep-all' to make it consistent with the prune dialog
in PBS.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 11:47:37 +01:00
984ac33d5c ui: subscription: usage chart: render date as ISO 8601
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 11:46:22 +01:00
0a4dfd63c9 ui: usage graph: show axis and set maximum
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 11:46:05 +01:00
a6e746f652 ui: datastore list summary: add more padding between elements
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 11:46:05 +01:00
30f73fa2e0 fix bug #3060: continue sync if we cannot aquire the group lock 2020-11-10 11:29:36 +01:00
9f0ee346e9 ui: Datastores Summary: change layout and chart
changes the layout to look i little bit more like the statistics panel
we have for ceph in pve, while changing to the UsageChart and adding
some more datastore infos (from last garbage collect)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-10 10:43:07 +01:00
48d6dede4a ui: refactor calculate_dedup_factor
so that we can reuse this

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-10 10:43:07 +01:00
8432e4a655 ui: add panel/UsageChart
heavily inspired by pveRunningChart, without the dynamically adding
of data and specific for the usage of datastores

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-10 10:43:07 +01:00
b35eb0a175 api2/status/datastore-usage: add gc-status and history start and delta
so that we can show more info and calculate the points in time for the
history

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-10 10:43:07 +01:00
c3a1b34ed3 ui: subscription: add more button icons, small UX fix
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 10:42:45 +01:00
bb26843cd6 ui/docs: add get help onlineHelp
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 10:35:35 +01:00
ee0ab12dd0 ui: move disks/directory stuff to tab panel
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 10:15:44 +01:00
d5f7755467 docs: online help scanner: also include help tool links
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 10:15:08 +01:00
5c64e83b1e ui: datastore: set onlineHelp for chaging group owner
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 09:53:05 +01:00
0f6f99b4ec ui: prune: set onlineHelp
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 09:51:30 +01:00
f668862ae0 ui: prune: add clear-trigger to keep fields
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 09:51:20 +01:00
c960d2b501 bail if mount point already exists for directories
similar to what we do for zfs. By bailing before partitioning, the disk is
still considered unused after a failed attempt.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 09:25:58 +01:00
f5d9f2534b mount zpools created via API under /mnt/datastore
as we do for other file systems

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 09:25:58 +01:00
9a3ddcea33 ui: utils: eslint format fixes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 09:24:35 +01:00
030464d3a9 docs: s/DataStore/Datastore/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 09:24:13 +01:00
3f30b32c2e ui: prune: show count for rule
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 09:24:13 +01:00
5eafe6aabc ui: prune: show which rule keeps backup
and adjust layout so the description fits.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
2020-11-10 09:24:13 +01:00
2c9f274efa ui: add help tool to user and remote config 2020-11-10 09:23:22 +01:00
31112c79ac ui: add help tool to datastore panel 2020-11-10 09:15:12 +01:00
d89f91b538 ui: acl editor: disallow path editing for datastore permission views
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 08:19:17 +01:00
a6310ec294 ui: fix widget height in dashboard 2020-11-10 08:12:35 +01:00
98d9323534 ui: add link to www.proxmox.com for subscription plans 2020-11-10 08:07:49 +01:00
09f1f28800 ui: ACL view: fix path filtering
and add some comments about actual behavior of those config
properties..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-10 07:33:20 +01:00
e1da9ca4bb ui: datastore dashboard: use gauge for usage, rework layout a bit
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 19:26:48 +01:00
625c7bfc0b ui: task summary: enable grid mouse track over
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 19:25:43 +01:00
d9503950e3 ui: tasl summary: add pointer cursor if clickable
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 18:09:05 +01:00
376e927980 ui: datastore summary: increase usage graph height
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 17:55:59 +01:00
5204cbcf0f ui: datastore summary: add line chart icon to full-estimation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 17:48:53 +01:00
e373dcc564 ui: datastore/content: improve action button layout
Fix font-size to 14px to improve font-awesome rendering, add some
slight margin between the buttons so that they are not glued
together, add a slight text-shadow on mouse over.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 17:45:08 +01:00
137a6ebcad apt: allow changelog retrieval from enterprise repo
If a package is or will be installed from the enterprise repo, retrieve
the changelog from there as well (securely via HTTPS and authenticated
with the subcription key).

Extends the get_string method to take additional headers, in this case
used for 'Authorization'. Hyper does not have built-in basic auth
support AFAICT but it's simple enough to just build the header manually.

Take the opportunity and also set the User-Agent sensibly for GET
requests, just like for POST.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2020-11-09 17:28:58 +01:00
ed1329ecf7 ui: make Datastore clickable again
by showing the previously added pbsDataStores panel

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
2371c1e371 ui: add Panels necessary for Datastores Overview
a panel for a single datastore that gets updated from an external caller
shows the usage, estimated full date, history and task summary grid

a panel that dynamically generates the panel above for each datastore

and a tabpanel that includes the panel above, as well as a global
syncview, verifiyview and aclview

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
63c07d950c ui: TaskSummary: handle less defined parameters of tasks
this makes it a little easier to provide good data, without
hardcoding all types in the source object

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
a3cdb19e33 ui: TaskSummary: add subPanelModal and datastore parameters
in preparation for the per-datastore grid

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
4623cd6497 ui: TaskSummary: move state/types/titles out of the controller
it seems that under certain circumstances, extjs does not initialize
or remove the content from objects in controllers

move it to the view, were they always exist

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
ab81bb13ad ui: make Sync/VerifyView and Edit usable without datastore
we want to use this panel again for a 'global' overview, without
any datastore preselected, so we have to handle that, and
adding a datastore selector in the editwindow

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
616650a198 ui: Utils: add parse_datastore_worker_id
to parse the datastore out of a worker_id
for this we need some regexes that are the same as in the backend

for now we only parse out the datastore, but we can extend this
in the future to parse relevant info (e.g. remote for syncs,
id/type for backups)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
78763d21b1 ui: refactor render_size_usage to Utils
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
f2d6324958 ui: refactor render_estimate
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
6e880f19cc api2/node/tasks: add check_job_store and use it
to easily check the store of a worker_id
this fixes the issue that one could not filter by type 'syncjob' and
datastore simultaneously

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-11-09 16:37:24 +01:00
64623f329e ui: recommit onlinehelp
now that the last commit fixed the title generation

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 16:36:00 +01:00
407f3fb994 scanrefs: remove term prefix from title
It can happen, that a title is defined as term in the following way:
:term:`My title`

This patch checks for it and strips the leading part and the last `.

Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
2020-11-09 16:35:29 +01:00
0eb0c4bd63 proxy: fix log message for auth log rotation
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 16:34:03 +01:00
82422c115a ui: admin/summary: add versions button/window
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 16:33:22 +01:00
ed2beb334d api: node/apt: add versions call
very basic, based on API/concepts of PVE one.

Still missing, addint an extra_info string option to APTUpdateInfo
and pass along running kernel/PBS version there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 16:31:56 +01:00
f3b4820d06 www: show more ACLs in datastore panel
since just the ACLs defined on the exact datastore path don't give
anywhere near a complete picture of who has access to it.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-11-09 15:19:15 +01:00
8f7cd96df4 installation: minor wording fix
very minor but worthwhile edits

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-11-09 15:18:44 +01:00
4accbc5853 backup-client: encryption: discuss paperkey command
adds a paragraph to the encryption section about
encoding the master key into a qr code for printing

Signed-off-by: Dylan Whyte <d.whyte@proxmox.com>
2020-11-09 15:18:44 +01:00
2791318ff1 fix bug #3121: forbid removing used reemotes 2020-11-09 12:48:29 +01:00
47208b4147 pxar: log when skipping mount points
Clippy complains about the number of paramters we have for
create_archive and it really does need to be made somewhat
less awkward and more usable. For now we just log to stderr
as we previously did. Added todo-comments for this.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-09 12:43:16 +01:00
b783591fb5 ui: datastore content: ensure action column is wide enough
with the "change owner" action added we now need more than the
default of 100 px, so increase to 120 px for now.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 12:31:14 +01:00
9dd6175808 ui: token selector: use same layout as auth id selector
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 12:24:54 +01:00
5e8b97178e ui: auth/token selector: tell ExtJS we injected data into the store
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 12:21:02 +01:00
38260cddf5 tools apt: include package name in filter data
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-11-09 08:55:08 +01:00
437 changed files with 45551 additions and 4952 deletions

10
.gitignore vendored
View File

@ -1,6 +1,16 @@
local.mak
/target
**/*.rs.bk
*~
*.backup
*.backup[0-9]
*.backup[0-9][0-9]
*.old
*.old[0-9]
*.old[0-9][0-9]
*.5
*.7
__pycache__/
/etc/proxmox-backup.service
/etc/proxmox-backup-proxy.service
build/

View File

@ -1,7 +1,16 @@
[package]
name = "proxmox-backup"
version = "0.9.7"
authors = ["Dietmar Maurer <dietmar@proxmox.com>"]
version = "1.0.11"
authors = [
"Dietmar Maurer <dietmar@proxmox.com>",
"Dominik Csapak <d.csapak@proxmox.com>",
"Christian Ebner <c.ebner@proxmox.com>",
"Fabian Grünbichler <f.gruenbichler@proxmox.com>",
"Stefan Reiter <s.reiter@proxmox.com>",
"Thomas Lamprecht <t.lamprecht@proxmox.com>",
"Wolfgang Bumiller <w.bumiller@proxmox.com>",
"Proxmox Support Team <support@proxmox.com>",
]
edition = "2018"
license = "AGPL-3"
description = "Proxmox Backup"
@ -14,22 +23,22 @@ name = "proxmox_backup"
path = "src/lib.rs"
[dependencies]
apt-pkg-native = "0.3.1" # custom patched version
apt-pkg-native = "0.3.2"
base64 = "0.12"
bitflags = "1.2.1"
bytes = "0.5"
bytes = "1.0"
crc32fast = "1"
endian_trait = { version = "0.6", features = ["arrays"] }
anyhow = "1.0"
futures = "0.3"
h2 = { version = "0.2", features = ["stream"] }
h2 = { version = "0.3", features = [ "stream" ] }
handlebars = "3.0"
http = "0.2"
hyper = "0.13.6"
hyper = { version = "0.14", features = [ "full" ] }
lazy_static = "1.4"
libc = "0.2"
log = "0.4"
nix = "0.19"
nix = "0.19.1"
num-traits = "0.2"
once_cell = "1.3.1"
openssl = "0.10"
@ -37,31 +46,34 @@ pam = "0.7"
pam-sys = "0.5"
percent-encoding = "2.1"
pin-utils = "0.1.0"
pin-project = "1.0"
pathpatterns = "0.1.2"
proxmox = { version = "0.7.0", features = [ "sortable-macro", "api-macro", "websocket" ] }
proxmox = { version = "0.11.0", features = [ "sortable-macro", "api-macro", "websocket" ] }
#proxmox = { git = "git://git.proxmox.com/git/proxmox", version = "0.1.2", features = [ "sortable-macro", "api-macro" ] }
#proxmox = { path = "../proxmox/proxmox", features = [ "sortable-macro", "api-macro", "websocket" ] }
proxmox-fuse = "0.1.0"
pxar = { version = "0.6.1", features = [ "tokio-io", "futures-io" ] }
#pxar = { path = "../pxar", features = [ "tokio-io", "futures-io" ] }
proxmox-fuse = "0.1.1"
pxar = { version = "0.10.0", features = [ "tokio-io" ] }
#pxar = { path = "../pxar", features = [ "tokio-io" ] }
regex = "1.2"
rustyline = "6"
rustyline = "7"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
siphasher = "0.3"
syslog = "4.0"
tokio = { version = "0.2.9", features = [ "blocking", "fs", "dns", "io-util", "macros", "process", "rt-threaded", "signal", "stream", "tcp", "time", "uds" ] }
tokio-openssl = "0.4.0"
tokio-util = { version = "0.3", features = [ "codec" ] }
tokio = { version = "1.0", features = [ "fs", "io-util", "macros", "net", "parking_lot", "process", "rt", "rt-multi-thread", "signal", "time" ] }
tokio-openssl = "0.6.1"
tokio-stream = "0.1.0"
tokio-util = { version = "0.6", features = [ "codec" ] }
tower-service = "0.3.0"
udev = ">= 0.3, <0.5"
url = "2.1"
#valgrind_request = { git = "https://github.com/edef1c/libvalgrind_request", version = "1.1.0", optional = true }
walkdir = "2"
webauthn-rs = "0.2.5"
xdg = "2.2"
zstd = { version = "0.4", features = [ "bindgen" ] }
nom = "5.1"
crossbeam-channel = "0.4"
crossbeam-channel = "0.5"
[features]
default = []

View File

@ -9,7 +9,10 @@ SUBDIRS := etc www docs
# Binaries usable by users
USR_BIN := \
proxmox-backup-client \
pxar
pxar \
proxmox-tape \
pmtx \
pmt
# Binaries usable by admins
USR_SBIN := \
@ -20,7 +23,7 @@ SERVICE_BIN := \
proxmox-backup-api \
proxmox-backup-banner \
proxmox-backup-proxy \
proxmox-daily-update \
proxmox-daily-update
ifeq ($(BUILD_MODE), release)
CARGO_BUILD_ARGS += --release
@ -141,6 +144,8 @@ install: $(COMPILED_BINS)
install -m755 $(COMPILEDIR)/$(i) $(DESTDIR)$(SBINDIR)/ ; \
install -m644 zsh-completions/_$(i) $(DESTDIR)$(ZSH_COMPL_DEST)/ ;)
install -dm755 $(DESTDIR)$(LIBEXECDIR)/proxmox-backup
# install sg-tape-cmd as setuid binary
install -m4755 -o root -g root $(COMPILEDIR)/sg-tape-cmd $(DESTDIR)$(LIBEXECDIR)/proxmox-backup/sg-tape-cmd
$(foreach i,$(SERVICE_BIN), \
install -m755 $(COMPILEDIR)/$(i) $(DESTDIR)$(LIBEXECDIR)/proxmox-backup/ ;)
$(MAKE) -C www install

View File

@ -53,3 +53,83 @@ Setup:
Note: 2. may be skipped if you already added the PVE or PBS package repository
You are now able to build using the Makefile or cargo itself.
Design Notes
============
Here are some random thought about the software design (unless I find a better place).
Large chunk sizes
-----------------
It is important to notice that large chunk sizes are crucial for
performance. We have a multi-user system, where different people can do
different operations on a datastore at the same time, and most operation
involves reading a series of chunks.
So what is the maximal theoretical speed we can get when reading a
series of chunks? Reading a chunk sequence need the following steps:
- seek to the first chunk start location
- read the chunk data
- seek to the first chunk start location
- read the chunk data
- ...
Lets use the following disk performance metrics:
:AST: Average Seek Time (second)
:MRS: Maximum sequential Read Speed (bytes/second)
:ACS: Average Chunk Size (bytes)
The maximum performance you can get is::
MAX(ACS) = ACS /(AST + ACS/MRS)
Please note that chunk data is likely to be sequential arranged on disk, but
this it is sort of a best case assumption.
For a typical rotational disk, we assume the following values::
AST: 10ms
MRS: 170MB/s
MAX(4MB) = 115.37 MB/s
MAX(1MB) = 61.85 MB/s;
MAX(64KB) = 6.02 MB/s;
MAX(4KB) = 0.39 MB/s;
MAX(1KB) = 0.10 MB/s;
Modern SSD are much faster, lets assume the following::
max IOPS: 20000 => AST = 0.00005
MRS: 500Mb/s
MAX(4MB) = 474 MB/s
MAX(1MB) = 465 MB/s;
MAX(64KB) = 354 MB/s;
MAX(4KB) = 67 MB/s;
MAX(1KB) = 18 MB/s;
Also, the average chunk directly relates to the number of chunks produced by
a backup::
CHUNK_COUNT = BACKUP_SIZE / ACS
Here are some staticics from my developer worstation::
Disk Usage: 65 GB
Directories: 58971
Files: 726314
Files < 64KB: 617541
As you see, there are really many small files. If we would do file
level deduplication, i.e. generate one chunk per file, we end up with
more than 700000 chunks.
Instead, our current algorithm only produce large chunks with an
average chunks size of 4MB. With above data, this produce about 15000
chunks (factor 50 less chunks).

View File

@ -35,3 +35,4 @@ Chores:
Suggestions
===========

238
debian/changelog vendored
View File

@ -1,3 +1,241 @@
rust-proxmox-backup (1.0.11-1) unstable; urgency=medium
* fix feature flag logic in pxar create
* tools/zip: add missing start_disk field for zip64 extension to improve
compatibility with some strict archive tools
* tape: speedup backup by doing read/write in parallel
* tape: store datastore name in tape archives and media catalog
-- Proxmox Support Team <support@proxmox.com> Thu, 18 Mar 2021 12:36:01 +0100
rust-proxmox-backup (1.0.10-1) unstable; urgency=medium
* tape: improve MediaPool allocation by sorting tapes by creation time and
label text
* api: tape backup: continue on vanishing snapshots, as a prune during long
running tape backup jobs is OK
* tape: fix scsi volume_statistics and cartridge_memory for quantum drives
* typo fixes all over the place
* d/postinst: restart, not reload, when updating from a to old version
-- Proxmox Support Team <support@proxmox.com> Thu, 11 Mar 2021 08:24:31 +0100
rust-proxmox-backup (1.0.9-1) unstable; urgency=medium
* client: track key source, print when used
* fix #3026: pxar: metadata: apply flags _after_ updating mtime
* docs: add acl.cfg, datastore.cfg, remote.cfg, sync.cfg, user.cfg and
verification.cfg manual page pages
* docs: add API viewer
* proxmox-backup-manger: add verify-job command group with various sub
commands
* add experimental opt-in tape backup support
* lto-barcode: fix page offset calibration
* lto-barcode: fix avery 3420 paper format properties
* asyncify pxar create archive
* client: raise HTTP_TIMEOUT for simple requests to 120s
* docs: depend on mathjax library package from debian instead of CDN
* fix #3321: docs: client: fix interactive restore command explanation
* ui: use shorter datetime format for encryption key creation time
* docs: TFA: improve language
* config/TFA: webauthn: disallow registering the same token more than once,
that can lead to buggy behavior in some token/browser combinations.
-- Proxmox Support Team <support@proxmox.com> Mon, 08 Mar 2021 15:54:47 +0100
rust-proxmox-backup (1.0.8-1) unstable; urgency=medium
* Https Connector: use hostname instead of URL again to avoid certificate
verification issues.
* ui: task summary: add verification jobs to count
* docs: explain some technical details about datastores/chunks
-- Proxmox Support Team <support@proxmox.com> Thu, 04 Feb 2021 12:39:49 +0100
rust-proxmox-backup (1.0.7-1) unstable; urgency=medium
* fix #3197: skip fingerprint check when restoring key
* client: add 'import-with-master-key' command
* fix #3192: correct sort in prune sim
* tools/daemon: improve reload behaviour
* http client: add timeouts for critical connects
* api: improve error messages for restricted endpoints
* api: allow tokens to list users
* ui: running tasks: Use gettext for column labels
* login: add two-factor authenication (TFA) and integrate in web-interface
* login: support webAuthn, recovery keys and TOTP as TFA methods
* make it possible to abort tasks with CTRL-C
* fix #3245: only use default schedule for new jobs
* manager CLI: user/token list: fix rendering 0 (never) expire date
* update the event-driven, non-blocking I/O tokio platform to 1.0
* access: limit editing all pam credentials to superuser
* access: restrict password changes on @pam realm to superuser
* patch out wrongly linked libraries from ELFs to avoid extra, bogus
dependencies in resulting package
* add "password hint" to encryption key config
* improve GC error handling
* cli: make it possible to abort tasks with CTRL-C
-- Proxmox Support Team <support@proxmox.com> Wed, 03 Feb 2021 10:34:23 +0100
rust-proxmox-backup (1.0.6-1) unstable; urgency=medium
* stricter handling of file-descriptors, fixes some cases where some could
leak
* ui: fix various usages of the findRecord emthod, ensuring it matches exact
* garbage collection: improve task log format
* verification: improve progress log, make it similar to what's logged on
pull (sync)
* datastore: move manifest locking to /run. This avoids issues with
filesystems which cannot natively handle removing in-use files ("delete on
last close"), and create a virtual, internal, replacement file to work
around that. This is done, for example, by NFS or CIFS (samba).
-- Proxmox Support Team <support@proxmox.com> Fri, 11 Dec 2020 12:51:33 +0100
rust-proxmox-backup (1.0.5-1) unstable; urgency=medium
* client: restore: print meta information exclusively to standard error
-- Proxmox Support Team <support@proxmox.com> Wed, 25 Nov 2020 15:29:58 +0100
rust-proxmox-backup (1.0.4-1) unstable; urgency=medium
* fingerprint: add bytes() accessor
* ui: fix broken gettext use
* cli: move more commands into "snapshot" sub-command
-- Proxmox Support Team <support@proxmox.com> Wed, 25 Nov 2020 06:37:41 +0100
rust-proxmox-backup (1.0.3-1) unstable; urgency=medium
* client: inform user when automatically using the default encryption key
* ui: UserView: render name as 'Firstname Lastname'
* proxmox-backup-manager: add versions command
* pxar: fix anchored exclusion at archive root
* pxar: include .pxarexclude files in the archive
* client: expose all-file-systems option
* api: make expensive parts of datastore status opt-in
* api: include datastore ID in invalid owner errors
* garbage collection: treat .bad files like regular chunks to ensure they
are removed if not referenced anymore
* client: fix issues with encoded UPID strings
* encryption: add fingerprint to key config
* client: add 'key show' command
* fix #3139: add key fingerprint to backup snapshot manifest and check it
when loading with a key
* ui: add snapshot/file fingerprint tooltip
-- Proxmox Support Team <support@proxmox.com> Tue, 24 Nov 2020 08:55:47 +0100
rust-proxmox-backup (1.0.1-1) unstable; urgency=medium
* ui: datastore summary: drop 'removed bytes' display
* ui: datastore add: set default schedule
* prune sim: make backup schedule a form, bind update button to its validity
* daemon: add workaround for race in reloading and systemd 'ready' notification
-- Proxmox Support Team <support@proxmox.com> Wed, 11 Nov 2020 10:18:12 +0100
rust-proxmox-backup (1.0.0-1) unstable; urgency=medium
* fix #3121: forbid removing used remotes
* docs: backup-client: encryption: discuss paperkey command
* pxar: log when skipping mount points
* ui: show also parent ACLs which affect a datastore in its panel
* api: node/apt: add versions call
* ui: make Datastore a selectable panel again. Show a datastore summary
list, and provide unfiltered access to all sync and verify jobs.
* ui: add help tool-button to various paneös
* ui: set various onlineHelp buttons
* zfs: mount new zpools created via API under /mnt/datastore/<id>
* ui: move disks/directory views to its own tab panel
* fix #3060: continue sync if we cannot aquire the group lock
* HttpsConnector: include destination on connect errors
* fix #3060:: improve get_owner error handling
* remote.cfg: rename userid to 'auth-id'
* verify: log/warn on invalid owner
-- Proxmox Support Team <support@proxmox.com> Tue, 10 Nov 2020 14:36:13 +0100
rust-proxmox-backup (0.9.7-1) unstable; urgency=medium
* ui: add remote store selector

90
debian/control vendored
View File

@ -7,24 +7,25 @@ Build-Depends: debhelper (>= 11),
rustc:native,
libstd-rust-dev,
librust-anyhow-1+default-dev,
librust-apt-pkg-native-0.3+default-dev (>= 0.3.1-~~),
librust-apt-pkg-native-0.3+default-dev (>= 0.3.2-~~),
librust-base64-0.12+default-dev,
librust-bitflags-1+default-dev (>= 1.2.1-~~),
librust-bytes-0.5+default-dev,
librust-bytes-1+default-dev,
librust-crc32fast-1+default-dev,
librust-crossbeam-channel-0.4+default-dev,
librust-crossbeam-channel-0.5+default-dev,
librust-endian-trait-0.6+arrays-dev,
librust-endian-trait-0.6+default-dev,
librust-futures-0.3+default-dev,
librust-h2-0.2+default-dev,
librust-h2-0.2+stream-dev,
librust-h2-0.3+default-dev,
librust-h2-0.3+stream-dev,
librust-handlebars-3+default-dev,
librust-http-0.2+default-dev,
librust-hyper-0.13+default-dev (>= 0.13.6-~~),
librust-hyper-0.14+default-dev,
librust-hyper-0.14+full-dev,
librust-lazy-static-1+default-dev (>= 1.4-~~),
librust-libc-0.2+default-dev,
librust-log-0.4+default-dev,
librust-nix-0.19+default-dev,
librust-nix-0.19+default-dev (>= 0.19.1-~~),
librust-nom-5+default-dev (>= 5.1-~~),
librust-num-traits-0.2+default-dev,
librust-once-cell-1+default-dev (>= 1.3.1-~~),
@ -33,42 +34,42 @@ Build-Depends: debhelper (>= 11),
librust-pam-sys-0.5+default-dev,
librust-pathpatterns-0.1+default-dev (>= 0.1.2-~~),
librust-percent-encoding-2+default-dev (>= 2.1-~~),
librust-pin-project-1+default-dev,
librust-pin-utils-0.1+default-dev,
librust-proxmox-0.7+api-macro-dev,
librust-proxmox-0.7+default-dev,
librust-proxmox-0.7+sortable-macro-dev,
librust-proxmox-0.7+websocket-dev,
librust-proxmox-fuse-0.1+default-dev,
librust-pxar-0.6+default-dev (>= 0.6.1-~~),
librust-pxar-0.6+futures-io-dev (>= 0.6.1-~~),
librust-pxar-0.6+tokio-io-dev (>= 0.6.1-~~),
librust-proxmox-0.11+api-macro-dev,
librust-proxmox-0.11+default-dev,
librust-proxmox-0.11+sortable-macro-dev,
librust-proxmox-0.11+websocket-dev,
librust-proxmox-fuse-0.1+default-dev (>= 0.1.1-~~),
librust-pxar-0.10+default-dev,
librust-pxar-0.10+tokio-io-dev,
librust-regex-1+default-dev (>= 1.2-~~),
librust-rustyline-6+default-dev,
librust-rustyline-7+default-dev,
librust-serde-1+default-dev,
librust-serde-1+derive-dev,
librust-serde-json-1+default-dev,
librust-siphasher-0.3+default-dev,
librust-syslog-4+default-dev,
librust-tokio-0.2+blocking-dev (>= 0.2.9-~~),
librust-tokio-0.2+default-dev (>= 0.2.9-~~),
librust-tokio-0.2+dns-dev (>= 0.2.9-~~),
librust-tokio-0.2+fs-dev (>= 0.2.9-~~),
librust-tokio-0.2+io-util-dev (>= 0.2.9-~~),
librust-tokio-0.2+macros-dev (>= 0.2.9-~~),
librust-tokio-0.2+process-dev (>= 0.2.9-~~),
librust-tokio-0.2+rt-threaded-dev (>= 0.2.9-~~),
librust-tokio-0.2+signal-dev (>= 0.2.9-~~),
librust-tokio-0.2+stream-dev (>= 0.2.9-~~),
librust-tokio-0.2+tcp-dev (>= 0.2.9-~~),
librust-tokio-0.2+time-dev (>= 0.2.9-~~),
librust-tokio-0.2+uds-dev (>= 0.2.9-~~),
librust-tokio-openssl-0.4+default-dev,
librust-tokio-util-0.3+codec-dev,
librust-tokio-util-0.3+default-dev,
librust-tokio-1+default-dev,
librust-tokio-1+fs-dev,
librust-tokio-1+io-util-dev,
librust-tokio-1+macros-dev,
librust-tokio-1+net-dev,
librust-tokio-1+parking-lot-dev,
librust-tokio-1+process-dev,
librust-tokio-1+rt-dev,
librust-tokio-1+rt-multi-thread-dev,
librust-tokio-1+signal-dev,
librust-tokio-1+time-dev,
librust-tokio-openssl-0.6+default-dev (>= 0.6.1-~~),
librust-tokio-stream-0.1+default-dev,
librust-tokio-util-0.6+codec-dev,
librust-tokio-util-0.6+default-dev,
librust-tower-service-0.3+default-dev,
librust-udev-0.4+default-dev | librust-udev-0.3+default-dev,
librust-url-2+default-dev (>= 2.1-~~),
librust-walkdir-2+default-dev,
librust-webauthn-rs-0.2+default-dev (>= 0.2.5-~~),
librust-xdg-2+default-dev (>= 2.2-~~),
librust-zstd-0.4+bindgen-dev,
librust-zstd-0.4+default-dev,
@ -76,43 +77,53 @@ Build-Depends: debhelper (>= 11),
libfuse3-dev,
libsystemd-dev,
uuid-dev,
debhelper (>= 12~),
libsgutils2-dev,
bash-completion,
pve-eslint,
python3-docutils,
python3-pygments,
rsync,
debhelper (>= 12~),
fonts-dejavu-core <!nodoc>,
fonts-lato <!nodoc>,
fonts-open-sans <!nodoc>,
graphviz <!nodoc>,
latexmk <!nodoc>,
patchelf,
pve-eslint (>= 7.18.0-1),
python3-docutils,
python3-pygments,
python3-sphinx <!nodoc>,
rsync,
texlive-fonts-extra <!nodoc>,
texlive-fonts-recommended <!nodoc>,
texlive-xetex <!nodoc>,
xindy <!nodoc>
Maintainer: Proxmox Support Team <support@proxmox.com>
Standards-Version: 4.4.1
Vcs-Git:
Vcs-Browser:
Vcs-Git: git://git.proxmox.com/git/proxmox-backup.git
Vcs-Browser: https://git.proxmox.com/?p=proxmox-backup.git;a=summary
Homepage: https://www.proxmox.com
Package: proxmox-backup-server
Architecture: any
Depends: fonts-font-awesome,
libjs-extjs (>= 6.0.1),
libjs-qrcodejs (>= 1.20201119),
libsgutils2-2,
libzstd1 (>= 1.3.8),
lvm2,
mt-st,
mtx,
openssh-server,
pbs-i18n,
postfix | mail-transport-agent,
proxmox-backup-docs,
proxmox-mini-journalreader,
proxmox-widget-toolkit (>= 2.3-6),
pve-xtermjs (>= 4.7.0-1),
sg3-utils,
smartmontools,
${misc:Depends},
${shlibs:Depends},
Recommends: zfsutils-linux,
ifupdown2,
Description: Proxmox Backup Server daemon with tools and GUI
This package contains the Proxmox Backup Server daemons and related
tools. This includes a web-based graphical user interface.
@ -130,6 +141,7 @@ Package: proxmox-backup-docs
Build-Profiles: <!nodoc>
Section: doc
Depends: libjs-extjs,
libjs-mathjax,
${misc:Depends},
Architecture: all
Description: Proxmox Backup Documentation

9
debian/control.in vendored
View File

@ -2,17 +2,25 @@ Package: proxmox-backup-server
Architecture: any
Depends: fonts-font-awesome,
libjs-extjs (>= 6.0.1),
libjs-qrcodejs (>= 1.20201119),
libsgutils2-2,
libzstd1 (>= 1.3.8),
lvm2,
mt-st,
mtx,
openssh-server,
pbs-i18n,
postfix | mail-transport-agent,
proxmox-backup-docs,
proxmox-mini-journalreader,
proxmox-widget-toolkit (>= 2.3-6),
pve-xtermjs (>= 4.7.0-1),
sg3-utils,
smartmontools,
${misc:Depends},
${shlibs:Depends},
Recommends: zfsutils-linux,
ifupdown2,
Description: Proxmox Backup Server daemon with tools and GUI
This package contains the Proxmox Backup Server daemons and related
tools. This includes a web-based graphical user interface.
@ -30,6 +38,7 @@ Package: proxmox-backup-docs
Build-Profiles: <!nodoc>
Section: doc
Depends: libjs-extjs,
libjs-mathjax,
${misc:Depends},
Architecture: all
Description: Proxmox Backup Documentation

2
debian/copyright vendored
View File

@ -1,4 +1,4 @@
Copyright (C) 2019 Proxmox Server Solutions GmbH
Copyright (C) 2019 - 2021 Proxmox Server Solutions GmbH
This software is written by Proxmox Server Solutions GmbH <support@proxmox.com>

22
debian/debcargo.toml vendored
View File

@ -2,33 +2,32 @@ overlay = "."
crate_src_path = ".."
whitelist = ["tests/*.c"]
# needed for pinutils alpha
allow_prerelease_deps = true
maintainer = "Proxmox Support Team <support@proxmox.com>"
[source]
# TODO: update once public
vcs_git = ""
vcs_browser = ""
maintainer = "Proxmox Support Team <support@proxmox.com>"
vcs_git = "git://git.proxmox.com/git/proxmox-backup.git"
vcs_browser = "https://git.proxmox.com/?p=proxmox-backup.git;a=summary"
section = "admin"
build_depends = [
"debhelper (>= 12~)",
"bash-completion",
"pve-eslint",
"python3-docutils",
"python3-pygments",
"rsync",
"debhelper (>= 12~)",
"fonts-dejavu-core <!nodoc>",
"fonts-lato <!nodoc>",
"fonts-open-sans <!nodoc>",
"graphviz <!nodoc>",
"latexmk <!nodoc>",
"patchelf",
"pve-eslint (>= 7.18.0-1)",
"python3-docutils",
"python3-pygments",
"python3-sphinx <!nodoc>",
"rsync",
"texlive-fonts-extra <!nodoc>",
"texlive-fonts-recommended <!nodoc>",
"texlive-xetex <!nodoc>",
"xindy <!nodoc>",
]
build_depends_excludes = [
"debhelper (>=11)",
]
@ -39,4 +38,5 @@ depends = [
"libfuse3-dev",
"libsystemd-dev",
"uuid-dev",
"libsgutils2-dev",
]

View File

@ -1,2 +1,2 @@
proxmox-backup-server: package-installs-apt-sources etc/apt/sources.list.d/pbstest-beta.list
proxmox-backup-server: package-installs-apt-sources etc/apt/sources.list.d/pbs-enterprise.list
proxmox-backup-server: systemd-service-file-refers-to-unusual-wantedby-target lib/systemd/system/proxmox-backup-banner.service getty.target

3
debian/pmt.bc vendored Normal file
View File

@ -0,0 +1,3 @@
# pmt bash completion
complete -C 'pmt bashcomplete' pmt

3
debian/pmtx.bc vendored Normal file
View File

@ -0,0 +1,3 @@
# pmtx bash completion
complete -C 'pmtx bashcomplete' pmtx

25
debian/postinst vendored
View File

@ -6,10 +6,21 @@ set -e
case "$1" in
configure)
# need to have user backup in the tape group
usermod -a -G tape backup
# modeled after dh_systemd_start output
systemctl --system daemon-reload >/dev/null || true
if [ -n "$2" ]; then
if dpkg --compare-versions "$2" 'lt' '1.0.7-1'; then
# there was an issue with reloading and systemd being confused in older daemon versions
# so restart instead of reload if upgrading from there, see commit 0ec79339f7aebf9
# FIXME: remove with PBS 2.1
echo "Upgrading from older proxmox-backup-server: restart (not reload) daemons"
_dh_action=try-restart
else
_dh_action=try-reload-or-restart
fi
else
_dh_action=start
fi
@ -28,12 +39,26 @@ case "$1" in
if dpkg --compare-versions "$2" 'le' '0.9.5-1'; then
chown --quiet backup:backup /var/log/proxmox-backup/api/auth.log || true
fi
if dpkg --compare-versions "$2" 'le' '0.9.7-1'; then
if [ -e /etc/proxmox-backup/remote.cfg ]; then
echo "NOTE: Switching over remote.cfg to new field names.."
flock -w 30 /etc/proxmox-backup/.remote.lck \
sed -i \
-e 's/^\s\+userid /\tauth-id /g' \
/etc/proxmox-backup/remote.cfg || true
fi
fi
# FIXME: remove with 2.0
if [ -d "/var/lib/proxmox-backup/tape" ] &&
[ "$(stat --printf '%a' '/var/lib/proxmox-backup/tape')" != "750" ]; then
chmod 0750 /var/lib/proxmox-backup/tape || true
fi
# FIXME: Remove in future version once we're sure no broken entries remain in anyone's files
if grep -q -e ':termproxy::[^@]\+: ' /var/log/proxmox-backup/tasks/active; then
echo "Fixing up termproxy user id in task log..."
flock -w 30 /var/log/proxmox-backup/tasks/active.lock sed -i 's/:termproxy::\([^@]\+\): /:termproxy::\1@pam: /' /var/log/proxmox-backup/tasks/active || true
fi
fi
;;
abort-upgrade|abort-remove|abort-deconfigure)

View File

@ -1,2 +1,5 @@
/usr/share/doc/proxmox-backup/proxmox-backup.pdf /usr/share/doc/proxmox-backup/html/proxmox-backup.pdf
/usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/prune-simulator/extjs
/usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/lto-barcode/extjs
/usr/share/javascript/extjs /usr/share/doc/proxmox-backup/html/api-viewer/extjs
/usr/share/javascript/mathjax /usr/share/doc/proxmox-backup/html/_static/mathjax

View File

@ -1 +1,4 @@
debian/proxmox-backup-manager.bc proxmox-backup-manager
debian/proxmox-tape.bc proxmox-tape
debian/pmtx.bc pmtx
debian/pmt.bc pmt

View File

@ -3,17 +3,35 @@ etc/proxmox-backup.service /lib/systemd/system/
etc/proxmox-backup-banner.service /lib/systemd/system/
etc/proxmox-backup-daily-update.service /lib/systemd/system/
etc/proxmox-backup-daily-update.timer /lib/systemd/system/
etc/pbstest-beta.list /etc/apt/sources.list.d/
etc/pbs-enterprise.list /etc/apt/sources.list.d/
usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-api
usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-proxy
usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-backup-banner
usr/lib/x86_64-linux-gnu/proxmox-backup/proxmox-daily-update
usr/lib/x86_64-linux-gnu/proxmox-backup/sg-tape-cmd
usr/sbin/proxmox-backup-manager
usr/bin/pmtx
usr/bin/pmt
usr/bin/proxmox-tape
usr/share/javascript/proxmox-backup/index.hbs
usr/share/javascript/proxmox-backup/css/ext6-pbs.css
usr/share/javascript/proxmox-backup/images/logo-128.png
usr/share/javascript/proxmox-backup/images/proxmox_logo.png
usr/share/javascript/proxmox-backup/images
usr/share/javascript/proxmox-backup/js/proxmox-backup-gui.js
usr/share/man/man1/proxmox-backup-manager.1
usr/share/man/man1/proxmox-backup-proxy.1
usr/share/man/man1/proxmox-tape.1
usr/share/man/man1/pmtx.1
usr/share/man/man1/pmt.1
usr/share/man/man5/acl.cfg.5
usr/share/man/man5/datastore.cfg.5
usr/share/man/man5/user.cfg.5
usr/share/man/man5/remote.cfg.5
usr/share/man/man5/sync.cfg.5
usr/share/man/man5/verification.cfg.5
usr/share/man/man5/media-pool.cfg.5
usr/share/man/man5/tape.cfg.5
usr/share/man/man5/tape-job.cfg.5
usr/share/zsh/vendor-completions/_proxmox-backup-manager
usr/share/zsh/vendor-completions/_proxmox-tape
usr/share/zsh/vendor-completions/_pmtx
usr/share/zsh/vendor-completions/_pmt

View File

@ -0,0 +1 @@
rm_conffile /etc/apt/sources.list.d/pbstest-beta.list 1.0.0~ proxmox-backup-server

3
debian/proxmox-tape.bc vendored Normal file
View File

@ -0,0 +1,3 @@
# proxmox-tape bash completion
complete -C 'proxmox-tape bashcomplete' proxmox-tape

10
debian/rules vendored
View File

@ -42,10 +42,20 @@ override_dh_installsystemd:
# note: we start/try-reload-restart services manually in postinst
dh_installsystemd --no-start --no-restart-after-upgrade
override_dh_fixperms:
dh_fixperms --exclude sg-tape-cmd
# workaround https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=933541
# TODO: remove once available (Debian 11 ?)
override_dh_dwz:
dh_dwz --no-dwz-multifile
override_dh_strip:
dh_strip
for exe in $$(find debian/proxmox-backup-client/usr \
debian/proxmox-backup-server/usr -executable -type f); do \
debian/scripts/elf-strip-unused-dependencies.sh "$$exe" || true; \
done
override_dh_compress:
dh_compress -X.pdf

View File

@ -0,0 +1,20 @@
#!/bin/bash
binary=$1
exec 3< <(ldd -u "$binary" | grep -oP '[^/:]+$')
patchargs=""
dropped=""
while read -r dep; do
dropped="$dep $dropped"
patchargs="--remove-needed $dep $patchargs"
done <&3
exec 3<&-
if [[ $dropped == "" ]]; then
exit 0
fi
echo -e "patchelf '$binary' - removing unused dependencies:\n $dropped"
patchelf $patchargs $binary

View File

@ -5,6 +5,7 @@ LIBDIR = $(PREFIX)/lib
LIBEXECDIR = $(LIBDIR)
DATAROOTDIR = $(PREFIX)/share
MAN1DIR = $(PREFIX)/share/man/man1
MAN5DIR = $(PREFIX)/share/man/man5
DOCDIR = $(PREFIX)/share/doc/proxmox-backup
JSDIR = $(DATAROOTDIR)/javascript/proxmox-backup
SYSCONFDIR = /etc

View File

@ -1,24 +1,66 @@
include ../defines.mk
GENERATED_SYNOPSIS := \
proxmox-tape/synopsis.rst \
proxmox-backup-client/synopsis.rst \
proxmox-backup-client/catalog-shell-synopsis.rst \
proxmox-backup-manager/synopsis.rst \
pxar/synopsis.rst \
backup-protocol-api.rst \
reader-protocol-api.rst
pmtx/synopsis.rst \
pmt/synopsis.rst \
config/media-pool/config.rst \
config/tape/config.rst \
config/tape-job/config.rst \
config/user/config.rst \
config/remote/config.rst \
config/sync/config.rst \
config/verification/config.rst \
config/acl/roles.rst \
config/datastore/config.rst
MANUAL_PAGES := \
MAN1_PAGES := \
pxar.1 \
pmtx.1 \
pmt.1 \
proxmox-tape.1 \
proxmox-backup-proxy.1 \
proxmox-backup-client.1 \
proxmox-backup-manager.1
MAN5_PAGES := \
media-pool.cfg.5 \
tape.cfg.5 \
tape-job.cfg.5 \
acl.cfg.5 \
user.cfg.5 \
remote.cfg.5 \
sync.cfg.5 \
verification.cfg.5 \
datastore.cfg.5
PRUNE_SIMULATOR_FILES := \
prune-simulator/index.html \
prune-simulator/documentation.html \
prune-simulator/clear-trigger.png \
prune-simulator/prune-simulator.js
LTO_BARCODE_FILES := \
lto-barcode/index.html \
lto-barcode/code39.js \
lto-barcode/prefix-field.js \
lto-barcode/label-style.js \
lto-barcode/tape-type.js \
lto-barcode/paper-size.js \
lto-barcode/page-layout.js \
lto-barcode/page-calibration.js \
lto-barcode/label-list.js \
lto-barcode/label-setup.js \
lto-barcode/lto-barcode.js
API_VIEWER_SOURCES= \
api-viewer/index.html \
api-viewer/apidoc.js
# Sphinx documentation setup
SPHINXOPTS =
SPHINXBUILD = sphinx-build
@ -35,15 +77,7 @@ endif
# Sphinx internal variables.
ALLSPHINXOPTS = -d $(BUILDDIR)/doctrees $(SPHINXOPTS) .
all: ${MANUAL_PAGES}
# Extract backup protocol docs
backup-protocol-api.rst: ${COMPILEDIR}/dump-backup-api
${COMPILEDIR}/dump-backup-api >$@
# Extract reader protocol docs
reader-protocol-api.rst: ${COMPILEDIR}/dump-reader-api
${COMPILEDIR}/dump-backup-api >$@
all: ${MAN1_PAGES} ${MAN5_PAGES}
# Build manual pages using rst2man
@ -53,6 +87,80 @@ pxar/synopsis.rst: ${COMPILEDIR}/pxar
pxar.1: pxar/man1.rst pxar/description.rst pxar/synopsis.rst
rst2man $< >$@
pmtx/synopsis.rst: ${COMPILEDIR}/pmtx
${COMPILEDIR}/pmtx printdoc > pmtx/synopsis.rst
pmtx.1: pmtx/man1.rst pmtx/description.rst pmtx/synopsis.rst
rst2man $< >$@
pmt/synopsis.rst: ${COMPILEDIR}/pmt
${COMPILEDIR}/pmt printdoc > pmt/synopsis.rst
pmt.1: pmt/man1.rst pmt/description.rst pmt/options.rst pmt/synopsis.rst
rst2man $< >$@
config/datastore/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen datastore.cfg >$@
datastore.cfg.5: config/datastore/man5.rst config/datastore/config.rst config/datastore/format.rst
rst2man $< >$@
config/user/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen user.cfg >$@
user.cfg.5: config/user/man5.rst config/user/config.rst config/user/format.rst
rst2man $< >$@
config/remote/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen remote.cfg >$@
remote.cfg.5: config/remote/man5.rst config/remote/config.rst config/remote/format.rst
rst2man $< >$@
config/sync/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen sync.cfg >$@
sync.cfg.5: config/sync/man5.rst config/sync/config.rst config/sync/format.rst
rst2man $< >$@
config/verification/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen verification.cfg >$@
verification.cfg.5: config/verification/man5.rst config/verification/config.rst config/verification/format.rst
rst2man $< >$@
config/acl/roles.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen "config::acl::Role" >$@
acl.cfg.5: config/acl/man5.rst config/acl/roles.rst config/acl/format.rst
rst2man $< >$@
config/media-pool/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen media-pool.cfg >$@
media-pool.cfg.5: config/media-pool/man5.rst config/media-pool/config.rst config/media-pool/format.rst
rst2man $< >$@
config/tape/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen tape.cfg >$@
tape.cfg.5: config/tape/man5.rst config/tape/config.rst config/tape/format.rst
rst2man $< >$@
config/tape-job/config.rst: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen tape-job.cfg >$@
tape-job.cfg.5: config/tape-job/man5.rst config/tape-job/config.rst config/tape-job/format.rst
rst2man $< >$@
proxmox-tape/synopsis.rst: ${COMPILEDIR}/proxmox-tape
${COMPILEDIR}/proxmox-tape printdoc > proxmox-tape/synopsis.rst
proxmox-tape.1: proxmox-tape/man1.rst proxmox-tape/description.rst proxmox-tape/synopsis.rst
rst2man $< >$@
proxmox-backup-client/synopsis.rst: ${COMPILEDIR}/proxmox-backup-client
${COMPILEDIR}/proxmox-backup-client printdoc > proxmox-backup-client/synopsis.rst
@ -77,12 +185,22 @@ onlinehelpinfo:
$(SPHINXBUILD) -b proxmox-scanrefs $(ALLSPHINXOPTS) $(BUILDDIR)/scanrefs
@echo "Build finished. OnlineHelpInfo.js is in $(BUILDDIR)/scanrefs."
api-viewer/apidata.js: ${COMPILEDIR}/docgen
${COMPILEDIR}/docgen apidata.js >$@
api-viewer/apidoc.js: api-viewer/apidata.js api-viewer/PBSAPI.js
cat api-viewer/apidata.js api-viewer/PBSAPI.js >$@
.PHONY: html
html: ${GENERATED_SYNOPSIS} images/proxmox-logo.svg custom.css conf.py ${PRUNE_SIMULATOR_FILES}
html: ${GENERATED_SYNOPSIS} images/proxmox-logo.svg custom.css conf.py ${PRUNE_SIMULATOR_FILES} ${LTO_BARCODE_FILES} ${API_VIEWER_SOURCES}
$(SPHINXBUILD) -b html $(ALLSPHINXOPTS) $(BUILDDIR)/html
install -m 0644 custom.js custom.css images/proxmox-logo.svg $(BUILDDIR)/html/_static/
install -dm 0755 $(BUILDDIR)/html/prune-simulator
install -m 0644 ${PRUNE_SIMULATOR_FILES} $(BUILDDIR)/html/prune-simulator
install -dm 0755 $(BUILDDIR)/html/lto-barcode
install -m 0644 ${LTO_BARCODE_FILES} $(BUILDDIR)/html/lto-barcode
install -dm 0755 $(BUILDDIR)/html/api-viewer
install -m 0644 ${API_VIEWER_SOURCES} $(BUILDDIR)/html/api-viewer
@echo
@echo "Build finished. The HTML pages are in $(BUILDDIR)/html."
@ -101,12 +219,14 @@ epub3: ${GENERATED_SYNOPSIS}
@echo "Build finished. The epub3 file is in $(BUILDDIR)/epub3."
clean:
rm -r -f *~ *.1 ${BUILDDIR} ${GENERATED_SYNOPSIS}
rm -r -f *~ *.1 ${BUILDDIR} ${GENERATED_SYNOPSIS} api-viewer/apidata.js
install_manual_pages: ${MANUAL_PAGES}
install_manual_pages: ${MAN1_PAGES} ${MAN5_PAGES}
install -dm755 $(DESTDIR)$(MAN1DIR)
for i in ${MANUAL_PAGES}; do install -m755 $$i $(DESTDIR)$(MAN1DIR)/ ; done
for i in ${MAN1_PAGES}; do install -m755 $$i $(DESTDIR)$(MAN1DIR)/ ; done
install -dm755 $(DESTDIR)$(MAN5DIR)
for i in ${MAN5_PAGES}; do install -m755 $$i $(DESTDIR)$(MAN5DIR)/ ; done
install_html: html
install -dm755 $(DESTDIR)$(DOCDIR)

View File

@ -44,7 +44,7 @@ def scan_extjs_files(wwwdir="../www"): # a bit rough i know, but we can optimize
js_files.append(os.path.join(root, filename))
for js_file in js_files:
fd = open(js_file).read()
allmatch = re.findall("onlineHelp:\s*[\'\"](.*?)[\'\"]", fd, re.M)
allmatch = re.findall("(?:onlineHelp:|get_help_tool\s*\()\s*[\'\"](.*?)[\'\"]", fd, re.M)
for match in allmatch:
anchor = match
anchor = re.sub('_', '-', anchor) # normalize labels
@ -73,7 +73,9 @@ class ReflabelMapper(Builder):
'link': '/docs/index.html',
'title': 'Proxmox Backup Server Documentation Index',
}
self.env.used_anchors = scan_extjs_files()
# Disabled until we find a sensible way to scan proxmox-widget-toolkit
# as well
#self.env.used_anchors = scan_extjs_files()
if not os.path.isdir(self.outdir):
os.mkdir(self.outdir)
@ -88,11 +90,25 @@ class ReflabelMapper(Builder):
if hasattr(node, 'expect_referenced_by_id') and len(node['ids']) > 1: # explicit labels
filename = self.env.doc2path(docname)
filename_html = re.sub('.rst', '.html', filename)
labelid = node['ids'][1] # [0] is predefined by sphinx, we need [1] for explicit ones
# node['ids'][0] contains a normalized version of the
# headline. If the ref and headline are the same
# (normalized) sphinx will set the node['ids'][1] to a
# generic id in the format `idX` where X is numeric. If the
# ref and headline are not the same, the ref name will be
# stored in node['ids'][1]
if re.match('^id[0-9]*$', node['ids'][1]):
labelid = node['ids'][0]
else:
labelid = node['ids'][1]
title = cast(nodes.title, node[0])
logger.info('traversing section {}'.format(title.astext()))
ref_name = getattr(title, 'rawsource', title.astext())
if (ref_name[:7] == ':term:`'):
ref_name = ref_name[7:-1]
self.env.online_help[labelid] = {'link': '', 'title': ''}
self.env.online_help[labelid]['link'] = "/docs/" + os.path.basename(filename_html) + "#{}".format(labelid)
self.env.online_help[labelid]['title'] = ref_name
@ -112,15 +128,18 @@ class ReflabelMapper(Builder):
def validate_anchors(self):
#pprint(self.env.online_help)
to_remove = []
for anchor in self.env.used_anchors:
if anchor not in self.env.online_help:
logger.info("[-] anchor {} is missing from onlinehelp!".format(anchor))
for anchor in self.env.online_help:
if anchor not in self.env.used_anchors and anchor != 'pbs_documentation_index':
logger.info("[*] anchor {} not used! deleting...".format(anchor))
to_remove.append(anchor)
for anchor in to_remove:
self.env.online_help.pop(anchor, None)
# Disabled until we find a sensible way to scan proxmox-widget-toolkit
# as well
#for anchor in self.env.used_anchors:
# if anchor not in self.env.online_help:
# logger.info("[-] anchor {} is missing from onlinehelp!".format(anchor))
#for anchor in self.env.online_help:
# if anchor not in self.env.used_anchors and anchor != 'pbs_documentation_index':
# logger.info("[*] anchor {} not used! deleting...".format(anchor))
# to_remove.append(anchor)
#for anchor in to_remove:
# self.env.online_help.pop(anchor, None)
return
def finish(self):

511
docs/api-viewer/PBSAPI.js Normal file
View File

@ -0,0 +1,511 @@
// avoid errors when running without development tools
if (!Ext.isDefined(Ext.global.console)) {
var console = {
dir: function() {},
log: function() {}
};
}
Ext.onReady(function() {
Ext.define('pve-param-schema', {
extend: 'Ext.data.Model',
fields: [
'name', 'type', 'typetext', 'description', 'verbose_description',
'enum', 'minimum', 'maximum', 'minLength', 'maxLength',
'pattern', 'title', 'requires', 'format', 'default',
'disallow', 'extends', 'links',
{
name: 'optional',
type: 'boolean'
}
]
});
var store = Ext.define('pve-updated-treestore', {
extend: 'Ext.data.TreeStore',
model: Ext.define('pve-api-doc', {
extend: 'Ext.data.Model',
fields: [
'path', 'info', 'text',
]
}),
proxy: {
type: 'memory',
data: pbsapi
},
sorters: [{
property: 'leaf',
direction: 'ASC'
}, {
property: 'text',
direction: 'ASC'
}],
filterer: 'bottomup',
doFilter: function(node) {
this.filterNodes(node, this.getFilters().getFilterFn(), true);
},
filterNodes: function(node, filterFn, parentVisible) {
var me = this,
bottomUpFiltering = me.filterer === 'bottomup',
match = filterFn(node) && parentVisible || (node.isRoot() && !me.getRootVisible()),
childNodes = node.childNodes,
len = childNodes && childNodes.length, i, matchingChildren;
if (len) {
for (i = 0; i < len; ++i) {
matchingChildren = me.filterNodes(childNodes[i], filterFn, match || bottomUpFiltering) || matchingChildren;
}
if (bottomUpFiltering) {
match = matchingChildren || match;
}
}
node.set("visible", match, me._silentOptions);
return match;
},
}).create();
var render_description = function(value, metaData, record) {
var pdef = record.data;
value = pdef.verbose_description || value;
// TODO: try to render asciidoc correctly
metaData.style = 'white-space:pre-wrap;'
return Ext.htmlEncode(value);
};
var render_type = function(value, metaData, record) {
var pdef = record.data;
return pdef['enum'] ? 'enum' : (pdef.type || 'string');
};
var render_format = function(value, metaData, record) {
var pdef = record.data;
metaData.style = 'white-space:normal;'
if (pdef.typetext)
return Ext.htmlEncode(pdef.typetext);
if (pdef['enum'])
return pdef['enum'].join(' | ');
if (pdef.format)
return pdef.format;
if (pdef.pattern)
return Ext.htmlEncode(pdef.pattern);
return '';
};
var real_path = function(path) {
return path.replace(/^.*\/_upgrade_(\/)?/, "/");
};
var permission_text = function(permission) {
let permhtml = "";
if (permission.user) {
if (!permission.description) {
if (permission.user === 'world') {
permhtml += "Accessible without any authentication.";
} else if (permission.user === 'all') {
permhtml += "Accessible by all authenticated users.";
} else {
permhtml += 'Onyl accessible by user "' +
permission.user + '"';
}
}
} else if (permission.check) {
permhtml += "<pre>Check: " +
Ext.htmlEncode(Ext.JSON.encode(permission.check)) + "</pre>";
} else if (permission.userParam) {
permhtml += `<div>Check if user matches parameter '${permission.userParam}'`;
} else if (permission.or) {
permhtml += "<div>Or<div style='padding-left: 10px;'>";
Ext.Array.each(permission.or, function(sub_permission) {
permhtml += permission_text(sub_permission);
})
permhtml += "</div></div>";
} else if (permission.and) {
permhtml += "<div>And<div style='padding-left: 10px;'>";
Ext.Array.each(permission.and, function(sub_permission) {
permhtml += permission_text(sub_permission);
})
permhtml += "</div></div>";
} else {
//console.log(permission);
permhtml += "Unknown systax!";
}
return permhtml;
};
var render_docu = function(data) {
var md = data.info;
// console.dir(data);
var items = [];
var clicmdhash = {
GET: 'get',
POST: 'create',
PUT: 'set',
DELETE: 'delete'
};
Ext.Array.each(['GET', 'POST', 'PUT', 'DELETE'], function(method) {
var info = md[method];
if (info) {
var usage = "";
usage += "<table><tr><td>HTTP:&nbsp;&nbsp;&nbsp;</td><td>"
+ method + " " + real_path("/api2/json" + data.path) + "</td></tr>";
var sections = [
{
title: 'Description',
html: Ext.htmlEncode(info.description),
bodyPadding: 10
},
{
title: 'Usage',
html: usage,
bodyPadding: 10
}
];
if (info.parameters && info.parameters.properties) {
var pstore = Ext.create('Ext.data.Store', {
model: 'pve-param-schema',
proxy: {
type: 'memory'
},
groupField: 'optional',
sorters: [
{
property: 'name',
direction: 'ASC'
}
]
});
Ext.Object.each(info.parameters.properties, function(name, pdef) {
pdef.name = name;
pstore.add(pdef);
});
pstore.sort();
var groupingFeature = Ext.create('Ext.grid.feature.Grouping',{
enableGroupingMenu: false,
groupHeaderTpl: '<tpl if="groupValue">Optional</tpl><tpl if="!groupValue">Required</tpl>'
});
sections.push({
xtype: 'gridpanel',
title: 'Parameters',
features: [groupingFeature],
store: pstore,
viewConfig: {
trackOver: false,
stripeRows: true
},
columns: [
{
header: 'Name',
dataIndex: 'name',
flex: 1
},
{
header: 'Type',
dataIndex: 'type',
renderer: render_type,
flex: 1
},
{
header: 'Default',
dataIndex: 'default',
flex: 1
},
{
header: 'Format',
dataIndex: 'type',
renderer: render_format,
flex: 2
},
{
header: 'Description',
dataIndex: 'description',
renderer: render_description,
flex: 6
}
]
});
}
if (info.returns) {
var retinf = info.returns;
var rtype = retinf.type;
if (!rtype && retinf.items)
rtype = 'array';
if (!rtype)
rtype = 'object';
var rpstore = Ext.create('Ext.data.Store', {
model: 'pve-param-schema',
proxy: {
type: 'memory'
},
groupField: 'optional',
sorters: [
{
property: 'name',
direction: 'ASC'
}
]
});
var properties;
if (rtype === 'array' && retinf.items.properties) {
properties = retinf.items.properties;
}
if (rtype === 'object' && retinf.properties) {
properties = retinf.properties;
}
Ext.Object.each(properties, function(name, pdef) {
pdef.name = name;
rpstore.add(pdef);
});
rpstore.sort();
var groupingFeature = Ext.create('Ext.grid.feature.Grouping',{
enableGroupingMenu: false,
groupHeaderTpl: '<tpl if="groupValue">Optional</tpl><tpl if="!groupValue">Obligatory</tpl>'
});
var returnhtml;
if (retinf.items) {
returnhtml = '<pre>items: ' + Ext.htmlEncode(JSON.stringify(retinf.items, null, 4)) + '</pre>';
}
if (retinf.properties) {
returnhtml = returnhtml || '';
returnhtml += '<pre>properties:' + Ext.htmlEncode(JSON.stringify(retinf.properties, null, 4)) + '</pre>';
}
var rawSection = Ext.create('Ext.panel.Panel', {
bodyPadding: '0px 10px 10px 10px',
html: returnhtml,
hidden: true
});
sections.push({
xtype: 'gridpanel',
title: 'Returns: ' + rtype,
features: [groupingFeature],
store: rpstore,
viewConfig: {
trackOver: false,
stripeRows: true
},
columns: [
{
header: 'Name',
dataIndex: 'name',
flex: 1
},
{
header: 'Type',
dataIndex: 'type',
renderer: render_type,
flex: 1
},
{
header: 'Default',
dataIndex: 'default',
flex: 1
},
{
header: 'Format',
dataIndex: 'type',
renderer: render_format,
flex: 2
},
{
header: 'Description',
dataIndex: 'description',
renderer: render_description,
flex: 6
}
],
bbar: [
{
xtype: 'button',
text: 'Show RAW',
handler: function(btn) {
rawSection.setVisible(!rawSection.isVisible());
btn.setText(rawSection.isVisible() ? 'Hide RAW' : 'Show RAW');
}}
]
});
sections.push(rawSection);
}
if (!data.path.match(/\/_upgrade_/)) {
var permhtml = '';
if (!info.permissions) {
permhtml = "Root only.";
} else {
if (info.permissions.description) {
permhtml += "<div style='white-space:pre-wrap;padding-bottom:10px;'>" +
Ext.htmlEncode(info.permissions.description) + "</div>";
}
permhtml += permission_text(info.permissions);
}
// we do not have this information for PBS api
//if (!info.allowtoken) {
// permhtml += "<br />This API endpoint is not available for API tokens."
//}
sections.push({
title: 'Required permissions',
bodyPadding: 10,
html: permhtml
});
}
items.push({
title: method,
autoScroll: true,
defaults: {
border: false
},
items: sections
});
}
});
var ct = Ext.getCmp('docview');
ct.setTitle("Path: " + real_path(data.path));
ct.removeAll(true);
ct.add(items);
ct.setActiveTab(0);
};
Ext.define('Ext.form.SearchField', {
extend: 'Ext.form.field.Text',
alias: 'widget.searchfield',
emptyText: 'Search...',
flex: 1,
inputType: 'search',
listeners: {
'change': function(){
var value = this.getValue();
if (!Ext.isEmpty(value)) {
store.filter({
property: 'path',
value: value,
anyMatch: true
});
} else {
store.clearFilter();
}
}
}
});
var tree = Ext.create('Ext.tree.Panel', {
title: 'Resource Tree',
tbar: [
{
xtype: 'searchfield',
}
],
tools: [
{
type: 'expand',
tooltip: 'Expand all',
tooltipType: 'title',
callback: (tree) => tree.expandAll(),
},
{
type: 'collapse',
tooltip: 'Collapse all',
tooltipType: 'title',
callback: (tree) => tree.collapseAll(),
},
],
store: store,
width: 200,
region: 'west',
split: true,
margins: '5 0 5 5',
rootVisible: false,
listeners: {
selectionchange: function(v, selections) {
if (!selections[0])
return;
var rec = selections[0];
render_docu(rec.data);
location.hash = '#' + rec.data.path;
}
}
});
Ext.create('Ext.container.Viewport', {
layout: 'border',
renderTo: Ext.getBody(),
items: [
tree,
{
xtype: 'tabpanel',
title: 'Documentation',
id: 'docview',
region: 'center',
margins: '5 5 5 0',
layout: 'fit',
items: []
}
]
});
var deepLink = function() {
var path = window.location.hash.substring(1).replace(/\/\s*$/, '')
var endpoint = store.findNode('path', path);
if (endpoint) {
tree.getSelectionModel().select(endpoint);
tree.expandPath(endpoint.getPath());
render_docu(endpoint.data);
}
}
window.onhashchange = deepLink;
deepLink();
});

View File

@ -0,0 +1,13 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<title>Proxmox Backup Server API Documentation</title>
<link rel="stylesheet" type="text/css" href="extjs/theme-crisp/resources/theme-crisp-all.css">
<script type="text/javascript" src="extjs/ext-all.js"></script>
<script type="text/javascript" src="apidoc.js"></script>
</head>
<body></body>
</html>

View File

@ -60,33 +60,10 @@ Environment Variables
Output Format
-------------
Most commands support the ``--output-format`` parameter. It accepts
the following values:
:``text``: Text format (default). Structured data is rendered as a table.
:``json``: JSON (single line).
:``json-pretty``: JSON (multiple lines, nicely formatted).
.. include:: output-format.rst
Please use the following environment variables to modify output behavior:
``PROXMOX_OUTPUT_FORMAT``
Defines the default output format.
``PROXMOX_OUTPUT_NO_BORDER``
If set (to any value), do not render table borders.
``PROXMOX_OUTPUT_NO_HEADER``
If set (to any value), do not render table headers.
.. note:: The ``text`` format is designed to be human readable, and
not meant to be parsed by automation tools. Please use the ``json``
format if you need to process the output.
.. _creating-backups:
.. _client_creating_backups:
Creating Backups
----------------
@ -246,7 +223,7 @@ Restoring this backup will result in:
. .. file2
.. _encryption:
.. _client_encryption:
Encryption
----------
@ -353,8 +330,10 @@ To set up a master key:
.. code-block:: console
# openssl rsautl -decrypt -inkey master-private.pem -in rsa-encrypted.key -out /path/to/target
Enter pass phrase for ./master-private.pem: *********
# proxmox-backup-client key import-with-master-key /path/to/target --master-keyfile /path/to/master-private.pem --encrypted-keyfile /path/to/rsa-encrypted.key
Master Key Password: ******
New Password: ******
Verify Password: ******
7. The target file will now contain the encryption key information in plain
text. The success of this can be confirmed by passing the resulting ``json``
@ -365,9 +344,22 @@ To set up a master key:
backed up. It can happen, for example, that you back up an entire system, using
a key on that system. If the system then becomes inaccessible for any reason
and needs to be restored, this will not be possible as the encryption key will be
lost along with the broken system. In preparation for the worst case scenario,
you should consider keeping a paper copy of this key locked away in
a safe place.
lost along with the broken system.
It is recommended that you keep your master key safe, but easily accessible, in
order for quick disaster recovery. For this reason, the best place to store it
is in your password manager, where it is immediately recoverable. As a backup to
this, you should also save the key to a USB drive and store that in a secure
place. This way, it is detached from any system, but is still easy to recover
from, in case of emergency. Finally, in preparation for the worst case scenario,
you should also consider keeping a paper copy of your master key locked away in
a safe place. The ``paperkey`` subcommand can be used to create a QR encoded
version of your master key. The following command sends the output of the
``paperkey`` command to a text file, for easy printing.
.. code-block:: console
proxmox-backup-client key paperkey --output-format text > qrkey.txt
Restoring Data
@ -379,11 +371,11 @@ periodic recovery tests to ensure that you can access the data in
case of problems.
First, you need to find the snapshot which you want to restore. The snapshot
command provides a list of all the snapshots on the server:
list command provides a list of all the snapshots on the server:
.. code-block:: console
# proxmox-backup-client snapshots
# proxmox-backup-client snapshot list
┌────────────────────────────────┬─────────────┬────────────────────────────────────┐
│ snapshot │ size │ files │
╞════════════════════════════════╪═════════════╪════════════════════════════════════╡
@ -468,16 +460,15 @@ subdirectory and add the corresponding pattern to the list for subsequent restor
all files in the archive matching the patterns to ``/target/path`` on the local
host. This will scan the whole archive.
With ``restore /target/path`` you can restore the sub-archive given by the current
working directory to the local target path ``/target/path`` on your host.
By additionally passing a glob pattern with ``--pattern <glob>``, the restore is
further limited to files matching the pattern.
For example:
The ``restore`` command can be used to restore all the files contained within
the backup archive. This is most helpful when paired with the ``--pattern
<glob>`` option, as it allows you to restore all files matching a specific
pattern. For example, if you wanted to restore configuration files
located in ``/etc``, you could do the following:
.. code-block:: console
pxar:/ > cd /etc/
pxar:/etc/ > restore /target/ --pattern **/*.conf
pxar:/ > restore target/ --pattern etc/**/*.conf
...
The above will scan trough all the directories below ``/etc`` and restore all
@ -568,7 +559,7 @@ command:
.. code-block:: console
# proxmox-backup-client forget <snapshot>
# proxmox-backup-client snapshot forget <snapshot>
.. caution:: This command removes all archives in this backup
@ -642,10 +633,10 @@ shows the list of existing snapshots and what actions prune would take.
.. note:: Neither the ``prune`` command nor the ``forget`` command free space
in the chunk-store. The chunk-store still contains the data blocks. To free
space you need to perform :ref:`garbage-collection`.
space you need to perform :ref:`client_garbage-collection`.
.. _garbage-collection:
.. _client_garbage-collection:
Garbage Collection
------------------
@ -706,32 +697,34 @@ benchmark using the ``benchmark`` subcommand of ``proxmox-backup-client``:
.. code-block:: console
# proxmox-backup-client benchmark
Uploaded 656 chunks in 5 seconds.
Time per request: 7659 microseconds.
TLS speed: 547.60 MB/s
SHA256 speed: 585.76 MB/s
Compression speed: 1923.96 MB/s
Decompress speed: 7885.24 MB/s
AES256/GCM speed: 3974.03 MB/s
Uploaded 1517 chunks in 5 seconds.
Time per request: 3309 microseconds.
TLS speed: 1267.41 MB/s
SHA256 speed: 2066.73 MB/s
Compression speed: 775.11 MB/s
Decompress speed: 1233.35 MB/s
AES256/GCM speed: 3688.27 MB/s
Verify speed: 783.43 MB/s
┌───────────────────────────────────┬─────────────────────┐
│ Name │ Value │
╞═══════════════════════════════════╪═════════════════════╡
│ TLS (maximal backup upload speed) │ 547.60 MB/s (93%)
│ TLS (maximal backup upload speed) │ 1267.41 MB/s (103%) │
├───────────────────────────────────┼─────────────────────┤
│ SHA256 checksum computation speed │ 585.76 MB/s (28%)
│ SHA256 checksum computation speed │ 2066.73 MB/s (102%)
├───────────────────────────────────┼─────────────────────┤
│ ZStd level 1 compression speed │ 1923.96 MB/s (89%) │
│ ZStd level 1 compression speed │ 775.11 MB/s (103%) │
├───────────────────────────────────┼─────────────────────┤
│ ZStd level 1 decompression speed │ 7885.24 MB/s (98%)
│ ZStd level 1 decompression speed │ 1233.35 MB/s (103%) │
├───────────────────────────────────┼─────────────────────┤
AES256 GCM encryption speed │ 3974.03 MB/s (104%) │
Chunk verification speed │ 783.43 MB/s (103%)
├───────────────────────────────────┼─────────────────────┤
│ AES256 GCM encryption speed │ 3688.27 MB/s (101%) │
└───────────────────────────────────┴─────────────────────┘
.. note:: The percentages given in the output table correspond to a
comparison against a Ryzen 7 2700X. The TLS test connects to the
local host, so there is no network involved.
You can also pass the ``--output-format`` parameter to output stats in ``json``,
rather than the default table format.

View File

@ -1,19 +1,140 @@
Backup Protocol
===============
.. todo:: add introduction to HTTP2 based backup protocols
Proxmox Backup Server uses a REST based API. While the management
interface use normal HTTP, the actual backup and restore interface use
HTTP/2 for improved performance. Both HTTP and HTTP/2 are well known
standards, so the following section assumes that you are familiar on
how to use them.
Backup Protocol API
-------------------
.. todo:: describe backup writer protocol
To start a new backup, the API call ``GET /api2/json/backup`` needs to
be upgraded to a HTTP/2 connection using
``proxmox-backup-protocol-v1`` as protocol name::
.. include:: backup-protocol-api.rst
GET /api2/json/backup HTTP/1.1
UPGRADE: proxmox-backup-protocol-v1
The server replies with HTTP 101 Switching Protocol status code,
and you can then issue REST commands on that updated HTTP/2 connection.
The backup protocol allows you to upload three different kind of files:
- Chunks and blobs (binary data)
- Fixed Indexes (List of chunks with fixed size)
- Dynamic Indexes (List of chunk with variable size)
The following section gives a short introduction how to upload such
files. Please use the `API Viewer <api-viewer/index.html>`_ for
details about available REST commands.
Reader Protocol API
-------------------
Upload Blobs
~~~~~~~~~~~~
.. todo:: describe backup reader protocol
Uploading blobs is done using ``POST /blob``. The HTTP body contains the
data encoded as :ref:`Data Blob <data-blob-format>`).
.. include:: reader-protocol-api.rst
The file name needs to end with ``.blob``, and is automatically added
to the backup manifest.
Upload Chunks
~~~~~~~~~~~~~
Chunks belong to an index, so you first need to open an index (see
below). After that, you can upload chunks using ``POST /fixed_chunk``
and ``POST /dynamic_chunk``. The HTTP body contains the chunk data
encoded as :ref:`Data Blob <data-blob-format>`).
Upload Fixed Indexes
~~~~~~~~~~~~~~~~~~~~
Fixed indexes are use to store VM image data. The VM image is split
into equally sized chunks, which are uploaded individually. The index
file simply contains a list to chunk digests.
You create a fixed index with ``POST /fixed_index``. Then upload
chunks with ``POST /fixed_chunk``, and append them to the index with
``PUT /fixed_index``. When finished, you need to close the index using
``POST /fixed_close``.
The file name needs to end with ``.fidx``, and is automatically added
to the backup manifest.
Upload Dynamic Indexes
~~~~~~~~~~~~~~~~~~~~~~
Dynamic indexes are use to store file archive data. The archive data
is split into dynamically sized chunks, which are uploaded
individually. The index file simply contains a list to chunk digests
and offsets.
You create a dynamic sized index with ``POST /dynamic_index``. Then
upload chunks with ``POST /dynamic_chunk``, and append them to the index with
``PUT /dynamic_index``. When finished, you need to close the index using
``POST /dynamic_close``.
The file name needs to end with ``.didx``, and is automatically added
to the backup manifest.
Finish Backup
~~~~~~~~~~~~~
Once you have uploaded all data, you need to call ``POST
/finish``. This commits all data and ends the backup protocol.
Restore/Reader Protocol API
---------------------------
To start a new reader, the API call ``GET /api2/json/reader`` needs to
be upgraded to a HTTP/2 connection using
``proxmox-backup-reader-protocol-v1`` as protocol name::
GET /api2/json/reader HTTP/1.1
UPGRADE: proxmox-backup-reader-protocol-v1
The server replies with HTTP 101 Switching Protocol status code,
and you can then issue REST commands on that updated HTTP/2 connection.
The reader protocol allows you to download three different kind of files:
- Chunks and blobs (binary data)
- Fixed Indexes (List of chunks with fixed size)
- Dynamic Indexes (List of chunk with variable size)
The following section gives a short introduction how to download such
files. Please use the `API Viewer <api-viewer/index.html>`_ for details about
available REST commands.
Download Blobs
~~~~~~~~~~~~~~
Downloading blobs is done using ``GET /download``. The HTTP body contains the
data encoded as :ref:`Data Blob <data-blob-format>`.
Download Chunks
~~~~~~~~~~~~~~~
Downloading chunks is done using ``GET /chunk``. The HTTP body contains the
data encoded as :ref:`Data Blob <data-blob-format>`).
Download Index Files
~~~~~~~~~~~~~~~~~~~~
Downloading index files is done using ``GET /download``. The HTTP body
contains the data encoded as :ref:`Fixed Index <fixed-index-format>`
or :ref:`Dynamic Index <dynamic-index-format>`.

View File

@ -1,5 +1,4 @@
.. _calendar-events:
.. _calendar-event-scheduling:
Calendar Events
===============

View File

@ -74,7 +74,7 @@ rst_epilog = epilog_file.read()
# General information about the project.
project = 'Proxmox Backup'
copyright = '2019-2020, Proxmox Server Solutions GmbH'
copyright = '2019-2021, Proxmox Server Solutions GmbH'
author = 'Proxmox Support Team'
# The version info for the project you're documenting, acts as replacement for
@ -107,10 +107,8 @@ today_fmt = '%A, %d %B %Y'
# This patterns also effect to html_static_path and html_extra_path
exclude_patterns = [
'_build', 'Thumbs.db', '.DS_Store',
'proxmox-backup-client/man1.rst',
'proxmox-backup-manager/man1.rst',
'proxmox-backup-proxy/man1.rst',
'pxar/man1.rst',
'*/man1.rst',
'config/*/man5.rst',
'epilog.rst',
'pbs-copyright.rst',
'local-zfs.rst'
@ -171,7 +169,9 @@ html_theme_options = {
'extra_nav_links': {
'Proxmox Homepage': 'https://proxmox.com',
'PDF': 'proxmox-backup.pdf',
'API Viewer' : 'api-viewer/index.html',
'Prune Simulator' : 'prune-simulator/index.html',
'LTO Barcode Generator' : 'lto-barcode/index.html',
},
'sidebar_width': '320px',
@ -245,10 +245,8 @@ html_js_files = [
#
# html_last_updated_fmt = None
# If true, SmartyPants will be used to convert quotes and dashes to
# typographically correct entities.
#
# html_use_smartypants = True
# We need to disable smatquotes, else Option Lists do not display long options
smartquotes = False
# Additional templates that should be rendered to pages, maps page names to
# template names.
@ -466,3 +464,6 @@ epub_exclude_files = ['search.html']
# If false, no index is generated.
#
# epub_use_index = True
# use local mathjax package, symlink comes from debian/proxmox-backup-docs.links
mathjax_path = "mathjax/MathJax.js?config=TeX-AMS-MML_HTMLorMML"

View File

@ -0,0 +1,22 @@
This file contains the access control list for the Proxmox Backup
Server API.
Each line starts with ``acl:``, followed by 4 additional values
separated by collon.
:propagate: Propagate permissions down the hierachrchy
:path: The object path
:User/Token: List of users and token
:Role: List of assigned roles
Here is an example list::
acl:1:/:root@pam!test:Admin
acl:1:/datastore/store1:user1@pbs:DatastoreAdmin
You can use the ``proxmox-backup-manager acl`` command to manipulate
this file.

35
docs/config/acl/man5.rst Normal file
View File

@ -0,0 +1,35 @@
==========================
acl.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Access Control Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/user.cfg is a configuration file for Proxmox
Backup Server. It contains the access control configuration for the API.
File Format
===========
.. include:: format.rst
Roles
=====
The following roles exist:
.. include:: roles.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,18 @@
The file contains a list of datastore configuration sections. Each
section starts with a header ``datastore: <name>``, followed by the
datastore configuration options.
::
datastore: <name1>
path <path1>
<option1> <value1>
...
datastore: <name2>
path <path2>
...
You can use the ``proxmox-backup-manager datastore`` command to manipulate
this file.

View File

@ -0,0 +1,33 @@
==========================
datastore.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Datastore Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/datastore.cfg is a configuration file for Proxmox
Backup Server. It contains the Datastore configuration.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,13 @@
Each entry starts with a header ``pool: <name>``, followed by the
media pool configuration options.
::
pool: company1
allocation always
retention overwrite
pool: ...
You can use the ``proxmox-tape pool`` command to manipulate this file.

View File

@ -0,0 +1,35 @@
==========================
media-pool.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Media Pool Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/media-pool.cfg is a configuration file
for Proxmox Backup Server. It contains the medila pool configuration
for tape backups.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,17 @@
This file contains information used to access remote servers.
Each entry starts with a header ``remote: <name>``, followed by the
remote configuration options.
::
remote: server1
host server1.local
auth-id sync@pbs
...
remote: ...
You can use the ``proxmox-backup-manager remote`` command to manipulate
this file.

View File

@ -0,0 +1,35 @@
==========================
remote.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Remote Server Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/remote.cfg is a configuration file for
Proxmox Backup Server. It contains information about remote servers,
usable for synchronization jobs.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,15 @@
Each entry starts with a header ``sync: <name>``, followed by the
job configuration options.
::
sync: job1
store store1
remote-store store1
remote lina
sync: ...
You can use the ``proxmox-backup-manager sync-job`` command to manipulate
this file.

35
docs/config/sync/man5.rst Normal file
View File

@ -0,0 +1,35 @@
==========================
sync.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Synchronization Job Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/sync.cfg is a configuration file for
Proxmox Backup Server. It contains the synchronization job
configuration.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,16 @@
Each entry starts with a header ``backup: <name>``, followed by the
job configuration options.
::
backup: job1
drive hh8
pool p4
store store3
schedule daily
backup: ...
You can use the ``proxmox-tape backup-job`` command to manipulate
this file.

View File

@ -0,0 +1,34 @@
==========================
tape-job.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Tape Job Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file ``/etc/proxmox-backup/tape-job.cfg`` is a configuration file for
Proxmox Backup Server. It contains the tape job configuration.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,22 @@
Each drive configuration section starts with a header ``linux: <name>``,
followed by the drive configuration options.
Tape changer configurations starts with ``changer: <name>``,
followed by the changer configuration options.
::
linux: hh8
changer sl3
path /dev/tape/by-id/scsi-10WT065325-nst
changer: sl3
export-slots 14,15,16
path /dev/tape/by-id/scsi-CJ0JBE0059
You can use the ``proxmox-tape drive`` and ``proxmox-tape changer``
commands to manipulate this file.
.. NOTE:: The ``virtual:`` drive type is experimental and onyl used
for debugging.

33
docs/config/tape/man5.rst Normal file
View File

@ -0,0 +1,33 @@
==========================
tape.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Tape Drive and Changer Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/tape.cfg is a configuration file for Proxmox
Backup Server. It contains the tape drive and changer configuration.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,28 @@
This file contains the list of API users and API tokens.
Each user configuration section starts with a header ``user: <name>``,
followed by the user configuration options.
API token configuration starts with a header ``token:
<userid!token_name>``, followed by the token configuration. The data
used to authenticate tokens is stored in a separate file
(``token.shadow``).
::
user: root@pam
comment Superuser
email test@example.local
...
token: root@pam!token1
comment API test token
enable true
expire 0
user: ...
You can use the ``proxmox-backup-manager user`` command to manipulate
this file.

33
docs/config/user/man5.rst Normal file
View File

@ -0,0 +1,33 @@
==========================
user.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
User Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/user.cfg is a configuration file for Proxmox
Backup Server. It contains the user configuration.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,16 @@
Each entry starts with a header ``verification: <name>``, followed by the
job configuration options.
::
verification: verify-store2
ignore-verified true
outdated-after 7
schedule daily
store store2
verification: ...
You can use the ``proxmox-backup-manager verify-job`` command to manipulate
this file.

View File

@ -0,0 +1,35 @@
==========================
verification.cfg
==========================
.. include:: ../../epilog.rst
-------------------------------------------------------------
Verification Job Configuration
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 5
Description
===========
The file /etc/proxmox-backup/sync.cfg is a configuration file for
Proxmox Backup Server. It contains the verification job
configuration.
File Format
===========
.. include:: format.rst
Options
=======
.. include:: config.rst
.. include:: ../../pbs-copyright.rst

View File

@ -0,0 +1,97 @@
Configuration Files
===================
All Proxmox Backup Server configuration files resides inside directory
``/etc/proxmox-backup/``.
``acl.cfg``
~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/acl/format.rst
Roles
^^^^^
The following roles exist:
.. include:: config/acl/roles.rst
``datastore.cfg``
~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/datastore/format.rst
Options
^^^^^^^
.. include:: config/datastore/config.rst
``user.cfg``
~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/user/format.rst
Options
^^^^^^^
.. include:: config/user/config.rst
``remote.cfg``
~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/remote/format.rst
Options
^^^^^^^
.. include:: config/remote/config.rst
``sync.cfg``
~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/sync/format.rst
Options
^^^^^^^
.. include:: config/sync/config.rst
``verification.cfg``
~~~~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/verification/format.rst
Options
^^^^^^^
.. include:: config/verification/config.rst

View File

@ -14,6 +14,10 @@ pre {
padding: 5px 10px;
}
div.topic {
background-color: #FAFAFA;
}
li a.current {
font-weight: bold;
border-bottom: 1px solid #000;
@ -25,6 +29,23 @@ ul li.toctree-l1 > a {
color: #000;
}
div.sphinxsidebar ul {
color: #444;
}
div.sphinxsidebar ul ul {
list-style: circle;
}
div.sphinxsidebar ul ul ul {
list-style: square;
}
div.sphinxsidebar ul a code {
font-weight: normal;
}
div.sphinxsidebar ul ul a {
border-bottom: 1px dotted #CCC;
}
div.sphinxsidebar form.search {
margin-bottom: 5px;
}

View File

@ -27,7 +27,7 @@ How long will my Proxmox Backup Server version be supported?
+-----------------------+--------------------+---------------+------------+--------------------+
|Proxmox Backup Version | Debian Version | First Release | Debian EOL | Proxmox Backup EOL |
+=======================+====================+===============+============+====================+
|Proxmox Backup 1.x | Debian 10 (Buster) | tba | tba | tba |
|Proxmox Backup 1.x | Debian 10 (Buster) | 2020-11 | tba | tba |
+-----------------------+--------------------+---------------+------------+--------------------+
@ -53,9 +53,12 @@ checksums. This manifest file is used to verify the integrity of each backup.
When backing up to remote servers, do I have to trust the remote server?
------------------------------------------------------------------------
Proxmox Backup Server supports client-side encryption, meaning your data is
encrypted before it reaches the server. Thus, in the event that an attacker
gains access to the server, they will not be able to read the data.
Proxmox Backup Server transfers data via `Transport Layer Security (TLS)
<https://en.wikipedia.org/wiki/Transport_Layer_Security>`_ and additionally
supports client-side encryption. This means that data is transferred securely
and can be encrypted before it reaches the server. Thus, in the event that an
attacker gains access to the server or any point of the network, they will not
be able to read the data.
.. note:: Encryption is not enabled by default. To set up encryption, see the
`Encryption

View File

@ -6,7 +6,113 @@ File Formats
Proxmox File Archive Format (``.pxar``)
---------------------------------------
.. graphviz:: pxar-format-overview.dot
.. _data-blob-format:
Data Blob Format (``.blob``)
----------------------------
The data blob format is used to store small binary data. The magic number decides the exact format:
.. list-table::
:widths: auto
* - ``[66, 171, 56, 7, 190, 131, 112, 161]``
- unencrypted
- uncompressed
* - ``[49, 185, 88, 66, 111, 182, 163, 127]``
- unencrypted
- compressed
* - ``[123, 103, 133, 190, 34, 45, 76, 240]``
- encrypted
- uncompressed
* - ``[230, 89, 27, 191, 11, 191, 216, 11]``
- encrypted
- compressed
Compression algorithm is ``zstd``. Encryption cipher is ``AES_256_GCM``.
Unencrypted blobs use the following format:
.. list-table::
:widths: auto
* - ``MAGIC: [u8; 8]``
* - ``CRC32: [u8; 4]``
* - ``Data: (max 16MiB)``
Encrypted blobs additionally contains a 16 byte IV, followed by a 16
byte Authenticated Encyryption (AE) tag, followed by the encrypted
data:
.. list-table::
* - ``MAGIC: [u8; 8]``
* - ``CRC32: [u8; 4]``
* - ``ÌV: [u8; 16]``
* - ``TAG: [u8; 16]``
* - ``Data: (max 16MiB)``
.. _fixed-index-format:
Fixed Index Format (``.fidx``)
-------------------------------
All numbers are stored as little-endian.
.. list-table::
* - ``MAGIC: [u8; 8]``
- ``[47, 127, 65, 237, 145, 253, 15, 205]``
* - ``uuid: [u8; 16]``,
- Unique ID
* - ``ctime: i64``,
- Creation Time (epoch)
* - ``index_csum: [u8; 32]``,
- Sha256 over the index (without header) ``SHA256(digest1||digest2||...)``
* - ``size: u64``,
- Image size
* - ``chunk_size: u64``,
- Chunk size
* - ``reserved: [u8; 4016]``,
- overall header size is one page (4096 bytes)
* - ``digest1: [u8; 32]``
- first chunk digest
* - ``digest2: [u8; 32]``
- next chunk
* - ...
- next chunk ...
.. _dynamic-index-format:
Dynamic Index Format (``.didx``)
--------------------------------
All numbers are stored as little-endian.
.. list-table::
* - ``MAGIC: [u8; 8]``
- ``[28, 145, 78, 165, 25, 186, 179, 205]``
* - ``uuid: [u8; 16]``,
- Unique ID
* - ``ctime: i64``,
- Creation Time (epoch)
* - ``index_csum: [u8; 32]``,
- Sha256 over the index (without header) ``SHA256(offset1||digest1||offset2||digest2||...)``
* - ``reserved: [u8; 4032]``,
- Overall header size is one page (4096 bytes)
* - ``offset1: u64``
- End of first chunk
* - ``digest1: [u8; 32]``
- first chunk digest
* - ``offset2: u64``
- End of second chunk
* - ``digest2: [u8; 32]``
- second chunk digest
* - ...
- next chunk offset/digest

View File

@ -129,8 +129,8 @@ top panel to view:
* **Content**: Information on the datastore's backup groups and their respective
contents
* **Prune & GC**: Schedule :ref:`pruning <backup-pruning>` and :ref:`garbage
collection <garbage-collection>` operations, and run garbage collection
collection <client_garbage-collection>` operations, and run garbage collection
manually
* **Sync Jobs**: Create, manage and run :ref:`syncjobs` from remote servers
* **Verify Jobs**: Create, manage and run :ref:`verification` jobs on the
* **Verify Jobs**: Create, manage and run :ref:`maintenance_verification` jobs on the
datastore

Binary file not shown.

After

Width:  |  Height:  |  Size: 36 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 31 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 16 KiB

View File

@ -2,7 +2,7 @@
Welcome to the Proxmox Backup documentation!
============================================
| Copyright (C) 2019-2020 Proxmox Server Solutions GmbH
| Copyright (C) 2019-2021 Proxmox Server Solutions GmbH
| Version |version| -- |today|
Permission is granted to copy, distribute and/or modify this document under the
@ -33,6 +33,7 @@ in the section entitled "GNU Free Documentation License".
pve-integration.rst
pxar-tool.rst
sysadmin.rst
technical-overview.rst
faq.rst
.. raw:: latex
@ -44,6 +45,7 @@ in the section entitled "GNU Free Documentation License".
:caption: Appendix
command-syntax.rst
configuration-files.rst
file-formats.rst
backup-protocol.rst
calendarevents.rst

View File

@ -37,16 +37,15 @@ Download the ISO from |DOWNLOADS|.
It includes the following:
* The `Proxmox Backup`_ server installer, which partitions the local
disk(s) with ext4, ext3, xfs or ZFS, and installs the operating
system
disk(s) with ext4, xfs or ZFS, and installs the operating system
* Complete operating system (Debian Linux, 64-bit)
* Our Linux kernel with ZFS support
* Proxmox Linux kernel with ZFS support
* Complete tool-set to administer backups and all necessary resources
* Web based GUI management interface
* Web based management interface
.. note:: During the installation process, the complete server
is used by default and all existing data is removed.

View File

@ -14,11 +14,12 @@ It supports deduplication, compression, and authenticated
encryption (AE_). Using :term:`Rust` as the implementation language guarantees high
performance, low resource usage, and a safe, high-quality codebase.
Proxmox Backup uses state of the art cryptography for client communication and
backup content :ref:`encryption <encryption>`. Encryption is done on the
client side, making it safer to back up data to targets that are not fully
trusted.
Proxmox Backup uses state of the art cryptography for both client-server
communication and backup content :ref:`encryption <client_encryption>`. All
client-server communication uses `TLS
<https://en.wikipedia.org/wiki/Transport_Layer_Security>`_, and backup data can
be encrypted on the client-side before sending, making it safer to back up data
to targets that are not fully trusted.
Architecture
------------
@ -64,9 +65,10 @@ Main Features
:Compression: The ultra-fast Zstandard_ compression is able to compress
several gigabytes of data per second.
:Encryption: Backups can be encrypted on the client-side, using AES-256 in
Galois/Counter Mode (GCM_) mode. This authenticated encryption (AE_) mode
provides very high performance on modern hardware.
:Encryption: Backups can be encrypted on the client-side, using AES-256 GCM_.
This authenticated encryption (AE_) mode provides very high performance on
modern hardware. In addition to client-side encryption, all data is
transferred via a secure TLS connection.
:Web interface: Manage the Proxmox Backup Server with the integrated, web-based
user interface.
@ -74,8 +76,16 @@ Main Features
:Open Source: No secrets. Proxmox Backup Server is free and open-source
software. The source code is licensed under AGPL, v3.
:Support: Enterprise support will be available from `Proxmox`_ once the beta
phase is over.
:No Limits: Proxmox Backup Server has no artifical limits for backup storage or
backup-clients.
:Enterprise Support: Proxmox Server Solutions GmbH offers enterprise support in
form of `Proxmox Backup Server Subscription Plans
<https://www.proxmox.com/en/proxmox-backup-server/pricing>`_. Users at every
subscription level get access to the Proxmox Backup :ref:`Enterprise
Repository <sysadmin_package_repos_enterprise>`. In addition, with a Basic,
Standard or Premium subscription, users have access to the :ref:`Proxmox
Customer Portal <get_help_enterprise_support>`.
Reasons for Data Backup?
@ -115,8 +125,8 @@ Proxmox Backup Server consists of multiple components:
* A client CLI tool (`proxmox-backup-client`) to access the server easily from
any `Linux amd64` environment
Aside from the web interface, everything is written in the Rust programming
language.
Aside from the web interface, most parts of Proxmox Backup Server are written in
the Rust programming language.
"The Rust programming language helps you write faster, more reliable software.
High-level ergonomics and low-level control are often at odds in programming
@ -127,12 +137,22 @@ language.
-- `The Rust Programming Language <https://doc.rust-lang.org/book/ch00-00-introduction.html>`_
.. todo:: further explain the software stack
.. _get_help:
Getting Help
------------
.. _get_help_enterprise_support:
Enterprise Support
~~~~~~~~~~~~~~~~~~
Users with a `Proxmox Backup Server Basic, Standard or Premium Subscription Plan
<https://www.proxmox.com/en/proxmox-backup-server/pricing>`_ have access to the
Proxmox Customer Portal. The Customer Portal provides support with guaranteed
response times from the Proxmox developers.
For more information or for volume discounts, please contact office@proxmox.com.
Community Support Forum
~~~~~~~~~~~~~~~~~~~~~~~
@ -160,7 +180,7 @@ of the issue and will send a notification once it has been solved.
License
-------
Copyright (C) 2019-2020 Proxmox Server Solutions GmbH
Copyright (C) 2019-2021 Proxmox Server Solutions GmbH
This software is written by Proxmox Server Solutions GmbH <support@proxmox.com>

344
docs/lto-barcode/code39.js Normal file
View File

@ -0,0 +1,344 @@
// Code39 barcode generator
// see https://en.wikipedia.org/wiki/Code_39
// IBM LTO Ultrium Cartridge Label Specification
// http://www-01.ibm.com/support/docview.wss?uid=ssg1S7000429
const code39_codes = {
"1": ['B', 's', 'b', 'S', 'b', 's', 'b', 's', 'B'],
"A": ['B', 's', 'b', 's', 'b', 'S', 'b', 's', 'B'],
"K": ['B', 's', 'b', 's', 'b', 's', 'b', 'S', 'B'],
"U": ['B', 'S', 'b', 's', 'b', 's', 'b', 's', 'B'],
"2": ['b', 's', 'B', 'S', 'b', 's', 'b', 's', 'B'],
"B": ['b', 's', 'B', 's', 'b', 'S', 'b', 's', 'B'],
"L": ['b', 's', 'B', 's', 'b', 's', 'b', 'S', 'B'],
"V": ['b', 'S', 'B', 's', 'b', 's', 'b', 's', 'B'],
"3": ['B', 's', 'B', 'S', 'b', 's', 'b', 's', 'b'],
"C": ['B', 's', 'B', 's', 'b', 'S', 'b', 's', 'b'],
"M": ['B', 's', 'B', 's', 'b', 's', 'b', 'S', 'b'],
"W": ['B', 'S', 'B', 's', 'b', 's', 'b', 's', 'b'],
"4": ['b', 's', 'b', 'S', 'B', 's', 'b', 's', 'B'],
"D": ['b', 's', 'b', 's', 'B', 'S', 'b', 's', 'B'],
"N": ['b', 's', 'b', 's', 'B', 's', 'b', 'S', 'B'],
"X": ['b', 'S', 'b', 's', 'B', 's', 'b', 's', 'B'],
"5": ['B', 's', 'b', 'S', 'B', 's', 'b', 's', 'b'],
"E": ['B', 's', 'b', 's', 'B', 'S', 'b', 's', 'b'],
"O": ['B', 's', 'b', 's', 'B', 's', 'b', 'S', 'b'],
"Y": ['B', 'S', 'b', 's', 'B', 's', 'b', 's', 'b'],
"6": ['b', 's', 'B', 'S', 'B', 's', 'b', 's', 'b'],
"F": ['b', 's', 'B', 's', 'B', 'S', 'b', 's', 'b'],
"P": ['b', 's', 'B', 's', 'B', 's', 'b', 'S', 'b'],
"Z": ['b', 'S', 'B', 's', 'B', 's', 'b', 's', 'b'],
"7": ['b', 's', 'b', 'S', 'b', 's', 'B', 's', 'B'],
"G": ['b', 's', 'b', 's', 'b', 'S', 'B', 's', 'B'],
"Q": ['b', 's', 'b', 's', 'b', 's', 'B', 'S', 'B'],
"-": ['b', 'S', 'b', 's', 'b', 's', 'B', 's', 'B'],
"8": ['B', 's', 'b', 'S', 'b', 's', 'B', 's', 'b'],
"H": ['B', 's', 'b', 's', 'b', 'S', 'B', 's', 'b'],
"R": ['B', 's', 'b', 's', 'b', 's', 'B', 'S', 'b'],
".": ['B', 'S', 'b', 's', 'b', 's', 'B', 's', 'b'],
"9": ['b', 's', 'B', 'S', 'b', 's', 'B', 's', 'b'],
"I": ['b', 's', 'B', 's', 'b', 'S', 'B', 's', 'b'],
"S": ['b', 's', 'B', 's', 'b', 's', 'B', 'S', 'b'],
" ": ['b', 'S', 'B', 's', 'b', 's', 'B', 's', 'b'],
"0": ['b', 's', 'b', 'S', 'B', 's', 'B', 's', 'b'],
"J": ['b', 's', 'b', 's', 'B', 'S', 'B', 's', 'b'],
"T": ['b', 's', 'b', 's', 'B', 's', 'B', 'S', 'b'],
"*": ['b', 'S', 'b', 's', 'B', 's', 'B', 's', 'b'],
};
const colors = [
'#BB282E',
'#FAE54A',
'#9AC653',
'#01A5E2',
'#9EAAB6',
'#D97E35',
'#E27B99',
'#67A945',
'#F6B855',
'#705A81',
];
const lto_label_width = 70;
const lto_label_height = 16.9;
function foreach_label(page_layout, callback) {
let count = 0;
let row = 0;
let height = page_layout.margin_top;
while ((height + page_layout.label_height) <= page_layout.page_height) {
let column = 0;
let width = page_layout.margin_left;
while ((width + page_layout.label_width) <= page_layout.page_width) {
callback(column, row, count, width, height);
count += 1;
column += 1;
width += page_layout.label_width;
width += page_layout.column_spacing;
}
row += 1;
height += page_layout.label_height;
height += page_layout.row_spacing;
}
}
function compute_max_labels(page_layout) {
let max_labels = 0;
foreach_label(page_layout, function() { max_labels += 1; });
return max_labels;
}
function svg_label(mode, label, label_type, pagex, pagey, label_borders) {
let svg = "";
if (label.length !== 6) {
throw "wrong label length";
}
if (label_type.length !== 2) {
throw "wrong label_type length";
}
let ratio = 2.75;
let parts = 3*ratio + 6; // 3*wide + 6*small;
let barcode_width = (lto_label_width/12)*10; // 10*code + 2margin
let small = barcode_width/(parts*10 + 9);
let code_width = small*parts;
let wide = small*ratio;
let xpos = pagex + code_width;
let height = 12;
let label_rect = `x='${pagex}' y='${pagey}' width='${lto_label_width}' height='${lto_label_height}'`;
if (mode === 'placeholder') {
if (label_borders) {
svg += `<rect class='unprintable' ${label_rect} fill='none' style='stroke:black;stroke-width:0.1;'/>`;
}
return svg;
}
if (label_borders) {
svg += `<rect ${label_rect} fill='none' style='stroke:black;stroke-width:0.1;'/>`;
}
if (mode === "color" || mode === "frame") {
let w = lto_label_width/8;
let h = lto_label_height - height;
for (let i = 0; i < 7; i++) {
let textx = w/2 + pagex + i*w;
let texty = pagey;
let fill = "none";
if (mode === "color" && (i < 6)) {
let letter = label.charAt(i);
if (letter >= '0' && letter <= '9') {
fill = colors[parseInt(letter, 10)];
}
}
svg += `<rect x='${textx}' y='${texty}' width='${w}' height='${h}' style='stroke:black;stroke-width:0.2;fill:${fill};'/>`;
if (i == 6) {
textx += 3;
texty += 3.7;
svg += `<text x='${textx}' y='${texty}' style='font-weight:bold;font-size:3px;font-family:sans-serif;'>${label_type}</text>`;
} else {
let letter = label.charAt(i);
textx += 3.5;
texty += 4;
svg += `<text x='${textx}' y='${texty}' style='font-weight:bold;font-size:4px;font-family:sans-serif;'>${letter}</text>`;
}
}
}
let raw_label = `*${label}${label_type}*`;
for (let i = 0; i < raw_label.length; i++) {
let letter = raw_label.charAt(i);
let code = code39_codes[letter];
if (code === undefined) {
throw `unable to encode letter '${letter}' with code39`;
}
if (mode === "simple") {
let textx = xpos + code_width/2;
let texty = pagey + 4;
if (i > 0 && (i+1) < raw_label.length) {
svg += `<text x='${textx}' y='${texty}' style='font-weight:bold;font-size:4px;font-family:sans-serif;'>${letter}</text>`;
}
}
for (let c of code) {
if (c === 's') {
xpos += small;
continue;
}
if (c === 'S') {
xpos += wide;
continue;
}
let w = c === 'B' ? wide : small;
let ypos = pagey + lto_label_height - height;
svg += `<rect x='${xpos}' y='${ypos}' width='${w}' height='${height}' style='fill:black'/>`;
xpos = xpos + w;
}
xpos += small;
}
return svg;
}
function html_page_header() {
let html = "<html5>";
html += "<style>";
/* no page margins */
html += "@page{margin-left: 0px;margin-right: 0px;margin-top: 0px;margin-bottom: 0px;}";
/* to hide things on printed page */
html += "@media print { .unprintable { visibility: hidden; } }";
html += "</style>";
//html += "<body onload='window.print()'>";
html += "<body style='background-color: white;'>";
return html;
}
function svg_page_header(page_width, page_height) {
let svg = "<svg version='1.1' xmlns='http://www.w3.org/2000/svg'";
svg += ` width='${page_width}mm' height='${page_height}mm' viewBox='0 0 ${page_width} ${page_height}'>`;
return svg;
}
function printBarcodePage() {
let frame = document.getElementById("print_frame");
let window = frame.contentWindow;
window.print();
}
function generate_barcode_page(target_id, page_layout, label_list, calibration) {
let svg = svg_page_header(page_layout.page_width, page_layout.page_height);
let c = calibration;
console.log(calibration);
svg += "<g id='barcode_page'";
if (c !== undefined) {
svg += ` transform='scale(${c.scalex}, ${c.scaley}),translate(${c.offsetx}, ${c.offsety})'`;
}
svg += '>';
foreach_label(page_layout, function(column, row, count, xpos, ypos) {
if (count >= label_list.length) { return; }
let item = label_list[count];
svg += svg_label(item.mode, item.label, item.tape_type, xpos, ypos, page_layout.label_borders);
});
svg += "</g>";
svg += "</svg>";
let html = html_page_header();
html += svg;
html += "</body>";
html += "</html>";
let frame = document.getElementById(target_id);
setupPrintFrame(frame, page_layout.page_width, page_layout.page_height);
let fwindow = frame.contentWindow;
fwindow.document.open();
fwindow.document.write(html);
fwindow.document.close();
}
function setupPrintFrame(frame, page_width, page_height) {
let dpi = 98;
let dpr = window.devicePixelRatio;
if (dpr !== undefined) {
dpi = dpi*dpr;
}
let ppmm = dpi/25.4;
frame.width = page_width*ppmm;
frame.height = page_height*ppmm;
}
function generate_calibration_page(target_id, page_layout, calibration) {
let frame = document.getElementById(target_id);
setupPrintFrame(frame, page_layout.page_width, page_layout.page_height);
let svg = svg_page_header(page_layout.page_width, page_layout.page_height);
svg += "<defs>";
svg += "<marker id='endarrow' markerWidth='10' markerHeight='7' ";
svg += "refX='10' refY='3.5' orient='auto'><polygon points='0 0, 10 3.5, 0 7' />";
svg += "</marker>";
svg += "<marker id='startarrow' markerWidth='10' markerHeight='7' ";
svg += "refX='0' refY='3.5' orient='auto'><polygon points='10 0, 10 7, 0 3.5' />";
svg += "</marker>";
svg += "</defs>";
svg += "<rect x='50' y='50' width='100' height='100' style='fill:none;stroke-width:0.05;stroke:rgb(0,0,0)'/>";
let text_style = "style='font-weight:bold;font-size:4;font-family:sans-serif;'";
svg += `<text x='10' y='99' ${text_style}>Sx = 50mm</text>`;
svg += "<line x1='0' y1='100' x2='50' y2='100' stroke='#000' marker-end='url(#endarrow)' stroke-width='.25'/>";
svg += `<text x='60' y='99' ${text_style}>Dx = 100mm</text>`;
svg += "<line x1='50' y1='100' x2='150' y2='100' stroke='#000' marker-start='url(#startarrow)' marker-end='url(#endarrow)' stroke-width='.25'/>";
svg += `<text x='142' y='10' ${text_style} writing-mode='tb'>Sy = 50mm</text>`;
svg += "<line x1='140' y1='0' x2='140' y2='50' stroke='#000' marker-end='url(#endarrow)' stroke-width='.25'/>";
svg += `<text x='142' y='60' ${text_style} writing-mode='tb'>Dy = 100mm</text>`;
svg += "<line x1='140' y1='50' x2='140' y2='150' stroke='#000' marker-start='url(#startarrow)' marker-end='url(#endarrow)' stroke-width='.25'/>";
let c = calibration;
if (c !== undefined) {
svg += `<rect x='50' y='50' width='100' height='100' style='fill:none;stroke-width:0.05;stroke:rgb(255,0,0)' `;
svg += `transform='scale(${c.scalex}, ${c.scaley}),translate(${c.offsetx}, ${c.offsety})'/>`;
}
svg += "</svg>";
let html = html_page_header();
html += svg;
html += "</body>";
html += "</html>";
let fwindow = frame.contentWindow;
fwindow.document.open();
fwindow.document.write(html);
fwindow.document.close();
}

View File

@ -0,0 +1,51 @@
<!DOCTYPE html>
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1, maximum-scale=1, user-scalable=no">
<title>Proxmox LTO Barcode Label Generator</title>
<link rel="stylesheet" type="text/css" href="extjs/theme-crisp/resources/theme-crisp-all.css">
<style>
/* fix action column icons */
.x-action-col-icon {
font-size: 13px;
height: 13px;
}
.x-grid-cell-inner-action-col {
padding: 6px 10px 5px;
}
.x-action-col-icon:before {
color: #555;
}
.x-action-col-icon {
color: #21BF4B;
}
.x-action-col-icon {
margin: 0 1px;
font-size: 14px;
}
.x-action-col-icon:before, .x-action-col-icon:after {
font-size: 14px;
}
.x-action-col-icon:hover:before, .x-action-col-icon:hover:after {
text-shadow: 1px 1px 1px #AAA;
font-weight: 800;
}
</style>
<link rel="stylesheet" type="text/css" href="font-awesome/css/font-awesome.css"/>
<script type="text/javascript" src="extjs/ext-all.js"></script>
<script type="text/javascript" src="code39.js"></script>
<script type="text/javascript" src="prefix-field.js"></script>
<script type="text/javascript" src="label-style.js"></script>
<script type="text/javascript" src="tape-type.js"></script>
<script type="text/javascript" src="paper-size.js"></script>
<script type="text/javascript" src="page-layout.js"></script>
<script type="text/javascript" src="page-calibration.js"></script>
<script type="text/javascript" src="label-list.js"></script>
<script type="text/javascript" src="label-setup.js"></script>
<script type="text/javascript" src="lto-barcode.js"></script>
</head>
<body>
</body>
</html>

View File

@ -0,0 +1,140 @@
Ext.define('LabelList', {
extend: 'Ext.grid.Panel',
alias: 'widget.labelList',
plugins: {
ptype: 'cellediting',
clicksToEdit: 1,
},
selModel: 'cellmodel',
store: {
field: [
'prefix',
'tape_type',
{
type: 'integer',
name: 'start',
},
{
type: 'integer',
name: 'end',
},
],
data: [],
},
listeners: {
validateedit: function(editor, context) {
console.log(context.field);
console.log(context.value);
context.record.set(context.field, context.value);
context.record.commit();
return true;
},
},
columns: [
{
text: 'Prefix',
dataIndex: 'prefix',
flex: 1,
editor: {
xtype: 'prefixfield',
allowBlank: false,
},
renderer: function(value, metaData, record) {
console.log(record);
if (record.data.mode === 'placeholder') {
return "-";
}
return value;
},
},
{
text: 'Type',
dataIndex: 'tape_type',
flex: 1,
editor: {
xtype: 'ltoTapeType',
allowBlank: false,
},
renderer: function(value, metaData, record) {
console.log(record);
if (record.data.mode === 'placeholder') {
return "-";
}
return value;
},
},
{
text: 'Mode',
dataIndex: 'mode',
flex: 1,
editor: {
xtype: 'ltoLabelStyle',
allowBlank: false,
},
},
{
text: 'Start',
dataIndex: 'start',
flex: 1,
editor: {
xtype: 'numberfield',
allowBlank: false,
},
},
{
text: 'End',
dataIndex: 'end',
flex: 1,
editor: {
xtype: 'numberfield',
},
renderer: function(value) {
if (value === null || value === '' || value === undefined) {
return "Fill";
}
return value;
},
},
{
xtype: 'actioncolumn',
width: 75,
items: [
{
tooltip: 'Move Up',
iconCls: 'fa fa-arrow-up',
handler: function(grid, rowIndex) {
if (rowIndex < 1) { return; }
let store = grid.getStore();
let record = store.getAt(rowIndex);
store.removeAt(rowIndex);
store.insert(rowIndex - 1, record);
},
},
{
tooltip: 'Move Down',
iconCls: 'fa fa-arrow-down',
handler: function(grid, rowIndex) {
let store = grid.getStore();
if (rowIndex >= store.getCount()) { return; }
let record = store.getAt(rowIndex);
store.removeAt(rowIndex);
store.insert(rowIndex + 1, record);
},
},
{
tooltip: 'Delete',
iconCls: 'fa fa-scissors',
//iconCls: 'fa critical fa-trash-o',
handler: function(grid, rowIndex) {
grid.getStore().removeAt(rowIndex);
},
},
],
},
],
});

View File

@ -0,0 +1,107 @@
Ext.define('LabelSetupPanel', {
extend: 'Ext.panel.Panel',
alias: 'widget.labelSetupPanel',
layout: {
type: 'hbox',
align: 'stretch',
pack: 'start',
},
getValues: function() {
let me = this;
let values = {};
Ext.Array.each(me.query('[isFormField]'), function(field) {
let data = field.getSubmitData();
Ext.Object.each(data, function(name, val) {
let parsed = parseInt(val, 10);
values[name] = isNaN(parsed) ? val : parsed;
});
});
return values;
},
controller: {
xclass: 'Ext.app.ViewController',
init: function() {
let me = this;
let view = me.getView();
let list = view.down("labelList");
let store = list.getStore();
store.on('datachanged', function(store) {
view.fireEvent("listchanged", store);
});
store.on('update', function(store) {
view.fireEvent("listchanged", store);
});
},
onAdd: function() {
let list = this.lookupReference('label_list');
let view = this.getView();
let params = view.getValues();
list.getStore().add(params);
},
},
items: [
{
border: false,
layout: {
type: 'vbox',
align: 'stretch',
pack: 'start',
},
items: [
{
xtype: 'prefixfield',
name: 'prefix',
value: 'TEST',
fieldLabel: 'Prefix',
},
{
xtype: 'ltoTapeType',
name: 'tape_type',
fieldLabel: 'Type',
value: 'L8',
},
{
xtype: 'ltoLabelStyle',
name: 'mode',
fieldLabel: 'Mode',
value: 'color',
},
{
xtype: 'numberfield',
name: 'start',
fieldLabel: 'Start',
minValue: 0,
allowBlank: false,
value: 0,
},
{
xtype: 'numberfield',
name: 'end',
fieldLabel: 'End',
minValue: 0,
emptyText: 'Fill',
},
{
xtype: 'button',
text: 'Add',
handler: 'onAdd',
},
],
},
{
margin: "0 0 0 10",
xtype: 'labelList',
reference: 'label_list',
flex: 1,
},
],
});

View File

@ -0,0 +1,20 @@
Ext.define('LtoLabelStyle', {
extend: 'Ext.form.field.ComboBox',
alias: 'widget.ltoLabelStyle',
editable: false,
displayField: 'text',
valueField: 'value',
queryMode: 'local',
store: {
field: ['value', 'text'],
data: [
{ value: 'simple', text: "Simple" },
{ value: 'color', text: 'Color (frames with color)' },
{ value: 'frame', text: 'Frame (no color)' },
{ value: 'placeholder', text: 'Placeholder (empty)' },
],
},
});

View File

@ -0,0 +1,206 @@
// FIXME: HACK! Makes scrolling in number spinner work again. fixed in ExtJS >= 6.1
if (Ext.isFirefox) {
Ext.$eventNameMap.DOMMouseScroll = 'DOMMouseScroll';
}
function draw_labels(target_id, label_list, page_layout, calibration) {
let max_labels = compute_max_labels(page_layout);
let count_fixed = 0;
let count_fill = 0;
for (i = 0; i < label_list.length; i++) {
let item = label_list[i];
if (item.end === null || item.end === '' || item.end === undefined) {
count_fill += 1;
continue;
}
if (item.end <= item.start) {
count_fixed += 1;
continue;
}
count_fixed += (item.end - item.start) + 1;
}
let rest = max_labels - count_fixed;
let fill_size = 1;
if (rest >= count_fill) {
fill_size = Math.floor(rest/count_fill);
}
let list = [];
let count_fill_2 = 0;
for (i = 0; i < label_list.length; i++) {
let item = label_list[i];
let count;
if (item.end === null || item.end === '' || item.end === undefined) {
count_fill_2 += 1;
if (count_fill_2 === count_fill) {
count = rest;
} else {
count = fill_size;
}
rest -= count;
} else if (item.end <= item.start) {
count = 1;
} else {
count = (item.end - item.start) + 1;
}
for (j = 0; j < count; j++) {
let id = item.start + j;
if (item.prefix.length == 6) {
list.push({
label: item.prefix,
tape_type: item.tape_type,
mode: item.mode,
id: id,
});
rest += count - j - 1;
break;
} else {
let pad_len = 6-item.prefix.length;
let label = item.prefix + id.toString().padStart(pad_len, 0);
if (label.length != 6) {
rest += count - j;
break;
}
list.push({
label: label,
tape_type: item.tape_type,
mode: item.mode,
id: id,
});
}
}
}
generate_barcode_page(target_id, page_layout, list, calibration);
}
Ext.define('MainView', {
extend: 'Ext.container.Viewport',
alias: 'widget.mainview',
layout: {
type: 'vbox',
align: 'stretch',
pack: 'start',
},
width: 800,
controller: {
xclass: 'Ext.app.ViewController',
update_barcode_preview: function() {
let me = this;
let view = me.getView();
let list_view = view.down("labelList");
let store = list_view.getStore();
let label_list = [];
store.each((record) => {
label_list.push(record.data);
});
let page_layout_view = view.down("pageLayoutPanel");
let page_layout = page_layout_view.getValues();
let calibration_view = view.down("pageCalibration");
let page_calibration = calibration_view.getValues();
draw_labels("print_frame", label_list, page_layout, page_calibration);
},
update_calibration_preview: function() {
let me = this;
let view = me.getView();
let page_layout_view = view.down("pageLayoutPanel");
let page_layout = page_layout_view.getValues();
let calibration_view = view.down("pageCalibration");
let page_calibration = calibration_view.getValues();
console.log(page_calibration);
generate_calibration_page('print_frame', page_layout, page_calibration);
},
control: {
labelSetupPanel: {
listchanged: function(store) {
this.update_barcode_preview();
},
activate: function() {
this.update_barcode_preview();
},
},
pageLayoutPanel: {
pagechanged: function(layout) {
this.update_barcode_preview();
},
activate: function() {
this.update_barcode_preview();
},
},
pageCalibration: {
calibrationchanged: function() {
this.update_calibration_preview();
},
activate: function() {
this.update_calibration_preview();
},
},
},
},
items: [
{
xtype: 'tabpanel',
items: [
{
xtype: 'labelSetupPanel',
title: 'Proxmox LTO Barcode Label Generator',
bodyPadding: 10,
},
{
xtype: 'pageLayoutPanel',
title: 'Page Layout',
bodyPadding: 10,
},
{
xtype: 'pageCalibration',
title: 'Printer Calibration',
bodyPadding: 10,
},
],
},
{
xtype: 'panel',
layout: "center",
title: 'Print Preview',
bodyStyle: "background-color: grey;",
bodyPadding: 10,
html: '<center><iframe id="print_frame" frameBorder="0"></iframe></center>',
border: false,
flex: 1,
scrollable: true,
tools: [{
type: 'print',
tooltip: 'Open Print Dialog',
handler: function(event, toolEl, panelHeader) {
printBarcodePage();
},
}],
},
],
});
Ext.onReady(function() {
Ext.create('MainView', {
renderTo: Ext.getBody(),
});
});

View File

@ -0,0 +1,142 @@
Ext.define('PageCalibration', {
extend: 'Ext.panel.Panel',
alias: 'widget.pageCalibration',
layout: {
type: 'hbox',
align: 'stretch',
pack: 'start',
},
getValues: function() {
let me = this;
let values = {};
Ext.Array.each(me.query('[isFormField]'), function(field) {
if (field.isValid()) {
let data = field.getSubmitData();
Ext.Object.each(data, function(name, val) {
let parsed = parseFloat(val, 10);
values[name] = isNaN(parsed) ? val : parsed;
});
}
});
if (values.d_x === undefined) { return; }
if (values.d_y === undefined) { return; }
if (values.s_x === undefined) { return; }
if (values.s_y === undefined) { return; }
scalex = 100/values.d_x;
scaley = 100/values.d_y;
let offsetx = ((50 - values.s_x) - (50*scalex - 50))/scalex;
let offsety = ((50 - values.s_y) - (50*scaley - 50))/scaley;
return {
scalex: scalex,
scaley: scaley,
offsetx: offsetx,
offsety: offsety,
};
},
controller: {
xclass: 'Ext.app.ViewController',
control: {
'field': {
change: function() {
let view = this.getView();
let param = view.getValues();
view.fireEvent("calibrationchanged", param);
},
},
},
},
items: [
{
border: false,
layout: {
type: 'vbox',
align: 'stretch',
pack: 'start',
},
items: [
{
xtype: 'displayfield',
value: 'a4',
fieldLabel: 'Start Offset Sx (mm)',
labelWidth: 150,
value: 50,
},
{
xtype: 'displayfield',
value: 'a4',
fieldLabel: 'Length Dx (mm)',
labelWidth: 150,
value: 100,
},
{
xtype: 'displayfield',
value: 'a4',
fieldLabel: 'Start Offset Sy (mm)',
labelWidth: 150,
value: 50,
},
{
xtype: 'displayfield',
value: 'a4',
fieldLabel: 'Length Dy (mm)',
labelWidth: 150,
value: 100,
},
],
},
{
border: false,
margin: '0 0 0 20',
layout: {
type: 'vbox',
align: 'stretch',
pack: 'start',
},
items: [
{
xtype: 'numberfield',
value: 'a4',
name: 's_x',
fieldLabel: 'Meassured Start Offset Sx (mm)',
allowBlank: false,
labelWidth: 200,
},
{
xtype: 'numberfield',
value: 'a4',
name: 'd_x',
fieldLabel: 'Meassured Length Dx (mm)',
allowBlank: false,
labelWidth: 200,
},
{
xtype: 'numberfield',
value: 'a4',
name: 's_y',
fieldLabel: 'Meassured Start Offset Sy (mm)',
allowBlank: false,
labelWidth: 200,
},
{
xtype: 'numberfield',
value: 'a4',
name: 'd_y',
fieldLabel: 'Meassured Length Dy (mm)',
allowBlank: false,
labelWidth: 200,
},
],
},
],
});

View File

@ -0,0 +1,167 @@
Ext.define('PageLayoutPanel', {
extend: 'Ext.panel.Panel',
alias: 'widget.pageLayoutPanel',
layout: {
type: 'hbox',
align: 'stretch',
pack: 'start',
},
getValues: function() {
let me = this;
let values = {};
Ext.Array.each(me.query('[isFormField]'), function(field) {
if (field.isValid()) {
let data = field.getSubmitData();
Ext.Object.each(data, function(name, val) {
values[name] = val;
});
}
});
let paper_size = values.paper_size || 'a4';
let param = Ext.apply({}, paper_sizes[paper_size]);
if (param === undefined) {
throw `unknown paper size ${paper_size}`;
}
param.paper_size = paper_size;
Ext.Object.each(values, function(name, val) {
let parsed = parseFloat(val, 10);
param[name] = isNaN(parsed) ? val : parsed;
});
return param;
},
controller: {
xclass: 'Ext.app.ViewController',
control: {
'paperSize': {
change: function(field, paper_size) {
let view = this.getView();
let defaults = paper_sizes[paper_size];
let names = [
'label_width',
'label_height',
'margin_left',
'margin_top',
'column_spacing',
'row_spacing',
];
for (i = 0; i < names.length; i++) {
let name = names[i];
let f = view.down(`field[name=${name}]`);
let v = defaults[name];
if (v != undefined) {
f.setValue(v);
f.setDisabled(defaults.fixed);
} else {
f.setDisabled(false);
}
}
},
},
'field': {
change: function() {
let view = this.getView();
let param = view.getValues();
view.fireEvent("pagechanged", param);
},
},
},
},
items: [
{
border: false,
layout: {
type: 'vbox',
align: 'stretch',
pack: 'start',
},
items: [
{
xtype: 'paperSize',
name: 'paper_size',
value: 'a4',
fieldLabel: 'Paper Size',
},
{
xtype: 'numberfield',
name: 'label_width',
fieldLabel: 'Label width',
minValue: 70,
allowBlank: false,
value: 70,
},
{
xtype: 'numberfield',
name: 'label_height',
fieldLabel: 'Label height',
minValue: 15,
allowBlank: false,
value: 17,
},
{
xtype: 'checkbox',
name: 'label_borders',
fieldLabel: 'Label borders',
value: true,
inputValue: true,
},
],
},
{
border: false,
margin: '0 0 0 10',
layout: {
type: 'vbox',
align: 'stretch',
pack: 'start',
},
items: [
{
xtype: 'numberfield',
name: 'margin_left',
fieldLabel: 'Left margin',
minValue: 0,
allowBlank: false,
value: 0,
},
{
xtype: 'numberfield',
name: 'margin_top',
fieldLabel: 'Top margin',
minValue: 0,
allowBlank: false,
value: 4,
},
{
xtype: 'numberfield',
name: 'column_spacing',
fieldLabel: 'Column spacing',
minValue: 0,
allowBlank: false,
value: 0,
},
{
xtype: 'numberfield',
name: 'row_spacing',
fieldLabel: 'Row spacing',
minValue: 0,
allowBlank: false,
value: 0,
},
],
},
],
});

View File

@ -0,0 +1,49 @@
const paper_sizes = {
a4: {
comment: 'A4 (plain)',
page_width: 210,
page_height: 297,
},
letter: {
comment: 'Letter (plain)',
page_width: 215.9,
page_height: 279.4,
},
avery3420: {
fixed: true,
comment: 'Avery Zweckform 3420',
page_width: 210,
page_height: 297,
label_width: 70,
label_height: 16.9,
margin_left: 0,
margin_top: 5,
column_spacing: 0,
row_spacing: 0,
},
};
function paper_size_combo_data() {
let data = [];
for (let [key, value] of Object.entries(paper_sizes)) {
data.push({ value: key, text: value.comment });
}
return data;
}
Ext.define('PaperSize', {
extend: 'Ext.form.field.ComboBox',
alias: 'widget.paperSize',
editable: false,
displayField: 'text',
valueField: 'value',
queryMode: 'local',
store: {
field: ['value', 'text'],
data: paper_size_combo_data(),
},
});

View File

@ -0,0 +1,15 @@
Ext.define('PrefixField', {
extend: 'Ext.form.field.Text',
alias: 'widget.prefixfield',
maxLength: 6,
allowBlank: false,
maskRe: /([A-Za-z]+)$/,
listeners: {
change: function(field) {
field.setValue(field.getValue().toUpperCase());
},
},
});

View File

@ -0,0 +1,23 @@
Ext.define('LtoTapeType', {
extend: 'Ext.form.field.ComboBox',
alias: 'widget.ltoTapeType',
editable: false,
displayField: 'text',
valueField: 'value',
queryMode: 'local',
store: {
field: ['value', 'text'],
data: [
{ value: 'L8', text: "LTO-8" },
{ value: 'L7', text: "LTO-7" },
{ value: 'L6', text: "LTO-6" },
{ value: 'L5', text: "LTO-5" },
{ value: 'L4', text: "LTO-4" },
{ value: 'L3', text: "LTO-3" },
{ value: 'CU', text: "Cleaning Unit" },
],
},
});

View File

@ -77,23 +77,59 @@ edit the interval at which pruning takes place.
:alt: Prune and garbage collection options
Retention Settings Example
^^^^^^^^^^^^^^^^^^^^^^^^^^
The backup frequency and retention of old backups may depend on how often data
changes, and how important an older state may be, in a specific work load.
When backups act as a company's document archive, there may also be legal
requirements for how long backup snapshots must be kept.
For this example, we assume that you are doing daily backups, have a retention
period of 10 years, and the period between backups stored gradually grows.
- **keep-last:** ``3`` - even if only daily backups, an admin may want to create
an extra one just before or after a big upgrade. Setting keep-last ensures
this.
- **keep-hourly:** not set - for daily backups this is not relevant. You cover
extra manual backups already, with keep-last.
- **keep-daily:** ``13`` - together with keep-last, which covers at least one
day, this ensures that you have at least two weeks of backups.
- **keep-weekly:** ``8`` - ensures that you have at least two full months of
weekly backups.
- **keep-monthly:** ``11`` - together with the previous keep settings, this
ensures that you have at least a year of monthly backups.
- **keep-yearly:** ``9`` - this is for the long term archive. As you covered the
current year with the previous options, you would set this to nine for the
remaining ones, giving you a total of at least 10 years of coverage.
We recommend that you use a higher retention period than is minimally required
by your environment; you can always reduce it if you find it is unnecessarily
high, but you cannot recreate backup snapshots from the past.
.. _maintenance_gc:
Garbage Collection
------------------
You can monitor and run :ref:`garbage collection <garbage-collection>` on the
You can monitor and run :ref:`garbage collection <client_garbage-collection>` on the
Proxmox Backup Server using the ``garbage-collection`` subcommand of
``proxmox-backup-manager``. You can use the ``start`` subcommand to manually
start garbage collection on an entire datastore and the ``status`` subcommand to
see attributes relating to the :ref:`garbage collection <garbage-collection>`.
see attributes relating to the :ref:`garbage collection <client_garbage-collection>`.
This functionality can also be accessed in the GUI, by navigating to **Prune &
GC** from the top panel. From here, you can edit the schedule at which garbage
collection runs and manually start the operation.
.. _verification:
.. _maintenance_verification:
Verification
------------
@ -106,7 +142,7 @@ Verification
Proxmox Backup offers various verification options to ensure that backup data is
intact. Verification is generally carried out through the creation of verify
jobs. These are scheduled tasks that run verification at a given interval (see
:ref:`calendar-events`). With these, you can set whether already verified
:ref:`calendar-event-scheduling`). With these, you can set whether already verified
snapshots are ignored, as well as set a time period, after which verified jobs
are checked again. The interface for creating verify jobs can be found under the
**Verify Jobs** tab of the datastore.

View File

@ -65,7 +65,7 @@ the ``proxmox-backup-manager sync-job`` command. The configuration information
for sync jobs is stored at ``/etc/proxmox-backup/sync.cfg``. To create a new
sync job, click the add button in the GUI, or use the ``create`` subcommand.
After creating a sync job, you can either start it manually from the GUI or
provide it with a schedule (see :ref:`calendar-events`) to run regularly.
provide it with a schedule (see :ref:`calendar-event-scheduling`) to run regularly.
.. code-block:: console

View File

@ -1,3 +1,5 @@
.. _sysadmin_network_configuration:
Network Management
==================

24
docs/output-format.rst Normal file
View File

@ -0,0 +1,24 @@
Most commands producing output supports the ``--output-format``
parameter. It accepts the following values:
:``text``: Text format (default). Structured data is rendered as a table.
:``json``: JSON (single line).
:``json-pretty``: JSON (multiple lines, nicely formatted).
Also, the following environment variables can modify output behavior:
``PROXMOX_OUTPUT_FORMAT``
Defines the default output format.
``PROXMOX_OUTPUT_NO_BORDER``
If set (to any value), do not render table borders.
``PROXMOX_OUTPUT_NO_HEADER``
If set (to any value), do not render table headers.
.. note:: The ``text`` format is designed to be human readable, and
not meant to be parsed by automation tools. Please use the ``json``
format if you need to process the output.

View File

@ -26,11 +26,8 @@ update``.
.. FIXME for 7.0: change security update suite to bullseye-security
In addition, you need a package repository from Proxmox to get Proxmox Backup updates.
During the Proxmox Backup beta phase, only one repository (pbstest) will be
available. Once released, an Enterprise repository for production use and a
no-subscription repository will be provided.
In addition, you need a package repository from Proxmox to get Proxmox Backup
updates.
SecureApt
~~~~~~~~~
@ -72,18 +69,16 @@ Here, the output should be:
f3f6c5a3a67baf38ad178e5ff1ee270c /etc/apt/trusted.gpg.d/proxmox-ve-release-6.x.gpg
.. comment
.. _sysadmin_package_repos_enterprise:
`Proxmox Backup`_ Enterprise Repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
This will be the default, stable, and recommended repository. It is available for
This is the stable, recommended repository. It is available for
all `Proxmox Backup`_ subscription users. It contains the most stable packages,
and is suitable for production use. The ``pbs-enterprise`` repository is
enabled by default:
.. note:: During the Proxmox Backup beta phase only one repository (pbstest)
will be available.
.. code-block:: sources.list
:caption: File: ``/etc/apt/sources.list.d/pbs-enterprise.list``
@ -96,12 +91,12 @@ Here, the output should be:
Please note that you need a valid subscription key to access this
repository. More information regarding subscription levels and pricing can be
found at https://www.proxmox.com/en/proxmox-backup/pricing.
found at https://www.proxmox.com/en/proxmox-backup-server/pricing
.. note:: You can disable this repository by commenting out the above
line using a `#` (at the start of the line). This prevents error
messages if you do not have a subscription key. Please configure the
``pbs-no-subscription`` repository in that case.
.. note:: You can disable this repository by commenting out the above line
using a `#` (at the start of the line). This prevents error messages if you do
not have a subscription key. Please configure the ``pbs-no-subscription``
repository in that case.
`Proxmox Backup`_ No-Subscription Repository
@ -128,12 +123,11 @@ Here, the output should be:
deb http://security.debian.org/debian-security buster/updates main contrib
`Proxmox Backup`_ Beta Repository
`Proxmox Backup`_ Test Repository
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
During the public beta, there is a repository called ``pbstest``. This one
contains the latest packages and is heavily used by developers to test new
features.
This repository contains the latest packages and is heavily used by developers
to test new features.
.. .. warning:: the ``pbstest`` repository should (as the name implies)
only be used to test new features or bug fixes.
@ -145,7 +139,3 @@ You can access this repository by adding the following line to
:caption: sources.list entry for ``pbstest``
deb http://download.proxmox.com/debian/pbs buster pbstest
If you installed Proxmox Backup Server from the official beta ISO, you should
have this repository already configured in
``/etc/apt/sources.list.d/pbstest-beta.list``

View File

@ -1,7 +1,7 @@
Copyright and Disclaimer
========================
Copyright (C) 2007-2019 Proxmox Server Solutions GmbH
Copyright (C) 2007-2021 Proxmox Server Solutions GmbH
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as

2
docs/pmt/description.rst Normal file
View File

@ -0,0 +1,2 @@
The ``pmt`` command controls Linux tape devices.

42
docs/pmt/man1.rst Normal file
View File

@ -0,0 +1,42 @@
==========================
pmt
==========================
.. include:: ../epilog.rst
-------------------------------------------------------------
Control Linux Tape Devices
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 1
Synopsis
========
.. include:: synopsis.rst
Common Options
==============
.. include:: options.rst
Description
===========
.. include:: description.rst
ENVIRONMENT
===========
:TAPE: If set, replaces the `--device` option.
:PROXMOX_TAPE_DRIVE: If set, replaces the `--drive` option.
.. include:: ../pbs-copyright.rst

51
docs/pmt/options.rst Normal file
View File

@ -0,0 +1,51 @@
All commands support the following parameters to specify the tape device:
--device <path> Path to the Linux tape device
--drive <name> Use drive from Proxmox Backup Server configuration.
Commands which generate output support the ``--output-format``
parameter. It accepts the following values:
:``text``: Text format (default). Human readable.
:``json``: JSON (single line).
:``json-pretty``: JSON (multiple lines, nicely formatted).
Device driver options can be specified as integer numbers (see
``/usr/include/linux/mtio.h``), or using symbolic names:
:``buffer-writes``: Enable buffered writes
:``async-writes``: Enable async writes
:``read-ahead``: Use read-ahead for fixed block size
:``debugging``: Enable debugging if compiled into the driver
:``two-fm``: Write two file marks when closing the file
:``fast-mteom``: Space directly to eod (and lose file number)
:``auto-lock``: Automatically lock/unlock drive door
:``def-writes``: Defaults are meant only for writes
:``can-bsr``: Indicates that the drive can space backwards
:``no-blklims``: Drive does not support read block limits
:``can-partitions``: Drive can handle partitioned tapes
:``scsi2locical``: Seek and tell use SCSI-2 logical block addresses
:``sysv``: Enable the System V semantics
:``nowait``: Do not wait for rewind, etc. to complete
:``sili``: Enables setting the SILI bit in SCSI commands when reading
in variable block mode to enhance performance when reading blocks
shorter than the byte count

View File

@ -0,0 +1,3 @@
The ``pmtx`` command controls SCSI media changer devices (tape
autoloader).

57
docs/pmtx/man1.rst Normal file
View File

@ -0,0 +1,57 @@
==========================
pmtx
==========================
.. include:: ../epilog.rst
-------------------------------------------------------------
Control SCSI media changer devices (tape autoloaders)
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 1
Synopsis
==========
.. include:: synopsis.rst
Common Options
==============
All command supports the following parameters to specify the changer device:
--device <path> Path to Linux generic SCSI device (e.g. '/dev/sg4')
--changer <name> Use changer from Proxmox Backup Server configuration.
Commands generating output supports the ``--output-format``
parameter. It accepts the following values:
:``text``: Text format (default). Human readable.
:``json``: JSON (single line).
:``json-pretty``: JSON (multiple lines, nicely formatted).
Description
============
.. include:: description.rst
ENVIRONMENT
===========
:CHANGER: If set, replaces the `--device` option
:PROXMOX_TAPE_DRIVE: If set, use the Proxmox Backup Server
configuration to find the associcated changer device.
.. include:: ../pbs-copyright.rst

View File

@ -1,4 +1,4 @@
This is just a test.
.. NOTE:: No further info.
This tool implements a backup server client, i.e. it can connect to a
backup servers to issue management commands and to create or restore
backups.

View File

@ -5,7 +5,7 @@ proxmox-backup-client
.. include:: ../epilog.rst
-------------------------------------------------------------
Command line toot for Backup and Restore
Command line tool for Backup and Restore
-------------------------------------------------------------
:Author: |AUTHOR|
@ -31,6 +31,12 @@ Those command are available when you start an intercative restore shell:
.. include:: catalog-shell-synopsis.rst
Common Options
==============
.. include:: ../output-format.rst
Description
============

View File

@ -1,4 +1,2 @@
This is just a test.
.. NOTE:: No further info.
This tool exposes the whole backup server management API on the
command line.

View File

@ -1,4 +1,5 @@
This is just a test.
.. NOTE:: No further info.
This daemon exposes the whole Proxmox Backup Server API on TCP port
8007 using HTTPS. It runs as user ``backup`` and has very limited
permissions. Operation requiring more permissions are forwarded to
the local ``proxmox-backup`` service.

View File

@ -0,0 +1,7 @@
This daemon exposes the Proxmox Backup Server management API on
``127.0.0.1:82``. It runs as ``root`` and has permission to do all
privileged operations.
NOTE: The daemon listens to a local address only, so you cannot access
it from outside. The ``proxmox-backup-proxy`` daemon exposes the API
to the outside world.

View File

@ -0,0 +1,41 @@
==========================
proxmox-backup
==========================
.. include:: ../epilog.rst
-------------------------------------------------------------
Proxmox Backup Local API Server
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 1
Synopsis
==========
This daemon is normally started and managed as ``systemd`` service::
systemctl start proxmox-backup
systemctl stop proxmox-backup
systemctl status proxmox-backup
For debugging, you can start the daemon in foreground using::
proxmox-backup-api
.. NOTE:: You need to stop the service before starting the daemon in
foreground.
Description
============
.. include:: description.rst
.. include:: ../pbs-copyright.rst

View File

@ -0,0 +1 @@
This tool can configure and manage tape backups.

View File

@ -0,0 +1,28 @@
==========================
proxmox-tape
==========================
.. include:: ../epilog.rst
-------------------------------------------------------------
Proxmox Tape Backup Command Line Tool
-------------------------------------------------------------
:Author: |AUTHOR|
:Version: Version |VERSION|
:Manual section: 1
Synopsis
========
.. include:: synopsis.rst
Description
===========
.. include:: description.rst
.. include:: ../pbs-copyright.rst

Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

View File

@ -13,6 +13,7 @@
.cal-day {
vertical-align: top;
width: 150px;
height: 75px; /* this is like min-height when used in tables */
border: #939393 1px solid;
color: #454545;
}
@ -32,6 +33,9 @@
.first-of-month {
border-right: dashed black 4px;
}
.clear-trigger {
background-image: url(./clear-trigger.png);
}
</style>
<script type="text/javascript" src="extjs/ext-all.js"></script>

View File

@ -152,7 +152,12 @@ Ext.onReady(function() {
dataIndex: 'mark',
renderer: function(value, metaData, record) {
if (record.data.mark === 'keep') {
if (record.data.keepCount) {
return 'keep (' + record.data.keepName +
': ' + record.data.keepCount + ')';
} else {
return 'keep (' + record.data.keepName + ')';
}
} else {
return value;
}
@ -212,8 +217,12 @@ Ext.onReady(function() {
let text = Ext.Date.format(backup.data.backuptime, 'H:i');
if (backup.data.mark === 'remove') {
html += `<span class="strikethrough">${text}</span>`;
} else {
if (backup.data.keepCount) {
text += ` (${backup.data.keepName} ${backup.data.keepCount})`;
} else {
text += ` (${backup.data.keepName})`;
}
if (me.useColors) {
let bgColor = COLORS[backup.data.keepName];
let textColor = TEXT_COLORS[backup.data.keepName];
@ -256,6 +265,34 @@ Ext.onReady(function() {
},
});
Ext.define('PBS.PruneSimulatorKeepInput', {
extend: 'Ext.form.field.Number',
alias: 'widget.prunesimulatorKeepInput',
allowBlank: true,
fieldGroup: 'keep',
minValue: 1,
listeners: {
afterrender: function(field) {
this.triggers.clear.setVisible(field.value !== null);
},
change: function(field, newValue, oldValue) {
this.triggers.clear.setVisible(newValue !== null);
},
},
triggers: {
clear: {
cls: 'clear-trigger',
weight: -1,
handler: function() {
this.triggers.clear.setVisible(false);
this.setValue(null);
},
},
},
});
Ext.define('PBS.PruneSimulatorPanel', {
extend: 'Ext.panel.Panel',
alias: 'widget.prunesimulatorPanel',
@ -412,11 +449,8 @@ Ext.onReady(function() {
});
});
// ordering here and iterating backwards through days
// ensures that everything is ordered
timesOnSingleDay.sort(function(a, b) {
return a < b;
});
// sort recent times first, backups array below is ordered now -> past
timesOnSingleDay.sort((a, b) => b - a);
let backups = [];
@ -448,16 +482,17 @@ Ext.onReady(function() {
backups.forEach(function(backup) {
let mark = backup.mark;
if (mark && mark === 'keep') {
let id = idFunc(backup);
if (finished || alreadyIncluded[id]) {
return;
}
if (mark) {
if (mark === 'keep') {
alreadyIncluded[id] = true;
}
});
backups.forEach(function(backup) {
let mark = backup.mark;
let id = idFunc(backup);
if (finished || alreadyIncluded[id] || mark) {
return;
}
@ -470,6 +505,7 @@ Ext.onReady(function() {
newlyIncludedCount++;
backup.mark = 'keep';
backup.keepName = keepName;
backup.keepCount = newlyIncludedCount;
} else {
backup.mark = 'remove';
}
@ -488,7 +524,7 @@ Ext.onReady(function() {
Number(keepParams['keep-yearly']) === 0) {
backups.forEach(function(backup) {
backup.mark = 'keep';
backup.keepName = 'all zero';
backup.keepName = 'keep-all';
});
return;
@ -550,58 +586,37 @@ Ext.onReady(function() {
keepItems: [
{
xtype: 'numberfield',
xtype: 'prunesimulatorKeepInput',
name: 'keep-last',
allowBlank: true,
fieldLabel: 'keep-last',
minValue: 0,
value: 4,
fieldGroup: 'keep',
},
{
xtype: 'numberfield',
xtype: 'prunesimulatorKeepInput',
name: 'keep-hourly',
allowBlank: true,
fieldLabel: 'keep-hourly',
minValue: 0,
value: 0,
fieldGroup: 'keep',
},
{
xtype: 'numberfield',
xtype: 'prunesimulatorKeepInput',
name: 'keep-daily',
allowBlank: true,
fieldLabel: 'keep-daily',
minValue: 0,
value: 5,
fieldGroup: 'keep',
},
{
xtype: 'numberfield',
xtype: 'prunesimulatorKeepInput',
name: 'keep-weekly',
allowBlank: true,
fieldLabel: 'keep-weekly',
minValue: 0,
value: 2,
fieldGroup: 'keep',
},
{
xtype: 'numberfield',
xtype: 'prunesimulatorKeepInput',
name: 'keep-monthly',
allowBlank: true,
fieldLabel: 'keep-monthly',
minValue: 0,
value: 0,
fieldGroup: 'keep',
},
{
xtype: 'numberfield',
xtype: 'prunesimulatorKeepInput',
name: 'keep-yearly',
allowBlank: true,
fieldLabel: 'keep-yearly',
minValue: 0,
value: 0,
fieldGroup: 'keep',
},
],
@ -613,42 +628,6 @@ Ext.onReady(function() {
sorters: { property: 'backuptime', direction: 'DESC' },
});
let scheduleItems = [
{
xtype: 'prunesimulatorDayOfWeekSelector',
name: 'schedule-weekdays',
fieldLabel: 'Day of week',
value: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun'],
allowBlank: false,
multiSelect: true,
padding: '0 0 0 10',
},
{
xtype: 'prunesimulatorCalendarEvent',
name: 'schedule-time',
allowBlank: false,
value: '0/6:00',
fieldLabel: 'Backup schedule',
padding: '0 0 0 10',
},
{
xtype: 'numberfield',
name: 'numberOfWeeks',
allowBlank: false,
fieldLabel: 'Number of weeks',
minValue: 1,
value: 15,
maxValue: 260, // five years
padding: '0 0 0 10',
},
{
xtype: 'button',
name: 'schedule-button',
text: 'Update Schedule',
handler: 'reloadFull',
},
];
me.items = [
{
xtype: 'panel',
@ -684,6 +663,7 @@ Ext.onReady(function() {
},
{ xtype: "panel", width: 1, border: 1 },
{
xtype: 'form',
layout: 'anchor',
flex: 1,
border: false,
@ -692,7 +672,42 @@ Ext.onReady(function() {
labelWidth: 120,
},
bodyPadding: 10,
items: scheduleItems,
items: [
{
xtype: 'prunesimulatorDayOfWeekSelector',
name: 'schedule-weekdays',
fieldLabel: 'Day of week',
value: ['mon', 'tue', 'wed', 'thu', 'fri', 'sat', 'sun'],
allowBlank: false,
multiSelect: true,
padding: '0 0 0 10',
},
{
xtype: 'prunesimulatorCalendarEvent',
name: 'schedule-time',
allowBlank: false,
value: '0/6:00',
fieldLabel: 'Backup schedule',
padding: '0 0 0 10',
},
{
xtype: 'numberfield',
name: 'numberOfWeeks',
allowBlank: false,
fieldLabel: 'Number of weeks',
minValue: 1,
value: 15,
maxValue: 260, // five years
padding: '0 0 0 10',
},
{
xtype: 'button',
name: 'schedule-button',
text: 'Update Schedule',
formBind: true,
handler: 'reloadFull',
},
],
},
],
},

View File

@ -1,6 +1,3 @@
Description
^^^^^^^^^^^
``pxar`` is a command line utility to create and manipulate archives in the
:ref:`pxar-format`.
It is inspired by `casync file archive format
@ -80,7 +77,7 @@ These files must contain one pattern per line, again later patterns win over
previous ones.
The patterns control file exclusions of files present within the given directory
or further below it in the tree.
The behavior is the same as described in :ref:`creating-backups`.
The behavior is the same as described in :ref:`client_creating_backups`.
Extracting an Archive
^^^^^^^^^^^^^^^^^^^^^

View File

@ -4,6 +4,9 @@ pxar
.. include:: ../epilog.rst
.. Avoid errors with sphinx ref role
.. role:: ref(emphasis)
-------------------------------------------------------------
Proxmox File Archive Command Line Tool
-------------------------------------------------------------
@ -25,4 +28,3 @@ Description
.. include:: ../pbs-copyright.rst

View File

@ -6,3 +6,9 @@ Service Daemons
.. include:: proxmox-backup-proxy/description.rst
``proxmox-backup``
~~~~~~~~~~~~~~~~~~
.. include:: proxmox-backup/description.rst

View File

@ -1,6 +1,8 @@
Storage
=======
.. _storage_disk_management:
Disk Management
---------------
@ -57,7 +59,7 @@ create a datastore at the location ``/mnt/datastore/store1``:
You can also create a ``zpool`` with various raid levels from **Administration
-> Disks -> Zpool** in the web interface, or by using ``zpool create``. The command
below creates a mirrored ``zpool`` using two disks (``sdb`` & ``sdc``) and
mounts it on the root directory (default):
mounts it under ``/mnt/datastore/zpool1``:
.. code-block:: console
@ -85,7 +87,7 @@ display S.M.A.R.T. attributes from the web interface or by using the command:
.. _datastore_intro:
:term:`DataStore`
:term:`Datastore`
-----------------
A datastore refers to a location at which backups are stored. The current
@ -117,10 +119,12 @@ directory on the filesystem. Each datastore also has associated retention
settings of how many backup snapshots for each interval of ``hourly``,
``daily``, ``weekly``, ``monthly``, ``yearly`` as well as a time-independent
number of backups to keep in that store. :ref:`backup-pruning` and
:ref:`garbage collection <garbage-collection>` can also be configured to run
periodically based on a configured schedule (see :ref:`calendar-events`) per datastore.
:ref:`garbage collection <client_garbage-collection>` can also be configured to run
periodically based on a configured schedule (see :ref:`calendar-event-scheduling`) per datastore.
.. _storage_datastore_create:
Creating a Datastore
^^^^^^^^^^^^^^^^^^^^
.. image:: images/screenshots/pbs-gui-datastore-create-general.png

View File

@ -1,3 +1,5 @@
.. _sysadmin_host_administration:
Host System Administration
==========================
@ -23,4 +25,7 @@ either explain things which are different on `Proxmox Backup`_, or
tasks which are commonly used on `Proxmox Backup`_. For other topics,
please refer to the standard Debian documentation.
.. include:: local-zfs.rst
.. include:: services.rst

960
docs/tape-backup.rst Normal file
View File

@ -0,0 +1,960 @@
.. _tape_backup:
Tape Backup
===========
.. CAUTION:: Tape Backup is a technical preview feature, not meant for
production use. To enable it in the GUI, you need to issue the
following command (as root user on the console):
.. code-block:: console
# touch /etc/proxmox-backup/tape.cfg
Proxmox tape backup provides an easy way to store datastore content
onto magnetic tapes. This increases data safety because you get:
- an additional copy of the data,
- on a different media type (tape),
- to an additional location (you can move tapes off-site)
In most restore jobs, only data from the last backup job is restored.
Restore requests further decline, the older the data
gets. Considering this, tape backup may also help to reduce disk
usage, because you can safely remove data from disk, once it's archived on
tape. This is especially true if you need to retain data for several
years.
Tape backups do not provide random access to the stored data. Instead,
you need to restore the data to disk, before you can access it
again. Also, if you store your tapes off-site (using some kind of tape
vaulting service), you need to bring them back on-site, before you can do any
restores. So please consider that restoring from tape can take much
longer than restoring from disk.
Tape Technology Primer
----------------------
.. _Linear Tape-Open: https://en.wikipedia.org/wiki/Linear_Tape-Open
As of 2021, the only widely available tape technology standard is
`Linear Tape-Open`_ (LTO). Different vendors offer LTO Ultrium tape
drives, auto-loaders, and LTO tape cartridges.
There are a few vendors that offer proprietary drives with
slight advantages in performance and capacity. Nevertheless, they have
significant disadvantages:
- proprietary (single vendor)
- a much higher purchase cost
So we currently do not test such drives.
In general, LTO tapes offer the following advantages:
- Durability (30 year lifespan)
- High Capacity (12 TB)
- Relatively low cost per TB
- Cold Media
- Movable (storable inside vault)
- Multiple vendors (for both media and drives)
- Build in AES-GCM Encryption engine
Note that `Proxmox Backup Server` already stores compressed data, so using the
tape compression feature has no advantage.
Supported Hardware
------------------
Proxmox Backup Server supports `Linear Tape-Open`_ generation 4 (LTO-4)
or later. In general, all SCSI-2 tape drives supported by the Linux
kernel should work, but features like hardware encryption need LTO-4
or later.
Tape changing is carried out using the Linux 'mtx' command line
tool, so any changer device supported by this tool should work.
Drive Performance
~~~~~~~~~~~~~~~~~
Current LTO-8 tapes provide read/write speeds of up to 360 MB/s. This means,
that it still takes a minimum of 9 hours to completely write or
read a single tape (even at maximum speed).
The only way to speed up that data rate is to use more than one
drive. That way, you can run several backup jobs in parallel, or run
restore jobs while the other dives are used for backups.
Also consider that you first need to read data from your datastore
(disk). However, a single spinning disk is unable to deliver data at this
rate. We measured a maximum rate of about 60MB/s to 100MB/s in practice,
so it takes 33 hours to read the 12TB needed to fill up an LTO-8 tape. If you want
to write to your tape at full speed, please make sure that the source
datastore is able to deliver that performance (e.g, by using SSDs).
Terminology
-----------
:Tape Labels: are used to uniquely identify a tape. You would normally apply a
sticky paper label to the front of the cartridge. We additionally store the
label text magnetically on the tape (first file on tape).
.. _Code 39: https://en.wikipedia.org/wiki/Code_39
.. _LTO Ultrium Cartridge Label Specification: https://www.ibm.com/support/pages/ibm-lto-ultrium-cartridge-label-specification
.. _LTO Barcode Generator: lto-barcode/index.html
:Barcodes: are a special form of tape labels, which are electronically
readable. Most LTO tape robots use an 8 character string encoded as
`Code 39`_, as defined in the `LTO Ultrium Cartridge Label
Specification`_.
You can either buy such barcode labels from your cartridge vendor,
or print them yourself. You can use our `LTO Barcode Generator`_
app, if you would like to print them yourself.
.. Note:: Physical labels and the associated adhesive should have an
environmental performance to match or exceed the environmental
specifications of the cartridge to which it is applied.
:Media Pools: A media pool is a logical container for tapes. A backup
job targets one media pool, so a job only uses tapes from that
pool. The pool additionally defines how long a backup job can
append data to tapes (allocation policy) and how long you want to
keep the data (retention policy).
:Media Set: A group of continuously written tapes (all from the same
media pool).
:Tape drive: The device used to read and write data to the tape. There
are standalone drives, but drives are usually shipped within tape libraries.
:Tape changer: A device which can change the tapes inside a tape drive
(tape robot). They are usually part of a tape library.
.. _Tape Library: https://en.wikipedia.org/wiki/Tape_library
:`Tape library`_: A storage device that contains one or more tape drives,
a number of slots to hold tape cartridges, a barcode reader to
identify tape cartridges, and an automated method for loading tapes
(a robot).
This is also commonly known as an 'autoloader', 'tape robot' or 'tape jukebox'.
:Inventory: The inventory stores the list of known tapes (with
additional status information).
:Catalog: A media catalog stores information about the media content.
Tape Quick Start
---------------
1. Configure your tape hardware (drives and changers)
2. Configure one or more media pools
3. Label your tape cartridges
4. Start your first tape backup job ...
Configuration
-------------
Please note that you can configure anything using the graphical user
interface or the command line interface. Both methods result in the
same configuration.
.. _tape_changer_config:
Tape changers
~~~~~~~~~~~~~
Tape changers (robots) are part of a `Tape Library`_. You can skip
this step if you are using a standalone drive.
Linux is able to auto detect these devices, and you can get a list
of available devices using:
.. code-block:: console
# proxmox-tape changer scan
┌─────────────────────────────┬─────────┬──────────────┬────────┐
│ path │ vendor │ model │ serial │
╞═════════════════════════════╪═════════╪══════════════╪════════╡
│ /dev/tape/by-id/scsi-CC2C52 │ Quantum │ Superloader3 │ CC2C52 │
└─────────────────────────────┴─────────┴──────────────┴────────┘
In order to use a device with Proxmox Backup Server, you need to create a
configuration entry:
.. code-block:: console
# proxmox-tape changer create sl3 --path /dev/tape/by-id/scsi-CC2C52
Where ``sl3`` is an arbitrary name you can choose.
.. Note:: Please use the persistent device path names from inside
``/dev/tape/by-id/``. Names like ``/dev/sg0`` may point to a
different device after reboot, and that is not what you want.
You can display the final configuration with:
.. code-block:: console
# proxmox-tape changer config sl3
┌──────┬─────────────────────────────┐
│ Name │ Value │
╞══════╪═════════════════════════════╡
│ name │ sl3 │
├──────┼─────────────────────────────┤
│ path │ /dev/tape/by-id/scsi-CC2C52 │
└──────┴─────────────────────────────┘
Or simply list all configured changer devices:
.. code-block:: console
# proxmox-tape changer list
┌──────┬─────────────────────────────┬─────────┬──────────────┬────────────┐
│ name │ path │ vendor │ model │ serial │
╞══════╪═════════════════════════════╪═════════╪══════════════╪════════════╡
│ sl3 │ /dev/tape/by-id/scsi-CC2C52 │ Quantum │ Superloader3 │ CC2C52 │
└──────┴─────────────────────────────┴─────────┴──────────────┴────────────┘
The Vendor, Model and Serial number are auto detected, but only shown
if the device is online.
To test your setup, please query the status of the changer device with:
.. code-block:: console
# proxmox-tape changer status sl3
┌───────────────┬──────────┬────────────┬─────────────┐
│ entry-kind │ entry-id │ changer-id │ loaded-slot │
╞═══════════════╪══════════╪════════════╪═════════════╡
│ drive │ 0 │ vtape1 │ 1 │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 1 │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 2 │ vtape2 │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ ... │ ... │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 16 │ │ │
└───────────────┴──────────┴────────────┴─────────────┘
Tape libraries usually provide some special import/export slots (also
called "mail slots"). Tapes inside those slots are accessible from
outside, making it easy to add/remove tapes to/from the library. Those
tapes are considered to be "offline", so backup jobs will not use
them. Those special slots are auto-detected and marked as an
``import-export`` slot in the status command.
It's worth noting that some of the smaller tape libraries don't have
such slots. While they have something called a "Mail Slot", that slot
is just a way to grab the tape from the gripper. They are unable
to hold media while the robot does other things. They also do not
expose that "Mail Slot" over the SCSI interface, so you wont see them in
the status output.
As a workaround, you can mark some of the normal slots as export
slot. The software treats those slots like real ``import-export``
slots, and the media inside those slots is considered to be 'offline'
(not available for backup):
.. code-block:: console
# proxmox-tape changer update sl3 --export-slots 15,16
After that, you can see those artificial ``import-export`` slots in
the status output:
.. code-block:: console
# proxmox-tape changer status sl3
┌───────────────┬──────────┬────────────┬─────────────┐
│ entry-kind │ entry-id │ changer-id │ loaded-slot │
╞═══════════════╪══════════╪════════════╪═════════════╡
│ drive │ 0 │ vtape1 │ 1 │
├───────────────┼──────────┼────────────┼─────────────┤
│ import-export │ 15 │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ import-export │ 16 │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 1 │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 2 │ vtape2 │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ ... │ ... │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 14 │ │ │
└───────────────┴──────────┴────────────┴─────────────┘
.. _tape_drive_config:
Tape drives
~~~~~~~~~~~
Linux is able to auto detect tape drives, and you can get a list
of available tape drives using:
.. code-block:: console
# proxmox-tape drive scan
┌────────────────────────────────┬────────┬─────────────┬────────┐
│ path │ vendor │ model │ serial │
╞════════════════════════════════╪════════╪═════════════╪════════╡
│ /dev/tape/by-id/scsi-12345-nst │ IBM │ ULT3580-TD4 │ 12345 │
└────────────────────────────────┴────────┴─────────────┴────────┘
In order to use that drive with Proxmox, you need to create a
configuration entry:
.. code-block:: console
# proxmox-tape drive create mydrive --path /dev/tape/by-id/scsi-12345-nst
.. Note:: Please use the persistent device path names from inside
``/dev/tape/by-id/``. Names like ``/dev/nst0`` may point to a
different device after reboot, and that is not what you want.
If you have a tape library, you also need to set the associated
changer device:
.. code-block:: console
# proxmox-tape drive update mydrive --changer sl3 --changer-drivenum 0
The ``--changer-drivenum`` is only necessary if the tape library
includes more than one drive (the changer status command lists all
drive numbers).
You can display the final configuration with:
.. code-block:: console
# proxmox-tape drive config mydrive
┌─────────┬────────────────────────────────┐
│ Name │ Value │
╞═════════╪════════════════════════════════╡
│ name │ mydrive │
├─────────┼────────────────────────────────┤
│ path │ /dev/tape/by-id/scsi-12345-nst │
├─────────┼────────────────────────────────┤
│ changer │ sl3 │
└─────────┴────────────────────────────────┘
.. NOTE:: The ``changer-drivenum`` value 0 is not stored in the
configuration, because it is the default.
To list all configured drives use:
.. code-block:: console
# proxmox-tape drive list
┌──────────┬────────────────────────────────┬─────────┬────────┬─────────────┬────────┐
│ name │ path │ changer │ vendor │ model │ serial │
╞══════════╪════════════════════════════════╪═════════╪════════╪═════════════╪════════╡
│ mydrive │ /dev/tape/by-id/scsi-12345-nst │ sl3 │ IBM │ ULT3580-TD4 │ 12345 │
└──────────┴────────────────────────────────┴─────────┴────────┴─────────────┴────────┘
The Vendor, Model and Serial number are auto detected, but only shown
if the device is online.
For testing, you can simply query the drive status with:
.. code-block:: console
# proxmox-tape status --drive mydrive
┌───────────┬────────────────────────┐
│ Name │ Value │
╞═══════════╪════════════════════════╡
│ blocksize │ 0 │
├───────────┼────────────────────────┤
│ status │ DRIVE_OPEN | IM_REP_EN │
└───────────┴────────────────────────┘
.. NOTE:: Blocksize should always be 0 (variable block size
mode). This is the default anyway.
.. _tape_media_pool_config:
Media Pools
~~~~~~~~~~~
A media pool is a logical container for tapes. A backup job targets
one media pool, so a job only uses tapes from that pool.
.. topic:: Media Set
A media set is a group of continuously written tapes, used to split
the larger pool into smaller, restorable units. One or more backup
jobs write to a media set, producing an ordered group of
tapes. Media sets are identified by a unique ID. That ID and the
sequence number are stored on each tape of that set (tape label).
Media sets are the basic unit for restore tasks. This means that you need
every tape in the set to restore the media set contents. Data is fully
deduplicated inside a media set.
.. topic:: Media Set Allocation Policy
The pool additionally defines how long backup jobs can append data
to a media set. The following settings are possible:
- Try to use the current media set.
This setting produces one large media set. While this is very
space efficient (deduplication, no unused space), it can lead to
long restore times, because restore jobs need to read all tapes in the
set.
.. NOTE:: Data is fully deduplicated inside a media set. This
also means that data is randomly distributed over the tapes in
the set. Thus, even if you restore a single VM, data may have to be
read from all tapes inside the media set.
Larger media sets are also more error-prone, because a single
damaged tape makes the restore fail.
Usage scenario: Mostly used with tape libraries. You manually
trigger new set creation by running a backup job with the
``--export`` option.
.. NOTE:: Retention period starts with the existence of a newer
media set.
- Always create a new media set.
With this setting, each backup job creates a new media set. This
is less space efficient, because the media from the last set
may not be fully written, leaving the remaining space unused.
The advantage is that this procudes media sets of minimal
size. Small sets are easier to handle, can be moved more conveniently
to an off-site vault, and can be restored much faster.
.. NOTE:: Retention period starts with the creation time of the
media set.
- Create a new set when the specified Calendar Event triggers.
.. _systemd.time manpage: https://manpages.debian.org/buster/systemd/systemd.time.7.en.html
This allows you to specify points in time by using systemd like
Calendar Event specifications (see `systemd.time manpage`_).
For example, the value ``weekly`` (or ``Mon *-*-* 00:00:00``)
will create a new set each week.
This balances between space efficiency and media count.
.. NOTE:: Retention period starts when the calendar event
triggers.
Additionally, the following events may allocate a new media set:
- Required tape is offline (and you use a tape library).
- Current set contains damaged or retired tapes.
- Media pool encryption has changed
- Database consistency errors, for example, if the inventory does not
contain the required media information, or it contains conflicting
information (outdated data).
.. topic:: Retention Policy
Defines how long we want to keep the data.
- Always overwrite media.
- Protect data for the duration specified.
We use systemd like time spans to specify durations, e.g. ``2
weeks`` (see `systemd.time manpage`_).
- Never overwrite data.
.. topic:: Hardware Encryption
LTO-4 (or later) tape drives support hardware encryption. If you
configure the media pool to use encryption, all data written to the
tapes is encrypted using the configured key.
This way, unauthorized users cannot read data from the media,
for example, if you loose a tape while shipping to an offsite location.
.. Note:: If the backup client also encrypts data, data on the tape
will be double encrypted.
The password protected key is stored on each medium, so that it is
possbible to `restore the key <tape_restore_encryption_key_>`_ using
the password. Please make sure to remember the password, in case
you need to restore the key.
.. NOTE:: We use global content namespace, meaning we do not store the
source datastore name. Because of this, it is impossible to distinguish
store1:/vm/100 from store2:/vm/100. Please use different media pools
if the sources are from different namespaces with conflicting names
(for example, if the sources are from different Proxmox VE clusters).
The following command creates a new media pool:
.. code-block:: console
// proxmox-tape pool create <name> --drive <string> [OPTIONS]
# proxmox-tape pool create daily --drive mydrive
Additional option can be set later, using the update command:
.. code-block:: console
# proxmox-tape pool update daily --allocation daily --retention 7days
To list all configured pools use:
.. code-block:: console
# proxmox-tape pool list
┌───────┬──────────┬────────────┬───────────┬──────────┐
│ name │ drive │ allocation │ retention │ template │
╞═══════╪══════════╪════════════╪═══════════╪══════════╡
│ daily │ mydrive │ daily │ 7days │ │
└───────┴──────────┴────────────┴───────────┴──────────┘
.. _tape_backup_job_config:
Tape Backup Jobs
~~~~~~~~~~~~~~~~
To automate tape backup, you can configure tape backup jobs which
write datastore content to a media pool, based on a specific time schedule.
The required settings are:
- ``store``: The datastore you want to backup
- ``pool``: The media pool - only tape cartridges from that pool are
used.
- ``drive``: The tape drive.
- ``schedule``: Job schedule (see :ref:`calendar-event-scheduling`)
For example, to configure a tape backup job for datastore ``vmstore1``
use:
.. code-block:: console
# proxmox-tape backup-job create job2 --store vmstore1 \
--pool yourpool --drive yourdrive --schedule daily
The backup includes all snapshots from a backup group by default. You can
set the ``latest-only`` flag to include only the latest snapshots:
.. code-block:: console
# proxmox-tape backup-job update job2 --latest-only
Backup jobs can use email to send tape request notifications or
report errors. You can set the notification user with:
.. code-block:: console
# proxmox-tape backup-job update job2 --notify-user root@pam
.. Note:: The email address is a property of the user (see :ref:`user_mgmt`).
It is sometimes useful to eject the tape from the drive after a
backup. For a standalone drive, the ``eject-media`` option ejects the
tape, making sure that the following backup cannot use the tape
(unless someone manually loads the tape again). For tape libraries,
this option unloads the tape to a free slot, which provides better
dust protection than inside a drive:
.. code-block:: console
# proxmox-tape backup-job update job2 --eject-media
.. Note:: For failed jobs, the tape remains in the drive.
For tape libraries, the ``export-media`` option moves all tapes from
the media set to an export slot, making sure that the following backup
cannot use the tapes. An operator can pick up those tapes and move them
to a vault.
.. code-block:: console
# proxmox-tape backup-job update job2 --export-media
.. Note:: The ``export-media`` option can be used to force the start
of a new media set, because tapes from the current set are no
longer online.
It is also possible to run backup jobs manually:
.. code-block:: console
# proxmox-tape backup-job run job2
To remove a job, please use:
.. code-block:: console
# proxmox-tape backup-job remove job2
Administration
--------------
Many sub-commands of the ``proxmox-tape`` command line tools take a
parameter called ``--drive``, which specifies the tape drive you want
to work on. For convenience, you can set this in an environment
variable:
.. code-block:: console
# export PROXMOX_TAPE_DRIVE=mydrive
You can then omit the ``--drive`` parameter from the command. If the
drive has an associated changer device, you may also omit the changer
parameter from commands that needs a changer device, for example:
.. code-block:: console
# proxmox-tape changer status
should display the changer status of the changer device associated with
drive ``mydrive``.
Label Tapes
~~~~~~~~~~~
By default, tape cartridges all look the same, so you need to put a
label on them for unique identification. First, put a sticky paper
label with some human readable text on the cartridge.
If you use a `Tape Library`_, you should use an 8 character string
encoded as `Code 39`_, as defined in the `LTO Ultrium Cartridge Label
Specification`_. You can either buy such barcode labels from your
cartridge vendor, or print them yourself. You can use our `LTO Barcode
Generator`_ app to print them.
Next, you need to write that same label text to the tape, so that the
software can uniquely identify the tape too.
For a standalone drive, manually insert the new tape cartridge into the
drive and run:
.. code-block:: console
# proxmox-tape label --changer-id <label-text> [--pool <pool-name>]
You may omit the ``--pool`` argument to allow the tape to be used by any pool.
.. Note:: For safety reasons, this command fails if the tape contains
any data. If you want to overwrite it anyway, erase the tape first.
You can verify success by reading back the label:
.. code-block:: console
# proxmox-tape read-label
┌─────────────────┬──────────────────────────────────────┐
│ Name │ Value │
╞═════════════════╪══════════════════════════════════════╡
│ changer-id │ vtape1 │
├─────────────────┼──────────────────────────────────────┤
│ uuid │ 7f42c4dd-9626-4d89-9f2b-c7bc6da7d533 │
├─────────────────┼──────────────────────────────────────┤
│ ctime │ Wed Jan 6 09:07:51 2021 │
├─────────────────┼──────────────────────────────────────┤
│ pool │ daily │
├─────────────────┼──────────────────────────────────────┤
│ media-set-uuid │ 00000000-0000-0000-0000-000000000000 │
├─────────────────┼──────────────────────────────────────┤
│ media-set-ctime │ Wed Jan 6 09:07:51 2021 │
└─────────────────┴──────────────────────────────────────┘
.. NOTE:: The ``media-set-uuid`` using all zeros indicates an empty
tape (not used by any media set).
If you have a tape library, apply the sticky barcode label to the tape
cartridges first. Then load those empty tapes into the library. You
can then label all unlabeled tapes with a single command:
.. code-block:: console
# proxmox-tape barcode-label [--pool <pool-name>]
Run Tape Backups
~~~~~~~~~~~~~~~~
To manually run a backup job use:
.. code-block:: console
# proxmox-tape backup <store> <pool> [OPTIONS]
The following options are available:
--eject-media Eject media upon job completion.
It is normally good practice to eject the tape after use. This unmounts the
tape from the drive and prevents the tape from getting dusty.
--export-media-set Export media set upon job completion.
After a successful backup job, this moves all tapes from the used
media set into import-export slots. The operator can then pick up
those tapes and move them to a media vault.
Restore from Tape
~~~~~~~~~~~~~~~~~
Restore is done at media-set granularity, so you first need to find
out which media set contains the data you want to restore. This
information is stored in the media catalog. If you do not have media
catalogs, you need to restore them first. Please note that you need
the catalog to find your data, but restoring a complete media-set does
not need media catalogs.
The following command lists the media content (from catalog):
.. code-block:: console
# proxmox-tape media content
┌────────────┬──────┬──────────────────────────┬────────┬────────────────────────────────┬──────────────────────────────────────┐
│ label-text │ pool │ media-set-name │ seq-nr │ snapshot │ media-set-uuid │
╞════════════╪══════╪══════════════════════════╪════════╪════════════════════════════════╪══════════════════════════════════════╡
│ TEST01L8 │ p2 │ Wed Jan 13 13:55:55 2021 │ 0 │ vm/201/2021-01-11T10:43:48Z │ 9da37a55-aac7-4deb-91c6-482b3b675f30 │
├────────────┼──────┼──────────────────────────┼────────┼────────────────────────────────┼──────────────────────────────────────┤
│ ... │ ... │ ... │ ... │ ... │ ... │
└────────────┴──────┴──────────────────────────┴────────┴────────────────────────────────┴──────────────────────────────────────┘
A restore job reads the data from the media set and moves it back to
data disk (datastore):
.. code-block:: console
// proxmox-tape restore <media-set-uuid> <datastore>
# proxmox-tape restore 9da37a55-aac7-4deb-91c6-482b3b675f30 mystore
Update Inventory
~~~~~~~~~~~~~~~~
Restore Catalog
~~~~~~~~~~~~~~~
Encryption Key Management
~~~~~~~~~~~~~~~~~~~~~~~~~
Creating a new encryption key:
.. code-block:: console
# proxmox-tape key create --hint "tape pw 2020"
Tape Encryption Key Password: **********
Verify Password: **********
"14:f8:79:b9:f5:13:e5:dc:bf:b6:f9:88:48:51:81:dc:79:bf:a0:22:68:47:d1:73:35:2d:b6:20:e1:7f:f5:0f"
List existing encryption keys:
.. code-block:: console
# proxmox-tape key list
┌───────────────────────────────────────────────────┬───────────────┐
│ fingerprint │ hint │
╞═══════════════════════════════════════════════════╪═══════════════╡
│ 14:f8:79:b9:f5:13:e5:dc: ... :b6:20:e1:7f:f5:0f │ tape pw 2020 │
└───────────────────────────────────────────────────┴───────────────┘
To show encryption key details:
.. code-block:: console
# proxmox-tape key show 14:f8:79:b9:f5:13:e5:dc:...:b6:20:e1:7f:f5:0f
┌─────────────┬───────────────────────────────────────────────┐
│ Name │ Value │
╞═════════════╪═══════════════════════════════════════════════╡
│ kdf │ scrypt │
├─────────────┼───────────────────────────────────────────────┤
│ created │ Sat Jan 23 14:47:21 2021 │
├─────────────┼───────────────────────────────────────────────┤
│ modified │ Sat Jan 23 14:47:21 2021 │
├─────────────┼───────────────────────────────────────────────┤
│ fingerprint │ 14:f8:79:b9:f5:13:e5:dc:...:b6:20:e1:7f:f5:0f │
├─────────────┼───────────────────────────────────────────────┤
│ hint │ tape pw 2020 │
└─────────────┴───────────────────────────────────────────────┘
The ``paperkey`` subcommand can be used to create a QR encoded
version of a tape encryption key. The following command sends the output of the
``paperkey`` command to a text file, for easy printing:
.. code-block:: console
proxmox-tape key paperkey <fingerprint> --output-format text > qrkey.txt
.. _tape_restore_encryption_key:
Restoring Encryption Keys
^^^^^^^^^^^^^^^^^^^^^^^^^
You can restore the encryption key from the tape, using the password
used to generate the key. First, load the tape you want to restore
into the drive. Then run:
.. code-block:: console
# proxmox-tape key restore
Tepe Encryption Key Password: ***********
If the password is correct, the key will get imported to the
database. Further restore jobs automatically use any available key.
Tape Cleaning
~~~~~~~~~~~~~
LTO tape drives require regular cleaning. This is done by loading a
cleaning cartridge into the drive, which is a manual task for
standalone drives.
For tape libraries, cleaning cartridges are identified using special
labels starting with letters "CLN". For example, our tape library has a
cleaning cartridge inside slot 3:
.. code-block:: console
# proxmox-tape changer status sl3
┌───────────────┬──────────┬────────────┬─────────────┐
│ entry-kind │ entry-id │ changer-id │ loaded-slot │
╞═══════════════╪══════════╪════════════╪═════════════╡
│ drive │ 0 │ vtape1 │ 1 │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 1 │ │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 2 │ vtape2 │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ slot │ 3 │ CLN001CU │ │
├───────────────┼──────────┼────────────┼─────────────┤
│ ... │ ... │ │ │
└───────────────┴──────────┴────────────┴─────────────┘
To initiate a cleaning operation simply run:
.. code-block:: console
# proxmox-tape clean
This command does the following:
- find the cleaning tape (in slot 3)
- unload the current media from the drive (back to slot 1)
- load the cleaning tape into the drive
- run drive cleaning operation
- unload the cleaning tape (to slot 3)
Configuration Files
-------------------
``media-pool.cfg``
~~~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/media-pool/format.rst
Options
^^^^^^^
.. include:: config/media-pool/config.rst
``tape.cfg``
~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/tape/format.rst
Options
^^^^^^^
.. include:: config/tape/config.rst
``tape-job.cfg``
~~~~~~~~~~~~~~~~
File Format
^^^^^^^^^^^
.. include:: config/tape-job/format.rst
Options
^^^^^^^
.. include:: config/tape-job/config.rst
Command Syntax
--------------
``proxmox-tape``
----------------
.. include:: proxmox-tape/synopsis.rst
``pmt``
-------
.. include:: pmt/options.rst
....
.. include:: pmt/synopsis.rst
``pmtx``
--------
.. include:: pmtx/synopsis.rst

166
docs/technical-overview.rst Normal file
View File

@ -0,0 +1,166 @@
.. _tech_design_overview:
Technical Overview
==================
Datastores
----------
A Datastore is the logical place where :ref:`Backup Snapshots
<term_backup_snapshot>` and their chunks are stored. Snapshots consist of a
manifest, blobs, dynamic- and fixed-indexes (see :ref:`terms`), and are
stored in the following directory structure:
<datastore-root>/<type>/<id>/<time>/
The deduplication of datastores is based on reusing chunks, which are
referenced by the indexes in a backup snapshot. This means that multiple
indexes can reference the same chunks, reducing the amount of space needed to
contain the data (even across backup snapshots).
Chunks
------
A chunk is some (possibly encrypted) data with a CRC-32 checksum at the end and
a type marker at the beginning. It is identified by the SHA-256 checksum of its
content.
To generate such chunks, backup data is split either into fixed-size or
dynamically sized chunks. The same content will be hashed to the same checksum.
The chunks of a datastore are found in
<datastore-root>/.chunks/
This chunk directory is further subdivided by the first four byte of the chunks
checksum, so the chunk with the checksum
a342e8151cbf439ce65f3df696b54c67a114982cc0aa751f2852c2f7acc19a8b
lives in
<datastore-root>/.chunks/a342/
This is done to reduce the number of files per directory, as having many files
per directory can be bad for file system performance.
These chunk directories ('0000'-'ffff') will be preallocated when a datastore
is created.
Fixed-sized Chunks
^^^^^^^^^^^^^^^^^^
For block based backups (like VMs), fixed-sized chunks are used. The content
(disk image), is split into chunks of the same length (typically 4 MiB).
This works very well for VM images, since the file system on the guest most
often tries to allocate files in contiguous pieces, so new files get new
blocks, and changing existing files changes only their own blocks.
As an optimization, VMs in `Proxmox VE`_ can make use of 'dirty bitmaps', which
can track the changed blocks of an image. Since these bitmap are also a
representation of the image split into chunks, there is a direct relation
between dirty blocks of the image and chunks which need to get uploaded, so
only modified chunks of the disk have to be uploaded for a backup.
Since the image is always split into chunks of the same size, unchanged blocks
will result in identical checksums for those chunks, so such chunks do not need
to be backed up again. This way storage snapshots are not needed to find the
changed blocks.
For consistency, `Proxmox VE`_ uses a QEMU internal snapshot mechanism, that
does not rely on storage snapshots either.
Dynamically sized Chunks
^^^^^^^^^^^^^^^^^^^^^^^^
If one does not want to backup block-based systems but rather file-based
systems, using fixed-sized chunks is not a good idea, since every time a file
would change in size, the remaining data gets shifted around and this would
result in many chunks changing, reducing the amount of deduplication.
To improve this, `Proxmox Backup`_ Server uses dynamically sized chunks
instead. Instead of splitting an image into fixed sizes, it first generates a
consistent file archive (:ref:`pxar <pxar-format>`) and uses a rolling hash
over this on-the-fly generated archive to calculate chunk boundaries.
We use a variant of Buzhash which is a cyclic polynomial algorithm. It works
by continuously calculating a checksum while iterating over the data, and on
certain conditions it triggers a hash boundary.
Assuming that most files of the system that is to be backed up have not
changed, eventually the algorithm triggers the boundary on the same data as a
previous backup, resulting in chunks that can be reused.
Encrypted Chunks
^^^^^^^^^^^^^^^^
Encrypted chunks are a special case. Both fixed- and dynamically sized chunks
can be encrypted, and they are handled in a slightly different manner than
normal chunks.
The hashes of encrypted chunks are calculated not with the actual (encrypted)
chunk content, but with the plaintext content concatenated with the encryption
key. This way, two chunks of the same data encrypted with different keys
generate two different checksums and no collisions occur for multiple
encryption keys.
This is done to speed up the client part of the backup, since it only needs to
encrypt chunks that are actually getting uploaded. Chunks that exist already in
the previous backup, do not need to be encrypted and uploaded.
Caveats and Limitations
-----------------------
Notes on hash collisions
^^^^^^^^^^^^^^^^^^^^^^^^
Every hashing algorithm has a chance to produce collisions, meaning two (or
more) inputs generate the same checksum. For SHA-256, this chance is
negligible. To calculate such a collision, one can use the ideas of the
'birthday problem' from probability theory. For big numbers, this is actually
infeasible to calculate with regular computers, but there is a good
approximation:
.. math::
p(n, d) = 1 - e^{-n^2/(2d)}
Where `n` is the number of tries, and `d` is the number of possibilities.
For a concrete example lets assume a large datastore of 1 PiB, and an average
chunk size of 4 MiB. That means :math:`n = 268435456` tries, and :math:`d =
2^{256}` possibilities. Inserting those values in the formula from earlier you
will see that the probability of a collision in that scenario is:
.. math::
3.1115 * 10^{-61}
For context, in a lottery game of guessing 6 out of 45, the chance to correctly
guess all 6 numbers is only :math:`1.2277 * 10^{-7}`, that means the chance of
collission is about the same as winning 13 such lotto games *in a row*.
In conclusion, it is extremely unlikely that such a collision would occur by
accident in a normal datastore.
Additionally, SHA-256 is prone to length extension attacks, but since there is
an upper limit for how big the chunk are, this is not a problem, since a
potential attacker cannot arbitrarily add content to the data beyond that
limit.
File-based Backup
^^^^^^^^^^^^^^^^^
Since dynamically sized chunks (for file-based backups) are created on a custom
archive format (pxar) and not over the files directly, there is no relation
between files and the chunks. This means that the Proxmox Backup client has to
read all files again for every backup, otherwise it would not be possible to
generate a consistent independent pxar archive where the original chunks can be
reused. Note that there will be still only new or change chunks be uploaded.
Verification of encrypted chunks
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
For encrypted chunks, only the checksum of the original (plaintext) data is
available, making it impossible for the server (without the encryption key), to
verify its content against it. Instead only the CRC-32 checksum gets checked.

View File

@ -1,3 +1,5 @@
.. _terms:
Terminology
===========
@ -99,6 +101,7 @@ Backup Group
The tuple ``<type>/<ID>`` is called a backup group. Such a group
may contain one or more backup snapshots.
.. _term_backup_snapshot:
Backup Snapshot
---------------

Some files were not shown because too many files have changed in this diff Show More