tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,890 Commits 2 Branches 54 Tags 14 MiB
26e949d5fe
Commit Graph

1118 Commits

Author SHA1 Message Date
Dietmar Maurer
5d6379f8db tape: implement locate_file without LOCATE(10) 2021-04-08 08:34:45 +02:00
Dietmar Maurer
566b946f9b tape: pmt - re-implement lock/unlock command 2021-04-08 07:28:30 +02:00
Dietmar Maurer
7f7459677d tape: pmt - re-implement fsr/bsr 2021-04-08 07:28:30 +02:00
Dietmar Maurer
0892a512bc tape: correctly set/display drive option 2021-04-08 07:28:30 +02:00
Dietmar Maurer
7b11a8098d tape: make sure there is a filemark at the end of the tape 2021-04-08 07:28:30 +02:00
Dietmar Maurer
e29f456efc tape: implement format/erase 2021-04-08 07:28:30 +02:00
Dietmar Maurer
a79082a0dd tape: implement LTO userspace driver 2021-04-08 07:28:30 +02:00
Thomas Lamprecht
2200a38671 code cleanup: drop extra newlines at EOF 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-04-01 11:27:07 +02:00
Stefan Reiter
ff8945fd2f proxmox_client_tools: move common key related functions to key_source.rs 
Add a new module containing key-related functions and schemata from all
over, code moved is not changed as much as possible.

Requires adapting some 'use' statements across proxmox-backup-client and
putting the XDG helpers quite cozily into proxmox_client_tools/mod.rs

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2021-04-01 11:09:28 +02:00
Dietmar Maurer
c553407e98 tape: add --scan option for catalog restore 2021-03-25 13:08:34 +01:00
Dietmar Maurer
589c4dad9e tape: add fsf/bsf to TapeDriver trait 2021-03-25 10:10:16 +01:00
Dominik Csapak
0320deb0a9 proxmox-tape: fix clean api call 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-25 08:14:13 +01:00
Dominik Csapak
4c4e5c2b1e api2/tape/restore: enable restore mapping of datastores 
by changing the 'store' parameter of the restore api call to a
list of mappings (or a single default datastore)

for example giving:
a=b,c=d,e

would restore
datastore 'a' from tape to local datastore 'b'
datastore 'c' from tape to local datastore 'e'
all other datastores to 'e'

this way, only a single datastore can also be restored, by only
giving a single mapping, e.g. 'a=b'

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-25 07:46:12 +01:00
Dominik Csapak
f1d76ecf6c fix #3359: fix blocking writes in async code during pxar create 
in commit `asyncify pxar create_archive`, we changed from a
separate thread for creating a pxar to using async code, but the
StdChannelWriter used for both pxar and catalog can block, which
may block the tokio runtime for single (and probably dual) core
environments

this patch adds a wrapper struct for any writer that implements
'std::io::Write' and wraps the write calls with 'block_in_place'
so that if called in a tokio runtime, it knows that this code
potentially blocks

Fixes: 6afb60abf5 ("asyncify pxar create_archive")

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-24 09:00:07 +01:00
Dietmar Maurer
ef4bdf6b8b tape: proxmox-tape media content - add 'store' attribute 2021-03-17 11:17:54 +01:00
Dominik Csapak
e36135031d ui: tape/Restore: let the user choose an owner 
so that the tape backup can be restored as any user, given
the current logged in user has the correct permission.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-16 08:55:42 +01:00
Dominik Csapak
5a5ee0326e proxmox-tape: add missing notify-user to 'proxmox-tape restore' 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-03-16 08:54:38 +01:00
Wolfgang Bumiller
591b120d35 fix feature flag logic in pxar create 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-03-12 10:17:51 +01:00
Thomas Lamprecht
d1d74c4367 typo fixes all over the place 
found and semi-manually replaced by using:
 codespell -L mut -L crate -i 3 -w

Mostly in comments, but also email notification and two occurrences
of misspelled  'reserved' struct member, which where not used and
cargo build did not complain about the change, soo ...

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-03-10 16:39:57 +01:00
Dietmar Maurer
5ca5f8daf3 pmt: add tape-alert-flags command 2021-03-08 07:37:05 +01:00
Dietmar Maurer
40bf636b47 ui: enable experimental tape UI if tape.cfg exists 2021-03-03 09:02:02 +01:00
Dietmar Maurer
3832911d50 docs: remove backup protocol api dumps 
We now have the api-viewer for that.
 2021-02-26 19:00:08 +01:00
Dietmar Maurer
0023cfa385 tape: fix 'proxmox-tape backup' cli command 2021-02-26 08:51:24 +01:00
Dietmar Maurer
5830e5620d tape: cleanup TapeJob implementation - uses AllOf Schema with SectionConfig 2021-02-25 09:44:31 +01:00
Wolfgang Bumiller
3554fe6480 update to schema changes in proxmox 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-02-24 14:50:08 +01:00
Dietmar Maurer
2037d9af03 api-viewer: show permissions 2021-02-22 12:01:24 +01:00
Dietmar Maurer
451856d21d api-viewer: fix path for upgraded HTTP/2 proptocls 2021-02-22 08:38:27 +01:00
Dietmar Maurer
aa30663ca5 docgen: improve regex format 2021-02-21 16:14:11 +01:00
Dietmar Maurer
8616a4afe5 docgen: dump string format (property strings) 2021-02-21 15:54:14 +01:00
Dietmar Maurer
bc2358319b docgen: improve api schema dump 2021-02-21 12:00:06 +01:00
Dietmar Maurer
0bf4b81370 docs: api-viewer - add backup and restore APIs 2021-02-21 10:38:33 +01:00
Dietmar Maurer
fee0fe5422 docgen: implement api schema generation (for api-viewer) 
Just a start - not complete jet.
 2021-02-21 09:06:58 +01:00
Dietmar Maurer
cd44fb8d84 tape: create tmp dirs early at server startup 2021-02-19 18:10:02 +01:00
Dietmar Maurer
4188fd59a0 tape: cache changer state 2021-02-19 16:48:19 +01:00
Dominik Csapak
926d05ef0b api2/tape/{backup, restore}, proxmox-tape: set device state 
set the drive device state everywhere we lock it, so that we
know what it currently does

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-19 10:16:30 +01:00
Dietmar Maurer
9bbd83b1f2 tape: media_list API - allow to update online status for a single changer 2021-02-18 10:59:33 +01:00
Dietmar Maurer
d0647e5a02 tape: use worker tasks for media load/unload 2021-02-18 09:04:51 +01:00
Stefan Reiter
f1a83e9759 client: extract common functions to proxmox_client_tools module 
...including common schemata, connect(), extract_*() and completion
functions.

For later use with proxmox-file-restore binary.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2021-02-17 10:13:10 +01:00
Stefan Reiter
6afb60abf5 asyncify pxar create_archive 
...to take advantage of the aio::Encoder from the pxar create.

Rather straightforward conversion, but does require getting rid of
references in the Archiver struct, and thus has to be given the Mutex
for the catalog directly. The callback is boxed.

archive_dir_contents can call itself recursively, and thus needs to
return a boxed future.

Users are adjusted, namely PxarBackupStream is converted to use an
Abortable future instead of a thread so it supports async in its handler
function, and the pxar bin create_archive is converted to an async API
function. One test case is made to just use 'block_on'.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-02-17 09:24:20 +01:00
Dietmar Maurer
41a8db3576 tape: implement API to run tape backup jobs 2021-02-15 11:09:30 +01:00
Dietmar Maurer
adb65b9889 tape: fix proxmox-tape pool config command 2021-02-15 08:58:05 +01:00
Dietmar Maurer
8513626b9f proxmox-backup-proxy: schedule tape backup jobs 2021-02-15 08:30:42 +01:00
Dietmar Maurer
7ca0ba4515 tape: add tape-job.cfg manual page 2021-02-15 07:05:39 +01:00
Dietmar Maurer
be327dbccd tape: add tape backup job configuration 2021-02-14 10:35:51 +01:00
Dominik Csapak
85cdc4f371 proxmox-tape: change lookup_drive_name to extract_drive_name 
in most uses, we want to remove the drive from the param afterwards
where we don't, we already overwrite it with the result of this function

this fixes some commands (like 'proxmox-tape read-label --drive foo')
that failed with:

parameter 'drive': duplicate parameter.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-02-11 14:22:50 +01:00
Dietmar Maurer
4a874665eb proxmox-backup-manger verify-job CLI 
Add missing command line interface to manage verification jobs.
 2021-02-11 14:06:44 +01:00
Dietmar Maurer
5b7f44555e docs: add manual page for verification.cfg 2021-02-11 13:40:11 +01:00
Dietmar Maurer
2ca396c015 docs: add manual page for acl.cfg 2021-02-11 13:13:34 +01:00
Dietmar Maurer
9f35e44681 fix proxmox_backup_manager remote list: s/userid/auth-id/ 2021-02-11 11:31:20 +01:00
Dietmar Maurer
6279b8f5a5 docs: add manukal page for remote.cfg 2021-02-11 11:25:15 +01:00
Dietmar Maurer
3084232cb5 depend on proxmox 0.10.5 2021-02-11 10:49:16 +01:00
Dietmar Maurer
2322a980d0 docs: add utility binary to generate docs 2021-02-10 11:05:02 +01:00
Fabian Grünbichler
15d2c7786e client: move test output files to target/testout 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-08 09:52:16 +01:00
Dietmar Maurer
fbf8779388 doc: improve pmt/pmtx manual pages 2021-02-06 16:50:56 +01:00
Fabian Grünbichler
2f26b8668a client: track key source, print when used 
to avoid confusing messages about using encryption keys when restoring
plaintext backups, or about loading master keys when they are not
actually used for the current operation.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 09:07:31 +01:00
Fabian Grünbichler
9432838914 client: refactor crypto_parameter handling 
pull out the crypt-mode to logically group arms and make the whole mess
a bit more "human-parsable".

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 09:06:28 +01:00
Fabian Grünbichler
1a89a7794e client: extend tests for master key handling 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 09:05:01 +01:00
Fabian Grünbichler
c0a87c12fb client: allow passing specific master key 
it's needed for PVE's LXC integration, and might be interesting for
other more special usage scenarios as well.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 09:04:42 +01:00
Fabian Grünbichler
c6a7ea0a2f client: refactor keyfile_parameters 
no semantic changes intended

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 09:03:03 +01:00
Fabian Grünbichler
5bb057e5a2 client: add test for keyfile_parameters 
this will get more complex soon, so add test to document current
behaviour.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 08:58:15 +01:00
Fabian Grünbichler
2924b37d6d key: rustfmt module 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 08:56:19 +01:00
Fabian Grünbichler
42c0f784e2 key: add show-master-pubkey command 
and print public key when generating/importing..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 08:55:21 +01:00
Fabian Grünbichler
05f17d1ec4 key: make 'default' master key explicit 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-02-06 08:52:11 +01:00
Dietmar Maurer
4c209d6b10 install pmt binary 2021-02-05 12:42:20 +01:00
Dietmar Maurer
25aa55b5f5 tape: correctly lock the tape device 2021-02-05 10:50:21 +01:00
Dietmar Maurer
f47e035721 tape: cleanup - move tape file readers/writers into src/tape/file_formats folder 2021-02-04 07:59:37 +01:00
Dietmar Maurer
0d4e4cae7f tape: improve pmt command line completion 2021-02-03 08:54:12 +01:00
Dietmar Maurer
8937c65951 tape: add pmt stoptions/stsethoptions/stclearoptions 2021-02-02 08:58:02 +01:00
Dietmar Maurer
6c6ad82d90 tape: add pmt setblk 2021-02-02 07:19:54 +01:00
Dietmar Maurer
d0f11b66f7 thape: add read_tapedev_options, display driver options with status command 2021-02-02 06:40:40 +01:00
Dietmar Maurer
19e4a36c70 tape: do not use drive.open() within pmt 
Do not fail if no media is loaded. Inportant for load command.
 2021-02-01 12:39:50 +01:00
Dietmar Maurer
90769e5694 tape: add pmt lock/unlock 2021-02-01 12:18:55 +01:00
Dietmar Maurer
d690d14568 tape: add pmt bsr/fsr 2021-02-01 10:39:04 +01:00
Dietmar Maurer
85ef624440 tape: add pmt asf 2021-02-01 10:32:21 +01:00
Dietmar Maurer
e995996290 tape: pmt - fix count parameter schema 2021-02-01 10:21:25 +01:00
Dietmar Maurer
8e6ad4301d tape: add pmt fsfm/bsfm, pass count as arg_param 2021-02-01 10:18:18 +01:00
Dietmar Maurer
9883b54cba tape: remove drive from pool config 2021-02-01 09:14:28 +01:00
Dietmar Maurer
83b8949a98 tape: add pmt weof 2021-01-31 17:33:07 +01:00
Dietmar Maurer
28f60e5291 cleanup: avoid compiler warnings 2021-01-31 17:02:55 +01:00
Dietmar Maurer
1f31d06f48 tape: add pmt bsf 2021-01-31 17:00:15 +01:00
Dietmar Maurer
2f2e83c890 tape: add pmt fsf 2021-01-31 16:54:16 +01:00
Dietmar Maurer
b22c618734 tape: add pmt erase 2021-01-31 16:34:10 +01:00
Dietmar Maurer
1e041082bb tape: add pmt command line tool 
Experimental, not installed by now.
 2021-01-31 16:19:53 +01:00
Dietmar Maurer
9586ce2f46 tape: move scan_drives API code to correct file 2021-01-30 08:03:17 +01:00
Dietmar Maurer
c297835b01 tape: proxmox-tape - use API instead of direct functions calls 2021-01-29 11:49:11 +01:00
Dietmar Maurer
e68269fcaf tape: proxmox-tape inventory: call API 2021-01-29 11:21:57 +01:00
Dietmar Maurer
5243df4712 tape: proxmox-tape - use API instead of direct functions calls 2021-01-29 10:50:11 +01:00
Dietmar Maurer
4470eba551 cleanup: factor out common client code to view task log/result 2021-01-29 10:10:04 +01:00
Dietmar Maurer
41dacd5d3d tape: use worker task for eject-media api 2021-01-28 16:49:08 +01:00
Dietmar Maurer
eb1dfb02b5 tape: proxmox-tape - use api for erase-media and rewind 2021-01-28 16:36:10 +01:00
Dominik Csapak
627d000098 tape: change changer-drive-id to changer-drivenum 
because it changed in the config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 15:11:22 +01:00
Dietmar Maurer
87f4be7998 tape: use api to run proxmox-tape backup 2021-01-28 14:56:42 +01:00
Dominik Csapak
5fdaecf6f4 api2/tape/drive: reorganize drive api 
similar to the changers, create a listing at /tape/drive and put
the specific api calls below that

move the scan api call up one level

remove the status info from the config listing

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-28 11:15:17 +01:00
Dietmar Maurer
651a61f559 pmtx: implement scan command 2021-01-27 12:40:51 +01:00
Dietmar Maurer
887f1cb90c cleanup: move scan changers API implementation 2021-01-27 09:58:16 +01:00
Dominik Csapak
740dc9d1d4 api2/tape/changer: reorganize api 
add a changer listing here (copied from api2/config/changer)
and put the status and transfer api calls below that

puts the changer scan into the top level tape api
and removes the (now redundant) info from the config api path

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2021-01-26 12:47:34 +01:00
Dietmar Maurer
c3747b93c8 tape: add new command line tool "pmtx" 
Also improve sgutil2 error reporting
 2021-01-26 11:57:15 +01:00
Fabian Grünbichler
4d104cd4d8 clippy: more misc fixes 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:55 +01:00
Fabian Grünbichler
367c0ff7c6 clippy: allow api functions with many arguments 
some of those can be reduced/cleaned up when we have updater support in
the api macro.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:52 +01:00
Fabian Grünbichler
93e3581ce7 derive/impl and use Default for some structs 
and revamp HttpClientOptions with two constructors for the common use
cases

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:45 +01:00
Fabian Grünbichler
72064fd0df pxar: extract PxarExtractOptions 
same as PxarCreateOptions, but for extraction/restore rather than
create.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:36 +01:00
Fabian Grünbichler
77486a608e pxar: factor out PxarCreateOptions 
containing the CLI parameters that are mostly passed-through from the
client to our pxar archive creation wrapper in pxar::create

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:32 +01:00
Fabian Grünbichler
e97025ab02 pxar: typedef on_error as ErrorHandler 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:26 +01:00
Fabian Grünbichler
e43b9175c0 client: factor out UploadOptions 
to reduce function signature complexity.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:54:18 +01:00
Fabian Grünbichler
6d233161b0 client: refactor catalog upload spawning 
by pulling out Result type into separate struct

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-26 09:53:51 +01:00
Dietmar Maurer
64b83c3d70 tape: implement paperkey command for tape encryption keys 2021-01-22 09:56:14 +01:00
Dietmar Maurer
639a6782bd paperkey: move code to src/tools/paperkey.rs 2021-01-22 09:42:59 +01:00
Dietmar Maurer
5f34d69bcc tape: add volume-statistics api/command 2021-01-22 08:45:35 +01:00
Dietmar Maurer
337ff5a3cc tape: add estimated medium wearout to status 2021-01-22 08:06:25 +01:00
Dietmar Maurer
645a044bf6 tape: further hierarchy improvements 2021-01-21 17:25:32 +01:00
Dietmar Maurer
37796ff73f tape: change code hierarchy to improve docs 2021-01-21 17:12:01 +01:00
Dietmar Maurer
1c86893d95 cleanup: always compute fingerprint in KeyConfig constructors 2021-01-21 11:56:54 +01:00
Dietmar Maurer
d543587d34 Merge branch 'master' of ssh://proxdev.maurer-it.com/rust/proxmox-backup 2021-01-21 10:56:52 +01:00
Dietmar Maurer
2b191385ea tape: use specialized encryption key per media-set 2021-01-21 10:31:49 +01:00
Fabian Grünbichler
ea368a06cd clippy: misc. fixes 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
17c7b46a69 clippy: use unwrap_or_default 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
a375df6f4c clippy: use copied/cloned instead of map 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
6334bdc1c5 clippy: collapse nested ifs 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
3b82f3eea5 clippy: avoid useless format! 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
38556bf60d clippy: remove explicit returns 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
3984a5fd77 clippy: is_some/none/ok/err/empty 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:23:54 +01:00
Fabian Grünbichler
22a9189ee0 clippy: remove unnecessary closures 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:22:59 +01:00
Fabian Grünbichler
4428818412 clippy: remove unnecessary clones 
and from::<T>(T)

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-20 16:22:59 +01:00
Dietmar Maurer
69b8bc3bfa tape: implemenmt show key 
Moved API types Kdf and KeyInfo to src/api2/types/mod.rs.
 2021-01-20 15:43:19 +01:00
Dietmar Maurer
301b8aa0a5 tape: implement change-passphrase for tape encryption keys 2021-01-20 15:43:19 +01:00
Dietmar Maurer
e5b6c93323 tape: add --kdf parameter to create key api 2021-01-20 15:43:19 +01:00
Dietmar Maurer
9a045790ed cleanup KeyConfig 2021-01-20 15:43:19 +01:00
Dietmar Maurer
82a103c8f9 add "password hint" to KeyConfig 2021-01-20 15:43:19 +01:00
Dietmar Maurer
feb1645f37 tape: generate random encryptions keys and store key_config on media 2021-01-19 11:20:07 +01:00
Dietmar Maurer
8a0046f519 tape: implement encrypted backup - simple version 
This is just a proof of concept, only storing the encryption key fingerprint
inside the media-set label.
 2021-01-18 13:38:22 +01:00
Dietmar Maurer
1e93fbb5c1 tape: add encrypt property to media pool configuration 2021-01-18 13:38:22 +01:00
Dietmar Maurer
d5a48b5ce4 tape: add hardware encryption key managenent api 2021-01-18 13:38:22 +01:00
Fabian Grünbichler
0bfcea6a11 cleanup: remove unnecessary 'mut' and '.clone()' 
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-14 16:01:33 +01:00
Fabian Grünbichler
b5a202acb6 tokio 1.0: update to new Signal interface 
Signal does not yet re-implement Stream (and is not yet wrapped in
tokio-stream either).

see https://github.com/tokio-rs/tokio/pull/3383

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-14 16:01:33 +01:00
Fabian Grünbichler
0f860f712f tokio 1.0: update to new tokio-openssl interface 
connect/accept are now happening on pinned SslStreams

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-14 16:01:33 +01:00
Fabian Grünbichler
7c66701366 tokio 1.0: use ReceiverStream from tokio-stream 
to wrap a Receiver in a Stream. this will likely move back into tokio
proper once we have a std Stream..

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-14 16:01:33 +01:00
Fabian Grünbichler
0a8d773ad0 tokio 1.0: delay -> sleep 
almost the same thing, new name(s), no longer Unpin

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2021-01-14 16:01:33 +01:00
Thomas Lamprecht
1a0b410554 manager: user/token list: fix rendering 0 (never) expire date 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-01-14 13:59:08 +01:00
Dietmar Maurer
2d50a6192f tape: sg-tape-cmd - add more ways to specify devices 2021-01-14 13:05:26 +01:00
Dietmar Maurer
781da7f6f0 tape: add --inventorize flag to read-label API/CLI 2021-01-14 11:51:23 +01:00
Dietmar Maurer
8446fbca85 tape: rename changer_id to label_text 2021-01-13 13:26:59 +01:00
Wolfgang Bumiller
7f066a9b21 proxy: expose qrcodejs 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2021-01-11 10:22:32 +01:00
Dietmar Maurer
483da89d03 tape: improve export media to directly export from drive, add CLI 2021-01-10 13:44:44 +01:00
Dietmar Maurer
edb90f6afa tape: backup - implement export-media-set option 2021-01-10 11:59:55 +01:00
Dietmar Maurer
42967bf185 tape: backup - implement --eject-media option 2021-01-09 15:17:03 +01:00
Dietmar Maurer
5843268c47 tape: abort backup when we detect critical tape alert flags 2021-01-09 12:34:00 +01:00
Dietmar Maurer
0bf1c314da tape: show catalog status in media list 2021-01-09 10:24:48 +01:00
Dietmar Maurer
df69a4fc59 tape: implement drive clean 2021-01-08 11:32:56 +01:00
Dietmar Maurer
879569d73f tape: changer transfer - make name parameter optional 2021-01-07 17:09:47 +01:00
Dietmar Maurer
b63f833d36 tape: fix paramater name - s/slot/source-slot/ 2021-01-07 15:39:25 +01:00
Dietmar Maurer
482c6e33dd tape: changer status command: make changer name optional 2021-01-07 15:12:19 +01:00
Dietmar Maurer
46a1863f88 tape: improve MediaChange trait 
We expose the whole MtxStatus, and we can load/store from/to
specified slot numbers.
 2021-01-07 14:26:43 +01:00
Dietmar Maurer
38ae42b11a tape: changer - add export-slot config 2021-01-06 11:06:50 +01:00
Dietmar Maurer
b40ab10d38 tape: add volume_mounts and medium_passes to LinuxDriveAndMediaStatus 2021-01-05 13:43:17 +01:00
Dietmar Maurer
f8ccbfdedd tape: implement read_volume_statistics 2021-01-05 12:58:18 +01:00
Dietmar Maurer
470f1c798a tape: status - show thape alert flags 2021-01-04 13:15:30 +01:00
Dietmar Maurer
165b641c1d tape: changer status - show full slots (for cartridge without barcode) 2021-01-04 12:06:05 +01:00
Dietmar Maurer
74595b8821 tape: sg-tape-cmd tape-alert-flags 2021-01-03 10:09:43 +01:00
Dietmar Maurer
b9b4b31284 tape: add basic restore api/command 2020-12-31 10:26:48 +01:00
Dietmar Maurer
b017bbc441 tape: add restore code, implement catalog api/command 2020-12-30 09:48:18 +01:00
Dietmar Maurer
a33389c391 tape: implement media content list api 2020-12-29 11:58:26 +01:00
Dietmar Maurer
0993923ed5 tape: factor out get_drive_and_media_status 2020-12-29 08:39:06 +01:00
Dietmar Maurer
e0362b0d0f tape: correctly parse mtx import/export slots 2020-12-28 13:32:56 +01:00
Dietmar Maurer
b27c32821c tape: install new sg-tape-cmd setuid binary 2020-12-28 11:10:25 +01:00
Dietmar Maurer
5ae86dfaa1 tape: return media usage info with status command 2020-12-23 11:24:34 +01:00
Dietmar Maurer
cb80d900b3 tape: add drive status api 2020-12-22 10:42:22 +01:00
Dietmar Maurer
ee01737e87 tape: rename 'mam' api to 'cartridge-memory' 2020-12-22 09:27:34 +01:00
Dietmar Maurer
2012825913 depend on proxmox 0.9.0 2020-12-22 08:52:24 +01:00
Wolfgang Bumiller
b2362a1207 adaptions for proxmox 0.9 and proxmox-api-macro 0.3 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-12-22 07:31:05 +01:00
Dietmar Maurer
1e20f819d5 tape: add command to read cartridge memory (MAM) 
Thsi add an additional dependency to sg3-utils (small).
 2020-12-21 12:12:33 +01:00
Dietmar Maurer
8835664653 tape: add tape backup api 2020-12-18 15:32:12 +01:00
Fabian Grünbichler
7137630d43 client: add 'import-with-master-key' command 
to import an encrypted encryption key using a master key.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-12-17 06:46:24 +01:00
Fabian Grünbichler
8acfd15d6e key: move RSA-encryption to KeyConfig 
since that is what gets encrypted, and not a CryptConfig.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-12-17 06:43:34 +01:00
Fabian Grünbichler
48fbbfeb7e fix #3197: skip fingerprint check when restoring key 
when restoring an encrypted key, the original one is obviously not
available to check the fingerprint with.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-12-17 06:37:54 +01:00
Fabian Grünbichler
9990af3042 master key: store blob name in constant 
since we will use it in more than one place.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-12-17 06:36:06 +01:00
Dietmar Maurer
c1c2c8f635 tape: cleanup MediaLocation type for direct use with API 2020-12-16 10:49:01 +01:00
Dietmar Maurer
af07ec8f29 tape: minor code cleanup 2020-12-14 16:56:26 +01:00
Dietmar Maurer
3f803af00b tape: scan - print more debug info 2020-12-14 13:16:18 +01:00
Dietmar Maurer
ac461bd651 tape: implement scan command (useful for debug) 2020-12-14 12:55:49 +01:00
Dietmar Maurer
ce955e1635 tape: implement eod cli command (debug tool) 2020-12-14 09:56:59 +01:00
Dietmar Maurer
e20d008c6a tape: rename cli 'media media-destroy' toö 'media destroy' 2020-12-14 09:30:32 +01:00
Dietmar Maurer
fb657d8ee5 tape: implement destroy_media 2020-12-14 08:58:40 +01:00
Dietmar Maurer
fba0b77469 tape: add media api 2020-12-14 07:55:57 +01:00
Dietmar Maurer
b5c1296eaa tape: make changer get_status async 2020-12-14 07:14:24 +01:00
Dietmar Maurer
7e1d4712b8 tape: rename CHANGER_ID_SCHEMA to CHANGER_NAME_SCHEMA 2020-12-13 09:22:08 +01:00
Dietmar Maurer
49c965a497 tape: rename DRIVE_ID_SCHEMA to DRIVE_NAME_SCHEMA 2020-12-13 09:18:16 +01:00
Dietmar Maurer
6fe9aedd0b tape: correctly call Async handler in proxmox-tape 2020-12-12 09:58:47 +01:00
Dietmar Maurer
663ef85992 tape: use WorkerTask for erase and rewind 2020-12-11 11:19:33 +01:00
Dietmar Maurer
e92c75815b tape: split inventory api 
inventory: sync, list labels with uuids,
update_inventory: WorkerTask, updates database
 2020-12-11 10:42:29 +01:00
Dietmar Maurer
6dbad5b4b5 tape: run label commands as WorkerTask (threads) 2020-12-11 09:10:22 +01:00
Dietmar Maurer
bff7e3f3e4 tape: implement barcode-label-mdedia 2020-12-11 07:50:19 +01:00
Dietmar Maurer
83abc7497d tape: implement inventory command 2020-12-11 07:39:28 +01:00
Dietmar Maurer
4606f34353 tape: implement read-label command 2020-12-10 13:20:39 +01:00
Dietmar Maurer
7bb720cb4d tape: implement label command 2020-12-10 12:30:27 +01:00
Dietmar Maurer
9700d5374a tape: add media pool cli 2020-12-10 11:13:12 +01:00
Dietmar Maurer
f70d8091d3 tape: implement option changer-drive-id 2020-12-10 09:09:06 +01:00
Dietmar Maurer
a3c709ef21 tape: cli cleanup - avoid api redefinition 2020-12-10 08:35:11 +01:00
Dietmar Maurer
93829fc680 tape: cleanup load-slot api 2020-12-10 08:04:55 +01:00
Dietmar Maurer
5605ca5619 tape: cli cleanup - rename scana-for-* into scan 2020-12-10 07:58:45 +01:00
Dietmar Maurer
e49f0c03d9 tape: implement load-media command 2020-12-10 07:52:56 +01:00
Dietmar Maurer
0098b712a5 tape: implement eject 2020-12-09 17:50:48 +01:00
Dietmar Maurer
5fb694e8c0 tape: implement rewind 2020-12-09 17:43:38 +01:00
Dietmar Maurer
583a68a446 tape: implement erase media 2020-12-09 17:35:31 +01:00
Dietmar Maurer
e6604cf391 tape: add command line interface proxmox-tape 2020-12-09 13:00:20 +01:00
Dietmar Maurer
6b68e5d597 client: move connect_to_localhost into client module 2020-12-09 11:59:50 +01:00
Dietmar Maurer
cafd51bf42 tape: add media state database 2020-12-09 11:21:56 +01:00
Wolfgang Bumiller
9b93c62044 remove unused descriptions from api macros 
these are now a hard error in the api macro

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-12-09 10:55:18 +01:00
Wolfgang Bumiller
35fe981c7d client: use tools::pipe instead of nix 
nix::unistd::pipe returns unguarded RawFds which should be
avoided

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-11-30 11:25:53 +01:00
Fabian Grünbichler
490560e0c6 restore: print to STDERR 
else restoring to STDOUT is broken..

Reported-by: Dominic Jäger <d.jaeger@proxmox.com>

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-25 14:38:02 +01:00
Dietmar Maurer
0c9209b04c cli: rename command "upload-log" to "snapshot upload-log" 2020-11-25 07:57:39 +01:00
Dietmar Maurer
edebd52374 cli: rename command "forget" to "snapshot forget" 2020-11-25 07:57:39 +01:00
Dietmar Maurer
61205f00fb cli: rename command "files" to "snapshot files" 2020-11-25 07:57:39 +01:00
Dietmar Maurer
731eeef25b cli: use new alias feature for "snapshots" 
Now maps to "snapshot list".
 2020-11-24 13:26:43 +01:00
Dominik Csapak
a65e3e4bc0 client: add 'snapshot notes show/update' command 
to show and update snapshot notes from the cli

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-11-24 11:44:19 +01:00
Fabian Grünbichler
8b7f8d3f3d expose previous backup time in backup env 
and use this information to add more information to client backup log
and guide the download manifest decision.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-24 08:44:55 +01:00
Fabian Grünbichler
f1e2904150 paperkey: refactor common code 
from formatting functions to main function, and pass along the key data
lines instead of the full string.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-24 07:57:21 +01:00
Fabian Grünbichler
23f9503a31 client: check fingerprint after downloading manifest 
this is stricter than the check that happened on manifest load, as it
also fails if the manifest is signed but we don't have a key available.

add some additional output at the start of a backup to indicate whether
a previous manifest is available to base the backup on.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-24 07:55:12 +01:00
Dietmar Maurer
5e17dbf2bb cli: cleanup 'key show' - use format_and_print_result_full 
We now expose all key derivation functions on the cli, so users can
choose between scrypt or pbkdf2.
 2020-11-24 07:32:34 +01:00
Fabian Grünbichler
dfb04575ad client: add 'key show' command 
for (pretty-)printing a keyfile.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-23 13:15:29 +01:00
Fabian Grünbichler
6f2626ae19 client: print key fingerprint and master key 
for operations where it makes sense.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-23 13:11:26 +01:00
Fabian Grünbichler
37e60ddcde key: add fingerprint to key config 
and set/generate it on
- key creation
- key passphrase change
- key decryption if not already set
- key encryption with master key

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-23 13:03:46 +01:00
Dominik Csapak
968a0ab261 fix systemd-encoded upid strings in http client 
since we systemd-encode parts of the upid string, and those can contain
characters that are invalid in urls (e.g. '\'), we have to percent encode
those

add a 'percent_encode_component' helper, so that we can maybe change
the AsciiSet for all uses at the same time

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-11-19 11:01:19 +01:00
Fabian Ebner
58fcbf5ab7 client: expose all-file-systems option 
Useful to avoid the need for a long (and possibly changing) list of include-dev
options in certain situations, e.g. nested ZFS file systems. The option is
already implemented and seems to work as expected. The checks for virtual
filesystems are not affected by this option.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
 2020-11-16 16:59:14 +01:00
Thomas Lamprecht
fc5a012068 manager: versions: non-verbose should actually print server pkg info 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-12 10:28:03 +01:00
Stefan Reiter
5e293f1315 apt: use typed response for get_versions 
...and cleanup get_versions for manager CLI.

Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
 2020-11-12 10:15:32 +01:00
Thomas Lamprecht
294466ee61 manager: versions: unify printing 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-11 18:30:33 +01:00
Mira Limbeck
c100fe9108 add versions command to proxmox-backup-manager 
Add the versions command to proxmox-backup-manager with a similar output
to pveversion [-v]. It prints the packages line by line with only the
package name, followed by the version and, for proxmox-backup and
proxmox-backup-server, some additional information (running kernel,
running version).

In addition it supports the optional output-format parameter which can
be used to print the complete data in either json, json-pretty or text
format. If output-format is specified, the --verbose parameter is
ignored and the detailed list of packages is printed.

With the addition of the versions command, the report is extended as
well.

Signed-off-by: Mira Limbeck <m.limbeck@proxmox.com>
 2020-11-11 18:30:33 +01:00
Stoiko Ivanov
beb07279b6 log source of encryption key 
This patch prints the source of the encryption key when running
operations with proxmox-backup-client.

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
 2020-11-11 16:35:20 +01:00
Stoiko Ivanov
8c6854c8fd inform user when using default encryption key 
Currently if you generate a default encryption key:
`proxmox-backup-client key create --kdf none`

all backup operations which don't explicitly disable encryption will be
encrypted with this key.

I found it quite surprising, that my backups were all encrypted without
me explicitly specfying neither key nor encryption mode

This patch informs the user when the default key is used (and no
crypt-mode is provided explicitly)

Signed-off-by: Stoiko Ivanov <s.ivanov@proxmox.com>
 2020-11-11 16:35:20 +01:00
Dominik Csapak
d7c6ad60dd daemon: add hack for sd_notify 
sd_notify is not synchronous, iow. it only waits until the message
reaches the queue not until it is processed by systemd

when the process that sent such a message exits before systemd could
process it, it cannot be associated to the correct pid

so in case of reloading, we send a message with 'MAINPID=<newpid>'
to signal that it will change. if now the old process exits before
systemd knows this, it will not accept the 'READY=1' message from the
child, since it rejects the MAINPID change

since there is no (AFAICS) library interface to check the unit status,
we use 'systemctl is-active <SERVICE_NAME>' to check the state until
it is not 'reloading' anymore.

on newer systemd versions, there is 'sd_notify_barrier' which would
allow us to wait for systemd to have all messages from the current
pid to be processed before acknowledging to the child, but on buster
the systemd version is to old...

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-11-11 09:43:00 +01:00
Wolfgang Bumiller
f3fde36beb client: error context when building HttpClient 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-11-10 11:58:19 +01:00
Thomas Lamprecht
0eb0c4bd63 proxy: fix log message for auth log rotation 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-09 16:34:03 +01:00
Dominik Csapak
db8a606707 proxmox-backup-proxy: remove unnecessary alias 
the basedir is already /usr/share/javascript/proxmox-backup/
so adding a subdir of that as alias is not needed

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-11-06 18:08:18 +01:00
Fabian Grünbichler
0224c3c273 client: properly complete new-owner 
with remote Authids, not local Userids.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-11-06 14:54:08 +01:00
Dietmar Maurer
ad54df3178 get rid of backup@pam 2020-11-06 08:39:30 +01:00
Fabian Grünbichler
e0100d618e api: refactor remote client and add remote scan 
to allow on-demand scanning of remote datastores accessible for the
configured remote user.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-11-05 12:56:20 +01:00
Thomas Lamprecht
fe7bdc9d29 proxy: also rotate auth.log file 
no need for triggering re-open here, we always re-open that file.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-04 17:26:34 +01:00
Thomas Lamprecht
546b6a23df proxy: logrotate: do not serialize sending async log-reopen commands 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-04 17:26:34 +01:00
Dietmar Maurer
ea93bea7bf proxy: log if there are too many open connections 2020-11-04 08:49:35 +01:00
Dietmar Maurer
e1d367df47 proxy: use env PROXMOX_DEBUG to enable/disable debug output 
We only print early connection errors when this env var is set.
 2020-11-04 06:55:57 +01:00
Dietmar Maurer
71f413cd27 cleanup: use Arc to count open connections 2020-11-04 06:35:44 +01:00
Dietmar Maurer
48aa2b93b7 fix #3106: correctly queue incoming connections 2020-11-04 06:24:42 +01:00
Thomas Lamprecht
941342f70e manager: report: call method directly, avoid HTTPS request 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-03 17:23:43 +01:00
Hannes Laimer
9a556c8a30 manager: add report cli command 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-11-03 15:16:42 +01:00
Thomas Lamprecht
a71a009313 proxy: drop now unused UPID import 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-02 21:08:38 +01:00
Dominik Csapak
b6ba5acd29 proxmox-backup-proxy: use only jobstate for garbage_collection schedule 
in case the garbage_collection errors out, we never set the in-memory
state, so if it failed, the last 'good' starttime was considered
for the schedule

this could lead to the job running every minute instead of the
correct schedule

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-11-02 21:08:38 +01:00
Dominik Csapak
c724f65805 server/gc_job: add 'to_stdout' 
we will use this for the manual api call

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-11-02 21:08:38 +01:00
Thomas Lamprecht
b7f2be5137 log rotate task: make task archive limits be binary based 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-02 19:53:30 +01:00
Thomas Lamprecht
72aa1834dc log rotate task: adapt internal jobstate ID, set worker one to None for now 
as we have only one logrotate task currently..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-02 19:53:30 +01:00
Thomas Lamprecht
fe4cc5b1a1 server: implement access log rotation with re-open via command socket 
re-use the future we already have for task log rotation to trigger
it.

Move the FileLogger in ApiConfig into an Arc, so that we can actually
update it and REST using the new one.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-11-02 19:53:30 +01:00