tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,953 Commits 2 Branches 54 Tags 14 MiB
a3eb7b2cea
Commit Graph

2444 Commits

Author SHA1 Message Date
Wolfgang Bumiller
d973aa827c introduce new runtime tokio helpers 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-01-20 13:12:40 +01:00
Christian Ebner
aac9dbf635 pxar::fuse: remove open, opendir and releasedir callbacks 
They are not needed as there is no internal state which needs to be tracked.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-20 12:56:21 +01:00
Christian Ebner
a8aff3535d pxar: Include symlink target in DirectoryEntry 
This allows to read the target path of a symbolic link in the
Decoder::read_directory_entry() function and stores it in the DirectoryEntry.
By this the Decoder::read_link() function becomes obsolete and is therefore
removed.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-20 12:56:10 +01:00
Dietmar Maurer
138910bcd4 src/client/pxar_backup_stream.rs: use std:🧵:Builder to name the thread 
To make debugging easier.
 2020-01-18 12:21:22 +01:00
Dietmar Maurer
4b4eba0b9e src/api2/pull.rs: implement delete flag for vanished groups 2020-01-17 11:24:55 +01:00
Dietmar Maurer
11d89239c3 src/backup/backup_info.rs: new list_groups helper 2020-01-17 10:42:03 +01:00
Dietmar Maurer
b31c8019d7 src/api2/types.rs: define and use GroupListItem 2020-01-17 10:17:18 +01:00
Christian Ebner
2bbbade367 pxar: Refactor fuse and remove unused code. 
By ambiguously using the Decoder::read_directory_entry() the code is simplified
and reading of the DirectoryEntry is concentrated into Context::run_in_context().

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-17 09:40:39 +01:00
Christian Ebner
95c9460c4a pxar::fuse: Always use start offset of entries as inode. 
Previously it was disciminated based on the entry mode.
For directories, the inode was the offset of the corresponding
goodbye tail mark while for all others it was the offset of the filename.

By simply using the start offset as calculated from the corresponding
goodbye table entry (which yields the archive offset of the filename),
the code is simplified and the more ambiguous read_directory_entry()
function can be used.
The disatvantage of this approach is the need to keep track of the
start and end offsets for each entry, as the end offset is needed in
order to access the goodbye table of directory entries.
The root node still has to be treated special, as it's inode is 1 as per fuse
definition and it has no filename as per the pxar file format definition.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-17 09:40:39 +01:00
Christian Ebner
6744440714 pxar::decoder: refactor Decoder::lookup() 
Use Decoder::read_directory_entry() instead of Decoder::attributes() as this
already returns the needed DirectoryEntry.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-17 09:40:39 +01:00
Dietmar Maurer
f357390c15 renamed: src/config/remotes.rs -> src/config/remote.rs 
And use 'remote' instead of 'remotes' everywhere.
 2020-01-16 14:32:06 +01:00
Dietmar Maurer
d2dd827877 src/backup/catalog.rs - SenderWriter: use tokio::task::block_in_place 
Make sure we do not block the executor.
 2020-01-16 14:24:15 +01:00
Dietmar Maurer
94609e2380 src/api2/pull.rs: use reference to remote configuration 2020-01-16 13:55:19 +01:00
Dietmar Maurer
625d19621f src/client/remote_chunk_reader.rs: use tokio::task::block_in_place to avoid blocking 
Else we block the current executor.
 2020-01-16 13:55:19 +01:00
Wolfgang Bumiller
6a0dc4a5bc bump proxmox crate version 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-01-16 11:40:23 +01:00
Wolfgang Bumiller
619495b25e fix use of deprecated items 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-01-16 08:58:54 +01:00
Wolfgang Bumiller
18c0df4c92 import ordering cleanup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-01-16 08:51:52 +01:00
Christian Ebner
ef2d7f2f96 pxar::fuse: remove readdir implementation 
By not implementing readdir but only readdirplus, the FUSE_CAP_READDIRPLUS flag
is set while the FUSE_CAP_READDIRPLUS_AUTO flag is not set.
Thereby the kernel will issue only readdirplus calls.
Documentation at:
https://libfuse.github.io/doxygen/fuse-3_88_80_2include_2fuse__common_8h.html#a9b90333ad08d0e1c2ed0134d9305ee87

As the expensive part for accessing and reading the attributes is seeking and
decoding each directory entry, it is usefull to force readdirplus calls.
By this a struct `EntryParam` is returned for each entry, therebye avoiding a
subsequent lookup call.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-15 14:44:53 +01:00
Dietmar Maurer
3be839c61c src/bin/proxmox-backup-manager.rs: add completion function for update commands 2020-01-15 12:42:13 +01:00
Dietmar Maurer
002a191abf src/api2/config: check digest 2020-01-15 12:27:05 +01:00
Dietmar Maurer
347834df25 src/api2/config: correctly lock files 2020-01-15 11:57:12 +01:00
Oguz Bektas
1c79e1017f services: fix indentation 
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
 2020-01-15 10:55:04 +01:00
Oguz Bektas
cbe01dc5c7 tools: add read_and_verify_password helper 
add a helper to perform some basic checks on password prompts.
- verification (asks for a 2nd time)
- check length

also use the new helper where password input in tty is taken to reduce
duplicate code.

this helper should be used when creating keys, changing passphrases etc.

note: this helper can be extended later on to provide better checks for
password strength.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
 2020-01-15 10:52:57 +01:00
Christian Ebner
ea5061979b pxar::fuse: Avoid macro to generate filler function, use closure instead. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-15 10:25:16 +01:00
Dietmar Maurer
f3ec5dae7f src/api2/config: improve api schema by adding return type 2020-01-14 14:55:58 +01:00
Dietmar Maurer
ddc526623f src/bin/proxmox-backup-manager.rs: add datastore update cli 2020-01-14 14:47:26 +01:00
Dietmar Maurer
c5799e4043 src/api2/config/datastore.rs: improve api, implement update and read 2020-01-14 14:45:56 +01:00
Dietmar Maurer
08195ac823 src/api2/config/remotes.rs: improve api, implement update and read 2020-01-14 14:20:16 +01:00
Dietmar Maurer
16f04b9d79 src/section_config.rs: add lookup_json() helper 2020-01-14 14:19:22 +01:00
Dietmar Maurer
71805bb7ec src/api2/config/remotes.rs: fix variable name -> s/remote/id/ 2020-01-14 13:20:13 +01:00
Dietmar Maurer
d0187a51a9 src/section_config.rs - convert_to_array: optionally add digest 
datastore::config() -> also return digest
remotes::config() -> also return digest
 2020-01-14 12:57:03 +01:00
Dietmar Maurer
4566303b05 src/section_config.rs - write: improve error message 2020-01-14 12:02:46 +01:00
Dietmar Maurer
f51420ba1f src/section_config.rs: do not allow control characters 2020-01-14 11:41:48 +01:00
Dietmar Maurer
7e7b781a18 src/api2/types.rs: also define PASSWORD_FORMAT and use it correctly 2020-01-14 11:32:02 +01:00
Dietmar Maurer
da4a15a351 src/api2/types.rs: define and use PASSWORD_REGEX 2020-01-14 11:22:42 +01:00
Dietmar Maurer
ff620a3d1a src/api2/types.rs: add regression test for PROXMOX_USER_ID_SCHEMA 2020-01-13 17:26:43 +01:00
Dietmar Maurer
ae62c4fe59 src/api2/types.rs: fix regex patterns 2020-01-13 14:38:31 +01:00
Dietmar Maurer
163dc16c0b src/api2/types.rs: define PROXMOX_AUTH_REALM_SCHEMA and PROXMOX_USER_ID_SCHEMA 
And try to use nbew schemas with config api...
 2020-01-13 14:18:19 +01:00
Dietmar Maurer
b25f313d66 src/api2/types.rs: define HOSTNAME_FORMAT, DNS_NAME_FORMAT and DNS_NAME_OR_IP_FORMAT 2020-01-13 13:10:52 +01:00
Dietmar Maurer
8aea35fcff src/config/*: add #[serde(skip_serializing_if="Option::is_none")] to optinal comment 2020-01-13 12:14:14 +01:00
Dietmar Maurer
454c13edce src/api2/types.rs: define SINGLE_LINE_COMMENT_SCHEMA 2020-01-13 12:02:13 +01:00
Dietmar Maurer
167971ed49 src/api2/types.rs: define REMOTE_ID_SCHEMA here 2020-01-13 11:47:07 +01:00
Christian Ebner
6de36b94b1 pxar::fuse: impl readdirplus callback 
readdirplus returns the entries together with their `EntryParam`, so subsequent
lookups for each of the entries are avoided.
In order to reduce code duplication, the code for filling the reply buffer is
moved into a macro.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-13 11:17:34 +01:00
Christian Ebner
caa201d895 pxar::fuse: Get rid of broken cacheing 
Remove the current caching of attrs and goodbye tables as it is broken anyway.
This will be replaced with a LRU cache.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-13 11:16:06 +01:00
Oguz Bektas
6d20a29d73 fix typo for function name load_and_decrypt_key 
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
 2020-01-13 11:14:47 +01:00
Dietmar Maurer
50af953e1b src/section_config.rs: make set_data generic 2020-01-11 11:09:27 +01:00
Dietmar Maurer
688fbe07a1 cleanup config api, add remotes config cli interface 2020-01-11 10:42:09 +01:00
Dietmar Maurer
9e9bc6525e src/config/datastore.rs: define DataStoreConfig using api macro 2020-01-11 09:18:42 +01:00
Dietmar Maurer
ca44172404 src/api2/config/remotes.rs: cleanup - avoid compiler warnings 2020-01-10 19:25:26 +01:00
Christian Ebner
c7257c5539 pxar::fuse: refactor HashMap to store child parent inode relation. 
By moving the HashMap into `Context`, the use of lazy_static as well as the
additional Mutex can be avoided (`Context` is already guarded by a Mutex).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-10 13:45:58 +01:00
Christian Ebner
90fc97af6a pxar::decoder::Decoder: include xattrs and payload size in DirectoryEntry. 
By reading and including xattrs and payload size in struct `DirectoryEntry`,
the tuple of return types is avoided and the code is simpler.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-10 13:45:40 +01:00
Christian Ebner
6fc053ed85 pxar: encoder: limit number of max entries held at once in memory during archive creation. 
Limit the total number of entries and therefore the approximate memory
consumption instead of doing this on a per directory basis as it was previously.
This makes more sense as it limits not only the width but also the depth of the
directory tree.

Further, instead of hardcoding this value, allow to pass this information as
additional optional parameter 'entires-max'.
By this, creation of the archive with directories containing a large number of
entries is possible.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-10 13:45:08 +01:00
Dietmar Maurer
141304d64e src/api2/config/remotes.rs: new API to configure remotes 2020-01-10 13:28:15 +01:00
Dietmar Maurer
331b869d96 src/bin/proxmox-backup-manager.rs: implement shell completion for remote datastore name 2020-01-10 11:58:31 +01:00
Dietmar Maurer
eb506c830c rename src/api2/sync.rs -> src/api2/pull.rs 
Use "pull" instead of "sync", because this also indicates a
direction (like "sync from").
 2020-01-10 11:15:16 +01:00
Dietmar Maurer
a81af92f9d src/section_config.rs: implement generic lookup 2020-01-09 17:35:44 +01:00
Dietmar Maurer
0eb0e02451 src/bin/proxmox-backup-manager.rs: implement cli sync command 2020-01-09 14:52:29 +01:00
Dietmar Maurer
a575320657 src/config/remotes.rs: implement SectionConfig for remote hosts 2020-01-09 14:51:02 +01:00
Christian Ebner
ecbc62261c pxar: encoder: warn on lacking read permissions instead of fail. 
If during creation of the archive, files/dirs with lacking read permissions are
encountered, the user is displayed a warning and the archive is created without
including the file/dir.
Previously this resulted in an error and the archive creation failed.

In order to implement this also for the .pxarexclude files, the Error type of
MatchPattern::from_file() and MatchPattern::from_line() was adopted accordingly.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-09 10:41:19 +01:00
Christian Ebner
88cee60bad pxar: encoder: fix incorrect goodbye table entries and update displayed path. 
If nodes are excluded by feature flags, they must not appear in the goodbye table.
This is fixed by continuing with the next entry in the for loop.

Further the relative path buffer is now poped in order to correctly display the path.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-09 10:39:41 +01:00
Christian Ebner
ec906dcf19 src/pxar/encoder.rs: avoid use of macro and initialize Vec with known capacity. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2020-01-09 10:38:58 +01:00
Dietmar Maurer
71da3d6a1b src/api2/types.rs: fix SnapshotListItem schema definition 2020-01-09 10:36:19 +01:00
Dietmar Maurer
de8ec04123 src/api2/sync.rs: implement remote sync 2020-01-08 14:03:52 +01:00
Dietmar Maurer
27c9affb23 src/bin/proxmox-backup-client.rs: cleanup - factor out api_datastore_latest_snapshot() 2020-01-07 15:18:36 +01:00
Dietmar Maurer
42af4b8f06 src/bin/proxmox-backup-client.rs: cleanup - factor out api_datastore_list_snapshots() 2020-01-07 15:07:57 +01:00
Dietmar Maurer
af9d4afc66 src/bin/proxmox-backup-client.rs: use SnapshotListItem 2020-01-07 13:25:25 +01:00
Dietmar Maurer
fc189b198c src/api2/types.rs: define and use new api type SnapshotListItem 2020-01-07 12:52:52 +01:00
Dietmar Maurer
8f14e8fe4c src/backup/backup_info.rs: implement last_successful_backup() 
A backup is considered successful if there is a manifest file.
 2020-01-06 11:35:22 +01:00
Dietmar Maurer
247a8ca5bb src/backup/manifest.rs: impl TryFrom DataBlob 2020-01-05 16:28:18 +01:00
Dietmar Maurer
7759eef552 src/backup/datastore.rs: implement cleanup_backup_dir() 2020-01-05 15:15:12 +01:00
Dietmar Maurer
5e1d1bead1 src/client/backup_reader.rs: make compute_file_csum() public 2020-01-05 14:50:54 +01:00
Dietmar Maurer
3743dee6de src/client/http_client.rs: allow access to AuthInfo fields 2020-01-05 10:31:19 +01:00
Dietmar Maurer
3758b398cd src/backup/read_chunk.rs: add read_raw_chunk() 
Allow to read chunk data without decoding.
 2020-01-02 13:29:10 +01:00
Dietmar Maurer
2585a8a4e2 src/backup/chunk_store.rs: implement cond_touch_chunk() 
This will be used by backup sync to test if a chunk already exists.
 2020-01-02 13:26:28 +01:00
Dietmar Maurer
1cf5178ac5 src/backup/datastore.rs: new helper try_shared_chunk_store_lock() 2020-01-02 11:00:33 +01:00
Dietmar Maurer
1e8da0a789 src/backup/manifest.rs: new helper archive_type() 2019-12-31 15:23:41 +01:00
Dietmar Maurer
94f085fab5 src/bin/proxmox-backup-proxy.rs: remove unused use clause 2019-12-30 11:50:24 +01:00
Dietmar Maurer
22be470d90 src/bin/proxmox-backup-api.rs: create cert here 2019-12-27 13:44:21 +01:00
Dietmar Maurer
ca6119551d src/client/http_client.rs - start_h2_connection: correctly return error during upgrade 2019-12-27 13:41:31 +01:00
Wolfgang Bumiller
a546a8a072 proxy/api: await running workers before shutdown 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-27 12:50:33 +01:00
Dietmar Maurer
f8fd5095d8 src/bin/proxmox-backup-proxy.rs: create self signed cert at startup. 2019-12-27 11:21:32 +01:00
Dietmar Maurer
afddffc7ec proxmox-backup-manager.rs: remove wrong fixme 2019-12-27 10:17:06 +01:00
Dietmar Maurer
e739a8d8b4 src/bin/proxmox-backup-manager.rs: impl. cert info 2019-12-23 13:06:43 +01:00
Dietmar Maurer
550e0d8870 src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate 
Without calling external openssl binary.
 2019-12-22 17:35:06 +01:00
Wolfgang Bumiller
5e62d19cfb fix create_datastore parameters: 
comment was declared as optional but unwrap()ed, causing
panics when not using it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-20 12:30:25 +01:00
Dietmar Maurer
645995634a src/api2/config/datastore.rs - create: pass uid and gid instead of User 2019-12-20 09:23:58 +01:00
Dietmar Maurer
e67770d496 src/backup/chunk_store.rs - create: pass User instead of CreateOptions 2019-12-20 09:11:40 +01:00
Thomas Lamprecht
bca9093520 api/ui: datastore: allow to set simple comment 
for now forbid all control characters[0] in the comment value, the
section config writer cannot cope with newlines in the value, it
writes them out literally, allowing "injection" or breaking the whole
config.

In the webinterface use also a textfield, not a textarea.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-19 17:58:01 +01:00
Wolfgang Bumiller
afdcfb5bc9 let ChunkStore::create take CreateOptions 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-19 13:14:49 +01:00
Dietmar Maurer
47d4712152 src/bin/proxmox-backup-manager.rs: add task management cli 2019-12-19 11:27:36 +01:00
Dietmar Maurer
d6c4a1198a src/bin/proxmox-backup-client.rs: use async fn 2019-12-19 11:20:59 +01:00
Dietmar Maurer
f74a03da1f remove tools::getpwnam_ugid, impl. crate::backup::backup_user() 
And use new nix::unistd::User struct.
 2019-12-19 10:20:13 +01:00
Dietmar Maurer
8bf4559b4e update to nix 0.16 2019-12-19 09:29:44 +01:00
Dietmar Maurer
769f8c9998 src/bin/proxmox-backup-manager.rs: connect to daemon to execute GC 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2019-12-19 08:45:27 +01:00
Dietmar Maurer
25de1c8005 src/client/backup_repo.rs: impl. new() 2019-12-19 08:44:38 +01:00
Dietmar Maurer
94913f35ca src/bin/proxmox-backup-client.rs - start_garbage_collection: add missing output-format parameter 2019-12-19 07:57:53 +01:00
Dietmar Maurer
868c585219 src/backup/datastore.rs: avoid divide by zero 2019-12-19 07:14:23 +01:00
Dietmar Maurer
217170e11e src/server/worker_task.rs: set worker thread name 
This gives better logs on thread panic...
 2019-12-19 07:07:39 +01:00
Dietmar Maurer
7e210bd0b4 src/backup/chunk_store.rs: create lock file with correct owner 2019-12-19 06:55:53 +01:00
Thomas Lamprecht
2337df7b3d api2: add very basic 'status/usage' endpoint for nodes 
For returning the nodes basic "usage status", for now one gets memory
and CPU utilization.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-18 17:24:32 +01:00
Dietmar Maurer
bce023c57c src/tools.rs: fix typo 2019-12-18 14:29:29 +01:00
Dietmar Maurer
0b97bc6158 src/backup/chunk_store.rs: use proxmox::tools::fs::create_path 2019-12-18 12:26:43 +01:00
Dietmar Maurer
f8ec1473be src/config/datastore.rs: use backup gid instead of uid 2019-12-18 12:21:44 +01:00
Wolfgang Bumiller
feaa1ad35f replace file_set_contents with replace_file 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-18 11:16:04 +01:00
Dietmar Maurer
a66e7920dc src/api2/config/datastore.rs: set protected flags for create/delete API 2019-12-18 10:52:45 +01:00
Dietmar Maurer
f0db500808 src/api2/config/datastore.rs: fix api docu 2019-12-18 10:48:19 +01:00
Dietmar Maurer
424766bc3b src/config/datastore.rs: change file owner/permissions 
owner(root) => read and write
group(backup) => read only
 2019-12-18 10:41:58 +01:00
Oguz Bektas
c07b458cac datastore: set correct owner on datastore.cfg 
since manager runs as root, we have to specify backup:root owner while
saving config.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
 2019-12-17 16:59:10 +01:00
Oguz Bektas
14f1e63067 chunk_store: create parent directories 
'datastore create storename /path/to/dir/that/may/not/exist' should
work.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
 2019-12-17 15:39:42 +01:00
Christian Ebner
ebfb2df22b binary_search_tree: add additional doctest for search_binary_tree_by() 
Make sure a start indexes larger or equal to the array size results in a return
value of `None`.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 15:37:02 +01:00
Christian Ebner
48f6d67764 binary_search_tree: fix off by one error in search_binary_tree_by() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 14:57:34 +01:00
Christian Ebner
38d9a69875 catalog: decoder and shell: fix incorrect logic in find matching 
The find matching was incorrectly performed starting from the parent directroy
and not as intended from the entries of the parent directory.

Further, the match pattern passed from the catalog shell contains the absolute
path of the search entry point as prefix, so find() must always start from the
archive root. This is because the match pattern has to be stored in the selected
list for a subsequent restore-selected command in the shell.
All matching paths are shown as absolute paths with all contents in the subdir,
equal to what would be restored by the given pattern.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 14:57:34 +01:00
Dietmar Maurer
81cc71c0d7 use proxmox-mini-journalreader to display syslog 2019-12-17 14:06:48 +01:00
Dietmar Maurer
20197f7cf5 src/api2/node/syslog.rs: use api macro 2019-12-17 13:26:49 +01:00
Thomas Lamprecht
9c01e73c2f ui: add "Documentation" button to main view 
Similar to PVE and PMG, for quick access when one has the basic
webinterface open anyway. Should move to the "proxmoxHelpButton" once
we have an onlineHelp mapping to the docs.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-17 11:22:34 +01:00
Thomas Lamprecht
2d694f8f1f add CSS file for PBS ExtJS6 basic ui 
some fitting rules copied over from PVE's ext6-pve.css file.
simply place it in the css subfolder where the proxmox-backup-gui.js
file is hosted and add a "css/" alias for that directory, the
formatter gets use the right content type with that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-17 11:20:32 +01:00
Christian Ebner
25cdd0e0a1 catalog_shell: impl find for catalog shell. 
Implements the find command which allows to find and select files for subsequent
restore.
Files selected for restore are now stored in a Vec instead of a HashSet.
This is needed, since instead of the full paths for each file, selected files are
now identified by a list of match pattern, where ordering matters.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 11:12:34 +01:00
Christian Ebner
90dfd0a71d catalog: impl find() for file search by match pattern. 
find() iterates over the file tree and matches each node against a list of match
patterns provided at function call.
For each matching node, a callback function with the current directroy stack is
called.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 11:12:34 +01:00
Christian Ebner
a333b4ccea pxar: match_pattern: impl invert() to switch the match type of a MatchPattern. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 11:12:34 +01:00
Christian Ebner
48af80b3ab pxar: match_pattern: impl traits needed for MatchPattern to be comparable. 
In order to check if two MatchPattern are identical, which will be used to avoid
duplicates.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 11:12:34 +01:00
Christian Ebner
255bb86030 pxar: match_pattern: refactor MatchPattern and introduce MatchPatternSlice. 
The MatchPattern impl heavily used copies and therefore was inefficient regarding
memory management.
This patch intoduces MatchPatternSlice as struct to avoid copies and perform the
same pattern matching functionality.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-17 11:12:34 +01:00
Wolfgang Bumiller
dc08934563 drop Cancellable future in favor of abortable 
futures-0.3 has a futures::future::abortable() function
which does the exact same, returns an Abortable future with
an AbortHandle providing an abort() method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-17 10:52:07 +01:00
Wolfgang Bumiller
8554ac5ec3 fix broadcast_future test case 
We used to await all the futures via the runtime's shutdown
method, which doesn't exist anymore, so await all the join
handles instead.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-17 10:47:12 +01:00
Thomas Lamprecht
ba3eb88d95 generate authkey: public part needs to be readable by backup group 
else the API proxy cannot use it and fails to run..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-17 10:17:03 +01:00
Thomas Lamprecht
9c18e93522 handle_static_file_download: move from and_then to await 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-17 08:56:55 +01:00
Thomas Lamprecht
4e8cff1c9d cipherbench: align/format output nicely 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-17 08:49:34 +01:00
Thomas Lamprecht
4832f53ab5 cipherbench: be conform with rfc7539, use 12 byte nonce 
else we get an error from this call, using a 16 byte (128 bit) nonce
is currently only supported by the still in draft
XChaCha20-Poly1305, not the current default specified by RFC 7539[0],
which uses a 12 byte (96 bit) nonce.

Fixes the following error:
> thread 'main' panicked at 'called `Result::unwrap()` on an `Err`
> value: ErrorStack([])', src/libcore/result.rs:1165:5

[0]: https://tools.ietf.org/html/rfc7539

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2019-12-17 08:49:07 +01:00
Dietmar Maurer
a47a02ae62 src/bin/proxmox-backup-client.rs: use #[api] macro 2019-12-16 13:49:18 +01:00
Wolfgang Bumiller
78d5436040 client: use api macro for catalog_shell async fn 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-16 11:24:44 +01:00
Dietmar Maurer
87c423751a src/bin/proxmox-backup-client.rs: use ApiHandler::Async for prune and status 2019-12-16 11:00:43 +01:00
Dietmar Maurer
8a8a470316 src/bin/proxmox-backup-client.rs: use new ApiHandler::Async 2019-12-16 10:06:26 +01:00
Dietmar Maurer
b9799012cf src/pxar/decoder.rs: require Send 2019-12-16 10:01:51 +01:00
Dietmar Maurer
bb084b9c91 api2: update for latest proxmox-api changes 
- rename ApiFuture into ApiResponseFuture
- impl. ApiHandler::Async
 2019-12-16 10:01:51 +01:00
Dietmar Maurer
236a396aa1 src/api2/admin/datastore.rs - prune: log retention options 2019-12-14 16:32:16 +01:00
Dietmar Maurer
503995c767 src/api2/admin/datastore.rs: fix prune worker id 2019-12-14 16:05:21 +01:00
Wolfgang Bumiller
f374ba4c0b update tests 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-13 12:42:02 +01:00
Wolfgang Bumiller
b5c9a706ce fix cancellable future 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-13 12:10:04 +01:00
Wolfgang Bumiller
8a1028e0e7 update to url/percent-encoding to 2.1 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-13 11:57:01 +01:00
Wolfgang Bumiller
db0cb9ce0b update a chunk of stuff to the hyper release 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-13 11:24:41 +01:00
Dietmar Maurer
3f1020b732 src/bin/proxmox-backup-client.rs: implement task stop 2019-12-13 09:43:08 +01:00
Dietmar Maurer
1ae5677de0 src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT 2019-12-12 14:15:40 +01:00
Dietmar Maurer
d0adf270fb src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA 2019-12-12 12:37:11 +01:00
Dietmar Maurer
66c49c21c3 src/api2/types.rs: factor out DATASTORE_SCHEMA 2019-12-11 13:45:23 +01:00
Dietmar Maurer
567d3e00fb src/api2/node/tasks.rs: new filter "store" to filter tasks for one store 2019-12-11 12:53:34 +01:00
Christian Ebner
d3dbe52f37 pxar: match_pattern: refactor match_filename 
The match_filename() in sequentail_decoder and encoder are moved to be static
functions of MatchPattern.
This allows to reuse the code also in the catalog find implementation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-11 10:55:01 +01:00
Christian Ebner
f084505ec5 src/backup/catalog_shell.rs: sort output of list-selected 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-11 10:54:20 +01:00
Dietmar Maurer
6d04612955 src/api2/node/tasks.rs: add "running" filter to list only running tasks 2019-12-11 10:52:27 +01:00
Dietmar Maurer
8560fe3e5f src/api2/node/tasks.rs - extract_upid: avoid duplicate error message 2019-12-10 13:45:56 +01:00
Dietmar Maurer
5830c20560 src/bin/proxmox-backup-client.rs: add simple task management cli 2019-12-10 13:43:53 +01:00
Dietmar Maurer
73e57f244e src/backup/datastore.rs - garbage-collection: fix average chunk size 2019-12-10 11:25:29 +01:00
Dietmar Maurer
d105176f45 src/bin/proxmox-backup-client.rs: cleanup - factor out view_task_result() 2019-12-09 18:00:22 +01:00
Dietmar Maurer
48ef3c3327 cli: avoid useless .into() 
This needs latest proxmox-api updates.
 2019-12-09 17:40:34 +01:00
Dietmar Maurer
eb7e2ee00b src/bin/proxmox-backup-client.rs: remove unnecessary .to_owned() 2019-12-09 17:11:48 +01:00
Dietmar Maurer
e5f7def40e src/bin/proxmox-backup-client.rs - garbage-collection: display task log 2019-12-09 13:29:23 +01:00
Dietmar Maurer
0ec9e1b010 src/bin/proxmox-backup-client.rs: impl. and use complete_pxar_archive_name 2019-12-09 13:14:32 +01:00
Dietmar Maurer
1c6ad6ef90 src/bin/proxmox-backup-client.rs: impl. catalog sub command 
catalog dump
catalog shell
 2019-12-09 13:07:25 +01:00
Dietmar Maurer
5a0b484bdd src/client/task_log.rs: move display_task_log into extra file 2019-12-08 11:27:15 +01:00
Dietmar Maurer
3b03abfe14 src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune 
And use display_task_log on the client side.
 2019-12-08 10:59:47 +01:00
Dietmar Maurer
ec137a99c6 src/bin/proxmox-backup-client.r - display_task_log: use string range instead of as_bytes 2019-12-08 10:25:26 +01:00
Dietmar Maurer
b73ff941f6 src/bin/proxmox-backup-client.r - display_task_log: add new strip_date parameter 2019-12-08 10:22:02 +01:00
Dietmar Maurer
163e9bbe91 src/bin/proxmox-backup-client.rs: new display task log helper 
New helper to display worker task logs. Use it for prune (experimental).
 2019-12-07 16:11:26 +01:00
Dietmar Maurer
6b508dd563 src/server/formatter.rs: impl. new result attribute "active" 2019-12-07 15:29:42 +01:00
Dietmar Maurer
102d8d4136 src/backup/prune.rs: implement --keep-hourly 2019-12-07 11:23:33 +01:00
Dietmar Maurer
052c4aa160 src/backup/prune.rs: prune - correctly use iso_week year 2019-12-06 19:41:38 +01:00
Christian Ebner
fa0b950070 pxar: fuse: add support for list- and getxattr 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 16:44:15 +01:00
Christian Ebner
25ad4cbf63 pxar: fuse: cache goodbye table for each directory on opendir and release it on releasedir 
Cache not only the goodbye table for the last directory but for each opened
directory.
The opendir fuse callback will fill the cache with the goodbye table and
releasedir will remove it from the cache.
This should reduce the number of chuncks fetched from the server in some cases.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 16:43:59 +01:00
Christian Ebner
02491b8fc6 pxar: fuse: use binary search instead of linear search in goodbye table. 
The goodbye table is layed out as binary search tree based on the hash, so use
this to be more efficient when looking up a hash in the table for directories
with a large number of entries.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 16:42:01 +01:00
Dietmar Maurer
7074a0b339 src/bin/proxmox-backup-client.rs: white space cleanup 2019-12-06 13:35:59 +01:00
Dietmar Maurer
74fa81b89c src/bin/proxmox-backup-client.rs: implement prune --dry-run 2019-12-06 13:35:25 +01:00
Dietmar Maurer
1f604f1a5a src/api2/admin/datastore.rs: add new api test_prune() 2019-12-06 13:06:56 +01:00
Dietmar Maurer
0ab08ac980 src/api2/admin/datastore.rs: cleanup - move schema defininition in front of rust method 2019-12-06 12:36:41 +01:00
Dietmar Maurer
1a0678b601 src/api2/admin/datastore.rs: remove prune parameters from status call 
Not idea why they are added there.
 2019-12-06 12:32:53 +01:00
Dietmar Maurer
9e3f008804 src/backup/prune.rs: add new helper keeps_something() 2019-12-06 12:28:31 +01:00
Dietmar Maurer
d75fbb0a21 src/bin/dump-catalog-shell-cli.rs: generate catalog shell docs 2019-12-06 09:54:13 +01:00
Dietmar Maurer
55c3cb69cd rename catalog_shell_api() into catalog_shell_cli() 2019-12-06 09:42:05 +01:00
Dietmar Maurer
ecbaa38fa4 src/backup/catalog_shell.rs: factor out catalog_shell_api() 
In order to extract documentation ...
 2019-12-06 09:35:47 +01:00
Christian Ebner
cca67dc0c3 src/backup/readline.rs: removed in favor of new readline impl 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 09:03:04 +01:00
Christian Ebner
6934c6fe77 src/backup/catalog_shell.rs: adapt to use API Schema definition and rustyline 
This major refactoring of the catalog based shell utilizes the new API macro and
the API Schema as well as rustyline instead of the old GNU readline C API.

The code now has these 3 main components:
 * The `Shell` which handles the readline loop via rustyline.
 * The shell functions defined via the API macro.
 * The `Context` which holds catalog and decoder instances.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-06 09:02:44 +01:00
Dietmar Maurer
9b78352188 src/backup/prune.rs: define new struct PruneOptions 2019-12-06 08:56:27 +01:00
Dietmar Maurer
92acbd69f7 src/backup/backup_info.rs: remove old prune code 2019-12-06 08:29:27 +01:00
Dietmar Maurer
408434360b src/backup/prune.rs: add PruneMark::KeepPartial 
Use separate mark for partial backup - dont mess them up with regular ones.
 2019-12-06 08:15:32 +01:00
Dietmar Maurer
a8c8366cfa src/backup/prune.rs: factor out code into remove_incomplete_snapshots() 2019-12-06 08:12:08 +01:00
Dietmar Maurer
dc18849156 src/backup/prune.rs: moved prune related code into extra file 2019-12-06 08:06:21 +01:00
Dietmar Maurer
6f47dd8a0f src/backup/backup_info.rs - prune: improve algorythm 2019-12-05 18:41:28 +01:00
Dietmar Maurer
8f0b4c1f90 src/api2/admin/datastore.rs - prune: log info about all snapshots 2019-12-05 15:35:11 +01:00
Dietmar Maurer
9ce42759ec src/backup/backup_info.rs - compute_prune_list: remove unfinished backups 2019-12-05 11:18:10 +01:00
Dietmar Maurer
2c034f8d0a src/backup/backup_info.rs: improve prune algorithm 2019-12-05 08:55:19 +01:00
Dietmar Maurer
aeeac29bb1 src/backup/backup_info.rs: new compute_prune_list helper 
So that we can write regression tests for this.
 2019-12-04 15:49:11 +01:00
Dietmar Maurer
3d229a4a01 Revert "test another api macro use" 
This reverts commit a9aa52e6a8.

Because we do not want to use macros for the backup protocol for now.

And because it crashes backup tasks for some unknown reason.
 2019-12-04 14:18:11 +01:00
Dietmar Maurer
4f1e40a29e src/api2/admin/datastore.rs - read_backup_index: add manifest with size 
So that the client can display the size of the index file.
 2019-12-04 10:03:52 +01:00
Wolfgang Bumiller
a9aa52e6a8 test another api macro use 
The api macro now supports hyphens in parameter names and
referencing externally defined `Schema`s, so here's an
example.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-12-03 13:41:40 +01:00
Christian Ebner
de0aa53174 pxar: decoder: fix incorrect read for hardlinks. 
Decoders read must check if the file is a hardlink and read data from the
corresponding offset if so.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-03 12:08:23 +01:00
Christian Ebner
b7e18004a2 pxar: decoder: return original name in attributes() 
The returned filename should be the one of the file given at the offset, not of
the one the hardlink points to.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-12-03 12:08:07 +01:00
Dietmar Maurer
7eea56ca00 use cli from proxmox::api 2019-12-02 11:56:29 +01:00
Dietmar Maurer
524b22df35 src/cli/completion.rs - get_help_completion: fix for "help -" expansion 2019-12-02 11:11:31 +01:00
Dietmar Maurer
e8e9bae4d1 src/bin/completion.rs: add echo command 2019-12-02 11:04:40 +01:00
Dietmar Maurer
19d7a59236 src/bin/completion.rs: use api macros 2019-12-02 10:57:19 +01:00
Dietmar Maurer
f9a51322cb src/cli/command.rs: use insert_help() 2019-12-02 08:52:08 +01:00
Dietmar Maurer
e3e72546dd src/cli/command.rs: cleanup, remove unused top_def parameter 2019-12-02 08:47:03 +01:00
Dietmar Maurer
2348e77edf src/cli/completion.rs - get_simple_completion: remove all_arg_param parameters 
Seems useless.
 2019-12-02 08:31:03 +01:00
Dietmar Maurer
28c855c0a2 src/cli: improve docs 2019-12-01 16:41:49 +01:00
Dietmar Maurer
3bf920527c src/cli/command.rs: make set_help_context private - no need to expose 2019-12-01 12:54:41 +01:00
Dietmar Maurer
bca6d9120a src/cli/command.rs: improve help command description 2019-12-01 12:40:37 +01:00
Dietmar Maurer
2f6ecc1ccd src/cli/completion.rs: moved all completion related code here 2019-12-01 11:44:31 +01:00
Dietmar Maurer
a810e05288 src/cli/readline.rs: moved readline related code here 2019-12-01 11:31:35 +01:00
Dietmar Maurer
5d64a0d083 src/cli/shellword.rs: moved shellword related code into this file 2019-12-01 11:17:57 +01:00
Dietmar Maurer
fb8a00bc00 src/cli/command.rs - get_nested_completion: allow fuzzy match 2019-12-01 11:06:54 +01:00
Dietmar Maurer
9498fcf9c5 src/cli/command.rs - get_nested_completion: return nothing if first argument does not match 2019-12-01 10:24:07 +01:00
Dietmar Maurer
309e449db2 src/cli/command.rs: improve get_help_completion 2019-12-01 10:18:47 +01:00
Dietmar Maurer
b8a82922c7 src/cli/command.rs: handle multilevel sub commands in help 2019-12-01 09:42:51 +01:00
Dietmar Maurer
b7bf71dfa1 src/cli/command.rs: fix help completions 2019-11-30 15:43:44 +01:00
Dietmar Maurer
7afc1af7ab src/bin/completion.rs: add history support 2019-11-30 15:08:37 +01:00
Dietmar Maurer
4b8573da65 src/cli/command.rs - handle_command: return Result instead of exit() 2019-11-30 15:08:00 +01:00
Dietmar Maurer
2b691daf6f src/cli.rs: add insert_help() method 2019-11-30 14:56:31 +01:00
Dietmar Maurer
1201abcffa src/cli/command.rs: implement help 2019-11-30 13:02:00 +01:00
Wolfgang Bumiller
8423c1fe64 cli/completion: use match statements and .contains() 
`match` is a bit more readable than the if-else chains,
also replace
    space_chars.iter().any(|s| c == *s)
with
    space_chars.contains(&c)
which is also more readable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-29 13:37:34 +01:00
Dietmar Maurer
e399398444 remove shellwords crate 2019-11-29 12:49:44 +01:00
Dietmar Maurer
ca60ac13d9 use rustyline to replace readline 2019-11-29 12:49:44 +01:00
Wolfgang Bumiller
41874331ed whitespace fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-29 09:52:15 +01:00
Wolfgang Bumiller
2905f2b5e6 update api macro example usage 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-29 09:51:27 +01:00
Dietmar Maurer
8b130e71da src/cli/command.rs: cleanup, make handlers more generic 2019-11-27 14:44:21 +01:00
Wolfgang Bumiller
7b6c41078b update api macro invocation to new style 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-27 14:19:46 +01:00
Christian Ebner
314bb35868 catalog_shell: fix several clippy warnings 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-27 10:13:50 +01:00
Wolfgang Bumiller
6486cb853f first api macro usage test/example 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-27 10:05:37 +01:00
Christian Ebner
951cf17ee3 catalog_shell: major refactoring of cli command definition and parsing 
By changing the way shell commands are defined and parsed, this makes it more
straight forward to extend the current functionality.
The readline input is parsed based on the provided command definition and the
given parameters and options are passed to a command specific callback function.
In addition, the provided command definition including its description is used
to generate a help string to display.
The help command shows a list of all supported commands or the help string for
the provided command.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-26 13:01:00 +01:00
Christian Ebner
59bc6ad676 catalog_shell: check and return on empty list in order to avoid division by zero 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-26 12:55:33 +01:00
Christian Ebner
4e56b2f792 catalog_shell: do not restore full archive if no entries are selected for restore-selected 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-26 12:55:18 +01:00
Dietmar Maurer
12b4098660 src/cli/format.rs: move doc generation code into extra file 2019-11-26 12:12:53 +01:00
Dietmar Maurer
2f2b87e659 src/pxar/fuse.rs: add impl Context 2019-11-26 11:16:46 +01:00
Dietmar Maurer
eedd1f95eb src/pxar/fuse.rs: move init and destroy into Session 2019-11-26 11:08:55 +01:00
Dietmar Maurer
970687c982 src/pxar/fuse.rs: cleanup - remove setup_session (moved into new) 2019-11-26 11:04:46 +01:00
Dietmar Maurer
2a11191072 src/pxar/fuse.rs: rename from_decoder into new, new into from_path 2019-11-26 10:56:41 +01:00
Dietmar Maurer
99b5b6cba9 src/pxar/decoder.rs: use trait object as reader 
So that the Decoder is no longer genertic.
 2019-11-26 10:45:11 +01:00
Dietmar Maurer
f701d0335e src/pxar/sequential_decoder.rs: remove callback from new() 
And use an extra functzion  set_callback() to configure that.

Also rewrite pxar/fuse.rs and implement a generic Session (will get
further cleanups with next patches).
 2019-11-26 09:56:48 +01:00
Dietmar Maurer
49fddd985c src/cli/command.rs: use static array for arg_param 2019-11-25 11:16:41 +01:00
Dietmar Maurer
329d40b50b rename ApiHandler::Async into ApiHandler::AsyncHttp 2019-11-23 09:03:21 +01:00
Dietmar Maurer
70fbac84da src/server/rest.rs: simplify code 2019-11-22 18:44:14 +01:00
Dietmar Maurer
2bbd835b9b src/server/rest.rs: rename get_request_parameters_async to get_request_parameters 2019-11-22 17:24:16 +01:00
Dietmar Maurer
75a5a68944 src/server/rest.rs - only pass ObjectSchema to get_request_parameters_async() 2019-11-22 17:22:07 +01:00
Dietmar Maurer
ad51d02aa9 src/server/rest.rs: cleanup async code 2019-11-22 13:02:05 +01:00
Wolfgang Bumiller
be2bb37205 remove api_schema submodule 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-22 09:31:45 +01:00
Dietmar Maurer
e57e1cd82d move src/api_schema/config.rs -> src/server/config.rs 2019-11-22 09:23:03 +01:00
Dietmar Maurer
8256b0e417 src/api_schema/registry.rs: remove dead code 2019-11-22 09:04:52 +01:00
Christian Ebner
3cf73c4e53 src/bin/proxmox-backup-client.rs: impl shell subcommand for client 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 08:22:50 +01:00
Christian Ebner
f14c96ea38 src/backup/catalog_shell.rs: impl shell to inspect and restore a snapshot via the catalog 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:54:04 +01:00
Christian Ebner
38446a9551 src/backup/readline.rs: impl wrapper for GNU readline 
In order to provide the context needed for tab completion via the readline
callback, the needed mut ref is passed via a thread local storage key.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:46:24 +01:00
Christian Ebner
6dba015043 src/backup/catalog.rs: derive Clone for DirEntry and DirEntryAttribute 
This is needed in order to explicitly clone the values when needed in the
catalog shell implementation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:15:39 +01:00
Christian Ebner
c9f002213e src/backup/catalog.rs: impl DirEntry::is_directory() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:14:00 +01:00
Christian Ebner
d3bcc7f261 src/pxar/sequential_decoder.rs: create all directories in target path 
Create the full target path and not fail if an intermediate directory does not
exist.
This is needed in order to restore multiple archives via the catalog, where the
target should further contain each archive name as subdir.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:11:49 +01:00
Christian Ebner
b6ca690095 src/pxar/decoder.rs: return also payload size in lookup 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-22 07:05:14 +01:00
Dietmar Maurer
45cc157f7c docs: add backup reader protocol 2019-11-22 06:59:37 +01:00
Wolfgang Bumiller
cad540e969 api/compat: remove remaining api_schema references 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:53:15 +01:00
Wolfgang Bumiller
a2479cfa1a api/compat: drop more compat imports from api_schema.rs 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:36:28 +01:00
Wolfgang Bumiller
cc4ecf80b0 api/compat: drop api_handler submodule 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:18:41 +01:00
Wolfgang Bumiller
4f1963d259 api/comapt: drop schema submodule 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:17:36 +01:00
Wolfgang Bumiller
2194824f77 api: BoxFut -> ApiFuture 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:16:37 +01:00
Wolfgang Bumiller
3d482025b3 move api schema into proxmox::api crate 
And leave some compat imports in api_schema.rs to get it to
build with minimal changes.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-21 14:14:54 +01:00
Dietmar Maurer
552c225948 sort all property lookup tables 
Required, because we use binary sreach to find items.
 2019-11-21 13:32:09 +01:00
Dietmar Maurer
255f378a1b use const api definitions 2019-11-21 13:32:09 +01:00
Wolfgang Bumiller
c5946faffd drop md5 crate dependency 
we already depend on openssl which also provides md5

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-14 11:19:43 +01:00
Wolfgang Bumiller
f569acc5e2 drop uuid crate dependency 
proxmox::tools now has a Uuid module using the native
libuuid.

Adds build dependency: libuuid1 (which is a Pre-Depends of
util-linux, so always installed anyway).

Drops uuid + 16 more crate dependencies.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-11-14 10:22:29 +01:00
Christian Ebner
b423958d8e catalog: fix issue with DirEntry start 
start has to be the parent start - offset

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-12 17:18:13 +01:00
Dietmar Maurer
780dd2b0a1 src/backup/catalog.rs: add some docs 2019-11-12 13:23:40 +01:00
Dietmar Maurer
6aa906b547 src/backup/catalog.rs - CatalogReader: verify magic number 2019-11-12 12:54:06 +01:00
Dietmar Maurer
2ec208aef5 src/backup/catalog.rs - CatalogReader::dump(): use root() 2019-11-12 12:47:21 +01:00
Dietmar Maurer
8f24a9ea18 src/backup/catalog.rs - Direntry::parse: abort if callback return false 2019-11-12 11:41:44 +01:00
Dietmar Maurer
7d017123fd src/backup/catalog.rs: impl read_dir() and lookup(), refactor common code 2019-11-12 11:41:44 +01:00
Dietmar Maurer
dc9596de45 src/backup/catalog.rs: move CatalogEntryType from src/pxar/catalog.rs 2019-11-12 11:41:44 +01:00
Dietmar Maurer
55c0b3cc7c src/backup/catalog.rs - DirEntry: factor out name attribute 
And avoid allocations inside parse().
 2019-11-12 11:41:44 +01:00
Christian Ebner
3f1c5b5e65 catalog: impl std::fmt::Display trait for CatalogEntryType 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-12 07:00:08 +01:00
Dietmar Maurer
f89359c2f3 src/backup/catalog.rs: clenup - use single impl block 2019-11-10 11:38:31 +01:00
Dietmar Maurer
e8d8fdc61d src/backup/catalog.rs: remove unused code 2019-11-10 11:37:06 +01:00
Dietmar Maurer
c74c074bfd src/backup/catalog.rs: write MAGIC at start of file 2019-11-10 11:36:45 +01:00
Dietmar Maurer
265664517a src/backup/catalog.rs: new parse() helper 2019-11-09 14:37:33 +01:00
Dietmar Maurer
5d92935e03 src/backup/catalog.rs - dump_dir: make sure we parsed the whole data block 2019-11-09 13:21:20 +01:00
Dietmar Maurer
12a1975af9 src/backup/catalog.rs: store u64 compressed 
This way catalogs get about 1/3 smaller.
 2019-11-09 13:07:47 +01:00
Dietmar Maurer
bb19af7302 src/client/backup_reader.rs: factor out download_blob() helper 2019-11-08 13:16:41 +01:00
Dietmar Maurer
7205050059 src/client/backup_reader.rs: factor out download_fixed_index() helper 2019-11-08 12:51:52 +01:00
Dietmar Maurer
c3d84a2281 src/client/backup_reader.rs: factor out download_dynamic_index() helper 2019-11-08 12:51:52 +01:00
Dietmar Maurer
89245fb530 renamed: src/backup/catalog_blob.rs -> src/backup/catalog.rs 2019-11-08 10:41:00 +01:00
Dietmar Maurer
bf6e321744 change catalog format, use dynamic index to store catalog. 
In order to remove size restriction of a single blob.
 2019-11-08 10:35:48 +01:00
Dietmar Maurer
e016f9ff2f src/client/backup_writer.rs: rename force_close to cancel 
And do not consume self.
 2019-11-08 10:34:16 +01:00
Dietmar Maurer
0997967d65 do no restrict index names too much 2019-11-08 10:28:14 +01:00
Christian Ebner
e66621182b src/pxar/decoder.rs: impl lookup for Decoder 
Allows to lookup an entry in a directory based on the provided `DirectoryEntry`.
This is needed to navigate the filesystem based on `DirectoryEntry`s and similar
to the find_goodbye_entry() function in src/pxar/fuse.rs

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-06 10:00:41 +01:00
Christian Ebner
fb2554de29 src/pxar/decoder.rs: fix wrong filename check for Decoder::restore() 
As it turns out the original implementation was correct and the start in
`DirectoryEntry` points to the `PxarEntry` and not as wrongly stated to the
filename.
This reverts the incorrect code and adds comments to the fields clarifying this.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-06 09:59:35 +01:00
Christian Ebner
33ad183a40 src/pxar/decoder.rs: allow to pass match pattern to restore 
In order to partially restore the archive starting from the directory provided as
parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-05 14:19:09 +01:00
Christian Ebner
5e8d600c71 src/pxar/decoder.rs: fix issue with restore 
`Decoder::restore()` calls the `SequentialDecoder::restore()` which expects to
encounter a `PxarEntry` at first. But the start of `DirectoryEntry` points to the
filename (except for the root dir), so skip over it.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-05 14:18:29 +01:00
Christian Ebner
9708b2d98c src/pxar/fuse.rs: refactor Iterator: use find() instead of position() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-11-04 13:59:16 +01:00
Christian Ebner
b82a101c86 catalog: store current writer position in CatalogBlobWriter 
Counts the bytes written by the CatalogBlobWriter in order to obtain the
stream position, needed to get offset to reference catalog items.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-31 09:11:51 +01:00
Dietmar Maurer
62ee2eb405 avoid some clippy warnings 2019-10-26 11:42:05 +02:00
Dietmar Maurer
834a2f95a0 avoid some clippy warnings 2019-10-25 18:44:51 +02:00
Dietmar Maurer
f58f426e86 src/pxar/binary_search_tree.rs: allow(clippy::many_single_char_names) 2019-10-25 18:22:19 +02:00
Dietmar Maurer
11377a47bb avoid some clippy warnings 2019-10-25 18:04:37 +02:00
Dietmar Maurer
9fe2f639b8 allow(clippy::cast_ptr_alignment) 2019-10-25 18:03:55 +02:00
Dietmar Maurer
cd69d36b45 src/backup/dynamic_index.rs: use write_all() 2019-10-25 17:22:41 +02:00
Christian Ebner
8abd12f1f3 pxar: decoder: cleanup un-needed Decoder::get_dir() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-23 08:02:23 +02:00
Christian Ebner
9c25c0330a pxar: fuse: optimize fuse readdir callback 
The expensive call to Decoder::read_directory_entry() can be omitted as
Decoder::attributes() returns all the information the fuse response needs.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-23 08:02:03 +02:00
Dietmar Maurer
ffe171c5db src/pxar/encoder.rs: avoid duplicate check for MAX_DIRECTORY_ENTRIES 
Simply by reordering code.
 2019-10-23 07:55:56 +02:00
Christian Ebner
62d123e50a pxar: add functionality to pass exclude MatchPatterns on create 
This exposes the option to pass a list of exclude MatchPattern via the
'--exclude' option.
The list is encoded as file '.pxarexclude-cli' in the archives root directory.
If such a file is present in the filesystem, it is skipped and not included in
the archive in order to avoid conflicting information.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-23 07:31:49 +02:00
Christian Ebner
ba8165c607 pxar: encoder: add encode_pxar_exclude_cli() function. 
This provides the functionality needed to encode MatchPatterns passed on the cli
in the root directory.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-23 07:30:30 +02:00
Dietmar Maurer
920243b12d pxar: simplify MatchPattern::to_bytes 2019-10-23 07:29:02 +02:00
Christian Ebner
e50a90e010 pxar: Implement to_bytes() for MatchPattern in order to write them to file. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-23 07:21:11 +02:00
Dietmar Maurer
a8f10f849e src/bin/proxmox-backup-client.rs: fix parameters for get_encryption_key_password 2019-10-17 10:28:21 +02:00
Christian Ebner
40c9ff9d81 src/pxar/encoder.rs: include .pxarexclude files into the catalog 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-17 10:25:20 +02:00
Christian Ebner
20feec9fac pxar: fuse: include nsec information in stat 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-10-17 10:24:13 +02:00
Dietmar Maurer
649c5ee7b1 load_and_decrtypt_key: allow function closures to readf passphrase 2019-10-16 13:23:52 +02:00
Dietmar Maurer
60ffa9649c src/backup/manifest.rs: fix property names (use '-' instead of '_') 2019-10-14 12:28:27 +02:00
Dietmar Maurer
4c9f753c56 src/backup/data_blob.rs: define create_zero_chunk as member function 
and rename it to build_zero_chunk()
 2019-10-14 10:58:26 +02:00
Dietmar Maurer
2745151d93 src/backup/data_blob.rs: add create_zero_chunk() helper 2019-10-14 10:44:46 +02:00
Dietmar Maurer
f06b820ac0 src/backup/manifest.rs: add verify_file 2019-10-13 10:09:12 +02:00
Dietmar Maurer
511a47bd73 cleanup 2019-10-13 09:39:21 +02:00
Dietmar Maurer
e17d5d864d src/backup/manifest.rs: improve parser (try_from) 2019-10-13 08:39:49 +02:00
Dietmar Maurer
59e9ba01c6 src/backup/manifest.rs: new class to generate/parse index.json 2019-10-12 17:58:08 +02:00
Dietmar Maurer
ad6e5a6f51 rename INDEX_BLOB_NAME to MANIFEST_BLOB_NAME 2019-10-12 15:53:27 +02:00
Dietmar Maurer
296c50ba3d src/client/backup_reader.rs: add download_manifest 
Also add crypt_config as member variable.
 2019-10-12 15:50:26 +02:00
Dietmar Maurer
cf9271e27c src/client/backup_writer.rs: move BackupClient code into extra file 
And rename it to BackupWriter.
 2019-10-12 13:53:11 +02:00
Dietmar Maurer
913acb41fd src/client/backup_reader.rs: add documentation 2019-10-12 13:26:47 +02:00
Dietmar Maurer
9e490a7479 src/client/backup_reader.rs: split BackupReader code into separate file 2019-10-12 12:57:08 +02:00
Dietmar Maurer
fd04ca7a5a src/bin/proxmox-backup-client.rs: pass verbose flag to dump_image 
And print some useful status to stderr.
 2019-10-09 12:04:02 +02:00
Dietmar Maurer
2c2097ee75 fix typo in error message 2019-10-07 12:48:56 +02:00
Dietmar Maurer
f7aa6f15d3 src/api2/reader.rs: use tokio::fs::read() to read file async 2019-10-07 12:24:06 +02:00
Dietmar Maurer
7123ff7d43 src/backup/data_blob.rs: avoid Arc<CryptConfig> 
use simple reference instead.
 2019-10-07 11:41:35 +02:00
Dietmar Maurer
4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
Dietmar Maurer
c26d4b4573 cleanup: remove dead code 2019-10-05 16:10:41 +02:00
Dietmar Maurer
88892ea801 src/bin/proxmox-backup-client.rs: cleanup restore 
- write directly to /dev/stdout (instead of using stdout line writer)
- avoid BufferedFixedReader
 2019-10-05 11:52:12 +02:00
Dietmar Maurer
62e05bb3b3 src/client/remote_chunk_reader.rs: pass reference to download_chunk 
Make sure we do not move data around.
 2019-10-05 11:45:20 +02:00
Dietmar Maurer
c0b1b14c05 src/api2/reader.rs - download_chunk: use blocking io 
Turns out to be twice as fast as tokio async io...
 2019-10-05 11:41:19 +02:00
Dietmar Maurer
ff01c1e393 src/client/http_client.rs: store raw data/blob size in index.json 2019-10-02 10:54:06 +02:00
Dietmar Maurer
f16aea687a src/client/http_client.rs: new helper send_upload_request() 2019-09-28 18:22:48 +02:00
Dietmar Maurer
9e603e254c src/backup/fixed_index.rs: do not use offset for checksum 2019-09-23 11:30:20 +02:00
Dietmar Maurer
fb6026b66b backup writer api: verify checksum inside close 2019-09-23 10:56:53 +02:00
Dietmar Maurer
c8c4051aa7 src/client/http_client.rs - H2Client::upload: fix parameter order 2019-09-20 13:19:55 +02:00
Dietmar Maurer
f011dba0f7 src/client/http_client.rs: add method parameter to H2 upload 2019-09-20 12:40:23 +02:00
Dietmar Maurer
792a70b9e1 src/client/http_client.rs: add content type parameter to H2 upload/requets_builder 2019-09-20 12:26:21 +02:00
Dietmar Maurer
a55b2975db src/client/http_client.rs: cleanup (rename parameter) 2019-09-20 11:57:03 +02:00
Dietmar Maurer
32efac1cce remove trailing whitespace 2019-09-20 08:12:31 +02:00
Christian Ebner
70235f72f4 src/bin/proxmox-backup-client.rs: add functionality to mount pxar archive. 
This adds the basic functionality to mount an archive stored in a repository
locally via fuse.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-20 08:01:01 +02:00
Christian Ebner
2fa91f5221 src/bin/pxar.rs: adapt to new deamonize flag 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-20 08:00:20 +02:00
Christian Ebner
d388e244f5 src/pxar/fuse.rs: make verbose flag independent from deamonize by adding this as new flag 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-20 08:00:04 +02:00
Christian Ebner
e717e9a3bf src/pxar/fuse.rs: refactor Session::new() 
and add Session::from_decoder() in order to be able to create a fuse session
with a `Decoder` given as argument.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-20 07:59:51 +02:00
Christian Ebner
946b72a6b2 src/pxar/fuse.rs: introduce Context in order to add caching. 
This patch introduces `Context` to hold the decoder, ino_offset and caches for
the attributes and the goodbye table.
By caching, certain callbacks can be handled without the need to read additional
data via the decoder, which improves performance.
The searching of the goodbye table is refactored as well, avoiding recursive
function calls in case of a hash collision.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-20 07:56:40 +02:00
Christian Ebner
1c93182371 src/pxar/decoder.rs: fix bug in decoder read 
`offset` points to the `PXAR_FILENAME`, therefore read the filename before
the `PXAR_ENTRY`.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-20 07:54:16 +02:00
Dietmar Maurer
2aba16bde1 src/backup/data_chunk.rs: add into_raw() 2019-09-19 14:24:17 +02:00
Dietmar Maurer
15bb1bba89 BackupClient: add upload() function 2019-09-19 13:59:24 +02:00
Dietmar Maurer
6d4df36c3c make download_chunk_list() public 2019-09-19 13:17:23 +02:00
Dietmar Maurer
cc2ce4a92f src/client/http_client.rs: add password parameter to HttpClient::new() 2019-09-16 12:35:23 +02:00
Christian Ebner
0e20b336e1 pxar: fuse: avoid possible hash collision in lookup by additional checking against filename 
The hash of the filename in the goodbye table items allows to quickly compare to
a hashed filename.
Unfortunately, a matching hash is no garantee for matching filenames as hash
collisions are possible.
This patch fixes such possible collisions by further checking the filenames once
a matching hash has been found.
This introduces no significant extra cost (except for the filename comparison)
for cases with matching hashes, as the lookup call has to seek and read the file
attributes (including the filename) anyway.
In cases with hash collision, the next matching item is read and treaded
analogously (what means we need at least one extra seek).
As collisions should be not that frequent, this should be an acceptable penalty.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-16 10:22:22 +02:00
Wolfgang Bumiller
ba5e67475a clippy: fix access to unaligned buffer in dynamic reader 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:50:45 +02:00
Wolfgang Bumiller
b526bd1408 backup: cleanup unused imports 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:46:46 +02:00
Wolfgang Bumiller
4dc79bb1d9 clippy: fix access to unaligned buffer 
via the newly introduced ReadExt::read_host_value_boxed()
from proxmox::tools::io.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:45:44 +02:00
Wolfgang Bumiller
ccdf3ad1ec backup: use static assertion in fixed index reader 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:43:25 +02:00
Wolfgang Bumiller
34a3845b01 clippy: don't use clone a double reference 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 14:02:30 +02:00
Wolfgang Bumiller
b21851cc5c clippy: collapse identical if branches 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 13:57:58 +02:00
Wolfgang Bumiller
299a2f18b2 fixup variable naming 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 13:57:23 +02:00
Wolfgang Bumiller
175eeb870d clippy: use write_all in file logger 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 13:56:09 +02:00
Wolfgang Bumiller
5a54935bc9 clippy: fix pointer to temporary value 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 13:55:56 +02:00
Wolfgang Bumiller
bd430c225b clippy: remove a loop{} which never actually loops 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 12:31:39 +02:00
Wolfgang Bumiller
44fed91e17 more clippy fixups 
mostly indentation changes, view with `-w`

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 12:23:12 +02:00
Wolfgang Bumiller
653b1ca10e some clippy fixups 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 12:14:05 +02:00
Christian Ebner
03310bea12 src/pxar/fuse.rs: s/Buf/ReplyBuf/ and s/BufState/ReplyBufState/ 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-11 11:27:18 +02:00
Christian Ebner
48cc1b8234 src/pxar/fuse.rs: refactor stat and fix i-node mapping 
The functionality of stat is split into smaller sub-functions, which allows
to reuse them more flexible, as the code flow is similar but not always the same.
By this, the ugly and incorrect re-setting of the i-node in the lookup callback
function is avoided.
The correct i-node is now calculated beforehand and stat simply creates a
`libc::stat` struct from the provided parameters.
Also, this fixes incorrect i-node assignments in the readdir callback function.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-11 11:26:55 +02:00
Christian Ebner
22eaa905a4 src/pxar/fuse.rs: impl readdir callback for fuse 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 11:38:58 +02:00
Christian Ebner
1652192f4d src/pxar/fuse.rs: impl readlink callback for fuse 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 11:38:48 +02:00
Christian Ebner
3710b23db5 src/pxar/decoder.rs: read_link has to read filename before entry. 
The offset points to the start of the item, which is the filename

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 11:36:04 +02:00
Christian Ebner
7eb9848ba7 src/pxar/fuse.rs: impl opendir callback for fuse 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 11:35:13 +02:00
Christian Ebner
2c5fd378d3 src/pxar/fuse.rs: impl read callback for fuse 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 11:34:55 +02:00
Christian Ebner
97e4db9dd5 src/pxar/fuse.rs: impl open callback for fuse 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 11:34:18 +02:00
Christian Ebner
b00689254a src/pxar/fuse.rs: impl lookup callback for fuse 
The lookup call checks if the given filename is found in the directory referenced
by the i-node by calclulating the filenames hash and looking it up in the
directories goodbye table.
If found, the entries parameters are returned.
In order to be able to lookup the parent offset by a given file offset in the
readdir callback, this also stores the corresponding values in a HashMap.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 10:54:23 +02:00
Christian Ebner
1443927c02 src/pxar/decoder.rs: correctly handle hardlinks in Decoders attribute function 
If the attributes are then returned by seeking to the new offset.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 10:46:39 +02:00
Christian Ebner
d00097a0e6 src/pxar/decoder.rs: split functionality of list_dir into list_dir and goodbye_table 
In order to read the contents of the goodbye table while keeping the
functionality of list_dir in place as is.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 10:46:29 +02:00
Christian Ebner
ac12570e99 src/pxar/decoder.rs: make read_directory_entry usable by fuse 
... and thereby allow it to read a single directory entry based on the
start and end archive offsets.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 10:32:48 +02:00
Christian Ebner
3f7f4f34be src/pxar/decoder.rs: remove DirectoryEntry start_offset and end_offset functions 
... as they are not needed with the latest iteration of the fuse callback
function impl (which never made it into the repos).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 10:30:45 +02:00
Christian Ebner
ac30bc3326 src/pxar/decoder.rs: correctly read, check and skip filename 
The previous implementation simply skipped over `size` bytes, which is not
correct as size includes also the header.
By relying on `SequentailDecoder`s read_filename function, this is correctly
taken care of plus some more integrity checks.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-10 10:30:21 +02:00
Dietmar Maurer
4a7de56e2f avoid injecting ENV vars from Makefile 
So that we can run "cargo build" without setting vars manually.
 2019-09-09 10:51:08 +02:00
Christian Ebner
d21ae955a6 src/pxar/fuse.rs: impl getattr callback for fuse 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-06 12:24:14 +02:00
Christian Ebner
ec04ea81f2 src/pxar/fuse.rs: Refactor run_in_context and remove inode_to_offset and offset_to_inode 
as their functionality is used only once so it makes more sense to keep them
inline.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-06 12:22:53 +02:00
Christian Ebner
8413083fc5 src/pxar/fuse.rs: Improve comments, add rustfmt skip tag and reafctor 
Also, removes an unused println statement in the decoder callback function and
fixes a typo.
Further, use ABI compatible Option<&T> for FFI to avoid use of raw pointers.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-06 12:22:17 +02:00
Dietmar Maurer
54a5a885a3 src/client/http_client.rs: use async for upload_speedtest() 2019-09-06 08:55:47 +02:00
Christian Ebner
bbd055bf45 src/pxar/decoder.rs: impl functionality needed for fuse implementation 
Implements functions attributes, open, read, read_link and get_dir
to be used by the fuse implementation which uses file offsets within the archive
as inodes to reference the archives items.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-06 07:18:14 +02:00
Christian Ebner
132cb0d0db src/pxar/decoder.rs: Add comment for unsolved issue and refactor if statement. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-06 07:16:31 +02:00
Dietmar Maurer
2a05048b93 src/client/http_client.rs: simplify send_request 2019-09-05 15:07:37 +02:00
Dietmar Maurer
9edd3bf1b8 src/client/http_client.rs: use async for h2api_response() 2019-09-05 14:56:52 +02:00
Dietmar Maurer
c18fddf80f src/client/http_client.rs - download_chunk_list: simplify async code 2019-09-05 14:16:45 +02:00
Dietmar Maurer
44f59dc731 src/client/http_client.rs - H2Client: fix error message in download() 2019-09-05 13:26:06 +02:00
Dietmar Maurer
c2a5a9f353 src/client/http_client.rs: use async for download_chunk_list 2019-09-05 13:12:03 +02:00
Dietmar Maurer
2f831baec0 src/client/http_client.rs - BackupClient: use async 2019-09-05 12:55:22 +02:00
Dietmar Maurer
2a1e6d7dea src/client/http_client.rs - BackupClient: use async 2019-09-05 11:42:12 +02:00
Dietmar Maurer
d4a085e564 src/client/http_client.rs - BackupReader: use async 2019-09-05 11:42:05 +02:00
Dietmar Maurer
9d456cf8ec src/backup/catalog_blob.rs: use write_all() instead of write() 
To correctly handle interrupted system calls.
 2019-09-05 11:40:48 +02:00
Dietmar Maurer
1a7a0e74c0 src/client/http_client.rs: use async for more functions 2019-09-04 13:48:16 +02:00
Dietmar Maurer
96f5e80abb src/client/http_client.rs: use async for login() 2019-09-04 10:01:46 +02:00
Dietmar Maurer
9d35dbbb8f src/client/http_client.rs: use async for credentials 2019-09-04 09:57:29 +02:00
Dietmar Maurer
d2267b112d src/bin/proxmox-backup-client.rs: verify blob/catlog checksums 2019-09-04 08:47:14 +02:00
Christian Ebner
70b03a07c8 src/pxar/sequential_decoder.rs: make functions needed in non-sequential decoder accessible from within crate 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-04 06:15:05 +02:00
Christian Ebner
f92e8266ae src/pxar/format_definitions.rs: move header type in PxarHeader and hash in PxarGoodbyeItem to top of struct 
By this it is possible to read and check just the first u64 of the corresponding
structs in order to identify the items.
This is needed for the fuse implementation in order to get entries based on the
archive offset, used as inode.
Directories are referenced by the offset to the goodbye tail while other items
are referenced by the offset of the filename followed by the entry.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-04 06:14:30 +02:00
Christian Ebner
75933d1e83 src/pxar/fuse.rs: add function to map offset to inode and vice versa 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-04 06:13:59 +02:00
Christian Ebner
c7fee396c6 src/pxar/decoder.rs: add getter functions to obtain start/end offsets for directories 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-04 06:10:56 +02:00
Christian Ebner
3626ac611f src/pxar/decoder.rs: rustfmt fixups 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-04 06:09:17 +02:00
Christian Ebner
3fa71727ee src/bin/proxmox-backup-client.rs: refactoring of match statement 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-09-04 06:08:00 +02:00
Dietmar Maurer
a40220c056 src/bin/proxmox-backup-client.rs: code cleanup - factor out some functions 2019-09-03 13:31:13 +02:00
Dietmar Maurer
36493d4d7e src/backup.rs: define INDEX_BLOB_NAME here 2019-09-03 13:15:44 +02:00
Dietmar Maurer
df65bd3da6 src/bin/proxmox-backup-client.rs - restore: verify checksums 2019-09-03 13:12:16 +02:00
Dietmar Maurer
0a51fe0011 src/backup/fixed_index.rs: new helper to compute checksum and file size 2019-09-03 13:11:45 +02:00
Dietmar Maurer
82c85a21a1 src/backup/dynamic_index.rs: new helper to compute checksum and file size 2019-09-03 13:11:09 +02:00
Dietmar Maurer
a3e032b754 src/client/http_client.rs: fix dynamic index checksum 2019-09-03 13:10:18 +02:00
Wolfgang Bumiller
c77563146b comment fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-03 11:17:33 +02:00
Wolfgang Bumiller
daef93f481 tools: add tokio::main() replacement 
to deal with block_on() not allowing blocking()

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-03 11:16:30 +02:00
Wolfgang Bumiller
66fbf5bad0 add WrappedReaderStream test 
Mostly to just document the rt.block_on() issue with
blocking().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-03 10:05:45 +02:00
Wolfgang Bumiller
2b92971fba merge fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:22:36 +02:00
Wolfgang Bumiller
083ff3fd5d update to tokio 0.2.0-alpha.4 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
1434f4f8df drop hyper_openssl 
We can use .await now, which means the whole connection
state machine doesn't need to be typed out as "types"
anymore, so, at least until hyper_openssl gets updated to
proper dependencies, let's drop it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
556eb70ea3 add generic either-or AsyncRead/Write type 
The HttpsConnector will use this. Instead of implementing a
specialized MaybeTlsStream, this is simply a generic "either
this or that kind of Async Read/Write type".

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
b30415d244 src/server/state.rs: update to tokio alpha.2 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
aadcf7c621 src/server/rest.rs: use tokio::timer::delay 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
0cdb8d9c5b src/tools/wrapped_reader_stream.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
e668912a99 src/tools/futures.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
0f5856acca src/tools/daemon.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
56e351c911 src/tools/broadcast_future.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
627bb7d114 src/tools/async_mutex.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
75fef4b463 src/server/worker_task.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
aa4110cc64 src/server/state.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
91e4587343 src/server/rest.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
b9b7f7ec1f src/server/h2service.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
160fc8147f src/server/command_socket.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
3c0facc787 src/client/remote_chunk_reader.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
369a87e3a2 src/client/pxar_backup_stream.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
2107bb40c1 src/client/pipe_to_stream.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
5b3911995b src/client/merge_known_chunks.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
a6782ca10b src/client/http_client.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
8630557918 src/bin/upload-speed.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
cab6816969 src/bin/test_chunk_speed2.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
fda5797b8a src/bin/proxmox-backup-proxy.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
e9722f8bde src/bin/proxmox-backup-client.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:21:26 +02:00
Wolfgang Bumiller
e76ac3a499 src/bin/proxmox-backup-api.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
e235c8f719 src/bin/h2s-server.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
74be6dc9b7 src/bin/h2server.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
55d8a631fc src/bin/h2s-client.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
15d0e4a3bd src/bin/h2client.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
b9203d87f4 src/bin/download-speed.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
745e652a7f src/backup/index.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
32bef1e2d1 src/backup/chunk_stream.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
f2d9b73c91 src/api_schema/router.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
ffb6434485 src/api2/reader.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
7622005574 src/api2/backup/upload_chunk.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
59b2baa0f6 src/api2/backup.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
6be147b78c src/api2/admin/datastore.rs: switch to async 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
5c1130df9f implement Sync for DynamicIndexReader and FixedIndexReader 
hyper's wrap_stream now needs this

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Wolfgang Bumiller
236761a3e6 drop src/storage/futures.rs 
it's unused and uses a tokio::Task I don't want to lookup
the std-future conversion for...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-02 15:17:58 +02:00
Dietmar Maurer
02fcf372e4 rc/bin/proxmox-backup-client.rs - restore: always download index.json.blob 2019-09-02 14:14:32 +02:00
Dietmar Maurer
0d9862803b src/bin/proxmox-backup-client.rs - reatore: avoid loading (large) blobs into memory 2019-09-02 14:13:31 +02:00
Christian Ebner
66c8eb9383 src/client/http_client.rs: Refactor handling Option and Result types 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-29 09:15:16 +02:00
Wolfgang Bumiller
40f8680e37 tools: remove SignalFd 
We don't use it anymore and it'll just be more work when
switching to async-await.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 13:44:53 +02:00
Wolfgang Bumiller
d21f8a5b5b backup/dynamic_index: explicitly zero-initialize the header 
The writer.write_all() call accessed data marked as
undefined to valgrind. Note that we shouldn't write out
uninitialized memory for security reasons anyway.
(note that vec::undefined already did zero-initialize the
data, but also marked it as undefined for valgrind when
compiling with the valgrind feature)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 11:42:35 +02:00
Wolfgang Bumiller
990b930f22 backup/dynamic_index: use static assertion, fix size 
The type was sized properly but the number was still wrong,
fixed this.
TODO! Once unions with non-Copy values are stable make this
a `union { full: [u8; 4096], data: TheActualHeader }`;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 11:42:35 +02:00
Christian Ebner
bcb664cb69 pxar/fuse: add pxar cli flag to set single- or multi-threaded session loop 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:43:14 +02:00
Christian Ebner
de2791868a src/pxar/sequentail_decoder.rs: followup: cleanup S_IFMT masking 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:42:22 +02:00
Christian Ebner
9cb9e1b7db src/pxar/sequential_decoder.rs: style fixups 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:42:19 +02:00
Christian Ebner
dac88033ba src/pxar/dir_stack.rs: style fixups 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:39:14 +02:00
Christian Ebner
b69169e7be src/pxar/fuse.rs: style fixups 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:37:58 +02:00
Christian Ebner
d792dc3cd2 src/pxar/match_pattern.rs: style fixups 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:37:56 +02:00
Christian Ebner
4ea22b68d9 src/pxar/format_definition.rs: style fixup 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-23 09:37:53 +02:00
Wolfgang Bumiller
7d83440c60 remove proxmox-protocol subcrate 
AFAICT we have no use for it anymore, its api entry points
are gone. If we do end up needing something from it, it's
still in the git history anyway. (And about two thirds of it
can be made much less awkward by utilizing async-await
anyway, so no love lost there...)

Moved the chunker back into src/backup/chunker.rs

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 14:08:25 +02:00
Wolfgang Bumiller
7a57cb77e1 more formatting & use statement fixups 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 13:50:27 +02:00
Christian Ebner
35dfd3ceb7 src/pxar/encoder.rs: style fixups by rustfmt 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 13:29:11 +02:00
Christian Ebner
1af30bc2fe (pxar: replace deprecated std::mem::uninitialized() 
... and use std::mem::MaybeUninit or proxmox::tools::vec::uninitialized() instead.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 13:27:25 +02:00
Wolfgang Bumiller
5e58e1bb7d {dynamic,fixed}_index: replace mem::uninitialized 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:35:36 +02:00
Wolfgang Bumiller
f93b55b057 src/backup/index: style fixup & unsafe copy removal 
We can use the safe .copy_from_slice alternative in this
case.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:31:45 +02:00
Wolfgang Bumiller
fa17b1ce2a src/backup/index: replace mem::uninitialized 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:27:35 +02:00
Wolfgang Bumiller
f3a8d1d7e0 api2/node/time: replace mem::uninitialized 
and fixup use statements

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:14:41 +02:00
Wolfgang Bumiller
48b85e8e3b tools: tty: replace mem::uninitialized and style fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:14:19 +02:00
Wolfgang Bumiller
b528fddfc6 tools: timer: style fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:07:56 +02:00
Wolfgang Bumiller
b7f44ace92 timer: replace mem::uninitialized 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 11:07:10 +02:00
Wolfgang Bumiller
6100071f4e tools: style & use statement fixups 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-22 10:57:56 +02:00
Wolfgang Bumiller
5c20e2da6b src/config: more style fixups 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 14:14:26 +02:00
Wolfgang Bumiller
91640ab567 src/config: style fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 14:11:07 +02:00
Wolfgang Bumiller
928650c4eb src/config: use statement cleanup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 14:10:50 +02:00
Wolfgang Bumiller
2ec979e4ed /var/run -> /run 
/var/run is considered deprecated and for instance in
systemd unit files lintian complains...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 12:28:24 +02:00
Wolfgang Bumiller
35238e234d switch from create_dir_chown to create_path 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 12:26:10 +02:00
Christian Ebner
e5471b4836 pxar/decoder: avoid failing on hardlinks and return corresponding target file entry 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
[Fixup: remove unnecessary 'mut']
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 09:32:52 +02:00
Christian Ebner
58262f40f7 src/pxar/decoder.rs: get correct stats for root directory 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 09:31:41 +02:00
Christian Ebner
7d26720e79 src/pxar/decoder.rs: move GOODBYE_ITEM_SIZE from function to module scope. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-21 09:31:39 +02:00
Wolfgang Bumiller
367f002ee2 use statement cleanup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-16 12:28:40 +02:00
Dietmar Maurer
9d135fe617 src/backup/catalog_blob.rs: moved catalog impl. from pxar 
And avoid loading catalog into memory.
 2019-08-16 12:27:17 +02:00
Wolfgang Bumiller
9025312aa6 avoid lifetimes in blob reader/writer 2019-08-16 10:07:24 +02:00
Dietmar Maurer
71d08e00b7 src/backup/data_blob_reader.rs: fix compiler warning 2019-08-16 08:10:39 +02:00
Christian Ebner
ac0b435344 src/pxar/fuse.rs: fix compiler warnings 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-14 15:14:57 +02:00
Dietmar Maurer
a84ef4c205 src/bin/proxmox-backup-client.rs: avoid loading catalog into memory 
We can use the new DataBlobReader instead.
 2019-08-14 15:07:28 +02:00
Dietmar Maurer
b791804f4b src/backup/data_blob_reader.rs: impl for Read instead of BufRead 2019-08-14 14:56:01 +02:00
Dietmar Maurer
39a4df61d6 improve docs 2019-08-14 14:08:27 +02:00
Dietmar Maurer
018d11bb84 src/backup/data_blob.rs: move parts into single files 2019-08-14 13:24:41 +02:00
Dietmar Maurer
e27c5a559f src/backup/data_blob.rs: avoid excessive stack usage 2019-08-14 12:44:08 +02:00
Dietmar Maurer
c638542b4b src/backup/file_formats.rs: moved header_size 2019-08-14 12:35:53 +02:00
Christian Ebner
41d08aa6de src/pxar/fuse.rs: enable libfuse debug output in verbose mode 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-14 08:59:15 +02:00
Christian Ebner
742e64ea0b src/pxar/fuse.rs: cleanup callback interface and store decoder within session context 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-14 08:58:44 +02:00
Christian Ebner
7750b7f2b7 pxar: decoder: take ownership of underlying reader 
By taking ownership it is easier to move the decoder into another struct,
e.g. into a session context in fuse.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-14 08:55:08 +02:00
Christian Ebner
f50b4fd6a0 src/pxar/decoder.rs: cleanup s/CaDirectoryEntry/DirectoryEntry 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-14 08:54:08 +02:00
Dietmar Maurer
6082216343 src/backup/data_blob.rs - CryptReader/CryptWriter: fix buffer size handling 2019-08-13 15:23:15 +02:00
Dietmar Maurer
548c9489d7 src/backup/data_blob.rs - DataBlobReader: impl compressed, encrypted blobs 2019-08-13 13:17:08 +02:00
Dietmar Maurer
2aa0bfff59 src/backup/data_blob.rs - DataBlobReader: implement reader for encrtypted blobs 2019-08-13 13:17:08 +02:00
Dietmar Maurer
e9a385a78e src/backup/data_blob.rs - DataBlobReader: impl compressed, signed blobs 2019-08-12 17:47:25 +02:00
Dietmar Maurer
4bfa147eaf src/backup/data_blob.rs - DataBlobReader: impl. reader for signed blobs 2019-08-12 17:41:25 +02:00
Dietmar Maurer
09785b2795 src/backup/data_blob.rs - DataBlobReader: start serious impl. 2019-08-12 17:20:41 +02:00
Dietmar Maurer
a32bd8a516 src/backup/data_blob.rs - DataBlobWriter: fix decryption 2019-08-12 11:57:29 +02:00
Dietmar Maurer
5d15cb49b4 src/backup/data_blob.rs - compute_crc: start after blob header 2019-08-12 11:32:26 +02:00
Dietmar Maurer
1cbdfd19a2 src/backup/file_formats.rs; use const instzead of static 2019-08-12 11:20:21 +02:00
Dietmar Maurer
18be4ec24a src/backup/data_blob.rs - DataBlobWriter: fix magic number for uncompressed blobs 2019-08-12 10:46:05 +02:00
Dietmar Maurer
5622a3fcdb src/backup/data_blob.rs - DataBlobWriter: impl. compressed encrypted blobs 2019-08-12 10:16:42 +02:00
Dietmar Maurer
f4942e9ffd src/backup/data_blob.rs - DataBlobWriter: impl. encrypted blobs 2019-08-12 10:08:00 +02:00
Dietmar Maurer
c57ec43a53 src/backup/crypt_config.rs: new helper data_crypter to get openssl Crypter 2019-08-12 10:06:51 +02:00
Dietmar Maurer
7776becf5d src/backup/data_blob.rs - DataBlobWriter: impl. compressed signed blobs 2019-08-12 09:01:08 +02:00
Dietmar Maurer
706638f803 src/backup/data_blob.rs: impl. ChecksumWriter 
To correctly compute crc,hmac of final data.
 2019-08-12 08:53:53 +02:00
Dietmar Maurer
f796351c54 src/backup/data_blob.rs: impl. DataBlobWriter for authenticated blobs 2019-08-12 07:34:41 +02:00
Dietmar Maurer
cb0eea29d9 src/backup/crypt_config.rs: new method to get hmac signer 2019-08-12 07:33:15 +02:00
Dietmar Maurer
a762ce54fd src/backup/data_blob.rs: make DataBlobWriter more generic 
Allow to write compressed and uncompressed blobs.
 2019-08-11 12:14:04 +02:00
Dietmar Maurer
1f26fdef19 src/backup/data_blob.rs: impl. simple reader/writer 
To avoid loading blob into memory.
 2019-08-11 11:32:36 +02:00
Dietmar Maurer
e240d8be0b src/bin/proxmox-backup-client.rs: implement login/logout 2019-08-10 09:12:17 +02:00
Christian Ebner
f71e8cc96f pxar: add command 'mount' to cli of pxar 
Allows to mount an archive to a specified mountpoint via the cli.
Once the archive is mounted, the process is send to the background.
By passing the --verbose flag, the process is kept in foreground and
debug output is provided.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-09 16:46:16 +02:00
Dietmar Maurer
02ba8a5d11 pxar: add fuse module and expose its pub functionality. 2019-08-09 16:45:13 +02:00
Christian Ebner
c50f87442c pxar: add basic code for FUSE implementation. 
This adds the basic code in order to create a fuse session and mount an archive.
It adds libfuse3-3 as runtime dependency and libfuse3-dev as build dependency.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-09 16:42:31 +02:00
Dietmar Maurer
781ac11c6a src/backup/data_blob.rs: define const MAX_BLOB_SIZE 2019-08-09 11:49:06 +02:00
Dietmar Maurer
9049a8cfe6 src/bin/proxmox-backup-client.rs: implement catalog command 2019-08-09 11:36:33 +02:00
Dietmar Maurer
7926a3a1cf src/bin/proxmox-backup-client.rs: add archive filename to catalog 2019-08-09 11:05:56 +02:00
Dietmar Maurer
863be2e6e2 src/backup/data_blob.rs: allow largert blob sizes (128MB) 
Need this for catalog files.
 2019-08-09 10:22:56 +02:00
Dietmar Maurer
3fb53e07b4 src/bin/proxmox-backup-client.rs: avoid compiler warning 2019-08-09 10:22:32 +02:00
Dietmar Maurer
543a260ff9 src/bin/proxmox-backup-client.rs: new completion helper for snapshots 2019-08-09 10:08:45 +02:00
Dietmar Maurer
2761d6a4f3 src/pxar/encoder.rs: use BackupCatalogWriter 2019-08-09 09:46:49 +02:00
Dietmar Maurer
aea0815d32 src/pxar/catalog.rs: new catalog helper 2019-08-09 09:41:43 +02:00
Dietmar Maurer
977eeb24f6 src/bin/proxmox-backup-client.rs: avoid doubled file extensions (.blob.blob) in index.json 2019-08-07 10:23:08 +02:00
Dietmar Maurer
4af0ee055a api: always use complete file names (including add exctensions) 2019-08-07 10:10:14 +02:00
Dietmar Maurer
d7c243977d src/api2/admin/datastore.rs - list_snapshot_files: list all files 
Add files that are not mentioned in index.json.
 2019-08-07 09:16:14 +02:00
Dietmar Maurer
34066e2eeb src/api2/backup/environment.rs - add_blob: always verify blob crc 2019-08-07 09:15:13 +02:00
Dietmar Maurer
eecb23560b src/backup/data_blob.rs - encode: always compute crc 2019-08-07 08:30:27 +02:00
Dietmar Maurer
38a6cddad7 src/backup/backup_info.rs: add BackupInfo::new constructor 2019-08-07 08:27:52 +02:00
Dietmar Maurer
c4f025ebbf src/bin/proxmox-backup-client.rs: fix remote filename completion 2019-08-07 06:48:18 +02:00
Dietmar Maurer
dbed4c8cd7 src/client/http_client.rs: compute checksums for chunk streams 2019-08-06 12:23:06 +02:00
Dietmar Maurer
bd15e96dd9 src/client/http_client.rs: minor code cleanup 2019-08-06 11:46:15 +02:00
Dietmar Maurer
f889b15867 src/backup/data_blob.rs: always compute crc 2019-08-06 11:42:14 +02:00
Dietmar Maurer
c807d231eb src/client/http_client.rs: compute file checksums 2019-08-06 11:26:06 +02:00
Dietmar Maurer
a17a0e7a9f src/api2/admin/datastore.rs: extract backup size from index.json 2019-08-06 10:56:21 +02:00
Christian Ebner
eecb182845 pxar: add initial docs for MatchPattern 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-06 09:03:02 +02:00
Christian Ebner
43e892d293 pxar: add error handling for MatchPattern::matches_filename() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-06 09:02:44 +02:00
Dietmar Maurer
8c70e3eb18 src/api2/admin/datastore.rs - files: return data from index.json 2019-08-05 13:22:19 +02:00
Dietmar Maurer
f69adc81a6 use new proxmox::tools::nodename 2019-08-03 17:06:23 +02:00
Dietmar Maurer
7f66c29e5c remove src/tools/common_regex.rs, use proxmox::tools::common_regex instead 2019-08-03 16:44:31 +02:00
Dietmar Maurer
fd329bdcd7 remove src/tools/procfs.rs, use proxmox::sys::linux::procfs instead 2019-08-03 16:26:44 +02:00
Dietmar Maurer
779e4c90b1 src/tools/vec.rs: remove unused file 
We already moved that to proxmox::tools
 2019-08-03 16:11:47 +02:00
Dietmar Maurer
e18a6c9ee5 update to nix 0.14, use code from proxmox:tools 2019-08-03 13:05:38 +02:00
Dietmar Maurer
b86f263ced use new proxmox::sys::linux::magic 2019-08-03 09:28:57 +02:00
Christian Ebner
4ea0f7d95b pxar: remove flag to change digest algorithm 
The flag CA_FORMAT_SHA512_256 is used to switch between sha512 and sha256 to
calculate digest in casync.

As we use sha256, we can get rid of this flag for now.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:57:53 +02:00
Christian Ebner
4d142ea79e pxar: cleanup: refactor and rename exclude pattern 
The original name PxarExcludePattern makes no sense anymore as the patterns are
also used to match filenames during restore of the archive.

Therefore, exclude_pattern.rs is moved to match_pattern.rs and PxarExcludePattern
rename to MatchPattern.
Further, since it makes more sense the MatchTypes are now declared as None,
Positive, Negative, PartialPositive or PartialNegative, as this makes more sense
and seems more readable.
Positive matches are those without '!' prefix, Negatives with '!' prefix.

This makes also the filename matching in the encoder/decoder more intuitive and
the logic was adapted accordingly.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:52:32 +02:00
Christian Ebner
fe076c8259 pxar: cleanup: s/PxarDirBuf/PxarDirStack/g and move dir_buffer.rs to dir_stack.rs 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:52:10 +02:00
Christian Ebner
ead7546a96 pxar: change uid and gid to u32 instead of u64 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:51:51 +02:00
Christian Ebner
5e50c606b0 pxar: cleanup: s/CA_FORMAT/PXAR/g and s/CaFormat/Pxar/g 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:51:33 +02:00
Christian Ebner
47651f9530 pxar: cleanup: move feature flags to src/pxar/flags.rs and omit CA_FORMAT prefix on all of them 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:51:06 +02:00
Christian Ebner
91451c4355 pxar: cleanup: move filesystem related magic numbers to tools/fs/magic.rs 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-03 08:48:12 +02:00
Dietmar Maurer
b335f5b713 src/bin/proxmox-backup-client.rs: sign index.json 2019-08-02 10:00:15 +02:00
Dietmar Maurer
69ecd8d5b0 src/backup/data_blob.rs: implement signed blobs 2019-08-02 09:56:01 +02:00
Dietmar Maurer
93205f942a src/backup/crypt_config.rs: new compute_auth_tag helper 2019-08-02 08:55:37 +02:00
Dietmar Maurer
c68d2170d5 src/backup/crypt_config.rs: fix typo 2019-08-02 08:29:40 +02:00
Christian Ebner
81a9905e0a pxar: implement feature flag support for device nodes, fifos and sockets 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:55:17 +02:00
Christian Ebner
8abc95a145 pxar: remove some unused feature flags from format definition 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:54:46 +02:00
Christian Ebner
3a905cac2b src/pxar/sequential_decoder.rs: remove of obsolete function dir_mkdirat 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:52:49 +02:00
Christian Ebner
beffac999f src/pxar/sequentail_decoder.rs: fix issue when restoring with glob pattern. 
Partial extraction of an archive with a glob pattern, e.g. '**/*.conf' lead to
the unexpected behaviour of restoring all partially matched directories (in this
example all of them).

This patch fixes this unexpected behaviour by only restoring those directories
were the directory or one of its sub-items fully matched the pattern and should
therefore be restored.

To achive this behavoiur, directory metadata is pushed onto a stack and restored
on demand.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:49:47 +02:00
Christian Ebner
51ac99c314 src/pxar/exclude_pattern.rs: add Clone and Copy trait to MatchType 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:49:47 +02:00
Christian Ebner
6a584cfd76 pxar: add PxarDir and PxarDirBuf to buffer directory metadata 
In order to restore only directories when some of their content fully matched
a match pattern on partial restores, these directories and their metadata are
pushed onto this buffer and only restored successivley on demand.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:45:12 +02:00
Dietmar Maurer
27e6e180b6 src/pxar/format_definition.rs - PxarAttributes: derive Default 
And remove unnecessary impl new().
 2019-08-02 07:16:10 +02:00
Christian Ebner
a92e6c9011 src/pxar/sequential_decoder.rs: split read and restore of attributes into separate functions 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:11:38 +02:00
Christian Ebner
e60cfbcdd0 src/pxar/format_definition.rs: add struct PxarAttributes 
This structure contains all the attributes allowing to easily store those within
a e.g. dir buffer.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 07:10:22 +02:00
Christian Ebner
2be3eff5c3 pxar: avoid taking ownership of xattrs, fcaps and quota_projid on corresponding restore functions. 
By borrowing these objects we preserve the functionality but make sure
that ownership doesn't change, avoiding problems when contained within other
structs such as e.g. a buffer storing these attributes.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-08-02 06:44:00 +02:00
Dietmar Maurer
2c3891d1c3 src/bin/proxmox-backup-client.rs: upload backup index.json 
The plan is to use this file to verify the backup content.
 2019-08-01 12:39:02 +02:00
Dietmar Maurer
953d5e1531 src/tools.rs - file_set_contents_full: only call fchmod when we pass permissions 
So that we can use it to write into /etc/pve/ (which does not support chmod).
 2019-07-31 12:44:27 +02:00
Dietmar Maurer
cec17a3ec7 src/bin/proxmox-backup-client.rs - files: allow --repository parameter 2019-07-31 11:45:55 +02:00
Dietmar Maurer
52c171e49d src/bin/proxmox-backup-client.rs: implement files command 2019-07-31 11:23:53 +02:00
Dietmar Maurer
43a406fd3f src/bin/proxmox-backup-client.rs: supress useless output from prune command 2019-07-31 10:15:16 +02:00
Christian Ebner
9731c8b862 pxar: add support for storing/restoring the quota project id on ZFS 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-30 09:40:48 +02:00
Christian Ebner
6a87910949 pxar: implement allow_existing_dirs for pxar decoder 
By default, restoring an archive will fail if files with the same filename
already exist in the target directory.

By setting the allow_existing_dirs flag, the restore will not fail if an
existing directory is encountered.
The metadata (permissions, acls, ...) of the existing directory will be set
to the ones from the archive.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-30 09:40:30 +02:00
Dietmar Maurer
46d5aa0a09 src/bin/proxmox-backup-client.rs: add restore flag --allow-existing-dirs 
Not jet implemented ...
 2019-07-29 12:49:15 +02:00
Dietmar Maurer
ea7a7ef299 src/bin/proxmox-backup-client.rs: fixup - correctly pass prune parameters 2019-07-27 09:24:23 +02:00
Dietmar Maurer
9fdc3ef46f src/api2/admin/datastore.rs: pass backup type/id (group) to prune 
IMHO, prune over all backup groups is a bit dangerous, considering
that more than one user might use a datastore.
 2019-07-27 08:49:14 +02:00
Dietmar Maurer
7a6cfbd98d src/bin/proxmox-backup-client.rs: correctly compute duration 
And format time with SecondsFormat::Secs
 2019-07-26 11:13:49 +02:00
Dietmar Maurer
bbf9e7e951 src/api2/backup.rs: cleanup schema definitions 2019-07-26 09:07:29 +02:00
Dietmar Maurer
ca5d0b61ca src/api2/backup.rs: new required backup-time parameter 
The client should pass the time as parameter.
 2019-07-25 13:44:01 +02:00
Dietmar Maurer
e128d4e84f src/api2/admin/datastore.rs: backup logs may not be written twice 2019-07-25 12:29:04 +02:00
Dietmar Maurer
ef3254c086 src/api2/admin/datastore.rs: use correct .blob extension for log 2019-07-25 12:22:50 +02:00
Dietmar Maurer
ec34f7eba0 src/bin/proxmox-backup-client.rs: implement upload-log 2019-07-25 12:18:15 +02:00
Dietmar Maurer
04512d3068 src/client/http_client.rs: allow upload with parameters 2019-07-25 12:17:35 +02:00
Dietmar Maurer
07ee223590 src/api2/admin/datastore.rs: add api to upload backup client log file 2019-07-25 12:15:30 +02:00
Dietmar Maurer
79679c2d6e src/bin/proxmox-backup-client.rs: allow to uploag .log files (as binary blobs) 2019-07-24 13:07:02 +02:00
Dietmar Maurer
5b72c9b4f7 src/pxar/encoder.rs: add new skip_lost_and_found parameter 2019-07-24 12:21:25 +02:00
Dietmar Maurer
6c3c9bceb5 pxar_backup_stream.rs: limit lock scope to avoid blocking forever 2019-07-24 11:30:43 +02:00
Dietmar Maurer
684233aa3b remove debug output 2019-07-24 09:33:52 +02:00
Dietmar Maurer
5be106eeae src/client/pxar_backup_stream.rs: correctly pass errors to stream 2019-07-24 09:24:35 +02:00
Dietmar Maurer
2eeaacb974 src/pxar/encoder.rs: allow to pass list of devices 
For better mount point include control...
 2019-07-24 08:11:59 +02:00
Dietmar Maurer
fa5d6977dd Revert previous, commit, use UTC RFC3339 without timezone (Z) 
We now have human readable data/time, and names are still sortable.
 2019-07-22 11:49:30 +02:00
Dietmar Maurer
10c2a21cfe src/backup/backup_info.rs: use unix epoch timestamps instead of RFC3339 
RFC3339 contains additional timezone information, but IMHO this just adds
more confusion. The API also uses timestamps.
 2019-07-22 07:57:03 +02:00
Christian Ebner
1fdef50f52 src/pxar/sequential_decoder.rs: Call callback only on restored paths. 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-19 11:58:51 +02:00
Dietmar Maurer
40a13369a3 src/bin/pxar.rs - extract: print archive name with --verbose flag 2019-07-19 07:01:57 +02:00
Dietmar Maurer
bbd4365c61 src/bin/pxar.rs: remove wrong debug message 2019-07-19 07:00:17 +02:00
Christian Ebner
a0ec687cd7 src/bin/pxar.rs: allow to pass paths and match patterns as args to pxar extract 
To improve usability it is now possible to directly pass paths or match patterns
as arguments to pxar extract to partially restore an archive.
The patterns provided via CLI are appended to the ones read from file by the
--files-from option in order to have priority over those.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-19 06:50:39 +02:00
Dietmar Maurer
9aa3f68278 src/bin/proxmox-backup-client.rs: code cleanup 2019-07-18 13:16:32 +02:00
Dietmar Maurer
f6ede796ca src/bin/proxmox-backup-client.rs - status: only print result[data] 2019-07-18 09:52:11 +02:00
Dietmar Maurer
15c847f136 src/api2/admin/datastore.rs: allow to list all snapshots 2019-07-18 09:11:43 +02:00
Dietmar Maurer
0f73ee73bb src/cli/command.rs: set exit status on error 2019-07-17 13:40:10 +02:00
Christian Ebner
129dda4729 src/bin/pxar.rs: add --files-from parameter to pxar extract 
Allows to pass a list of match pattern to the pxar extract call by reading them
from file.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:20:29 +02:00
Christian Ebner
fa7e957ca3 pxar: Adapt SequentialDecoder::restore() calls to pass empty match pattern for full restore 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:19:31 +02:00
Christian Ebner
d32c2e4d27 src/pxar/sequential_decoder.rs: adapt code for partial restore by match pattern 
Allows to partially restore an archive by passing match patterns to the restore
function.
The whole restore is performed in sequential, therefore the whole archive has to
be read.
By wrapping the RawFd into an Option it can be controlled if the corresponding
part is restored (in case of Some(fd)) or if the Reader reads over it
without restore (in case of None).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:11:37 +02:00
Christian Ebner
4902291673 src/pxar/sequential_decoder.rs: implement match_filename to match include pattern for partial restores 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:07:54 +02:00
Christian Ebner
894cd49ac4 src/pxar/sequential_decoder.rs: Also check for nul bytes when reading the filename 
Check if the filename does not contain invalid nul byes when reading it from the
archive.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 12:04:41 +02:00
Christian Ebner
0b39674895 src/pxar/sequential_decoder.rs: remove old comments and refactoring 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 09:46:02 +02:00
Christian Ebner
9f8fcdd050 src/pxar/encoder.rs: refactoring of match_exclude_pattern() 
match_exclude_pattern() does not need a '&mut self' reference to the encoder,
move it therefore out of the impl.
Further, this patch contains some naming and formatting cosmetics.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 09:45:19 +02:00
Christian Ebner
3ff4ef28e6 src/pxar/encoder.rs: remove unneeded dir_count 
dir_count was used to track the number of directory entries to store in the
archive and bail if the maximum is exceeded.
As the number of entries is equally obtained from the list of the filenames to
include, use that one instead.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 09:44:59 +02:00
Christian Ebner
a771f90745 src/pxar/exclude_pattern.rs: do not return partial matches for files 
Partial matches make only sense for directories, files are always leafs of the
tree. Take this into account in order to avoid restoring of files which only
matched the front of a match pattern.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 09:44:06 +02:00
Christian Ebner
bdf0d82ced src/bin/pxar.rs: Make pxar extract target optional 
In order to improve usablity, the target on archive extraction will be the
current working directory by default.
A different target can be provided via the optional --target <PATH> parameter.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-17 09:40:02 +02:00
Dietmar Maurer
34a816cc7b src/bin/proxmox-backup-client.rs: add output-fromat options, implement status api 2019-07-16 13:35:25 +02:00
Dietmar Maurer
0eecf38fbf src/api2/admin/datastore.rs: add status api call 2019-07-16 13:34:38 +02:00
Dietmar Maurer
25500cfa26 src/cli/command.rs: add helpers/schema to format output 2019-07-16 11:46:58 +02:00
Dietmar Maurer
aad2ee4947 src/bin/pxar.rs: only print archive name in verbose mode 2019-07-16 08:38:02 +02:00
Dietmar Maurer
6b9a071028 src/bin/pxar.rs: add --verbose parameter to list (replace dump) 2019-07-16 08:31:40 +02:00
Dietmar Maurer
2665cef703 src/bin/proxmox-backup-client.rs: make repository parameter optional 
And allow to read it from $ENV{PBS_REPOSITORY}
 2019-07-16 07:30:04 +02:00
Christian Ebner
3f5192730e src/pxar/format_definition.rs: fix typo 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-15 10:49:34 +02:00
Dietmar Maurer
9eae781ab8 src/bin/pxar.rs - extract: allow to read from stdin 2019-07-05 12:39:25 +02:00
Dietmar Maurer
c6c9e093d0 src/bin/pxar.rs - dump: allow to read from stdin 2019-07-05 12:32:15 +02:00
Dietmar Maurer
bf1252619a src/bin/proxmox-backup-client.rs - restore: allow to pipe output to stdout 2019-07-05 12:14:50 +02:00
Dietmar Maurer
f8100e9623 src/bin/proxmox-backup-client.rs: remove download cli, do everything in restore 2019-07-05 11:36:45 +02:00
Dietmar Maurer
f4bf7dfcc7 src/client/remote_chunk_reader.rs: implement simple caching 2019-07-05 10:42:46 +02:00
Dietmar Maurer
afb4cd28be src/backup/fixed_index.rs: implement BufferedFixedReader 2019-07-05 09:19:56 +02:00
Christian Ebner
be9b39e17a src/pxar/sequential_decoder.rs: major refactoring of restore_sequential() 
This splits the functionality of restore_sequential() into several smaller
functions in order to allow to reuse them when restoring by seeking based on
the goodbye table offsets.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-05 06:45:50 +02:00
Christian Ebner
48a4509c67 src/pxar/encoder.rs: fix bug with incorrect offset stored in goodbye table 
Commit cd7dc87903 introduced the special treatment
for .pxarexclude files when stored in the archive.
The incorrect placement of a code snipplet from this path leads to an incorrect
offset and size stored in the goodbye table.
This fix places the start to the correct position, restoring the previously
correct behaviour.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-05 06:39:39 +02:00
Christian Ebner
7dcbe051e9 pxar: Refactor SequentialDecoder to store the callback function within the struct 
Reduces the number of arguments for the function calls within the decoder.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-05 06:38:22 +02:00
Dietmar Maurer
b46c3fad03 src/backup/fixed_index.rs: use correct size 
We need to consider that the last chunk may have smaller size.
 2019-07-04 15:13:22 +02:00
Dietmar Maurer
9cc88a7c00 src/client/http_client.rs - finish: hold reference to avoid early connection close 2019-07-04 15:11:23 +02:00
Dietmar Maurer
5e04ec707d src/api2/backup/environment.rs: improve chunk alignment/size checks 2019-07-04 13:40:43 +02:00
Dietmar Maurer
a24e3993e0 src/backup/chunk_store.rs: coding style fixes 2019-07-04 11:39:10 +02:00
Dietmar Maurer
e4c2fbf170 src/backup/chunk_store.rs: additionally log chunk count 2019-07-04 11:27:11 +02:00
Dietmar Maurer
9850bcdf19 src/backup/chunk_store.rs: improve error reporting 2019-07-04 11:21:54 +02:00
Wolfgang Bumiller
a3f3e91da2 backup/chunk_store: rework chunk iterator 
We can now use iter::from_fn() which makes for a much nicer
logic. The only thing better is going to be when we can use
generators with `yield`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-04 10:55:17 +02:00
Dietmar Maurer
99168f43e6 src/client/http_client.rs: s/set_recv_buf_size/set_recv_buffer_size/ 2019-07-04 10:51:18 +02:00
Dietmar Maurer
c698636a5d src/tools/fs.rs - read_dir: return nix::Result 2019-07-04 10:47:16 +02:00
Dietmar Maurer
a57360983b src/backup/chunk_store.rs - get_chunk_iterator: return percentage inside iterator item 2019-07-04 09:26:44 +02:00
Dietmar Maurer
0f0a35b390 src/backup/dynamic_index.rs: lock file inside new, code cleanup 2019-07-04 08:17:30 +02:00
Dietmar Maurer
a7c72ad9eb src/backup/fixed_index.rs - FixedIndexReader: remove reference to chunk store 2019-07-04 08:13:16 +02:00
Dietmar Maurer
a660978c9a src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats 2019-07-04 07:57:43 +02:00
Dietmar Maurer
86eda3eb0d src/bin/proxmox-backup-client.rs: implement restore using BackupReader 2019-07-03 15:45:12 +02:00
Dietmar Maurer
b2a6897982 src/client/remote_chunk_reader.rs: remove dead code 2019-07-03 14:56:24 +02:00
Dietmar Maurer
7f99bf691a src/client/remote_chunk_reader.rs: implement remote chunk reader 2019-07-03 14:39:13 +02:00
Dietmar Maurer
4f6aaf542c src/client/http_client.rs: avoid compiler warning 2019-07-03 14:36:02 +02:00
Dietmar Maurer
3467cd91e9 src/client/http_client.rs: return Arc<Self> 
Avoid to call canceller for each clone on drop ...
 2019-07-03 14:26:07 +02:00
Dietmar Maurer
1e0784841c src/backup/read_chunk.rs: use &mut self 2019-07-03 14:25:18 +02:00
Christian Ebner
0d70535a36 src/pxar/encoder.rs: refactor simple match statements to map_err() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-03 12:18:36 +02:00
Christian Ebner
6e1c31163b src/pxar/sequential_decoder.rs: refactoring of simple match statements to map_err() 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-03 12:18:24 +02:00
Christian Ebner
9d227350a3 src/pxar/sequential_decoder.rs: Cleanup and fix incorrect error message 
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-07-03 12:16:59 +02:00
Dietmar Maurer
80af046794 src/server/rest.rs: avoid unwrap 2019-07-03 12:00:43 +02:00
Dietmar Maurer
7fb4f5642a src/server/rest.rs: log peer address, use hyper MakeService 2019-07-03 11:54:35 +02:00
Dietmar Maurer
6d1f61b208 use hyper/tokio-openssl instead of hyper/tokio-tls 
This exposes the complete SSL setup. And download is much faster
now (600MB/s instead of 130MB/s)!
 2019-07-02 13:36:28 +02:00
Dietmar Maurer
74d0a6bc23 src/bin/proxmox-backup-proxy.rs: set socket options for max. performance 2019-07-02 10:48:58 +02:00
Dietmar Maurer
93f8bb0445 src/bin/h2s-client.rs: avoid compiler warnings 2019-07-02 08:59:33 +02:00
Dietmar Maurer
877dd11133 src/backup/{fixed, dynamic}_index.rs: improve error messages 2019-07-02 08:56:56 +02:00
Dietmar Maurer
5c593a4d56 src/backup/read_chunk.rs: use Arc for DataStore 2019-07-02 08:49:16 +02:00
Dietmar Maurer
b850673634 src/backup/read_chunk.rs: move read chunk trait into extra file 
And implement LocalChunkReader.
 2019-07-02 08:22:29 +02:00
Dietmar Maurer
8fad30a4b1 use latest hyper master branch to include socket buffer size fix 2019-07-02 07:09:46 +02:00
Wolfgang Bumiller
f35197f449 replace tools::vec with proxmox::tools::vec 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-01 11:03:25 +02:00
Wolfgang Bumiller
d0162d53d3 tools: remove io module 
now completely replaced by proxmox::tools::io.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-01 10:44:12 +02:00
Wolfgang Bumiller
9b2b627fe0 update remaining users of tools::io::ops 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-01 10:44:03 +02:00
Wolfgang Bumiller
9110a69bd8 tools: remove read/write 
now completely replaced by
proxmox::tools::io::{ReadExt, WriteExt}

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-01 10:39:42 +02:00
Wolfgang Bumiller
ca3c3ce997 daemon: remove last use of tools::read/write 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-01 10:39:13 +02:00
Wolfgang Bumiller
5485b579a1 backup: use new proxmox::tools::io traits 
This replaces all `write_value` calls with `write_le_value`
calls!

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-07-01 10:37:02 +02:00
Dietmar Maurer
dc9775d1b0 src/bin/h2s-client.rs: increase tokio tcp stream read buffer size 
This improves download speed by factor 10.
 2019-06-29 18:09:50 +02:00
Dietmar Maurer
34f7131160 src/bin/h2s-client.rs: avoid hyper, use h2 directly 
But performance is still bad.
 2019-06-29 15:58:18 +02:00
Dietmar Maurer
5ed2277f0e src/bin/h2s-server.rs: test to reproduce slow h2 download 
Simplified code, use with h2s-client.rs.
 2019-06-29 14:57:54 +02:00
Dietmar Maurer
17243003b3 src/bin/download-speed.rs: h2 download speedtest for our server 
I get about 30MB/s (much too slow)
 2019-06-29 13:43:10 +02:00
Dietmar Maurer
a2b29b68d4 src/client/http_client.rs: use maximal h2 window sizes 2019-06-29 11:05:36 +02:00
Dietmar Maurer
872debdefa src/bin/h2client.rs: tune h2 client parameters to get reasonable speed 
We now get about 2.5GB/s (not really good, but better)
 2019-06-29 10:41:24 +02:00
Dietmar Maurer
38294bcc56 src/client.rs: make module put_to_stream public 2019-06-29 10:40:43 +02:00
Dietmar Maurer
fded1f3154 add code to test H2 speed 
I currently get about 1MB/s, which is ways too slow (must be a bug)?
 2019-06-29 10:08:12 +02:00
Dietmar Maurer
d48a9955a5 src/backup/dynamic_index.rs: introduce ReadChunk trait 2019-06-28 16:35:00 +02:00
Dietmar Maurer
09d7dc5024 src/api2/reader.rs: add download chunk api 2019-06-28 16:27:01 +02:00
Dietmar Maurer
fcf5dea56d src/client/http_client.rs: increase h2 window size, set tcp nodelay option 
To increase h2 download speed.
 2019-06-28 16:00:58 +02:00
Dietmar Maurer
81a6ce6fde src/backup/chunk_store.rs: new method chunk_path() 
Returns the absolute path.
 2019-06-28 15:48:09 +02:00
Dietmar Maurer
fcfb84fedf file download: avoid unnecessary copy 2019-06-28 07:07:52 +02:00
Dietmar Maurer
984a7c3502 src/client/http_client.rs - h2 download: implement flow control 2019-06-28 07:02:43 +02:00
Dietmar Maurer
dd066d28e2 src/api2/reader.rs: implement backup reader protocol 2019-06-27 09:01:41 +02:00
Dietmar Maurer
42a87f7b96 src/server/h2service.rs: implement generic h2 service 2019-06-26 17:38:33 +02:00
Dietmar Maurer
fb0470837b src/client/http_client.rs: split out code to start new h2 connections 2019-06-26 12:09:18 +02:00
Dietmar Maurer
b208da8393 src/backup/data_{chunk,blob}.rs: add verify_crc method 2019-06-26 09:54:25 +02:00
Dietmar Maurer
fef44d4f78 src/bin/proxmox-backup-client.rs: decode blobs on download 
No more need for that clumsy blob cli (removed).
 2019-06-26 09:18:59 +02:00
Dietmar Maurer
c2b945341c src/client/http_client.rs - download: use generic Write type, return writer. 
Make it possible to write int Vec<u8>.
 2019-06-26 09:17:13 +02:00
Dietmar Maurer
bb8231409e src/backup/crypt_config.rs - generate_rsa_encoded_key: store as json 
Use the KeyConfig serialization with kdf = None.
 2019-06-26 07:32:34 +02:00
Dietmar Maurer
3031e44c58 src/bin/proxmox-backup-client.rs: remove stale download file, start blob cli 2019-06-25 13:00:37 +02:00
Dietmar Maurer
f14a8c9a85 src/api2/admin/datastore.rs: use correct path for download 2019-06-25 12:59:36 +02:00
Dietmar Maurer
81820b0d4d src/client/http_client.rs - download: fix error handling 2019-06-25 12:43:55 +02:00
Dietmar Maurer
45db6f89e1 src/bin/proxmox-backup-client.rs: implement download command 2019-06-25 11:17:24 +02:00
Dietmar Maurer
9e47c0a56e src/api2/admin/datastore.rs: implement generic download method 
We simply allow to download raw binary data.
 2019-06-25 10:16:59 +02:00
Dietmar Maurer
da7d67213e src/api2/admin/datastore/pxar.rs: remove pxar upload api 
Upload needs to be done using the backup protocol.
 2019-06-25 09:45:35 +02:00
Dietmar Maurer
dd8e744f9a src/api2/admin/datastore.rs: use a worker log log prune actions 2019-06-25 08:12:25 +02:00
Dietmar Maurer
6899dbfb47 src/bin/proxmox-backup-client.rs: strip .blob extension 2019-06-25 07:11:02 +02:00
Dietmar Maurer
c6d203bbd3 src/backup/backup_info.rs: also list .blob files 2019-06-25 07:07:45 +02:00
Dietmar Maurer
045e8a568d src/bin/proxmox-backup-client.rs: change key name to rsa-encrypted.key 2019-06-25 07:00:07 +02:00
Dietmar Maurer
79bc7345e8 src/cli/command.rs: fix ReST output format 2019-06-25 06:19:51 +02:00
Dietmar Maurer
9f46c7de4b src/bin/proxmox-backup-client.rs: upload rsa encoded key after backup 2019-06-24 13:56:37 +02:00
Christian Ebner
cd7dc87903 pxar: impl .pxarexclude parsing and exclude matching 
.pxarexclude files allow to exclude or include parts of a subtree by matching
with a glob pattern. The globs are used according to the matches of fnmatch.
In addition '**' can be used to match multiple directories within the path.

Order of the entries matter, as later ones win over previous ones.
As the .pxarexclude files can be placed at any node of the directory hirarchy,
this implies that matching child entries win over parent entries.
The only exception to this behaviour is, when a parent entry already fully
matched the path, thereby excluding the child entries which would match
otherwise.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
 2019-06-24 10:26:20 +02:00
Dietmar Maurer
46bd880041 src/api2/backup/environment.rs: new helper add_blob() 2019-06-24 09:35:37 +02:00
Dietmar Maurer
cb08ac3efe src/api2/backup.rs: replace upload_config with upload_blob 2019-06-23 11:44:30 +02:00
Dietmar Maurer
a38c5d4d12 src/backup/data_blob.rs: implement from_raw() 2019-06-23 09:35:44 +02:00
Dietmar Maurer
6f083b7a92 src/api2/backup/upload_chunk.rs: compute crc on upload 2019-06-23 09:04:42 +02:00
Dietmar Maurer
ba01828d38 try to use use proxmox::tools::io::ops::ReadExtOps 2019-06-22 16:29:10 +02:00
Dietmar Maurer
27042ce637 src/backup/file_formats.rs: improve docs 2019-06-22 15:45:36 +02:00
Dietmar Maurer
9f83e0f7ee src/backup/crypt_config.rs: simplify decode interface 2019-06-22 13:24:29 +02:00
Dietmar Maurer
0066c6d972 src/backup/crypt_config.rs: remove encode_chunk, use encrypt_to instead 2019-06-22 13:02:53 +02:00
Dietmar Maurer
ee8a7e8080 src/backup/crypt_config.rs: implement encrypt_to (output to writer) 2019-06-22 12:25:04 +02:00
Dietmar Maurer
991abfa8b4 src/backup/file_formats.rs: split out file format data 2019-06-22 09:12:25 +02:00
Dietmar Maurer
b7f4f27d6c add crc field for binary blobs formats 2019-06-21 18:49:04 +02:00
Dietmar Maurer
a7f67a9a9c depend on crc32fast 
We can use this to compute fast checksums to test file integrity (not crypto safe).
 2019-06-21 15:58:00 +02:00
Dietmar Maurer
3025b3a53c src/backup/data_blob.rs: new file format for binary blobs 2019-06-21 11:32:07 +02:00
Dietmar Maurer
077a8cae63 src/backup/crypt_config.rs: make code more flexible 
Allow to pass magic numbers as parameters.
 2019-06-21 10:50:48 +02:00
Dietmar Maurer
c38266c18d renamed: src/backup/crypt_setup.rs -> src/backup/crypt_config.rs 2019-06-21 09:51:18 +02:00
Dietmar Maurer
3ea8bfc9c0 src/bin/proxmox-backup-client.rs: add import-master-pubkey command 2019-06-21 07:34:17 +02:00
Dietmar Maurer
37c5a17505 src/bin/proxmox-backup-client.rs: helper to generate RSA master key 2019-06-21 07:12:19 +02:00
Dietmar Maurer
ab44acff57 src/backup/key_derivation.rs: add modified field to key file 2019-06-20 07:58:33 +02:00
Dietmar Maurer
6d0983dbe1 src/bin/proxmox-backup-client.rs: add keyfile parameter for backup 2019-06-19 17:16:41 +02:00
Dietmar Maurer
181f097af9 src/bin/proxmox-backup-client.rs - key API: pass kdf parameter 
Allow to store keys without password.
 2019-06-18 18:21:01 +02:00
Dietmar Maurer
9b06db4563 src/bin/proxmox-backup-client.rs: use file system path to reference keys 2019-06-18 17:38:50 +02:00
Dietmar Maurer
091540298e src/bin/proxmox-backup-client.rs - create_key: always read password from tty 2019-06-18 17:27:52 +02:00
Dietmar Maurer
826f309bf5 src/backup/key_derivation.rs: move kdf code into separate file 2019-06-18 11:17:22 +02:00
Dietmar Maurer
11515438cc Cargo.toml: use serde feature derive 2019-06-18 06:23:25 +02:00
Dietmar Maurer
ac71623483 src/bin/proxmox-backup-client.rs: implement "key change-passphrase" 2019-06-17 13:10:00 +02:00
Dietmar Maurer
f2401311b0 src/bin/proxmox-backup-client.rs: start key management 2019-06-17 10:33:24 +02:00
Dietmar Maurer
a7dd483097 src/backup/*_index.rs: used generated magic numbers 2019-06-14 14:58:37 +02:00
Dietmar Maurer
bffd40d6b7 src/tools.rs: move hex_to_digest and digest_to_hex to proxmox::tools 2019-06-14 11:40:04 +02:00
Dietmar Maurer
00388226e6 src/api2/backup/environment.rs: log index checksums 2019-06-14 10:36:20 +02:00
Dietmar Maurer
9335d74eba src/backup/fixed_index.rs: compute checksum over the index 2019-06-14 10:34:29 +02:00
Dietmar Maurer
16ff6b7cd8 src/backup/dynamic_index.rs: compute checksum over the index 2019-06-14 10:33:59 +02:00
Dietmar Maurer
36075475a6 src/api2/backup/environment.rs: log duplicate chunks 2019-06-14 07:12:30 +02:00
Dietmar Maurer
fa148dbd11 src/api2/backup/upload_chunk.rs: verify chunk data if possible 2019-06-14 06:23:46 +02:00
Dietmar Maurer
0f37577092 src/bin/cipherbench.rs: add test for zstd compression 2019-06-13 17:24:57 +02:00
Dietmar Maurer
9399c98f82 src/bin/cipherbench.rs: cipher speed test with large blocks 2019-06-13 17:16:43 +02:00
Dietmar Maurer
f640f23ac5 src/backup/data_chunk.rs: fix typo 2019-06-13 16:42:55 +02:00
Dietmar Maurer
dba72d5016 src/backup/data_chunk.rs: only use compressed data if it is shorter than uncompressed 2019-06-13 12:56:50 +02:00
Dietmar Maurer
68ab376ae9 src/backup/crypt_setup.rs: only use compressed data if it is shorter than uncompressed 2019-06-13 12:47:54 +02:00
Dietmar Maurer
f98ac774ee backup: Add support for client side encryption 
first try ...
 2019-06-13 11:47:23 +02:00
Dietmar Maurer
51929e4532 src/backup/data_chunk.rs - decode: make crypt_config optional 2019-06-13 11:44:38 +02:00
Dietmar Maurer
046bcb9773 src/backup/crypt_setup.rs: add comment for possible improvements 2019-06-13 07:35:59 +02:00
Dietmar Maurer
bd0e3c7cfa src/backup/data_chunk.rs: Moved ChunkInfo from merge_known_chunks.rs 2019-06-13 07:35:02 +02:00
Dietmar Maurer
6762db70d6 src/api2/types.rs: define CHUNK_DIGEST_SCHEMA 2019-06-13 07:30:42 +02:00
Dietmar Maurer
b595cb9d2c src/backup/data_chunk.rs: Data Chunk builder 
Make handling encoded chunks easier.
 2019-06-12 13:37:01 +02:00
Dietmar Maurer
ef27200cec src/backup/crypt_setup.rs: fix encoder for uncompressed chunks 2019-06-10 09:56:06 +02:00
Dietmar Maurer
74792b95b2 src/backup/crypt_setup.rs: allow compressed and uncompressed chunks 2019-06-10 08:27:35 +02:00
Dietmar Maurer
9e0187a203 src/backup/crypt_setup.rs: remove unused CryptData struct 2019-06-10 07:55:36 +02:00
Dietmar Maurer
bec8498ae3 src/backup/crypt_setup.rs: avoid useless allocations 2019-06-09 11:44:17 +02:00
Dietmar Maurer
c20b17b460 src/backup.rs: define magic numbers for chunk store file format 2019-06-08 10:34:20 +02:00
Dietmar Maurer
48b4b40b57 src/backup/crypt_setup.rs: crypto helpers 2019-06-08 09:56:51 +02:00