Commit Graph

1852 Commits

Author SHA1 Message Date
Christian Ebner 6de36b94b1 pxar::fuse: impl readdirplus callback
readdirplus returns the entries together with their `EntryParam`, so subsequent
lookups for each of the entries are avoided.
In order to reduce code duplication, the code for filling the reply buffer is
moved into a macro.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-13 11:17:34 +01:00
Christian Ebner caa201d895 pxar::fuse: Get rid of broken cacheing
Remove the current caching of attrs and goodbye tables as it is broken anyway.
This will be replaced with a LRU cache.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-13 11:16:06 +01:00
Oguz Bektas 6d20a29d73 fix typo for function name load_and_decrypt_key
Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-01-13 11:14:47 +01:00
Dietmar Maurer 50af953e1b src/section_config.rs: make set_data generic 2020-01-11 11:09:27 +01:00
Dietmar Maurer 688fbe07a1 cleanup config api, add remotes config cli interface 2020-01-11 10:42:09 +01:00
Dietmar Maurer 9e9bc6525e src/config/datastore.rs: define DataStoreConfig using api macro 2020-01-11 09:18:42 +01:00
Dietmar Maurer ca44172404 src/api2/config/remotes.rs: cleanup - avoid compiler warnings 2020-01-10 19:25:26 +01:00
Christian Ebner c7257c5539 pxar::fuse: refactor HashMap to store child parent inode relation.
By moving the HashMap into `Context`, the use of lazy_static as well as the
additional Mutex can be avoided (`Context` is already guarded by a Mutex).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-10 13:45:58 +01:00
Christian Ebner 90fc97af6a pxar::decoder::Decoder: include xattrs and payload size in `DirectoryEntry`.
By reading and including xattrs and payload size in struct `DirectoryEntry`,
the tuple of return types is avoided and the code is simpler.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-10 13:45:40 +01:00
Christian Ebner 6fc053ed85 pxar: encoder: limit number of max entries held at once in memory during archive creation.
Limit the total number of entries and therefore the approximate memory
consumption instead of doing this on a per directory basis as it was previously.
This makes more sense as it limits not only the width but also the depth of the
directory tree.

Further, instead of hardcoding this value, allow to pass this information as
additional optional parameter 'entires-max'.
By this, creation of the archive with directories containing a large number of
entries is possible.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-10 13:45:08 +01:00
Dietmar Maurer 141304d64e src/api2/config/remotes.rs: new API to configure remotes 2020-01-10 13:28:15 +01:00
Dietmar Maurer 331b869d96 src/bin/proxmox-backup-manager.rs: implement shell completion for remote datastore name 2020-01-10 11:58:31 +01:00
Dietmar Maurer eb506c830c rename src/api2/sync.rs -> src/api2/pull.rs
Use "pull" instead of "sync", because this also indicates a
direction (like "sync from").
2020-01-10 11:15:16 +01:00
Dietmar Maurer a81af92f9d src/section_config.rs: implement generic lookup 2020-01-09 17:35:44 +01:00
Dietmar Maurer 0eb0e02451 src/bin/proxmox-backup-manager.rs: implement cli sync command 2020-01-09 14:52:29 +01:00
Dietmar Maurer a575320657 src/config/remotes.rs: implement SectionConfig for remote hosts 2020-01-09 14:51:02 +01:00
Christian Ebner ecbc62261c pxar: encoder: warn on lacking read permissions instead of fail.
If during creation of the archive, files/dirs with lacking read permissions are
encountered, the user is displayed a warning and the archive is created without
including the file/dir.
Previously this resulted in an error and the archive creation failed.

In order to implement this also for the .pxarexclude files, the Error type of
MatchPattern::from_file() and MatchPattern::from_line() was adopted accordingly.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-09 10:41:19 +01:00
Christian Ebner 88cee60bad pxar: encoder: fix incorrect goodbye table entries and update displayed path.
If nodes are excluded by feature flags, they must not appear in the goodbye table.
This is fixed by continuing with the next entry in the for loop.

Further the relative path buffer is now poped in order to correctly display the path.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-09 10:39:41 +01:00
Christian Ebner ec906dcf19 src/pxar/encoder.rs: avoid use of macro and initialize Vec with known capacity.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-09 10:38:58 +01:00
Dietmar Maurer 71da3d6a1b src/api2/types.rs: fix SnapshotListItem schema definition 2020-01-09 10:36:19 +01:00
Dietmar Maurer de8ec04123 src/api2/sync.rs: implement remote sync 2020-01-08 14:03:52 +01:00
Dietmar Maurer 27c9affb23 src/bin/proxmox-backup-client.rs: cleanup - factor out api_datastore_latest_snapshot() 2020-01-07 15:18:36 +01:00
Dietmar Maurer 42af4b8f06 src/bin/proxmox-backup-client.rs: cleanup - factor out api_datastore_list_snapshots() 2020-01-07 15:07:57 +01:00
Dietmar Maurer af9d4afc66 src/bin/proxmox-backup-client.rs: use SnapshotListItem 2020-01-07 13:25:25 +01:00
Dietmar Maurer fc189b198c src/api2/types.rs: define and use new api type SnapshotListItem 2020-01-07 12:52:52 +01:00
Dietmar Maurer 8f14e8fe4c src/backup/backup_info.rs: implement last_successful_backup()
A backup is considered successful if there is a manifest file.
2020-01-06 11:35:22 +01:00
Dietmar Maurer 247a8ca5bb src/backup/manifest.rs: impl TryFrom DataBlob 2020-01-05 16:28:18 +01:00
Dietmar Maurer 7759eef552 src/backup/datastore.rs: implement cleanup_backup_dir() 2020-01-05 15:15:12 +01:00
Dietmar Maurer 5e1d1bead1 src/client/backup_reader.rs: make compute_file_csum() public 2020-01-05 14:50:54 +01:00
Dietmar Maurer 3743dee6de src/client/http_client.rs: allow access to AuthInfo fields 2020-01-05 10:31:19 +01:00
Dietmar Maurer 3758b398cd src/backup/read_chunk.rs: add read_raw_chunk()
Allow to read chunk data without decoding.
2020-01-02 13:29:10 +01:00
Dietmar Maurer 2585a8a4e2 src/backup/chunk_store.rs: implement cond_touch_chunk()
This will be used by backup sync to test if a chunk already exists.
2020-01-02 13:26:28 +01:00
Dietmar Maurer 1cf5178ac5 src/backup/datastore.rs: new helper try_shared_chunk_store_lock() 2020-01-02 11:00:33 +01:00
Dietmar Maurer 1e8da0a789 src/backup/manifest.rs: new helper archive_type() 2019-12-31 15:23:41 +01:00
Dietmar Maurer 94f085fab5 src/bin/proxmox-backup-proxy.rs: remove unused use clause 2019-12-30 11:50:24 +01:00
Dietmar Maurer 22be470d90 src/bin/proxmox-backup-api.rs: create cert here 2019-12-27 13:44:21 +01:00
Dietmar Maurer ca6119551d src/client/http_client.rs - start_h2_connection: correctly return error during upgrade 2019-12-27 13:41:31 +01:00
Wolfgang Bumiller a546a8a072 proxy/api: await running workers before shutdown
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-27 12:50:33 +01:00
Dietmar Maurer f8fd5095d8 src/bin/proxmox-backup-proxy.rs: create self signed cert at startup. 2019-12-27 11:21:32 +01:00
Dietmar Maurer afddffc7ec proxmox-backup-manager.rs: remove wrong fixme 2019-12-27 10:17:06 +01:00
Dietmar Maurer e739a8d8b4 src/bin/proxmox-backup-manager.rs: impl. cert info 2019-12-23 13:06:43 +01:00
Dietmar Maurer 550e0d8870 src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate
Without calling external openssl binary.
2019-12-22 17:35:06 +01:00
Wolfgang Bumiller 5e62d19cfb fix create_datastore parameters:
comment was declared as optional but unwrap()ed, causing
panics when not using it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-20 12:30:25 +01:00
Dietmar Maurer 645995634a src/api2/config/datastore.rs - create: pass uid and gid instead of User 2019-12-20 09:23:58 +01:00
Dietmar Maurer e67770d496 src/backup/chunk_store.rs - create: pass User instead of CreateOptions 2019-12-20 09:11:40 +01:00
Thomas Lamprecht bca9093520 api/ui: datastore: allow to set simple comment
for now forbid all control characters[0] in the comment value, the
section config writer cannot cope with newlines in the value, it
writes them out literally, allowing "injection" or breaking the whole
config.

In the webinterface use also a textfield, not a textarea.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-19 17:58:01 +01:00
Wolfgang Bumiller afdcfb5bc9 let ChunkStore::create take CreateOptions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-19 13:14:49 +01:00
Dietmar Maurer 47d4712152 src/bin/proxmox-backup-manager.rs: add task management cli 2019-12-19 11:27:36 +01:00
Dietmar Maurer d6c4a1198a src/bin/proxmox-backup-client.rs: use async fn 2019-12-19 11:20:59 +01:00
Dietmar Maurer f74a03da1f remove tools::getpwnam_ugid, impl. crate::backup::backup_user()
And use new nix::unistd::User struct.
2019-12-19 10:20:13 +01:00
Dietmar Maurer 8bf4559b4e update to nix 0.16 2019-12-19 09:29:44 +01:00
Dietmar Maurer 769f8c9998 src/bin/proxmox-backup-manager.rs: connect to daemon to execute GC
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2019-12-19 08:45:27 +01:00
Dietmar Maurer 25de1c8005 src/client/backup_repo.rs: impl. new() 2019-12-19 08:44:38 +01:00
Dietmar Maurer 94913f35ca src/bin/proxmox-backup-client.rs - start_garbage_collection: add missing output-format parameter 2019-12-19 07:57:53 +01:00
Dietmar Maurer 868c585219 src/backup/datastore.rs: avoid divide by zero 2019-12-19 07:14:23 +01:00
Dietmar Maurer 217170e11e src/server/worker_task.rs: set worker thread name
This gives better logs on thread panic...
2019-12-19 07:07:39 +01:00
Dietmar Maurer 7e210bd0b4 src/backup/chunk_store.rs: create lock file with correct owner 2019-12-19 06:55:53 +01:00
Thomas Lamprecht 2337df7b3d api2: add very basic 'status/usage' endpoint for nodes
For returning the nodes basic "usage status", for now one gets memory
and CPU utilization.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-18 17:24:32 +01:00
Dietmar Maurer bce023c57c src/tools.rs: fix typo 2019-12-18 14:29:29 +01:00
Dietmar Maurer 0b97bc6158 src/backup/chunk_store.rs: use proxmox::tools::fs::create_path 2019-12-18 12:26:43 +01:00
Dietmar Maurer f8ec1473be src/config/datastore.rs: use backup gid instead of uid 2019-12-18 12:21:44 +01:00
Wolfgang Bumiller feaa1ad35f replace file_set_contents with replace_file
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-18 11:16:04 +01:00
Dietmar Maurer a66e7920dc src/api2/config/datastore.rs: set protected flags for create/delete API 2019-12-18 10:52:45 +01:00
Dietmar Maurer f0db500808 src/api2/config/datastore.rs: fix api docu 2019-12-18 10:48:19 +01:00
Dietmar Maurer 424766bc3b src/config/datastore.rs: change file owner/permissions
owner(root) => read and write
group(backup) => read only
2019-12-18 10:41:58 +01:00
Oguz Bektas c07b458cac datastore: set correct owner on datastore.cfg
since manager runs as root, we have to specify backup:root owner while
saving config.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-12-17 16:59:10 +01:00
Oguz Bektas 14f1e63067 chunk_store: create parent directories
'datastore create storename /path/to/dir/that/may/not/exist' should
work.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-12-17 15:39:42 +01:00
Christian Ebner ebfb2df22b binary_search_tree: add additional doctest for search_binary_tree_by()
Make sure a start indexes larger or equal to the array size results in a return
value of `None`.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 15:37:02 +01:00
Christian Ebner 48f6d67764 binary_search_tree: fix off by one error in search_binary_tree_by()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 14:57:34 +01:00
Christian Ebner 38d9a69875 catalog: decoder and shell: fix incorrect logic in find matching
The find matching was incorrectly performed starting from the parent directroy
and not as intended from the entries of the parent directory.

Further, the match pattern passed from the catalog shell contains the absolute
path of the search entry point as prefix, so find() must always start from the
archive root. This is because the match pattern has to be stored in the selected
list for a subsequent restore-selected command in the shell.
All matching paths are shown as absolute paths with all contents in the subdir,
equal to what would be restored by the given pattern.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 14:57:34 +01:00
Dietmar Maurer 81cc71c0d7 use proxmox-mini-journalreader to display syslog 2019-12-17 14:06:48 +01:00
Dietmar Maurer 20197f7cf5 src/api2/node/syslog.rs: use api macro 2019-12-17 13:26:49 +01:00
Thomas Lamprecht 9c01e73c2f ui: add "Documentation" button to main view
Similar to PVE and PMG, for quick access when one has the basic
webinterface open anyway. Should move to the "proxmoxHelpButton" once
we have an onlineHelp mapping to the docs.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 11:22:34 +01:00
Thomas Lamprecht 2d694f8f1f add CSS file for PBS ExtJS6 basic ui
some fitting rules copied over from PVE's ext6-pve.css file.
simply place it in the css subfolder where the proxmox-backup-gui.js
file is hosted and add a "css/" alias for that directory, the
formatter gets use the right content type with that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 11:20:32 +01:00
Christian Ebner 25cdd0e0a1 catalog_shell: impl find for catalog shell.
Implements the find command which allows to find and select files for subsequent
restore.
Files selected for restore are now stored in a Vec instead of a HashSet.
This is needed, since instead of the full paths for each file, selected files are
now identified by a list of match pattern, where ordering matters.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner 90dfd0a71d catalog: impl find() for file search by match pattern.
find() iterates over the file tree and matches each node against a list of match
patterns provided at function call.
For each matching node, a callback function with the current directroy stack is
called.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner a333b4ccea pxar: match_pattern: impl invert() to switch the match type of a MatchPattern.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner 48af80b3ab pxar: match_pattern: impl traits needed for MatchPattern to be comparable.
In order to check if two MatchPattern are identical, which will be used to avoid
duplicates.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner 255bb86030 pxar: match_pattern: refactor MatchPattern and introduce MatchPatternSlice.
The MatchPattern impl heavily used copies and therefore was inefficient regarding
memory management.
This patch intoduces MatchPatternSlice as struct to avoid copies and perform the
same pattern matching functionality.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Wolfgang Bumiller dc08934563 drop Cancellable future in favor of abortable
futures-0.3 has a futures::future::abortable() function
which does the exact same, returns an Abortable future with
an AbortHandle providing an abort() method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:52:07 +01:00
Wolfgang Bumiller 8554ac5ec3 fix broadcast_future test case
We used to await all the futures via the runtime's shutdown
method, which doesn't exist anymore, so await all the join
handles instead.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:47:12 +01:00
Thomas Lamprecht ba3eb88d95 generate authkey: public part needs to be readable by backup group
else the API proxy cannot use it and fails to run..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 10:17:03 +01:00
Thomas Lamprecht 9c18e93522 handle_static_file_download: move from and_then to await
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 08:56:55 +01:00
Thomas Lamprecht 4e8cff1c9d cipherbench: align/format output nicely
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 08:49:34 +01:00
Thomas Lamprecht 4832f53ab5 cipherbench: be conform with rfc7539, use 12 byte nonce
else we get an error from this call, using a 16 byte (128 bit) nonce
is currently only supported by the still in draft
XChaCha20-Poly1305, not the current default specified by RFC 7539[0],
which uses a 12 byte (96 bit) nonce.

Fixes the following error:
> thread 'main' panicked at 'called `Result::unwrap()` on an `Err`
> value: ErrorStack([])', src/libcore/result.rs:1165:5

[0]: https://tools.ietf.org/html/rfc7539

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 08:49:07 +01:00
Dietmar Maurer a47a02ae62 src/bin/proxmox-backup-client.rs: use #[api] macro 2019-12-16 13:49:18 +01:00
Wolfgang Bumiller 78d5436040 client: use api macro for catalog_shell async fn
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-16 11:24:44 +01:00
Dietmar Maurer 87c423751a src/bin/proxmox-backup-client.rs: use ApiHandler::Async for prune and status 2019-12-16 11:00:43 +01:00
Dietmar Maurer 8a8a470316 src/bin/proxmox-backup-client.rs: use new ApiHandler::Async 2019-12-16 10:06:26 +01:00
Dietmar Maurer b9799012cf src/pxar/decoder.rs: require Send 2019-12-16 10:01:51 +01:00
Dietmar Maurer bb084b9c91 api2: update for latest proxmox-api changes
- rename ApiFuture into ApiResponseFuture
- impl. ApiHandler::Async
2019-12-16 10:01:51 +01:00
Dietmar Maurer 236a396aa1 src/api2/admin/datastore.rs - prune: log retention options 2019-12-14 16:32:16 +01:00
Dietmar Maurer 503995c767 src/api2/admin/datastore.rs: fix prune worker id 2019-12-14 16:05:21 +01:00
Wolfgang Bumiller f374ba4c0b update tests
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 12:42:02 +01:00
Wolfgang Bumiller b5c9a706ce fix cancellable future
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 12:10:04 +01:00
Wolfgang Bumiller 8a1028e0e7 update to url/percent-encoding to 2.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:57:01 +01:00
Wolfgang Bumiller db0cb9ce0b update a chunk of stuff to the hyper release
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
Dietmar Maurer 3f1020b732 src/bin/proxmox-backup-client.rs: implement task stop 2019-12-13 09:43:08 +01:00
Dietmar Maurer 1ae5677de0 src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT 2019-12-12 14:15:40 +01:00
Dietmar Maurer d0adf270fb src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA 2019-12-12 12:37:11 +01:00
Dietmar Maurer 66c49c21c3 src/api2/types.rs: factor out DATASTORE_SCHEMA 2019-12-11 13:45:23 +01:00
Dietmar Maurer 567d3e00fb src/api2/node/tasks.rs: new filter "store" to filter tasks for one store 2019-12-11 12:53:34 +01:00
Christian Ebner d3dbe52f37 pxar: match_pattern: refactor match_filename
The match_filename() in sequentail_decoder and encoder are moved to be static
functions of MatchPattern.
This allows to reuse the code also in the catalog find implementation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-11 10:55:01 +01:00
Christian Ebner f084505ec5 src/backup/catalog_shell.rs: sort output of list-selected
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-11 10:54:20 +01:00
Dietmar Maurer 6d04612955 src/api2/node/tasks.rs: add "running" filter to list only running tasks 2019-12-11 10:52:27 +01:00
Dietmar Maurer 8560fe3e5f src/api2/node/tasks.rs - extract_upid: avoid duplicate error message 2019-12-10 13:45:56 +01:00
Dietmar Maurer 5830c20560 src/bin/proxmox-backup-client.rs: add simple task management cli 2019-12-10 13:43:53 +01:00
Dietmar Maurer 73e57f244e src/backup/datastore.rs - garbage-collection: fix average chunk size 2019-12-10 11:25:29 +01:00
Dietmar Maurer d105176f45 src/bin/proxmox-backup-client.rs: cleanup - factor out view_task_result() 2019-12-09 18:00:22 +01:00
Dietmar Maurer 48ef3c3327 cli: avoid useless .into()
This needs latest proxmox-api updates.
2019-12-09 17:40:34 +01:00
Dietmar Maurer eb7e2ee00b src/bin/proxmox-backup-client.rs: remove unnecessary .to_owned() 2019-12-09 17:11:48 +01:00
Dietmar Maurer e5f7def40e src/bin/proxmox-backup-client.rs - garbage-collection: display task log 2019-12-09 13:29:23 +01:00
Dietmar Maurer 0ec9e1b010 src/bin/proxmox-backup-client.rs: impl. and use complete_pxar_archive_name 2019-12-09 13:14:32 +01:00
Dietmar Maurer 1c6ad6ef90 src/bin/proxmox-backup-client.rs: impl. catalog sub command
catalog dump
catalog shell
2019-12-09 13:07:25 +01:00
Dietmar Maurer 5a0b484bdd src/client/task_log.rs: move display_task_log into extra file 2019-12-08 11:27:15 +01:00
Dietmar Maurer 3b03abfe14 src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune
And use display_task_log on the client side.
2019-12-08 10:59:47 +01:00
Dietmar Maurer ec137a99c6 src/bin/proxmox-backup-client.r - display_task_log: use string range instead of as_bytes 2019-12-08 10:25:26 +01:00
Dietmar Maurer b73ff941f6 src/bin/proxmox-backup-client.r - display_task_log: add new strip_date parameter 2019-12-08 10:22:02 +01:00
Dietmar Maurer 163e9bbe91 src/bin/proxmox-backup-client.rs: new display task log helper
New helper to display worker task logs. Use it for prune (experimental).
2019-12-07 16:11:26 +01:00
Dietmar Maurer 6b508dd563 src/server/formatter.rs: impl. new result attribute "active" 2019-12-07 15:29:42 +01:00
Dietmar Maurer 102d8d4136 src/backup/prune.rs: implement --keep-hourly 2019-12-07 11:23:33 +01:00
Dietmar Maurer 052c4aa160 src/backup/prune.rs: prune - correctly use iso_week year 2019-12-06 19:41:38 +01:00
Christian Ebner fa0b950070 pxar: fuse: add support for list- and getxattr
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 16:44:15 +01:00
Christian Ebner 25ad4cbf63 pxar: fuse: cache goodbye table for each directory on opendir and release it on releasedir
Cache not only the goodbye table for the last directory but for each opened
directory.
The opendir fuse callback will fill the cache with the goodbye table and
releasedir will remove it from the cache.
This should reduce the number of chuncks fetched from the server in some cases.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 16:43:59 +01:00
Christian Ebner 02491b8fc6 pxar: fuse: use binary search instead of linear search in goodbye table.
The goodbye table is layed out as binary search tree based on the hash, so use
this to be more efficient when looking up a hash in the table for directories
with a large number of entries.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 16:42:01 +01:00
Dietmar Maurer 7074a0b339 src/bin/proxmox-backup-client.rs: white space cleanup 2019-12-06 13:35:59 +01:00
Dietmar Maurer 74fa81b89c src/bin/proxmox-backup-client.rs: implement prune --dry-run 2019-12-06 13:35:25 +01:00
Dietmar Maurer 1f604f1a5a src/api2/admin/datastore.rs: add new api test_prune() 2019-12-06 13:06:56 +01:00
Dietmar Maurer 0ab08ac980 src/api2/admin/datastore.rs: cleanup - move schema defininition in front of rust method 2019-12-06 12:36:41 +01:00
Dietmar Maurer 1a0678b601 src/api2/admin/datastore.rs: remove prune parameters from status call
Not idea why they are added there.
2019-12-06 12:32:53 +01:00
Dietmar Maurer 9e3f008804 src/backup/prune.rs: add new helper keeps_something() 2019-12-06 12:28:31 +01:00
Dietmar Maurer d75fbb0a21 src/bin/dump-catalog-shell-cli.rs: generate catalog shell docs 2019-12-06 09:54:13 +01:00
Dietmar Maurer 55c3cb69cd rename catalog_shell_api() into catalog_shell_cli() 2019-12-06 09:42:05 +01:00
Dietmar Maurer ecbaa38fa4 src/backup/catalog_shell.rs: factor out catalog_shell_api()
In order to extract documentation ...
2019-12-06 09:35:47 +01:00
Christian Ebner cca67dc0c3 src/backup/readline.rs: removed in favor of new readline impl
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:03:04 +01:00
Christian Ebner 6934c6fe77 src/backup/catalog_shell.rs: adapt to use API Schema definition and rustyline
This major refactoring of the catalog based shell utilizes the new API macro and
the API Schema as well as rustyline instead of the old GNU readline C API.

The code now has these 3 main components:
 * The `Shell` which handles the readline loop via rustyline.
 * The shell functions defined via the API macro.
 * The `Context` which holds catalog and decoder instances.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 09:02:44 +01:00
Dietmar Maurer 9b78352188 src/backup/prune.rs: define new struct PruneOptions 2019-12-06 08:56:27 +01:00
Dietmar Maurer 92acbd69f7 src/backup/backup_info.rs: remove old prune code 2019-12-06 08:29:27 +01:00
Dietmar Maurer 408434360b src/backup/prune.rs: add PruneMark::KeepPartial
Use separate mark for partial backup - dont mess them up with regular ones.
2019-12-06 08:15:32 +01:00
Dietmar Maurer a8c8366cfa src/backup/prune.rs: factor out code into remove_incomplete_snapshots() 2019-12-06 08:12:08 +01:00
Dietmar Maurer dc18849156 src/backup/prune.rs: moved prune related code into extra file 2019-12-06 08:06:21 +01:00
Dietmar Maurer 6f47dd8a0f src/backup/backup_info.rs - prune: improve algorythm 2019-12-05 18:41:28 +01:00
Dietmar Maurer 8f0b4c1f90 src/api2/admin/datastore.rs - prune: log info about all snapshots 2019-12-05 15:35:11 +01:00
Dietmar Maurer 9ce42759ec src/backup/backup_info.rs - compute_prune_list: remove unfinished backups 2019-12-05 11:18:10 +01:00
Dietmar Maurer 2c034f8d0a src/backup/backup_info.rs: improve prune algorithm 2019-12-05 08:55:19 +01:00
Dietmar Maurer aeeac29bb1 src/backup/backup_info.rs: new compute_prune_list helper
So that we can write regression tests for this.
2019-12-04 15:49:11 +01:00
Dietmar Maurer 3d229a4a01 Revert "test another api macro use"
This reverts commit a9aa52e6a8.

Because we do not want to use macros for the backup protocol for now.

And because it crashes backup tasks for some unknown reason.
2019-12-04 14:18:11 +01:00
Dietmar Maurer 4f1e40a29e src/api2/admin/datastore.rs - read_backup_index: add manifest with size
So that the client can display the size of the index file.
2019-12-04 10:03:52 +01:00
Wolfgang Bumiller a9aa52e6a8 test another api macro use
The api macro now supports hyphens in parameter names and
referencing externally defined `Schema`s, so here's an
example.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-03 13:41:40 +01:00
Christian Ebner de0aa53174 pxar: decoder: fix incorrect read for hardlinks.
Decoders read must check if the file is a hardlink and read data from the
corresponding offset if so.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-03 12:08:23 +01:00
Christian Ebner b7e18004a2 pxar: decoder: return original name in attributes()
The returned filename should be the one of the file given at the offset, not of
the one the hardlink points to.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-03 12:08:07 +01:00
Dietmar Maurer 7eea56ca00 use cli from proxmox::api 2019-12-02 11:56:29 +01:00
Dietmar Maurer 524b22df35 src/cli/completion.rs - get_help_completion: fix for "help -" expansion 2019-12-02 11:11:31 +01:00
Dietmar Maurer e8e9bae4d1 src/bin/completion.rs: add echo command 2019-12-02 11:04:40 +01:00
Dietmar Maurer 19d7a59236 src/bin/completion.rs: use api macros 2019-12-02 10:57:19 +01:00
Dietmar Maurer f9a51322cb src/cli/command.rs: use insert_help() 2019-12-02 08:52:08 +01:00
Dietmar Maurer e3e72546dd src/cli/command.rs: cleanup, remove unused top_def parameter 2019-12-02 08:47:03 +01:00
Dietmar Maurer 2348e77edf src/cli/completion.rs - get_simple_completion: remove all_arg_param parameters
Seems useless.
2019-12-02 08:31:03 +01:00
Dietmar Maurer 28c855c0a2 src/cli: improve docs 2019-12-01 16:41:49 +01:00
Dietmar Maurer 3bf920527c src/cli/command.rs: make set_help_context private - no need to expose 2019-12-01 12:54:41 +01:00
Dietmar Maurer bca6d9120a src/cli/command.rs: improve help command description 2019-12-01 12:40:37 +01:00
Dietmar Maurer 2f6ecc1ccd src/cli/completion.rs: moved all completion related code here 2019-12-01 11:44:31 +01:00
Dietmar Maurer a810e05288 src/cli/readline.rs: moved readline related code here 2019-12-01 11:31:35 +01:00
Dietmar Maurer 5d64a0d083 src/cli/shellword.rs: moved shellword related code into this file 2019-12-01 11:17:57 +01:00
Dietmar Maurer fb8a00bc00 src/cli/command.rs - get_nested_completion: allow fuzzy match 2019-12-01 11:06:54 +01:00
Dietmar Maurer 9498fcf9c5 src/cli/command.rs - get_nested_completion: return nothing if first argument does not match 2019-12-01 10:24:07 +01:00
Dietmar Maurer 309e449db2 src/cli/command.rs: improve get_help_completion 2019-12-01 10:18:47 +01:00
Dietmar Maurer b8a82922c7 src/cli/command.rs: handle multilevel sub commands in help 2019-12-01 09:42:51 +01:00
Dietmar Maurer b7bf71dfa1 src/cli/command.rs: fix help completions 2019-11-30 15:43:44 +01:00
Dietmar Maurer 7afc1af7ab src/bin/completion.rs: add history support 2019-11-30 15:08:37 +01:00
Dietmar Maurer 4b8573da65 src/cli/command.rs - handle_command: return Result instead of exit() 2019-11-30 15:08:00 +01:00
Dietmar Maurer 2b691daf6f src/cli.rs: add insert_help() method 2019-11-30 14:56:31 +01:00
Dietmar Maurer 1201abcffa src/cli/command.rs: implement help 2019-11-30 13:02:00 +01:00
Wolfgang Bumiller 8423c1fe64 cli/completion: use match statements and .contains()
`match` is a bit more readable than the if-else chains,
also replace
    space_chars.iter().any(|s| c == *s)
with
    space_chars.contains(&c)
which is also more readable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 13:37:34 +01:00
Dietmar Maurer e399398444 remove shellwords crate 2019-11-29 12:49:44 +01:00
Dietmar Maurer ca60ac13d9 use rustyline to replace readline 2019-11-29 12:49:44 +01:00
Wolfgang Bumiller 41874331ed whitespace fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:52:15 +01:00
Wolfgang Bumiller 2905f2b5e6 update api macro example usage
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-29 09:51:27 +01:00
Dietmar Maurer 8b130e71da src/cli/command.rs: cleanup, make handlers more generic 2019-11-27 14:44:21 +01:00
Wolfgang Bumiller 7b6c41078b update api macro invocation to new style
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 14:19:46 +01:00
Christian Ebner 314bb35868 catalog_shell: fix several clippy warnings
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-27 10:13:50 +01:00
Wolfgang Bumiller 6486cb853f first api macro usage test/example
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-27 10:05:37 +01:00
Christian Ebner 951cf17ee3 catalog_shell: major refactoring of cli command definition and parsing
By changing the way shell commands are defined and parsed, this makes it more
straight forward to extend the current functionality.
The readline input is parsed based on the provided command definition and the
given parameters and options are passed to a command specific callback function.
In addition, the provided command definition including its description is used
to generate a help string to display.
The help command shows a list of all supported commands or the help string for
the provided command.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 13:01:00 +01:00
Christian Ebner 59bc6ad676 catalog_shell: check and return on empty list in order to avoid division by zero
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:33 +01:00
Christian Ebner 4e56b2f792 catalog_shell: do not restore full archive if no entries are selected for restore-selected
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-11-26 12:55:18 +01:00
Dietmar Maurer 12b4098660 src/cli/format.rs: move doc generation code into extra file 2019-11-26 12:12:53 +01:00
Dietmar Maurer 2f2b87e659 src/pxar/fuse.rs: add impl Context 2019-11-26 11:16:46 +01:00
Dietmar Maurer eedd1f95eb src/pxar/fuse.rs: move init and destroy into Session 2019-11-26 11:08:55 +01:00
Dietmar Maurer 970687c982 src/pxar/fuse.rs: cleanup - remove setup_session (moved into new) 2019-11-26 11:04:46 +01:00
Dietmar Maurer 2a11191072 src/pxar/fuse.rs: rename from_decoder into new, new into from_path 2019-11-26 10:56:41 +01:00
Dietmar Maurer 99b5b6cba9 src/pxar/decoder.rs: use trait object as reader
So that the Decoder is no longer genertic.
2019-11-26 10:45:11 +01:00
Dietmar Maurer f701d0335e src/pxar/sequential_decoder.rs: remove callback from new()
And use an extra functzion  set_callback() to configure that.

Also rewrite pxar/fuse.rs and implement a generic Session (will get
further cleanups with next patches).
2019-11-26 09:56:48 +01:00
Dietmar Maurer 49fddd985c src/cli/command.rs: use static array for arg_param 2019-11-25 11:16:41 +01:00
Dietmar Maurer 329d40b50b rename ApiHandler::Async into ApiHandler::AsyncHttp 2019-11-23 09:03:21 +01:00
Dietmar Maurer 70fbac84da src/server/rest.rs: simplify code 2019-11-22 18:44:14 +01:00
Dietmar Maurer 2bbd835b9b src/server/rest.rs: rename get_request_parameters_async to get_request_parameters 2019-11-22 17:24:16 +01:00
Dietmar Maurer 75a5a68944 src/server/rest.rs - only pass ObjectSchema to get_request_parameters_async() 2019-11-22 17:22:07 +01:00
Dietmar Maurer ad51d02aa9 src/server/rest.rs: cleanup async code 2019-11-22 13:02:05 +01:00
Wolfgang Bumiller be2bb37205 remove api_schema submodule
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-11-22 09:31:45 +01:00
Dietmar Maurer e57e1cd82d move src/api_schema/config.rs -> src/server/config.rs 2019-11-22 09:23:03 +01:00