Commit Graph

1724 Commits

Author SHA1 Message Date
Dietmar Maurer 5e1d1bead1 src/client/backup_reader.rs: make compute_file_csum() public 2020-01-05 14:50:54 +01:00
Dietmar Maurer 3743dee6de src/client/http_client.rs: allow access to AuthInfo fields 2020-01-05 10:31:19 +01:00
Dietmar Maurer 3758b398cd src/backup/read_chunk.rs: add read_raw_chunk()
Allow to read chunk data without decoding.
2020-01-02 13:29:10 +01:00
Dietmar Maurer 2585a8a4e2 src/backup/chunk_store.rs: implement cond_touch_chunk()
This will be used by backup sync to test if a chunk already exists.
2020-01-02 13:26:28 +01:00
Dietmar Maurer 1cf5178ac5 src/backup/datastore.rs: new helper try_shared_chunk_store_lock() 2020-01-02 11:00:33 +01:00
Dietmar Maurer 1e8da0a789 src/backup/manifest.rs: new helper archive_type() 2019-12-31 15:23:41 +01:00
Dietmar Maurer 94f085fab5 src/bin/proxmox-backup-proxy.rs: remove unused use clause 2019-12-30 11:50:24 +01:00
Dietmar Maurer 22be470d90 src/bin/proxmox-backup-api.rs: create cert here 2019-12-27 13:44:21 +01:00
Dietmar Maurer ca6119551d src/client/http_client.rs - start_h2_connection: correctly return error during upgrade 2019-12-27 13:41:31 +01:00
Wolfgang Bumiller a546a8a072 proxy/api: await running workers before shutdown
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-27 12:50:33 +01:00
Dietmar Maurer f8fd5095d8 src/bin/proxmox-backup-proxy.rs: create self signed cert at startup. 2019-12-27 11:21:32 +01:00
Dietmar Maurer afddffc7ec proxmox-backup-manager.rs: remove wrong fixme 2019-12-27 10:17:06 +01:00
Dietmar Maurer e739a8d8b4 src/bin/proxmox-backup-manager.rs: impl. cert info 2019-12-23 13:06:43 +01:00
Dietmar Maurer 550e0d8870 src/bin/proxmox-backup-manager.rs: add cli to generate proxy certificate
Without calling external openssl binary.
2019-12-22 17:35:06 +01:00
Wolfgang Bumiller 5e62d19cfb fix create_datastore parameters:
comment was declared as optional but unwrap()ed, causing
panics when not using it

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-20 12:30:25 +01:00
Dietmar Maurer 645995634a src/api2/config/datastore.rs - create: pass uid and gid instead of User 2019-12-20 09:23:58 +01:00
Dietmar Maurer e67770d496 src/backup/chunk_store.rs - create: pass User instead of CreateOptions 2019-12-20 09:11:40 +01:00
Thomas Lamprecht bca9093520 api/ui: datastore: allow to set simple comment
for now forbid all control characters[0] in the comment value, the
section config writer cannot cope with newlines in the value, it
writes them out literally, allowing "injection" or breaking the whole
config.

In the webinterface use also a textfield, not a textarea.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-19 17:58:01 +01:00
Wolfgang Bumiller afdcfb5bc9 let ChunkStore::create take CreateOptions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-19 13:14:49 +01:00
Dietmar Maurer 47d4712152 src/bin/proxmox-backup-manager.rs: add task management cli 2019-12-19 11:27:36 +01:00
Dietmar Maurer d6c4a1198a src/bin/proxmox-backup-client.rs: use async fn 2019-12-19 11:20:59 +01:00
Dietmar Maurer f74a03da1f remove tools::getpwnam_ugid, impl. crate::backup::backup_user()
And use new nix::unistd::User struct.
2019-12-19 10:20:13 +01:00
Dietmar Maurer 8bf4559b4e update to nix 0.16 2019-12-19 09:29:44 +01:00
Dietmar Maurer 769f8c9998 src/bin/proxmox-backup-manager.rs: connect to daemon to execute GC
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2019-12-19 08:45:27 +01:00
Dietmar Maurer 25de1c8005 src/client/backup_repo.rs: impl. new() 2019-12-19 08:44:38 +01:00
Dietmar Maurer 94913f35ca src/bin/proxmox-backup-client.rs - start_garbage_collection: add missing output-format parameter 2019-12-19 07:57:53 +01:00
Dietmar Maurer 868c585219 src/backup/datastore.rs: avoid divide by zero 2019-12-19 07:14:23 +01:00
Dietmar Maurer 217170e11e src/server/worker_task.rs: set worker thread name
This gives better logs on thread panic...
2019-12-19 07:07:39 +01:00
Dietmar Maurer 7e210bd0b4 src/backup/chunk_store.rs: create lock file with correct owner 2019-12-19 06:55:53 +01:00
Thomas Lamprecht 2337df7b3d api2: add very basic 'status/usage' endpoint for nodes
For returning the nodes basic "usage status", for now one gets memory
and CPU utilization.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-18 17:24:32 +01:00
Dietmar Maurer bce023c57c src/tools.rs: fix typo 2019-12-18 14:29:29 +01:00
Dietmar Maurer 0b97bc6158 src/backup/chunk_store.rs: use proxmox::tools::fs::create_path 2019-12-18 12:26:43 +01:00
Dietmar Maurer f8ec1473be src/config/datastore.rs: use backup gid instead of uid 2019-12-18 12:21:44 +01:00
Wolfgang Bumiller feaa1ad35f replace file_set_contents with replace_file
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-18 11:16:04 +01:00
Dietmar Maurer a66e7920dc src/api2/config/datastore.rs: set protected flags for create/delete API 2019-12-18 10:52:45 +01:00
Dietmar Maurer f0db500808 src/api2/config/datastore.rs: fix api docu 2019-12-18 10:48:19 +01:00
Dietmar Maurer 424766bc3b src/config/datastore.rs: change file owner/permissions
owner(root) => read and write
group(backup) => read only
2019-12-18 10:41:58 +01:00
Oguz Bektas c07b458cac datastore: set correct owner on datastore.cfg
since manager runs as root, we have to specify backup:root owner while
saving config.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-12-17 16:59:10 +01:00
Oguz Bektas 14f1e63067 chunk_store: create parent directories
'datastore create storename /path/to/dir/that/may/not/exist' should
work.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2019-12-17 15:39:42 +01:00
Christian Ebner ebfb2df22b binary_search_tree: add additional doctest for search_binary_tree_by()
Make sure a start indexes larger or equal to the array size results in a return
value of `None`.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 15:37:02 +01:00
Christian Ebner 48f6d67764 binary_search_tree: fix off by one error in search_binary_tree_by()
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 14:57:34 +01:00
Christian Ebner 38d9a69875 catalog: decoder and shell: fix incorrect logic in find matching
The find matching was incorrectly performed starting from the parent directroy
and not as intended from the entries of the parent directory.

Further, the match pattern passed from the catalog shell contains the absolute
path of the search entry point as prefix, so find() must always start from the
archive root. This is because the match pattern has to be stored in the selected
list for a subsequent restore-selected command in the shell.
All matching paths are shown as absolute paths with all contents in the subdir,
equal to what would be restored by the given pattern.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 14:57:34 +01:00
Dietmar Maurer 81cc71c0d7 use proxmox-mini-journalreader to display syslog 2019-12-17 14:06:48 +01:00
Dietmar Maurer 20197f7cf5 src/api2/node/syslog.rs: use api macro 2019-12-17 13:26:49 +01:00
Thomas Lamprecht 9c01e73c2f ui: add "Documentation" button to main view
Similar to PVE and PMG, for quick access when one has the basic
webinterface open anyway. Should move to the "proxmoxHelpButton" once
we have an onlineHelp mapping to the docs.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 11:22:34 +01:00
Thomas Lamprecht 2d694f8f1f add CSS file for PBS ExtJS6 basic ui
some fitting rules copied over from PVE's ext6-pve.css file.
simply place it in the css subfolder where the proxmox-backup-gui.js
file is hosted and add a "css/" alias for that directory, the
formatter gets use the right content type with that.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 11:20:32 +01:00
Christian Ebner 25cdd0e0a1 catalog_shell: impl find for catalog shell.
Implements the find command which allows to find and select files for subsequent
restore.
Files selected for restore are now stored in a Vec instead of a HashSet.
This is needed, since instead of the full paths for each file, selected files are
now identified by a list of match pattern, where ordering matters.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner 90dfd0a71d catalog: impl find() for file search by match pattern.
find() iterates over the file tree and matches each node against a list of match
patterns provided at function call.
For each matching node, a callback function with the current directroy stack is
called.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner a333b4ccea pxar: match_pattern: impl invert() to switch the match type of a MatchPattern.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner 48af80b3ab pxar: match_pattern: impl traits needed for MatchPattern to be comparable.
In order to check if two MatchPattern are identical, which will be used to avoid
duplicates.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Christian Ebner 255bb86030 pxar: match_pattern: refactor MatchPattern and introduce MatchPatternSlice.
The MatchPattern impl heavily used copies and therefore was inefficient regarding
memory management.
This patch intoduces MatchPatternSlice as struct to avoid copies and perform the
same pattern matching functionality.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-17 11:12:34 +01:00
Wolfgang Bumiller dc08934563 drop Cancellable future in favor of abortable
futures-0.3 has a futures::future::abortable() function
which does the exact same, returns an Abortable future with
an AbortHandle providing an abort() method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:52:07 +01:00
Wolfgang Bumiller 8554ac5ec3 fix broadcast_future test case
We used to await all the futures via the runtime's shutdown
method, which doesn't exist anymore, so await all the join
handles instead.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-17 10:47:12 +01:00
Thomas Lamprecht ba3eb88d95 generate authkey: public part needs to be readable by backup group
else the API proxy cannot use it and fails to run..

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 10:17:03 +01:00
Thomas Lamprecht 9c18e93522 handle_static_file_download: move from and_then to await
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 08:56:55 +01:00
Thomas Lamprecht 4e8cff1c9d cipherbench: align/format output nicely
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 08:49:34 +01:00
Thomas Lamprecht 4832f53ab5 cipherbench: be conform with rfc7539, use 12 byte nonce
else we get an error from this call, using a 16 byte (128 bit) nonce
is currently only supported by the still in draft
XChaCha20-Poly1305, not the current default specified by RFC 7539[0],
which uses a 12 byte (96 bit) nonce.

Fixes the following error:
> thread 'main' panicked at 'called `Result::unwrap()` on an `Err`
> value: ErrorStack([])', src/libcore/result.rs:1165:5

[0]: https://tools.ietf.org/html/rfc7539

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2019-12-17 08:49:07 +01:00
Dietmar Maurer a47a02ae62 src/bin/proxmox-backup-client.rs: use #[api] macro 2019-12-16 13:49:18 +01:00
Wolfgang Bumiller 78d5436040 client: use api macro for catalog_shell async fn
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-16 11:24:44 +01:00
Dietmar Maurer 87c423751a src/bin/proxmox-backup-client.rs: use ApiHandler::Async for prune and status 2019-12-16 11:00:43 +01:00
Dietmar Maurer 8a8a470316 src/bin/proxmox-backup-client.rs: use new ApiHandler::Async 2019-12-16 10:06:26 +01:00
Dietmar Maurer b9799012cf src/pxar/decoder.rs: require Send 2019-12-16 10:01:51 +01:00
Dietmar Maurer bb084b9c91 api2: update for latest proxmox-api changes
- rename ApiFuture into ApiResponseFuture
- impl. ApiHandler::Async
2019-12-16 10:01:51 +01:00
Dietmar Maurer 236a396aa1 src/api2/admin/datastore.rs - prune: log retention options 2019-12-14 16:32:16 +01:00
Dietmar Maurer 503995c767 src/api2/admin/datastore.rs: fix prune worker id 2019-12-14 16:05:21 +01:00
Wolfgang Bumiller f374ba4c0b update tests
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 12:42:02 +01:00
Wolfgang Bumiller b5c9a706ce fix cancellable future
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 12:10:04 +01:00
Wolfgang Bumiller 8a1028e0e7 update to url/percent-encoding to 2.1
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:57:01 +01:00
Wolfgang Bumiller db0cb9ce0b update a chunk of stuff to the hyper release
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-12-13 11:24:41 +01:00
Dietmar Maurer 3f1020b732 src/bin/proxmox-backup-client.rs: implement task stop 2019-12-13 09:43:08 +01:00
Dietmar Maurer 1ae5677de0 src/api2/types.rs: use PROXMOX_SAFE_ID_FORMAT instead of FILENAME_FORMAT 2019-12-12 14:15:40 +01:00
Dietmar Maurer d0adf270fb src/api2/types.rs: introduce PROXMOX_SAFE_ID_REGEX, use it for DATASTORE_SCHEMA 2019-12-12 12:37:11 +01:00
Dietmar Maurer 66c49c21c3 src/api2/types.rs: factor out DATASTORE_SCHEMA 2019-12-11 13:45:23 +01:00
Dietmar Maurer 567d3e00fb src/api2/node/tasks.rs: new filter "store" to filter tasks for one store 2019-12-11 12:53:34 +01:00
Christian Ebner d3dbe52f37 pxar: match_pattern: refactor match_filename
The match_filename() in sequentail_decoder and encoder are moved to be static
functions of MatchPattern.
This allows to reuse the code also in the catalog find implementation.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-11 10:55:01 +01:00
Christian Ebner f084505ec5 src/backup/catalog_shell.rs: sort output of list-selected
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-11 10:54:20 +01:00
Dietmar Maurer 6d04612955 src/api2/node/tasks.rs: add "running" filter to list only running tasks 2019-12-11 10:52:27 +01:00
Dietmar Maurer 8560fe3e5f src/api2/node/tasks.rs - extract_upid: avoid duplicate error message 2019-12-10 13:45:56 +01:00
Dietmar Maurer 5830c20560 src/bin/proxmox-backup-client.rs: add simple task management cli 2019-12-10 13:43:53 +01:00
Dietmar Maurer 73e57f244e src/backup/datastore.rs - garbage-collection: fix average chunk size 2019-12-10 11:25:29 +01:00
Dietmar Maurer d105176f45 src/bin/proxmox-backup-client.rs: cleanup - factor out view_task_result() 2019-12-09 18:00:22 +01:00
Dietmar Maurer 48ef3c3327 cli: avoid useless .into()
This needs latest proxmox-api updates.
2019-12-09 17:40:34 +01:00
Dietmar Maurer eb7e2ee00b src/bin/proxmox-backup-client.rs: remove unnecessary .to_owned() 2019-12-09 17:11:48 +01:00
Dietmar Maurer e5f7def40e src/bin/proxmox-backup-client.rs - garbage-collection: display task log 2019-12-09 13:29:23 +01:00
Dietmar Maurer 0ec9e1b010 src/bin/proxmox-backup-client.rs: impl. and use complete_pxar_archive_name 2019-12-09 13:14:32 +01:00
Dietmar Maurer 1c6ad6ef90 src/bin/proxmox-backup-client.rs: impl. catalog sub command
catalog dump
catalog shell
2019-12-09 13:07:25 +01:00
Dietmar Maurer 5a0b484bdd src/client/task_log.rs: move display_task_log into extra file 2019-12-08 11:27:15 +01:00
Dietmar Maurer 3b03abfe14 src/api2/admin/datastore.rs: remove test_prune, add dry-run to normal prune
And use display_task_log on the client side.
2019-12-08 10:59:47 +01:00
Dietmar Maurer ec137a99c6 src/bin/proxmox-backup-client.r - display_task_log: use string range instead of as_bytes 2019-12-08 10:25:26 +01:00
Dietmar Maurer b73ff941f6 src/bin/proxmox-backup-client.r - display_task_log: add new strip_date parameter 2019-12-08 10:22:02 +01:00
Dietmar Maurer 163e9bbe91 src/bin/proxmox-backup-client.rs: new display task log helper
New helper to display worker task logs. Use it for prune (experimental).
2019-12-07 16:11:26 +01:00
Dietmar Maurer 6b508dd563 src/server/formatter.rs: impl. new result attribute "active" 2019-12-07 15:29:42 +01:00
Dietmar Maurer 102d8d4136 src/backup/prune.rs: implement --keep-hourly 2019-12-07 11:23:33 +01:00
Dietmar Maurer 052c4aa160 src/backup/prune.rs: prune - correctly use iso_week year 2019-12-06 19:41:38 +01:00
Christian Ebner fa0b950070 pxar: fuse: add support for list- and getxattr
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 16:44:15 +01:00
Christian Ebner 25ad4cbf63 pxar: fuse: cache goodbye table for each directory on opendir and release it on releasedir
Cache not only the goodbye table for the last directory but for each opened
directory.
The opendir fuse callback will fill the cache with the goodbye table and
releasedir will remove it from the cache.
This should reduce the number of chuncks fetched from the server in some cases.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 16:43:59 +01:00
Christian Ebner 02491b8fc6 pxar: fuse: use binary search instead of linear search in goodbye table.
The goodbye table is layed out as binary search tree based on the hash, so use
this to be more efficient when looking up a hash in the table for directories
with a large number of entries.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-12-06 16:42:01 +01:00
Dietmar Maurer 7074a0b339 src/bin/proxmox-backup-client.rs: white space cleanup 2019-12-06 13:35:59 +01:00
Dietmar Maurer 74fa81b89c src/bin/proxmox-backup-client.rs: implement prune --dry-run 2019-12-06 13:35:25 +01:00
Dietmar Maurer 1f604f1a5a src/api2/admin/datastore.rs: add new api test_prune() 2019-12-06 13:06:56 +01:00