Commit Graph

776 Commits

Author SHA1 Message Date
Dietmar Maurer a6b7551311 src/client/http_client.rs: impl download 2019-03-06 11:02:08 +01:00
Wolfgang Bumiller 6716f30bbc add protocol test client
This uses futures for everything which is mostly useful as a
test to see if the protocol crate's non-blocking I/O support
can handle it...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
Wolfgang Bumiller e2d007f76e api2/admin/datastore: add a backup protocol test api path
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
Wolfgang Bumiller e3062f87b1 fixed_index: split add_chunk/add_digest
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Wolfgang Bumiller c3bb97e59c fixed_index: impl Send for FixedIndexWriter
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Wolfgang Bumiller 3b7ade9e78 backup/datastore: expose insert_chunk method
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Wolfgang Bumiller 91ebea8262 router: add upgrade method
This is just an Async GET method, but semantically
different, so it gets its own router method.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Dietmar Maurer 80822b95b1 src/bin/proxmox-backup-client.rs: sort list results 2019-03-06 07:20:51 +01:00
Dietmar Maurer fba30411fb src/bin/proxmox-backup-client.rs: allow to set backup-id 2019-03-06 07:02:52 +01:00
Dietmar Maurer 21ea0158b4 src/client/http_client.rs: remove debug statement 2019-03-06 06:45:51 +01:00
Dietmar Maurer ba3a60b2a4 src/client/http_client.rs: store/load ticket in xdg runtime dir, depend on crate xdg 2019-03-05 12:56:21 +01:00
Dietmar Maurer f8f94534d4 src/api2/access.rs: authenticate_user() - add ticket login 2019-03-05 12:56:21 +01:00
Dietmar Maurer e5662b04e9 src/tools/ticket.rs: define const TICKET_LIFETIME 2019-03-05 12:56:21 +01:00
Dietmar Maurer 53cafb5925 src/tools.rs: impl file_get_json() 2019-03-05 12:56:21 +01:00
Wolfgang Bumiller 391d310741 backup_info: BackupDir shortcut
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
Wolfgang Bumiller 93b49ce38a backup_info: use two independent Into<String> types
there's no reason to force them to be the same input type

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 09:13:08 +01:00
Dietmar Maurer 3ec3ec3f9a src/bin/proxmox-backup-client.rs: print backup duration 2019-03-05 08:11:40 +01:00
Dietmar Maurer d57474e055 src/backup/backup_info.rs: improve docs 2019-03-05 07:28:13 +01:00
Dietmar Maurer b3483782d1 src/backup/backup_info.rs: move code into separate file
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
Dietmar Maurer 2b01a22507 src/backup/datastore.rs: add helper to sort backup lists 2019-03-04 18:20:57 +01:00
Dietmar Maurer 875fb1c01a src/backup/datastore.rs: use unix epoch to create DateTime
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
Dietmar Maurer 38f8815925 src/backup/datastore.rs: use new() function to create structs 2019-03-04 17:44:27 +01:00
Dietmar Maurer 9b492eb256 src/backup/datastore.rs: protect BackupDir fields, impl new() 2019-03-04 13:51:36 +01:00
Dietmar Maurer 1e9a94e579 src/backup/datastore.rs: protect BackupGroup fields, impl new() 2019-03-04 13:38:23 +01:00
Dietmar Maurer cdebd467e6 src/bin/proxmox-backup-client.rs: simply log to stdout 2019-03-04 09:38:34 +01:00
Dietmar Maurer f72dfd2d10 src/backup/datastore.rs: make sure timestamp does not contains nanoseconds
Because we want simple, readable file names.
2019-03-04 09:37:10 +01:00
Dietmar Maurer 219ef0e6ca src/catar/encoder.rs: add verbose flags 2019-03-04 08:01:09 +01:00
Dietmar Maurer e0b8ce6424 src/catar/encoder.rs: skip RAMFS
unless explicitly selected.
2019-03-04 06:48:47 +01:00
Dietmar Maurer 597a92039e src/bin/proxmox-backup-client.rs: rename create into backup 2019-03-03 11:46:53 +01:00
Dietmar Maurer 6f62c9240a src/api2/admin/datastore.rs: imp delete_snapshot 2019-03-03 11:29:00 +01:00
Dietmar Maurer af926291e0 avoid compiler warning 2019-03-03 09:52:06 +01:00
Dietmar Maurer e77a02ed87 catar api: cleanup parameter names 2019-03-03 09:48:32 +01:00
Dietmar Maurer 421c9ba1d9 src/bin/proxmox-backup-client.rs: use new json_object_to_query 2019-03-03 09:44:25 +01:00
Dietmar Maurer f5f13ebc5a src/tools.rs: new helper json_object_to_query 2019-03-03 08:51:37 +01:00
Dietmar Maurer 49dc074007 parameter name cleanups (use '-' instead of '_') 2019-03-02 16:34:10 +01:00
Dietmar Maurer ad20d19877 src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups 2019-03-02 16:28:36 +01:00
Dietmar Maurer 184f17afad src/api2/admin/datastore.rs: impl list_snapshots 2019-03-02 16:20:50 +01:00
Dietmar Maurer af53186e6a src/tools.rs: add simply string join 2019-03-02 16:15:45 +01:00
Dietmar Maurer 812c6f8716 src/api2/admin/datastore.rs: impl list_backup_groups 2019-03-02 11:29:05 +01:00
Dietmar Maurer e909522f06 src/backup/datastore.rs: use timestamp with zone offset 2019-03-01 17:36:40 +01:00
Dietmar Maurer dc4c09faae src/backup/datastore.rs: split BackupDir 2019-03-01 16:26:35 +01:00
Dietmar Maurer 38b0dfa511 src/backup/datastore.rs: split BackupInfo 2019-03-01 16:26:35 +01:00
Wolfgang Bumiller dcd15d10bb backup/index: remove IndexIterator
removed the need for it, to avoid copies

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-01 13:29:36 +01:00
Dietmar Maurer 17d6979afd src/bin/proxmox-backup-client.rs: generate a log file 2019-03-01 09:35:41 +01:00
Dietmar Maurer 3b151414f8 src/tools/file_logger.rs: class to log into files 2019-03-01 09:34:29 +01:00
Dietmar Maurer 5a2892f0ae src/bin/proxmox-backup-client.rs: use chrono::DateTime 2019-03-01 06:59:32 +01:00
Dietmar Maurer 51144821f9 src/bin/proxmox-backup-client.rs: login before starting backup 2019-03-01 06:48:41 +01:00
Dietmar Maurer 541b908ea4 src/backup/datastore.rs: use DateTime<Utc> instead of unix epoch 2019-02-28 17:03:48 +01:00
Dietmar Maurer b55cee9227 src/cli/command.rs: generate_usage_str() - fix option separator (newline) 2019-02-28 16:20:00 +01:00
Dietmar Maurer c4c5174000 src/cli/command.rs: get_schema_type_text() - display min/max 2019-02-28 16:11:49 +01:00
Wolfgang Bumiller df67070b1c client/backup_repo: allow ip addresses for BackupRepository
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
Wolfgang Bumiller 5de2bced2d datastore: add generic open_index
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
Wolfgang Bumiller 7bc1d72778 add IndexFile trait
We want to be able to iterate through digests of index
files, but without always having to distinguish between
dynamic and fixed types, so add a trait we can use as a
trait object.

Unfortunately the iterator needs to yield copies as
iterators cannot yield values with life times when
represented as trait objects (Box<dyn Iterator<Item = ?>>)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
Wolfgang Bumiller 5be4065baf make FixedIndexReader Send and update comments
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
Wolfgang Bumiller fded74d0c7 style: BufferedReader -> BufferedRead
Single-method traits usually use the same name as their
method and aren't usually the 'noun' (which is usually an
implementation of them instead).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
Dietmar Maurer 8f57971714 src/api2/admin/datastore.rs: implement prune 2019-02-28 12:51:27 +01:00
Dietmar Maurer 83b7db0209 src/api2/admin/datastore.rs: start prune api
just a dummy for now.
2019-02-27 16:53:17 +01:00
Dietmar Maurer 0ffbccce5e rc/server/rest.rs: allow to pass parameters as application/json 2019-02-27 12:37:53 +01:00
Dietmar Maurer f98ea63de2 src/bin/proxmox-backup-client.rs: fix bash completion 2019-02-27 12:37:53 +01:00
Wolfgang Bumiller 8731e40a7f backup/datastore: create_backup_dir: show if path existed
To enable asserting the creation of a new backup rather than
adding to an existing one.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-27 10:15:36 +01:00
Dietmar Maurer 25f1650b71 src/bin/proxmox-backup-client.rs: minor code cleanup 2019-02-27 07:21:57 +01:00
Dietmar Maurer 9bc3ddb822 src/cli/command.rs: improve generate_usage_str
correctly format positional array arguments ({<name>}).
2019-02-26 12:40:51 +01:00
Dietmar Maurer ae0be2dda9 src/bin/proxmox-backup-client.rs: allow to upload more that one directory 2019-02-26 12:27:28 +01:00
Dietmar Maurer a477d6889a src/client/http_client.rs: login once, store and reuse ticket/token 2019-02-26 11:59:10 +01:00
Dietmar Maurer d02c2ed701 src/cli/getopts.rs: cleanup, avoid compiler warning 2019-02-26 11:55:06 +01:00
Dietmar Maurer f8dfbb45c0 src/tools.rs: new helper required_array_param() 2019-02-26 08:48:17 +01:00
Dietmar Maurer aaba6faace src/cli/getopts.rs: support positional array arguments 2019-02-26 08:47:34 +01:00
Dietmar Maurer f33fa27307 src/cli/command.rs: improve error message 2019-02-26 08:46:36 +01:00
Dietmar Maurer 92a173feb0 src/api_schema/schema.rs: improve ParameterError display function 2019-02-26 07:11:24 +01:00
Wolfgang Bumiller f661f37427 backup/dynamic_index: split out add_chunk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:50:45 +01:00
Wolfgang Bumiller 9ac6ec868a backup/chunk_store: split insert_chunk
The protocol handler will receive chunk data plus a hash
pre-calculated by the client. It will verify the hash before
sending it up to the datastore in order to respond to the
client with an error on a mismatch, so there's no need to
recalculate the hash another time.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
Wolfgang Bumiller 3a50ddd0c9 backup/chunk_store: mark ChunkIterator as fused
Ranges are FusedIterators and it can be useful to propagate
that property to users.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
Wolfgang Bumiller eff25ecaf2 backup/chunk_store: make percentage output optional
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
Wolfgang Bumiller d59397e60e backup/datastore: expose chunk_store.get_chunk_iterator
will be used by the protocol handler to iterate over
existing chunks

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
Dietmar Maurer 8eb1203bda src/backup/chunk_stat.rs: log write speed 2019-02-25 13:21:00 +01:00
Dietmar Maurer 7e3365554e src/backup/chunk_stat.rs: new struct to track chunk statistics 2019-02-25 12:52:10 +01:00
Dietmar Maurer 798f7fa065 src/backup/chunk_store.rs: return info about compressed chunk size
So that we can generate better statistics ..
2019-02-25 11:36:05 +01:00
Dietmar Maurer 78216a5ab1 src/backup/chunk_store.rs: use lz4 for compression
And depend on lz4 lib now.
2019-02-25 10:00:28 +01:00
Dietmar Maurer 793b0f4d77 src/cli/command.rs: improve help command completion 2019-02-23 18:07:32 +01:00
Dietmar Maurer 6949d91564 src/cli/command.rs: cleanup, avoid array manipulation 2019-02-23 17:50:33 +01:00
Dietmar Maurer 9d78d57960 src/cli/command.rs: implement bash completion for help 2019-02-23 17:38:10 +01:00
Dietmar Maurer 698d9d4402 src/cli/command.rs: add help command 2019-02-23 15:10:48 +01:00
Dietmar Maurer 8b6dd2240d src/cli/command.rs::allow to generate ReST docu 2019-02-23 11:29:18 +01:00
Dietmar Maurer 2f3f2bb77f src/cli/command.rs: improve usage imformation 2019-02-22 17:40:37 +01:00
Dietmar Maurer f252ca654c src/backup/chunker.rs: simplify break test
To make it a little bit faster ...
2019-02-22 13:02:11 +01:00
Dietmar Maurer 6adb40eefc src/cli/command.rs: start implementing generate_usage_str() 2019-02-22 12:02:59 +01:00
Dietmar Maurer 33256db6dd src/cli/command.rs: simplify cli handler 2019-02-22 11:15:10 +01:00
Dietmar Maurer 61834c1cbf src/backup/chunker.rs: another try to speedup the chunker 2019-02-22 11:15:10 +01:00
Dietmar Maurer 532a69eb04 src/cli/getopts.rs: improve docs 2019-02-21 09:36:52 +01:00
Dietmar Maurer 4de0e142a0 cleanup cli library structure 2019-02-21 09:08:03 +01:00
Wolfgang Bumiller 56458d9764 http_client: on a tty, read password if no env var is set
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 15:03:34 +01:00
Wolfgang Bumiller c9b296f117 tools: add tty helper module
with read_password() and stdin_isatty() functions

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 14:43:07 +01:00
Dietmar Maurer 8cc0d6aff3 src/bin/proxmox-backup-client.rs: implement garbage-collection sub-command 2019-02-20 14:10:45 +01:00
Dietmar Maurer 81da38c143 src/client/http_client.rs: implement post
without parameters for now...
2019-02-20 14:09:55 +01:00
Dietmar Maurer 8c75372b79 src/backup/datastore.rs: list_backups() - include list of index files 2019-02-20 13:37:44 +01:00
Dietmar Maurer 14691fc1f3 src/bin/test_chunk_size.rs: avoid compilöer warnings, exit after 1GB 2019-02-20 12:05:28 +01:00
Dietmar Maurer 1604ec2654 src/bin/test_chunk_size.rs: another chunker test - gather some stats 2019-02-20 11:50:10 +01:00
Dietmar Maurer fa4ebf0fce src/backup/chunker.rs: simplify code 2019-02-20 11:17:19 +01:00
Dietmar Maurer 1b9c959c32 src/backup/chunker.rs: change window size back to 48
Using 64 is a bad idea, because it always computes hash 0 for
constant data streams.
2019-02-20 11:01:09 +01:00
Dietmar Maurer 247cdbce72 src/api2/admin/datastore/catar.rs: allow to configure chunk-size 2019-02-19 15:19:12 +01:00
Dietmar Maurer c584aa21fd src/backup/chunker.rs: start() - correctly store hash in self.h 2019-02-19 12:40:12 +01:00
Dietmar Maurer cd1598302a src/backup/chunker.rs: use 64 byte chunker window size
This results in faster code.
2019-02-19 12:27:37 +01:00
Dietmar Maurer 9a3e869cf5 src/backup/chunker.rs: simplify buzzhash start code 2019-02-19 12:17:17 +01:00
Dietmar Maurer 1c7a88aeff UploadCaTar: currectly write all data using write_all() 2019-02-18 18:31:02 +01:00
Dietmar Maurer 4d2881dd4c src/backup/chunker.rs: correctly compute window index 'idx'
And add a simple test.
2019-02-18 16:59:23 +01:00
Dietmar Maurer a4a5c78c01 src/client/http_client.rs: send csrf token 2019-02-18 16:59:23 +01:00
Wolfgang Bumiller fc7f03523c cleanup
Error::from is already a function taking 1 parameter,
there's no need to wrap it with `|e| Error::from(e)`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:21:27 +01:00
Wolfgang Bumiller 8fdef1a81c fix typo: confi{g}dir
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:11:42 +01:00
Dietmar Maurer 1cb99c234e src/server/rest.rs: correctly insert NoLogExtension() 2019-02-18 06:54:12 +01:00
Dietmar Maurer 591f570b22 src/client/http_client.rs: trim path 2019-02-18 06:24:28 +01:00
Dietmar Maurer 7f1685230a src/server/rest.rs: get_index() include username and CSRF token
When we have an valid ticket. Also delay get_index() if called with
an invalid ticket.
2019-02-17 19:28:32 +01:00
Dietmar Maurer 141de8374a src/server/rest.rs: factor our normalize_path() 2019-02-17 17:31:53 +01:00
Dietmar Maurer 8225aa2ff6 src/server/rest.rs: improve logs for unauthorized request 2019-02-17 17:18:44 +01:00
Dietmar Maurer 54d15ab5c5 src/api_schema/schema.rs: improve docu 2019-02-17 12:13:06 +01:00
Dietmar Maurer 51ebd07974 src/api_schema/router.rs: add docu for RpcEnvironmentType 2019-02-17 11:59:18 +01:00
Dietmar Maurer d21c2710da api_schema/router.rs: improve doc 2019-02-17 10:30:41 +01:00
Dietmar Maurer ef2f2efbcc improve api_schema module structure 2019-02-17 10:16:33 +01:00
Dietmar Maurer dc9a007b11 rename src/api to src/api_schema 2019-02-17 09:59:20 +01:00
Dietmar Maurer 5ddf8cb10a cleanup auth code, verify CSRF prevention token 2019-02-16 15:52:55 +01:00
Dietmar Maurer 95aee7e032 tools/common_regex.rs: add more docu 2019-02-16 12:36:01 +01:00
Dietmar Maurer 99be8844c7 buildcfg.rs: improve doc 2019-02-16 12:31:39 +01:00
Dietmar Maurer a8f268afbb fix configuration dir permission
Carefully set and check permissions ...
2019-02-16 12:19:13 +01:00
Dietmar Maurer aada2a9719 config/datastore.rs: remove unused code 2019-02-16 10:10:45 +01:00
Dietmar Maurer d11594db4c config/datastore.rs: return empty config if file does not exist 2019-02-16 10:06:08 +01:00
Dietmar Maurer 12400210a2 tools::file_get_contents - improve error message 2019-02-16 09:42:00 +01:00
Dietmar Maurer 6235a41862 tools::file_read_firstline - improve error message 2019-02-16 09:36:29 +01:00
Dietmar Maurer 17ed456c2e change proxy user from www-data to backup 2019-02-16 09:29:04 +01:00
Dietmar Maurer 5d14eb6a76 tools.rs: new helper to get uid/gid for the sepcified system user.
Or is there an easier way to get that info?
2019-02-15 15:33:12 +01:00
Dietmar Maurer f0a6112451 backup/datastore.rs: improve error reporting 2019-02-15 14:33:03 +01:00
Dietmar Maurer 9b50c16103 section_config.rs: simplify parser by using new try_block macro 2019-02-15 12:13:15 +01:00
Dietmar Maurer fe651dd6e5 tools.rs: fix doc tests 2019-02-15 11:33:51 +01:00
Dietmar Maurer e80503d2a6 tools.rs: document try_block macro 2019-02-15 11:20:45 +01:00
Dietmar Maurer f0dbba8cbe tools.rs: new try_block macro 2019-02-15 10:59:12 +01:00
Wolfgang Bumiller 1d77b6cf6b update to pam 0.7 (renamed from pam-auth)
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-15 10:34:40 +01:00
Dietmar Maurer d4736445fe server/rest.rs: add method to log message 2019-02-15 10:16:12 +01:00
Dietmar Maurer f02e6fc478 api2/admin/datastore/catar.rs: simplify/fix debug message 2019-02-15 10:04:37 +01:00
Dietmar Maurer 44c00c0dfd server/rest.rs: log full error messages 2019-02-15 09:55:12 +01:00
Dietmar Maurer 7171b3e079 server/rest.rs: use a protocol extension to avoid double log
Instead of modifying the response header itself.
2019-02-14 16:04:24 +01:00
Dietmar Maurer 7e03988caf avoid double logging of proxied requests 2019-02-14 13:28:41 +01:00
Dietmar Maurer af6f80d3db Revert "backup/datastore: fn open is a private implementation detail"
This reverts commit 3dacd9a660.

I currently use this in some test ...
2019-02-14 13:13:49 +01:00
Dietmar Maurer 78a1fa6722 src/server/rest.rs: log failed requests 2019-02-14 13:07:34 +01:00
Wolfgang Bumiller 2cc90617dd tools/fs: mark ReadDir as Send
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
Wolfgang Bumiller 62f2422f6a backup/chunk_store: verify chunk file names
get_chunk_iterator() should skip over files which aren't an
actual chunk

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
Wolfgang Bumiller 3dacd9a660 backup/datastore: fn open is a private implementation detail
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
Wolfgang Bumiller 9739aca41f backup/chunk_store: open base dir in get_chunk_iterator
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
Dietmar Maurer db4868ef4f src/client/backup_repo.rs - improve docu 2019-02-14 11:17:08 +01:00
Dietmar Maurer 151c6ce27d src/client.rs: improve library structure and docu 2019-02-14 11:11:39 +01:00
Wolfgang Bumiller dd1bfe6c12 tools/timer: add module documentation
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:13:42 +01:00
Wolfgang Bumiller c7f481b642 backup: replace ChunkIterator
Instead we just build the iterator using our
tools::fs::read_subdir iterator as follows:
Use the index range (0..0x10000) as an iterator, map() it
to yield a subdirectory iterator for each index, then
flatten it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller 121f18efb7 tools: use scan_subdir iterator for scandir
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller 32286b039c tools/fs: add file_name_utf8_unchecked shortcut
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller 806d7a6a35 tools/fs: change scan_subdir to use filter_file_name_regex
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller 53e1e7ca72 tools/fs: add filter_file_name_regex to FileIterOps
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller 23fba9d777 tools/fs: add FileIterOps trait and FileTypeFilter
FileIterOps will provide operations on file entry iterators.
For now this provides a file type filter.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller 1db416150d tools/fs: let ReadDir iterator yield ReadDirEntry
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller b12505da69 tools/fs: add ReadDirEntry helper
This wraps nix::dir::Entry with the parent and additionally
exposes the parent directory file descriptor since we need
it to perform openat().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller a25f863afd tools/fs: add scan_subdir helper
This filters the results of read_subdir with a regex.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Wolfgang Bumiller b4d5787de9 tools: start fs submodule, add read_subdir
This creates an iterator relative to a RawFd via
nix::dir::Dir over nix::dir::Entrys.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
Dietmar Maurer 0dffe3f99a src/client/http_client.rs: try to login
use an environment var to store passphrase (PBS_PASSWORD)
2019-02-13 14:31:43 +01:00
Dietmar Maurer 33d64b81e0 src/bin/proxmox-backup-client.rs: define and use BackupRepository 2019-02-13 12:30:52 +01:00
Wolfgang Bumiller 4a3f65172c client: use hyper-tls for now
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
Wolfgang Bumiller 42d6e4fb05 tools: borrow: implement Deref for Tied
Eg. if you have an x: Tied<Dir, Iterator> now you can simply
call x.next() as it'll dereference to the Iterator.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
Dietmar Maurer 541a302224 api2/admin/datastore/catar.rs: use nested file extensions
i.e. archive.catar.didx
2019-02-13 10:37:00 +01:00
Dietmar Maurer e5064ba607 simplify backup lib structure (pub use xxx:*), improve doc 2019-02-12 14:13:31 +01:00
Dietmar Maurer d78345bcfc src/backup.rs - improve doc 2019-02-12 13:27:11 +01:00
Dietmar Maurer 93d5d77952 rename ArchiveIndex to DynamicIndex
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00
Dietmar Maurer 91a905b6dd rename ImageIndex to FixedIndex
also changed the file extension fron .iidx to .fidx
2019-02-12 11:50:45 +01:00
Wolfgang Bumiller 82bc0ad40c backup: fixup warnings and readd lost file type check
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:43:43 +01:00
Wolfgang Bumiller 9349d2a1d2 backup: cleanup long lines
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
Wolfgang Bumiller fdd71f5286 backup: use ChunkIterator for sweep_unused_chunks
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
Wolfgang Bumiller c9e7f4dc8b backup: add optional progress callback to ChunkIterator
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
Wolfgang Bumiller 4070096c6a backup: add ChunkIterator
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
Wolfgang Bumiller 6ed25cbe3a tools: add Tied value
Tie two dependent values together, such as a nix::Dir with
its nix::dir::Iter<'a> where 'a is tied to the Dir's
lifetime, making it otherwise impossible to return them or
store them in a struct.

Alternatively we could try the 'rental' crate.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
Wolfgang Bumiller 9ac1045c7b bin: manager: fixup datastore parameter name
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
Dietmar Maurer 8f8d5a420c avoid compiler warning 2019-02-11 15:12:01 +01:00
Dietmar Maurer aa5a4060ab bin/proxmox-backup-api.rs: improve error handling 2019-02-11 15:11:42 +01:00
Dietmar Maurer 4223d9f800 bin/proxmox-backup-proxy.rs: improve error handling 2019-02-11 14:43:26 +01:00
Wolfgang Bumiller ace9e3531a access: use proxmox-backup-auth for pam
allows customization via /etc/pam.d/proxmox-backup-auth

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-07 13:40:22 +01:00
Wolfgang Bumiller 13f8310cac api2: time: use /etc/localtime as fallback
in case there is no /etc/timezone

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-05 14:14:56 +01:00
Wolfgang Bumiller a7cac14ed9 tools: use std::fs::read for file_get_contents
Until we add the extra options like we have in pve, just use
the already existing (since 1.26) shortcut.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:35:32 +01:00
Wolfgang Bumiller e937034c86 remove some rather inconvenient debug output
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:34:38 +01:00
Wolfgang Bumiller 9d02e5c38a proxy: use <configdir>/server.pfx for tls
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:19:32 +01:00
Wolfgang Bumiller 9f4962d396 introduce buildcfg module and PROXMOX_CONFIGDIR
buildcfg.rs should contain convenience variables or macros
for using build-time configured variables

For now we replace hardcoded "/etc/proxmox-backup/<foo>"
with configdir!("<foo>").

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:13:08 +01:00
Wolfgang Bumiller 0d176f3681 proxy: use TLS via tokio-tls
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 14:56:07 +01:00
Wolfgang Bumiller e0a5d1ca2f tests: chunk_store: cleanup .testdir
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
Wolfgang Bumiller a4f9188300 tests: import lazy_static as needed
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
Wolfgang Bumiller 6285b251e7 don't hardcode /usr/share paths
Add a defines.mk file defining the standard default layout
for installation (PREFIX, DATAROOTDIR, LIBDIR etc.).

Add a $(JSDIR) variable and use env!("JSDIR") in place of
the hardcoded /usr/share/javascript/proxmox-backup path.

Have defines.mk include an *optional* local.mak to override
JSDIR to www/ during local development.

Add `make install` targets and remove debian/install.

Note that service files are now in $libdir/proxmox-backup/
as there's usually no point in starting them from the
command line, so /usr/sbin makes no sense.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
Wolfgang Bumiller 8e4df103dd remove useless command option from daemons
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
Dietmar Maurer 6278365fd9 improve ApiMethod docu 2019-02-01 10:38:07 +01:00
Dietmar Maurer 4b2cdeb9a6 implement relead_timezone flag 2019-02-01 10:04:46 +01:00
Dietmar Maurer c82bc1a1f9 depend on pam-auth, use pam for root login
Please use username "root@pam" for login via GUI.
2019-02-01 09:30:50 +01:00
Dietmar Maurer a154a8e8a4 delay unauthorized request (rate limit) 2019-01-31 14:34:21 +01:00
Dietmar Maurer 5996577ab6 move http error class to router.rs 2019-01-31 13:22:30 +01:00
Dietmar Maurer b9903d6331 server/rest.rs: verify auth cookie 2019-01-31 12:22:00 +01:00
Dietmar Maurer f484eed3c2 Utils.js: fix cookie handling
Use unsecure cookie foör testing.
2019-01-31 10:08:08 +01:00
Dietmar Maurer 9f49fe1d5d avoid compiler warnings 2019-01-30 18:25:37 +01:00
Dietmar Maurer 34f956bc25 api2/access.rs: add ticket api 2019-01-30 15:16:10 +01:00
Wolfgang Bumiller 1bf446a33e fix timer_delete call
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-30 12:04:20 +01:00
Dietmar Maurer cf671670a0 auth_helpers.rs: add timestamp to csrf token 2019-01-29 17:50:03 +01:00
Dietmar Maurer 1e76cbc6a2 auth_helpers.rs: implement assemble_csrf_prevention_token 2019-01-29 17:41:45 +01:00