tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
690 Commits 2 Branches 54 Tags 14 MiB
91ebea8262
Commit Graph

620 Commits

Author SHA1 Message Date
Dietmar Maurer
83b7db0209 src/api2/admin/datastore.rs: start prune api 
just a dummy for now.
 2019-02-27 16:53:17 +01:00
Dietmar Maurer
0ffbccce5e rc/server/rest.rs: allow to pass parameters as application/json 2019-02-27 12:37:53 +01:00
Dietmar Maurer
f98ea63de2 src/bin/proxmox-backup-client.rs: fix bash completion 2019-02-27 12:37:53 +01:00
Wolfgang Bumiller
8731e40a7f backup/datastore: create_backup_dir: show if path existed 
To enable asserting the creation of a new backup rather than
adding to an existing one.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-27 10:15:36 +01:00
Dietmar Maurer
25f1650b71 src/bin/proxmox-backup-client.rs: minor code cleanup 2019-02-27 07:21:57 +01:00
Dietmar Maurer
9bc3ddb822 src/cli/command.rs: improve generate_usage_str 
correctly format positional array arguments ({<name>}).
 2019-02-26 12:40:51 +01:00
Dietmar Maurer
ae0be2dda9 src/bin/proxmox-backup-client.rs: allow to upload more that one directory 2019-02-26 12:27:28 +01:00
Dietmar Maurer
a477d6889a src/client/http_client.rs: login once, store and reuse ticket/token 2019-02-26 11:59:10 +01:00
Dietmar Maurer
d02c2ed701 src/cli/getopts.rs: cleanup, avoid compiler warning 2019-02-26 11:55:06 +01:00
Dietmar Maurer
f8dfbb45c0 src/tools.rs: new helper required_array_param() 2019-02-26 08:48:17 +01:00
Dietmar Maurer
aaba6faace src/cli/getopts.rs: support positional array arguments 2019-02-26 08:47:34 +01:00
Dietmar Maurer
f33fa27307 src/cli/command.rs: improve error message 2019-02-26 08:46:36 +01:00
Dietmar Maurer
92a173feb0 src/api_schema/schema.rs: improve ParameterError display function 2019-02-26 07:11:24 +01:00
Wolfgang Bumiller
f661f37427 backup/dynamic_index: split out add_chunk 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-25 13:50:45 +01:00
Wolfgang Bumiller
9ac6ec868a backup/chunk_store: split insert_chunk 
The protocol handler will receive chunk data plus a hash
pre-calculated by the client. It will verify the hash before
sending it up to the datastore in order to respond to the
client with an error on a mismatch, so there's no need to
recalculate the hash another time.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-25 13:44:29 +01:00
Wolfgang Bumiller
3a50ddd0c9 backup/chunk_store: mark ChunkIterator as fused 
Ranges are FusedIterators and it can be useful to propagate
that property to users.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-25 13:44:29 +01:00
Wolfgang Bumiller
eff25ecaf2 backup/chunk_store: make percentage output optional 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-25 13:44:29 +01:00
Wolfgang Bumiller
d59397e60e backup/datastore: expose chunk_store.get_chunk_iterator 
will be used by the protocol handler to iterate over
existing chunks

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-25 13:44:29 +01:00
Dietmar Maurer
8eb1203bda src/backup/chunk_stat.rs: log write speed 2019-02-25 13:21:00 +01:00
Dietmar Maurer
7e3365554e src/backup/chunk_stat.rs: new struct to track chunk statistics 2019-02-25 12:52:10 +01:00
Dietmar Maurer
798f7fa065 src/backup/chunk_store.rs: return info about compressed chunk size 
So that we can generate better statistics ..
 2019-02-25 11:36:05 +01:00
Dietmar Maurer
78216a5ab1 src/backup/chunk_store.rs: use lz4 for compression 
And depend on lz4 lib now.
 2019-02-25 10:00:28 +01:00
Dietmar Maurer
793b0f4d77 src/cli/command.rs: improve help command completion 2019-02-23 18:07:32 +01:00
Dietmar Maurer
6949d91564 src/cli/command.rs: cleanup, avoid array manipulation 2019-02-23 17:50:33 +01:00
Dietmar Maurer
9d78d57960 src/cli/command.rs: implement bash completion for help 2019-02-23 17:38:10 +01:00
Dietmar Maurer
698d9d4402 src/cli/command.rs: add help command 2019-02-23 15:10:48 +01:00
Dietmar Maurer
8b6dd2240d src/cli/command.rs::allow to generate ReST docu 2019-02-23 11:29:18 +01:00
Dietmar Maurer
2f3f2bb77f src/cli/command.rs: improve usage imformation 2019-02-22 17:40:37 +01:00
Dietmar Maurer
f252ca654c src/backup/chunker.rs: simplify break test 
To make it a little bit faster ...
 2019-02-22 13:02:11 +01:00
Dietmar Maurer
6adb40eefc src/cli/command.rs: start implementing generate_usage_str() 2019-02-22 12:02:59 +01:00
Dietmar Maurer
33256db6dd src/cli/command.rs: simplify cli handler 2019-02-22 11:15:10 +01:00
Dietmar Maurer
61834c1cbf src/backup/chunker.rs: another try to speedup the chunker 2019-02-22 11:15:10 +01:00
Dietmar Maurer
532a69eb04 src/cli/getopts.rs: improve docs 2019-02-21 09:36:52 +01:00
Dietmar Maurer
4de0e142a0 cleanup cli library structure 2019-02-21 09:08:03 +01:00
Wolfgang Bumiller
56458d9764 http_client: on a tty, read password if no env var is set 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-20 15:03:34 +01:00
Wolfgang Bumiller
c9b296f117 tools: add tty helper module 
with read_password() and stdin_isatty() functions

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-20 14:43:07 +01:00
Dietmar Maurer
8cc0d6aff3 src/bin/proxmox-backup-client.rs: implement garbage-collection sub-command 2019-02-20 14:10:45 +01:00
Dietmar Maurer
81da38c143 src/client/http_client.rs: implement post 
without parameters for now...
 2019-02-20 14:09:55 +01:00
Dietmar Maurer
8c75372b79 src/backup/datastore.rs: list_backups() - include list of index files 2019-02-20 13:37:44 +01:00
Dietmar Maurer
14691fc1f3 src/bin/test_chunk_size.rs: avoid compilöer warnings, exit after 1GB 2019-02-20 12:05:28 +01:00
Dietmar Maurer
1604ec2654 src/bin/test_chunk_size.rs: another chunker test - gather some stats 2019-02-20 11:50:10 +01:00
Dietmar Maurer
fa4ebf0fce src/backup/chunker.rs: simplify code 2019-02-20 11:17:19 +01:00
Dietmar Maurer
1b9c959c32 src/backup/chunker.rs: change window size back to 48 
Using 64 is a bad idea, because it always computes hash 0 for
constant data streams.
 2019-02-20 11:01:09 +01:00
Dietmar Maurer
247cdbce72 src/api2/admin/datastore/catar.rs: allow to configure chunk-size 2019-02-19 15:19:12 +01:00
Dietmar Maurer
c584aa21fd src/backup/chunker.rs: start() - correctly store hash in self.h 2019-02-19 12:40:12 +01:00
Dietmar Maurer
cd1598302a src/backup/chunker.rs: use 64 byte chunker window size 
This results in faster code.
 2019-02-19 12:27:37 +01:00
Dietmar Maurer
9a3e869cf5 src/backup/chunker.rs: simplify buzzhash start code 2019-02-19 12:17:17 +01:00
Dietmar Maurer
1c7a88aeff UploadCaTar: currectly write all data using write_all() 2019-02-18 18:31:02 +01:00
Dietmar Maurer
4d2881dd4c src/backup/chunker.rs: correctly compute window index 'idx' 
And add a simple test.
 2019-02-18 16:59:23 +01:00
Dietmar Maurer
a4a5c78c01 src/client/http_client.rs: send csrf token 2019-02-18 16:59:23 +01:00
Wolfgang Bumiller
fc7f03523c cleanup 
Error::from is already a function taking 1 parameter,
there's no need to wrap it with `|e| Error::from(e)`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-18 13:21:27 +01:00
Wolfgang Bumiller
8fdef1a81c fix typo: confi{g}dir 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-18 13:11:42 +01:00
Dietmar Maurer
1cb99c234e src/server/rest.rs: correctly insert NoLogExtension() 2019-02-18 06:54:12 +01:00
Dietmar Maurer
591f570b22 src/client/http_client.rs: trim path 2019-02-18 06:24:28 +01:00
Dietmar Maurer
7f1685230a src/server/rest.rs: get_index() include username and CSRF token 
When we have an valid ticket. Also delay get_index() if called with
an invalid ticket.
 2019-02-17 19:28:32 +01:00
Dietmar Maurer
141de8374a src/server/rest.rs: factor our normalize_path() 2019-02-17 17:31:53 +01:00
Dietmar Maurer
8225aa2ff6 src/server/rest.rs: improve logs for unauthorized request 2019-02-17 17:18:44 +01:00
Dietmar Maurer
54d15ab5c5 src/api_schema/schema.rs: improve docu 2019-02-17 12:13:06 +01:00
Dietmar Maurer
51ebd07974 src/api_schema/router.rs: add docu for RpcEnvironmentType 2019-02-17 11:59:18 +01:00
Dietmar Maurer
d21c2710da api_schema/router.rs: improve doc 2019-02-17 10:30:41 +01:00
Dietmar Maurer
ef2f2efbcc improve api_schema module structure 2019-02-17 10:16:33 +01:00
Dietmar Maurer
dc9a007b11 rename src/api to src/api_schema 2019-02-17 09:59:20 +01:00
Dietmar Maurer
5ddf8cb10a cleanup auth code, verify CSRF prevention token 2019-02-16 15:52:55 +01:00
Dietmar Maurer
95aee7e032 tools/common_regex.rs: add more docu 2019-02-16 12:36:01 +01:00
Dietmar Maurer
99be8844c7 buildcfg.rs: improve doc 2019-02-16 12:31:39 +01:00
Dietmar Maurer
a8f268afbb fix configuration dir permission 
Carefully set and check permissions ...
 2019-02-16 12:19:13 +01:00
Dietmar Maurer
aada2a9719 config/datastore.rs: remove unused code 2019-02-16 10:10:45 +01:00
Dietmar Maurer
d11594db4c config/datastore.rs: return empty config if file does not exist 2019-02-16 10:06:08 +01:00
Dietmar Maurer
12400210a2 tools::file_get_contents - improve error message 2019-02-16 09:42:00 +01:00
Dietmar Maurer
6235a41862 tools::file_read_firstline - improve error message 2019-02-16 09:36:29 +01:00
Dietmar Maurer
17ed456c2e change proxy user from www-data to backup 2019-02-16 09:29:04 +01:00
Dietmar Maurer
5d14eb6a76 tools.rs: new helper to get uid/gid for the sepcified system user. 
Or is there an easier way to get that info?
 2019-02-15 15:33:12 +01:00
Dietmar Maurer
f0a6112451 backup/datastore.rs: improve error reporting 2019-02-15 14:33:03 +01:00
Dietmar Maurer
9b50c16103 section_config.rs: simplify parser by using new try_block macro 2019-02-15 12:13:15 +01:00
Dietmar Maurer
fe651dd6e5 tools.rs: fix doc tests 2019-02-15 11:33:51 +01:00
Dietmar Maurer
e80503d2a6 tools.rs: document try_block macro 2019-02-15 11:20:45 +01:00
Dietmar Maurer
f0dbba8cbe tools.rs: new try_block macro 2019-02-15 10:59:12 +01:00
Wolfgang Bumiller
1d77b6cf6b update to pam 0.7 (renamed from pam-auth) 
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-15 10:34:40 +01:00
Dietmar Maurer
d4736445fe server/rest.rs: add method to log message 2019-02-15 10:16:12 +01:00
Dietmar Maurer
f02e6fc478 api2/admin/datastore/catar.rs: simplify/fix debug message 2019-02-15 10:04:37 +01:00
Dietmar Maurer
44c00c0dfd server/rest.rs: log full error messages 2019-02-15 09:55:12 +01:00
Dietmar Maurer
7171b3e079 server/rest.rs: use a protocol extension to avoid double log 
Instead of modifying the response header itself.
 2019-02-14 16:04:24 +01:00
Dietmar Maurer
7e03988caf avoid double logging of proxied requests 2019-02-14 13:28:41 +01:00
Dietmar Maurer
af6f80d3db Revert "backup/datastore: fn open is a private implementation detail" 
This reverts commit 3dacd9a660.

I currently use this in some test ...
 2019-02-14 13:13:49 +01:00
Dietmar Maurer
78a1fa6722 src/server/rest.rs: log failed requests 2019-02-14 13:07:34 +01:00
Wolfgang Bumiller
2cc90617dd tools/fs: mark ReadDir as Send 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 12:37:31 +01:00
Wolfgang Bumiller
62f2422f6a backup/chunk_store: verify chunk file names 
get_chunk_iterator() should skip over files which aren't an
actual chunk

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 12:37:31 +01:00
Wolfgang Bumiller
3dacd9a660 backup/datastore: fn open is a private implementation detail 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 12:37:31 +01:00
Wolfgang Bumiller
9739aca41f backup/chunk_store: open base dir in get_chunk_iterator 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 12:37:31 +01:00
Dietmar Maurer
db4868ef4f src/client/backup_repo.rs - improve docu 2019-02-14 11:17:08 +01:00
Dietmar Maurer
151c6ce27d src/client.rs: improve library structure and docu 2019-02-14 11:11:39 +01:00
Wolfgang Bumiller
dd1bfe6c12 tools/timer: add module documentation 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:13:42 +01:00
Wolfgang Bumiller
c7f481b642 backup: replace ChunkIterator 
Instead we just build the iterator using our
tools::fs::read_subdir iterator as follows:
Use the index range (0..0x10000) as an iterator, map() it
to yield a subdirectory iterator for each index, then
flatten it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
121f18efb7 tools: use scan_subdir iterator for scandir 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
32286b039c tools/fs: add file_name_utf8_unchecked shortcut 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
806d7a6a35 tools/fs: change scan_subdir to use filter_file_name_regex 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
53e1e7ca72 tools/fs: add filter_file_name_regex to FileIterOps 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
23fba9d777 tools/fs: add FileIterOps trait and FileTypeFilter 
FileIterOps will provide operations on file entry iterators.
For now this provides a file type filter.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
1db416150d tools/fs: let ReadDir iterator yield ReadDirEntry 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00
Wolfgang Bumiller
b12505da69 tools/fs: add ReadDirEntry helper 
This wraps nix::dir::Entry with the parent and additionally
exposes the parent directory file descriptor since we need
it to perform openat().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-02-14 10:09:59 +01:00