f54c19989c
tools: add async signalfd handler
...
This provides a Stream<Item = siginfo> via nix' signalfd,
by wrapping it in tokio's PollEvented2 struct to allow
polling via tokio's event loop.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-12 08:04:28 +01:00
d5c34d98c0
src/catar/decoder.rs: simplify public restore API
2019-03-11 14:31:01 +01:00
8e39232acc
src/bin/proxmox-backup-client.rs: strip .didx file extensions
2019-03-11 10:54:02 +01:00
74cdb52107
src/bin/proxmox-backup-client.rs: include file extension in backupspec
2019-03-11 10:27:28 +01:00
fc2bf37ead
src/catar/decoder.rs: cleanup style
2019-03-11 08:14:28 +01:00
656b23e107
src/catar/decoder.rs: dir_mkdirat - add create_new flag
2019-03-11 07:27:45 +01:00
e9c9409a99
src/catar/decoder.rs: implement real sequential decoder (no Seek)
...
I copied the old code to src/catar/inspector.rs. Will modify
that later to implement a random access decoder ...
2019-03-08 16:55:54 +01:00
eed6db3923
src/bin/proxmox-backup-client.rs: impl option all-file-systems
2019-03-08 09:36:40 +01:00
06178f13fa
move chunker into proxmox_protocol
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-08 09:29:47 +01:00
e3c30c50a6
src/bin/catar.rs: implement all-file-systems option
2019-03-08 09:28:12 +01:00
65092b1e27
src/catar/encoder.rs: log skipped mount points
2019-03-08 09:25:00 +01:00
6b64d25d64
src/catar/encoder.rs: remove debug output
2019-03-08 08:23:07 +01:00
2689810c92
src/bin/catar.rs: implement verbose flags
2019-03-08 08:20:56 +01:00
79c281fdac
src/catar/encoder.rs: use simply flag "all_file_systems"
2019-03-08 08:14:26 +01:00
25e77d3877
src/cli/command.rs: print_bash_completion - avoid crash
2019-03-07 12:14:26 +01:00
141f062e08
src/backup/chunk_store.rs: use zstd compression insteadf of lz4
...
Provides better compressionm rate, and is still fast.
2019-03-07 11:42:59 +01:00
d2690f74bb
src/backup/chunk_store.rs: read_chunk - clear buffer before calling read_to_end
2019-03-07 10:40:01 +01:00
2ae7d196b9
src/bin/proxmox-backup-client.rs: correctly strip file extension
2019-03-06 11:18:46 +01:00
9f91249351
src/bin/proxmox-backup-client.rs: impl restore
2019-03-06 11:02:08 +01:00
01a1342364
src/api2/admin/datastore.rs: impl list_snapshot_files
2019-03-06 11:02:08 +01:00
58e99e138e
src/backup/backup_info.rs: new helper to list files
2019-03-06 11:02:08 +01:00
dd79265a8a
src/api2/admin/datastore/catar.rs: use archive_name directly
2019-03-06 11:02:08 +01:00
a6b7551311
src/client/http_client.rs: impl download
2019-03-06 11:02:08 +01:00
6716f30bbc
add protocol test client
...
This uses futures for everything which is mostly useful as a
test to see if the protocol crate's non-blocking I/O support
can handle it...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
e2d007f76e
api2/admin/datastore: add a backup protocol test api path
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
e3062f87b1
fixed_index: split add_chunk/add_digest
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
c3bb97e59c
fixed_index: impl Send for FixedIndexWriter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
3b7ade9e78
backup/datastore: expose insert_chunk method
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
91ebea8262
router: add upgrade method
...
This is just an Async GET method, but semantically
different, so it gets its own router method.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
80822b95b1
src/bin/proxmox-backup-client.rs: sort list results
2019-03-06 07:20:51 +01:00
fba30411fb
src/bin/proxmox-backup-client.rs: allow to set backup-id
2019-03-06 07:02:52 +01:00
21ea0158b4
src/client/http_client.rs: remove debug statement
2019-03-06 06:45:51 +01:00
ba3a60b2a4
src/client/http_client.rs: store/load ticket in xdg runtime dir, depend on crate xdg
2019-03-05 12:56:21 +01:00
f8f94534d4
src/api2/access.rs: authenticate_user() - add ticket login
2019-03-05 12:56:21 +01:00
e5662b04e9
src/tools/ticket.rs: define const TICKET_LIFETIME
2019-03-05 12:56:21 +01:00
53cafb5925
src/tools.rs: impl file_get_json()
2019-03-05 12:56:21 +01:00
391d310741
backup_info: BackupDir shortcut
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
93b49ce38a
backup_info: use two independent Into<String> types
...
there's no reason to force them to be the same input type
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 09:13:08 +01:00
3ec3ec3f9a
src/bin/proxmox-backup-client.rs: print backup duration
2019-03-05 08:11:40 +01:00
d57474e055
src/backup/backup_info.rs: improve docs
2019-03-05 07:28:13 +01:00
b3483782d1
src/backup/backup_info.rs: move code into separate file
...
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
2b01a22507
src/backup/datastore.rs: add helper to sort backup lists
2019-03-04 18:20:57 +01:00
875fb1c01a
src/backup/datastore.rs: use unix epoch to create DateTime
...
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
38f8815925
src/backup/datastore.rs: use new() function to create structs
2019-03-04 17:44:27 +01:00
9b492eb256
src/backup/datastore.rs: protect BackupDir fields, impl new()
2019-03-04 13:51:36 +01:00
1e9a94e579
src/backup/datastore.rs: protect BackupGroup fields, impl new()
2019-03-04 13:38:23 +01:00
cdebd467e6
src/bin/proxmox-backup-client.rs: simply log to stdout
2019-03-04 09:38:34 +01:00
f72dfd2d10
src/backup/datastore.rs: make sure timestamp does not contains nanoseconds
...
Because we want simple, readable file names.
2019-03-04 09:37:10 +01:00
219ef0e6ca
src/catar/encoder.rs: add verbose flags
2019-03-04 08:01:09 +01:00
e0b8ce6424
src/catar/encoder.rs: skip RAMFS
...
unless explicitly selected.
2019-03-04 06:48:47 +01:00
597a92039e
src/bin/proxmox-backup-client.rs: rename create into backup
2019-03-03 11:46:53 +01:00
6f62c9240a
src/api2/admin/datastore.rs: imp delete_snapshot
2019-03-03 11:29:00 +01:00
af926291e0
avoid compiler warning
2019-03-03 09:52:06 +01:00
e77a02ed87
catar api: cleanup parameter names
2019-03-03 09:48:32 +01:00
421c9ba1d9
src/bin/proxmox-backup-client.rs: use new json_object_to_query
2019-03-03 09:44:25 +01:00
f5f13ebc5a
src/tools.rs: new helper json_object_to_query
2019-03-03 08:51:37 +01:00
49dc074007
parameter name cleanups (use '-' instead of '_')
2019-03-02 16:34:10 +01:00
ad20d19877
src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups
2019-03-02 16:28:36 +01:00
184f17afad
src/api2/admin/datastore.rs: impl list_snapshots
2019-03-02 16:20:50 +01:00
af53186e6a
src/tools.rs: add simply string join
2019-03-02 16:15:45 +01:00
812c6f8716
src/api2/admin/datastore.rs: impl list_backup_groups
2019-03-02 11:29:05 +01:00
e909522f06
src/backup/datastore.rs: use timestamp with zone offset
2019-03-01 17:36:40 +01:00
dc4c09faae
src/backup/datastore.rs: split BackupDir
2019-03-01 16:26:35 +01:00
38b0dfa511
src/backup/datastore.rs: split BackupInfo
2019-03-01 16:26:35 +01:00
dcd15d10bb
backup/index: remove IndexIterator
...
removed the need for it, to avoid copies
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-01 13:29:36 +01:00
17d6979afd
src/bin/proxmox-backup-client.rs: generate a log file
2019-03-01 09:35:41 +01:00
3b151414f8
src/tools/file_logger.rs: class to log into files
2019-03-01 09:34:29 +01:00
5a2892f0ae
src/bin/proxmox-backup-client.rs: use chrono::DateTime
2019-03-01 06:59:32 +01:00
51144821f9
src/bin/proxmox-backup-client.rs: login before starting backup
2019-03-01 06:48:41 +01:00
541b908ea4
src/backup/datastore.rs: use DateTime<Utc> instead of unix epoch
2019-02-28 17:03:48 +01:00
b55cee9227
src/cli/command.rs: generate_usage_str() - fix option separator (newline)
2019-02-28 16:20:00 +01:00
c4c5174000
src/cli/command.rs: get_schema_type_text() - display min/max
2019-02-28 16:11:49 +01:00
df67070b1c
client/backup_repo: allow ip addresses for BackupRepository
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
5de2bced2d
datastore: add generic open_index
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
7bc1d72778
add IndexFile trait
...
We want to be able to iterate through digests of index
files, but without always having to distinguish between
dynamic and fixed types, so add a trait we can use as a
trait object.
Unfortunately the iterator needs to yield copies as
iterators cannot yield values with life times when
represented as trait objects (Box<dyn Iterator<Item = ?>>)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
5be4065baf
make FixedIndexReader Send and update comments
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
fded74d0c7
style: BufferedReader -> BufferedRead
...
Single-method traits usually use the same name as their
method and aren't usually the 'noun' (which is usually an
implementation of them instead).
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
8f57971714
src/api2/admin/datastore.rs: implement prune
2019-02-28 12:51:27 +01:00
83b7db0209
src/api2/admin/datastore.rs: start prune api
...
just a dummy for now.
2019-02-27 16:53:17 +01:00
0ffbccce5e
rc/server/rest.rs: allow to pass parameters as application/json
2019-02-27 12:37:53 +01:00
f98ea63de2
src/bin/proxmox-backup-client.rs: fix bash completion
2019-02-27 12:37:53 +01:00
8731e40a7f
backup/datastore: create_backup_dir: show if path existed
...
To enable asserting the creation of a new backup rather than
adding to an existing one.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-27 10:15:36 +01:00
25f1650b71
src/bin/proxmox-backup-client.rs: minor code cleanup
2019-02-27 07:21:57 +01:00
9bc3ddb822
src/cli/command.rs: improve generate_usage_str
...
correctly format positional array arguments ({<name>}).
2019-02-26 12:40:51 +01:00
ae0be2dda9
src/bin/proxmox-backup-client.rs: allow to upload more that one directory
2019-02-26 12:27:28 +01:00
a477d6889a
src/client/http_client.rs: login once, store and reuse ticket/token
2019-02-26 11:59:10 +01:00
d02c2ed701
src/cli/getopts.rs: cleanup, avoid compiler warning
2019-02-26 11:55:06 +01:00
f8dfbb45c0
src/tools.rs: new helper required_array_param()
2019-02-26 08:48:17 +01:00
aaba6faace
src/cli/getopts.rs: support positional array arguments
2019-02-26 08:47:34 +01:00
f33fa27307
src/cli/command.rs: improve error message
2019-02-26 08:46:36 +01:00
92a173feb0
src/api_schema/schema.rs: improve ParameterError display function
2019-02-26 07:11:24 +01:00
f661f37427
backup/dynamic_index: split out add_chunk
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:50:45 +01:00
9ac6ec868a
backup/chunk_store: split insert_chunk
...
The protocol handler will receive chunk data plus a hash
pre-calculated by the client. It will verify the hash before
sending it up to the datastore in order to respond to the
client with an error on a mismatch, so there's no need to
recalculate the hash another time.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
3a50ddd0c9
backup/chunk_store: mark ChunkIterator as fused
...
Ranges are FusedIterators and it can be useful to propagate
that property to users.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
eff25ecaf2
backup/chunk_store: make percentage output optional
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
d59397e60e
backup/datastore: expose chunk_store.get_chunk_iterator
...
will be used by the protocol handler to iterate over
existing chunks
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
8eb1203bda
src/backup/chunk_stat.rs: log write speed
2019-02-25 13:21:00 +01:00
7e3365554e
src/backup/chunk_stat.rs: new struct to track chunk statistics
2019-02-25 12:52:10 +01:00
798f7fa065
src/backup/chunk_store.rs: return info about compressed chunk size
...
So that we can generate better statistics ..
2019-02-25 11:36:05 +01:00
78216a5ab1
src/backup/chunk_store.rs: use lz4 for compression
...
And depend on lz4 lib now.
2019-02-25 10:00:28 +01:00
793b0f4d77
src/cli/command.rs: improve help command completion
2019-02-23 18:07:32 +01:00
6949d91564
src/cli/command.rs: cleanup, avoid array manipulation
2019-02-23 17:50:33 +01:00
9d78d57960
src/cli/command.rs: implement bash completion for help
2019-02-23 17:38:10 +01:00
698d9d4402
src/cli/command.rs: add help command
2019-02-23 15:10:48 +01:00
8b6dd2240d
src/cli/command.rs::allow to generate ReST docu
2019-02-23 11:29:18 +01:00
2f3f2bb77f
src/cli/command.rs: improve usage imformation
2019-02-22 17:40:37 +01:00
f252ca654c
src/backup/chunker.rs: simplify break test
...
To make it a little bit faster ...
2019-02-22 13:02:11 +01:00
6adb40eefc
src/cli/command.rs: start implementing generate_usage_str()
2019-02-22 12:02:59 +01:00
33256db6dd
src/cli/command.rs: simplify cli handler
2019-02-22 11:15:10 +01:00
61834c1cbf
src/backup/chunker.rs: another try to speedup the chunker
2019-02-22 11:15:10 +01:00
532a69eb04
src/cli/getopts.rs: improve docs
2019-02-21 09:36:52 +01:00
4de0e142a0
cleanup cli library structure
2019-02-21 09:08:03 +01:00
56458d9764
http_client: on a tty, read password if no env var is set
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 15:03:34 +01:00
c9b296f117
tools: add tty helper module
...
with read_password() and stdin_isatty() functions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 14:43:07 +01:00
8cc0d6aff3
src/bin/proxmox-backup-client.rs: implement garbage-collection sub-command
2019-02-20 14:10:45 +01:00
81da38c143
src/client/http_client.rs: implement post
...
without parameters for now...
2019-02-20 14:09:55 +01:00
8c75372b79
src/backup/datastore.rs: list_backups() - include list of index files
2019-02-20 13:37:44 +01:00
14691fc1f3
src/bin/test_chunk_size.rs: avoid compilöer warnings, exit after 1GB
2019-02-20 12:05:28 +01:00
1604ec2654
src/bin/test_chunk_size.rs: another chunker test - gather some stats
2019-02-20 11:50:10 +01:00
fa4ebf0fce
src/backup/chunker.rs: simplify code
2019-02-20 11:17:19 +01:00
1b9c959c32
src/backup/chunker.rs: change window size back to 48
...
Using 64 is a bad idea, because it always computes hash 0 for
constant data streams.
2019-02-20 11:01:09 +01:00
247cdbce72
src/api2/admin/datastore/catar.rs: allow to configure chunk-size
2019-02-19 15:19:12 +01:00
c584aa21fd
src/backup/chunker.rs: start() - correctly store hash in self.h
2019-02-19 12:40:12 +01:00
cd1598302a
src/backup/chunker.rs: use 64 byte chunker window size
...
This results in faster code.
2019-02-19 12:27:37 +01:00
9a3e869cf5
src/backup/chunker.rs: simplify buzzhash start code
2019-02-19 12:17:17 +01:00
1c7a88aeff
UploadCaTar: currectly write all data using write_all()
2019-02-18 18:31:02 +01:00
4d2881dd4c
src/backup/chunker.rs: correctly compute window index 'idx'
...
And add a simple test.
2019-02-18 16:59:23 +01:00
a4a5c78c01
src/client/http_client.rs: send csrf token
2019-02-18 16:59:23 +01:00
fc7f03523c
cleanup
...
Error::from is already a function taking 1 parameter,
there's no need to wrap it with `|e| Error::from(e)`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:21:27 +01:00
8fdef1a81c
fix typo: confi{g}dir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:11:42 +01:00
1cb99c234e
src/server/rest.rs: correctly insert NoLogExtension()
2019-02-18 06:54:12 +01:00
591f570b22
src/client/http_client.rs: trim path
2019-02-18 06:24:28 +01:00
7f1685230a
src/server/rest.rs: get_index() include username and CSRF token
...
When we have an valid ticket. Also delay get_index() if called with
an invalid ticket.
2019-02-17 19:28:32 +01:00
141de8374a
src/server/rest.rs: factor our normalize_path()
2019-02-17 17:31:53 +01:00
8225aa2ff6
src/server/rest.rs: improve logs for unauthorized request
2019-02-17 17:18:44 +01:00
54d15ab5c5
src/api_schema/schema.rs: improve docu
2019-02-17 12:13:06 +01:00
51ebd07974
src/api_schema/router.rs: add docu for RpcEnvironmentType
2019-02-17 11:59:18 +01:00
d21c2710da
api_schema/router.rs: improve doc
2019-02-17 10:30:41 +01:00
ef2f2efbcc
improve api_schema module structure
2019-02-17 10:16:33 +01:00
dc9a007b11
rename src/api to src/api_schema
2019-02-17 09:59:20 +01:00
5ddf8cb10a
cleanup auth code, verify CSRF prevention token
2019-02-16 15:52:55 +01:00
95aee7e032
tools/common_regex.rs: add more docu
2019-02-16 12:36:01 +01:00
99be8844c7
buildcfg.rs: improve doc
2019-02-16 12:31:39 +01:00
a8f268afbb
fix configuration dir permission
...
Carefully set and check permissions ...
2019-02-16 12:19:13 +01:00
aada2a9719
config/datastore.rs: remove unused code
2019-02-16 10:10:45 +01:00
d11594db4c
config/datastore.rs: return empty config if file does not exist
2019-02-16 10:06:08 +01:00
12400210a2
tools::file_get_contents - improve error message
2019-02-16 09:42:00 +01:00
6235a41862
tools::file_read_firstline - improve error message
2019-02-16 09:36:29 +01:00
17ed456c2e
change proxy user from www-data to backup
2019-02-16 09:29:04 +01:00
5d14eb6a76
tools.rs: new helper to get uid/gid for the sepcified system user.
...
Or is there an easier way to get that info?
2019-02-15 15:33:12 +01:00
f0a6112451
backup/datastore.rs: improve error reporting
2019-02-15 14:33:03 +01:00
9b50c16103
section_config.rs: simplify parser by using new try_block macro
2019-02-15 12:13:15 +01:00
fe651dd6e5
tools.rs: fix doc tests
2019-02-15 11:33:51 +01:00
e80503d2a6
tools.rs: document try_block macro
2019-02-15 11:20:45 +01:00
f0dbba8cbe
tools.rs: new try_block macro
2019-02-15 10:59:12 +01:00
1d77b6cf6b
update to pam 0.7 (renamed from pam-auth)
...
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-15 10:34:40 +01:00
d4736445fe
server/rest.rs: add method to log message
2019-02-15 10:16:12 +01:00
f02e6fc478
api2/admin/datastore/catar.rs: simplify/fix debug message
2019-02-15 10:04:37 +01:00
44c00c0dfd
server/rest.rs: log full error messages
2019-02-15 09:55:12 +01:00
7171b3e079
server/rest.rs: use a protocol extension to avoid double log
...
Instead of modifying the response header itself.
2019-02-14 16:04:24 +01:00
7e03988caf
avoid double logging of proxied requests
2019-02-14 13:28:41 +01:00
af6f80d3db
Revert "backup/datastore: fn open is a private implementation detail"
...
This reverts commit 3dacd9a660
2019-02-14 13:13:49 +01:00
78a1fa6722
src/server/rest.rs: log failed requests
2019-02-14 13:07:34 +01:00
2cc90617dd
tools/fs: mark ReadDir as Send
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
62f2422f6a
backup/chunk_store: verify chunk file names
...
get_chunk_iterator() should skip over files which aren't an
actual chunk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
3dacd9a660
backup/datastore: fn open is a private implementation detail
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
9739aca41f
backup/chunk_store: open base dir in get_chunk_iterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
db4868ef4f
src/client/backup_repo.rs - improve docu
2019-02-14 11:17:08 +01:00
151c6ce27d
src/client.rs: improve library structure and docu
2019-02-14 11:11:39 +01:00
dd1bfe6c12
tools/timer: add module documentation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:13:42 +01:00
c7f481b642
backup: replace ChunkIterator
...
Instead we just build the iterator using our
tools::fs::read_subdir iterator as follows:
Use the index range (0..0x10000) as an iterator, map() it
to yield a subdirectory iterator for each index, then
flatten it.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
121f18efb7
tools: use scan_subdir iterator for scandir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
32286b039c
tools/fs: add file_name_utf8_unchecked shortcut
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
806d7a6a35
tools/fs: change scan_subdir to use filter_file_name_regex
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
53e1e7ca72
tools/fs: add filter_file_name_regex to FileIterOps
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
23fba9d777
tools/fs: add FileIterOps trait and FileTypeFilter
...
FileIterOps will provide operations on file entry iterators.
For now this provides a file type filter.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
1db416150d
tools/fs: let ReadDir iterator yield ReadDirEntry
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
b12505da69
tools/fs: add ReadDirEntry helper
...
This wraps nix::dir::Entry with the parent and additionally
exposes the parent directory file descriptor since we need
it to perform openat().
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
a25f863afd
tools/fs: add scan_subdir helper
...
This filters the results of read_subdir with a regex.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
b4d5787de9
tools: start fs submodule, add read_subdir
...
This creates an iterator relative to a RawFd via
nix::dir::Dir over nix::dir::Entrys.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
0dffe3f99a
src/client/http_client.rs: try to login
...
use an environment var to store passphrase (PBS_PASSWORD)
2019-02-13 14:31:43 +01:00
33d64b81e0
src/bin/proxmox-backup-client.rs: define and use BackupRepository
2019-02-13 12:30:52 +01:00
4a3f65172c
client: use hyper-tls for now
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
42d6e4fb05
tools: borrow: implement Deref for Tied
...
Eg. if you have an x: Tied<Dir, Iterator> now you can simply
call x.next() as it'll dereference to the Iterator.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
541a302224
api2/admin/datastore/catar.rs: use nested file extensions
...
i.e. archive.catar.didx
2019-02-13 10:37:00 +01:00
e5064ba607
simplify backup lib structure (pub use xxx:*), improve doc
2019-02-12 14:13:31 +01:00
d78345bcfc
src/backup.rs - improve doc
2019-02-12 13:27:11 +01:00
93d5d77952
rename ArchiveIndex to DynamicIndex
...
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00
91a905b6dd
rename ImageIndex to FixedIndex
...
also changed the file extension fron .iidx to .fidx
2019-02-12 11:50:45 +01:00
82bc0ad40c
backup: fixup warnings and readd lost file type check
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:43:43 +01:00
9349d2a1d2
backup: cleanup long lines
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
fdd71f5286
backup: use ChunkIterator for sweep_unused_chunks
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
c9e7f4dc8b
backup: add optional progress callback to ChunkIterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
4070096c6a
backup: add ChunkIterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
6ed25cbe3a
tools: add Tied value
...
Tie two dependent values together, such as a nix::Dir with
its nix::dir::Iter<'a> where 'a is tied to the Dir's
lifetime, making it otherwise impossible to return them or
store them in a struct.
Alternatively we could try the 'rental' crate.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
9ac1045c7b
bin: manager: fixup datastore parameter name
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
8f8d5a420c
avoid compiler warning
2019-02-11 15:12:01 +01:00
aa5a4060ab
bin/proxmox-backup-api.rs: improve error handling
2019-02-11 15:11:42 +01:00
4223d9f800
bin/proxmox-backup-proxy.rs: improve error handling
2019-02-11 14:43:26 +01:00
ace9e3531a
access: use proxmox-backup-auth for pam
...
allows customization via /etc/pam.d/proxmox-backup-auth
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-07 13:40:22 +01:00
13f8310cac
api2: time: use /etc/localtime as fallback
...
in case there is no /etc/timezone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-05 14:14:56 +01:00
a7cac14ed9
tools: use std::fs::read for file_get_contents
...
Until we add the extra options like we have in pve, just use
the already existing (since 1.26) shortcut.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:35:32 +01:00
e937034c86
remove some rather inconvenient debug output
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:34:38 +01:00
9d02e5c38a
proxy: use <configdir>/server.pfx for tls
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:19:32 +01:00
9f4962d396
introduce buildcfg module and PROXMOX_CONFIGDIR
...
buildcfg.rs should contain convenience variables or macros
for using build-time configured variables
For now we replace hardcoded "/etc/proxmox-backup/<foo>"
with configdir!("<foo>").
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:13:08 +01:00
0d176f3681
proxy: use TLS via tokio-tls
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 14:56:07 +01:00
e0a5d1ca2f
tests: chunk_store: cleanup .testdir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
a4f9188300
tests: import lazy_static as needed
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
6285b251e7
don't hardcode /usr/share paths
...
Add a defines.mk file defining the standard default layout
for installation (PREFIX, DATAROOTDIR, LIBDIR etc.).
Add a $(JSDIR) variable and use env!("JSDIR") in place of
the hardcoded /usr/share/javascript/proxmox-backup path.
Have defines.mk include an *optional* local.mak to override
JSDIR to www/ during local development.
Add `make install` targets and remove debian/install.
Note that service files are now in $libdir/proxmox-backup/
as there's usually no point in starting them from the
command line, so /usr/sbin makes no sense.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
8e4df103dd
remove useless command option from daemons
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
6278365fd9
improve ApiMethod docu
2019-02-01 10:38:07 +01:00
4b2cdeb9a6
implement relead_timezone flag
2019-02-01 10:04:46 +01:00
c82bc1a1f9
depend on pam-auth, use pam for root login
...
Please use username "root@pam" for login via GUI.
2019-02-01 09:30:50 +01:00
a154a8e8a4
delay unauthorized request (rate limit)
2019-01-31 14:34:21 +01:00
5996577ab6
move http error class to router.rs
2019-01-31 13:22:30 +01:00
b9903d6331
server/rest.rs: verify auth cookie
2019-01-31 12:22:00 +01:00
f484eed3c2
Utils.js: fix cookie handling
...
Use unsecure cookie foör testing.
2019-01-31 10:08:08 +01:00
9f49fe1d5d
avoid compiler warnings
2019-01-30 18:25:37 +01:00
34f956bc25
api2/access.rs: add ticket api
2019-01-30 15:16:10 +01:00
1bf446a33e
fix timer_delete call
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-30 12:04:20 +01:00
cf671670a0
auth_helpers.rs: add timestamp to csrf token
2019-01-29 17:50:03 +01:00
1e76cbc6a2
auth_helpers.rs: implement assemble_csrf_prevention_token
2019-01-29 17:41:45 +01:00
d01e2420f7
load auth keys on startup
2019-01-29 17:21:58 +01:00
6c30068ebf
auth_helpers.rs: split code into separate file
2019-01-29 16:55:49 +01:00
39a90ca6c5
bin/proxmox-backup-api.rs: gererate secret for csrf token
2019-01-29 13:15:33 +01:00
26027a151e
tools/ticket.rs: remove debug statements
2019-01-29 13:04:01 +01:00
8d04280b29
tools/ticket.rs: add basic ticket support
2019-01-29 12:59:07 +01:00
c8f3f9b1af
server/rest.rs: simplify proxy code
...
Only pass neccessary parameters.
2019-01-28 18:22:16 +01:00
a859fa95a9
add protected flag for some api methods
2019-01-28 18:16:47 +01:00
a3da38ddfd
server/rest.rs: implement proxy_sync_api_request
2019-01-28 18:06:42 +01:00
f120483353
server/rest.rs: add proxy_sync_api_request() dummy
2019-01-28 17:30:39 +01:00
4e4df8e2f0
api/router.rs: add new 'protected' flag to ApiMethod
2019-01-28 17:18:42 +01:00
5bb3398171
server/formatter.rs: fix extjs error format
2019-01-28 13:44:48 +01:00
02c7a75520
src/bin/proxmox-backup-proxy.rs: implement unpriviledged server
...
We want to run the public server as user www-data. Requests needing
root priviledges needs to be proxied to the proxmox-backup.service, which
now listens to 127.0.0.1:82.
2019-01-28 13:29:58 +01:00
48849593ee
api2/node/services.rs: implement service commands
2019-01-27 12:40:31 +01:00
d7d23785f0
RpcEnvironment: implement set_user() and get_user()
2019-01-27 10:52:26 +01:00
162b979394
RpcEnvironment: add environment type enum RpcEnvironmentType
2019-01-27 10:33:42 +01:00
e82dad9700
also pass rpcenv to async handlers
2019-01-27 10:18:52 +01:00
d2ab5f19e2
api2/node/services.rs: start services api
2019-01-26 17:54:18 +01:00
0f253593c6
move rpc environment implementation to separate files
2019-01-26 15:08:02 +01:00
6049b71f41
api: pass RpcEnvirnment to api handlers
2019-01-26 14:50:37 +01:00
d96d82736d
use crate log and syslog
2019-01-26 10:56:11 +01:00
e182ab4a4a
api2/node/syslog.rs: improve error handling
2019-01-26 10:23:52 +01:00
0441886842
api2/node/syslog.rs: implement dump journal
2019-01-25 18:20:51 +01:00
4f9a726833
api2/node/syslog.rs: add syslog api schema
2019-01-25 17:17:30 +01:00
1631c54fca
tools::get_hardware_address - fix FILENAME, use file_get_contents()
2019-01-25 12:23:47 +01:00
af2fddea4f
api2/node/dns.rs: implement concurrent update protection
2019-01-25 11:38:59 +01:00
de6b0721fa
pi2/node/dns.rs: return file digest
2019-01-25 11:03:41 +01:00
2296860027
tools.rs: move digest_to_hex() into tools
2019-01-25 10:58:28 +01:00
53157ca6cf
tools.rs: implement file_get_contents()
2019-01-25 10:44:40 +01:00
46b79b9ee5
api2/node/dns.rs: improve dns api
2019-01-25 10:15:32 +01:00
332dcc2277
backup/chunk_store.rs: fix test case - use absolute path
2019-01-25 10:14:25 +01:00
8f973f8181
api2/node/dns.rs: implement resolv.conf parser
2019-01-24 14:59:40 +01:00
784252dbc5
backup/datastore.rs: use single lazy_static block
2019-01-24 14:58:47 +01:00
e6ffeb918c
api2/node/time.rs: implement set timezone
2019-01-24 12:05:06 +01:00
cd8b91f78c
api2/node/time.rs: add return schema
2019-01-24 11:07:11 +01:00
447787ab7c
tools.rs: implement file_read_firstline
2019-01-24 10:43:30 +01:00
0463602a79
api2/node/time.rs: implement read api
2019-01-23 15:14:14 +01:00
b2b3485d5f
src/api2/node: start node configuration api
2019-01-23 13:09:17 +01:00
d15009c0ce
server/rest.rs: fake login cookie
2019-01-23 12:49:10 +01:00
248cb51862
add missing api2.rs
2019-01-22 13:56:27 +01:00
7e13b2d67f
api2/subscription.rs: add subscription api class
2019-01-22 12:50:19 +01:00
576e3bf252
rename api3 back to api2
...
There is no real need to change the path, so using api2 we can reuse
all helpers (like tools from proxmox widget toolkit).
2019-01-22 12:10:38 +01:00
1adb353d6f
lient/http_client.rs: simplify code
2019-01-22 10:36:04 +01:00
41c039e143
bin/proxmox-backup-client.rs: implement list command
2019-01-21 18:58:14 +01:00
1fdb4c6ffa
client/http_client.rs: implement get, share common code
2019-01-21 18:56:48 +01:00
83dbd80bd8
api3/admin/datastore.rs: implement list backups
2019-01-21 11:22:55 +01:00
7ca80246ea
backup/datastore.rs: use DateTime<Utc> to store backup time
2019-01-21 10:51:52 +01:00
be0084b001
white space fix
2019-01-21 10:32:07 +01:00
e25736b4c3
backup/datastore.rs: implement list_backups
2019-01-20 18:02:27 +01:00
cce1676a9e
tools.rs: use FnMut for scandir callback
2019-01-20 17:49:11 +01:00
443f3743bb
tools.rs: new scandir() helper
2019-01-20 17:31:43 +01:00
68469eebb4
backup/chunk_store.rs: verify base path is absolute
2019-01-20 16:49:22 +01:00
059ca7c3a1
remove crate tokio-codec (seems to be part of tokio now)
2019-01-20 14:28:06 +01:00
8eeef8492a
tools/wrapped_reader_stream.rs: allocate Vec<u8> for buffer
2019-01-20 11:01:18 +01:00
ff8bdf3b64
tools/wrapped_reader_stream.rs: only implement Stream (without AsyncRead)
...
Looks more efficent ...
2019-01-20 10:41:21 +01:00
8e89d9cafe
tools/wrapped_reader_stream.rs: use tokio_threadpool::blocking
2019-01-20 10:28:51 +01:00
ce7ba139f1
catar: improve bash completion
2019-01-20 09:55:20 +01:00
29f34b8e1a
/api3/admin/datastore.rs: add missing subdir to directory index
2019-01-20 09:47:32 +01:00
0b05fd5830
api3/admin/datastore/catar.rs: implement download
2019-01-20 09:39:32 +01:00
7f0d67cf79
tools/wrapped_reader_stream.rs: new helper class
...
Will use that to download catar files.
2019-01-20 09:38:28 +01:00
6a4c091616
api3/admin/datastore/catar.rs: start impl. download
2019-01-19 17:27:02 +01:00
50cfb695ae
api/router.rs: rename ApiUploadMethod to ApiAsyncMethod
...
We can use this for uploads and downloads ...
2019-01-19 16:42:43 +01:00
379ea0edb6
Revert "/api/schema.rs: implement Schema::Option"
...
This reverts commit 0a35462c1e
2019-01-19 12:53:32 +01:00
80f069656d
Revert "api/schema.rs: avoid Option(Option( nesting"
...
This reverts commit ffdac1af01
2019-01-19 12:52:51 +01:00
ca3d861513
remove unused type PropertyMap
2019-01-18 17:54:06 +01:00
ffdac1af01
api/schema.rs: avoid Option(Option( nesting
2019-01-18 17:49:19 +01:00
0a35462c1e
/api/schema.rs: implement Schema::Option
2019-01-18 17:40:37 +01:00
728797d0c1
reduce compiler warnings
2019-01-18 16:50:15 +01:00
383e857749
move complete_file_name() helper into tools.rs
2019-01-18 13:42:52 +01:00
73e844dbfe
bin/proxmox-backup-client.rs: improve file name completion
2019-01-18 13:40:05 +01:00
95cea65b04
backup/datastore.rs: list all index files using walkdir crate
2019-01-18 12:24:58 +01:00
ff3d3100d4
backup/datastore.rs: try to create useful directory layout
...
store/type/ID/TIME/xxx.yyy
2019-01-18 12:01:37 +01:00
0d38dcb43c
cleanup nodename()
2019-01-18 10:13:45 +01:00
74a69302a7
tools.rs: new helper nodename()
2019-01-18 09:58:15 +01:00
5a778d92b3
api3/admin/datastore.rs: rename parameter "name" to "store"
2019-01-18 08:33:11 +01:00
244d9b17a8
bin/proxmox-backup-client.rs: implement file name completions - first try ...
2019-01-17 14:24:20 +01:00
83bdac1e3b
api3/admin/datastore/upload_catar.rs: verify content type ("application/x-proxmox-backup-catar")
2019-01-17 12:43:29 +01:00
06aeb76a5c
client/catar_backup_stream.rs: remove debug code
2019-01-17 12:16:36 +01:00
0fe5d605e7
tools.rs: new helper required_string_param()
...
To extract parameters from Value.
2019-01-17 12:14:02 +01:00
Wolfgang Bumiller
Dietmar Maurer