Dietmar Maurer
b9903d6331
server/rest.rs: verify auth cookie
2019-01-31 12:22:00 +01:00
Dietmar Maurer
f484eed3c2
Utils.js: fix cookie handling
...
Use unsecure cookie foör testing.
2019-01-31 10:08:08 +01:00
Dietmar Maurer
9f49fe1d5d
avoid compiler warnings
2019-01-30 18:25:37 +01:00
Dietmar Maurer
34f956bc25
api2/access.rs: add ticket api
2019-01-30 15:16:10 +01:00
Wolfgang Bumiller
1bf446a33e
fix timer_delete call
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-30 12:04:20 +01:00
Dietmar Maurer
cf671670a0
auth_helpers.rs: add timestamp to csrf token
2019-01-29 17:50:03 +01:00
Dietmar Maurer
1e76cbc6a2
auth_helpers.rs: implement assemble_csrf_prevention_token
2019-01-29 17:41:45 +01:00
Dietmar Maurer
d01e2420f7
load auth keys on startup
2019-01-29 17:21:58 +01:00
Dietmar Maurer
6c30068ebf
auth_helpers.rs: split code into separate file
2019-01-29 16:55:49 +01:00
Dietmar Maurer
39a90ca6c5
bin/proxmox-backup-api.rs: gererate secret for csrf token
2019-01-29 13:15:33 +01:00
Dietmar Maurer
26027a151e
tools/ticket.rs: remove debug statements
2019-01-29 13:04:01 +01:00
Dietmar Maurer
8d04280b29
tools/ticket.rs: add basic ticket support
2019-01-29 12:59:07 +01:00
Dietmar Maurer
c8f3f9b1af
server/rest.rs: simplify proxy code
...
Only pass neccessary parameters.
2019-01-28 18:22:16 +01:00
Dietmar Maurer
a859fa95a9
add protected flag for some api methods
2019-01-28 18:16:47 +01:00
Dietmar Maurer
a3da38ddfd
server/rest.rs: implement proxy_sync_api_request
2019-01-28 18:06:42 +01:00
Dietmar Maurer
f120483353
server/rest.rs: add proxy_sync_api_request() dummy
2019-01-28 17:30:39 +01:00
Dietmar Maurer
4e4df8e2f0
api/router.rs: add new 'protected' flag to ApiMethod
2019-01-28 17:18:42 +01:00
Dietmar Maurer
5bb3398171
server/formatter.rs: fix extjs error format
2019-01-28 13:44:48 +01:00
Dietmar Maurer
02c7a75520
src/bin/proxmox-backup-proxy.rs: implement unpriviledged server
...
We want to run the public server as user www-data. Requests needing
root priviledges needs to be proxied to the proxmox-backup.service, which
now listens to 127.0.0.1:82.
2019-01-28 13:29:58 +01:00
Dietmar Maurer
48849593ee
api2/node/services.rs: implement service commands
2019-01-27 12:40:31 +01:00
Dietmar Maurer
d7d23785f0
RpcEnvironment: implement set_user() and get_user()
2019-01-27 10:52:26 +01:00
Dietmar Maurer
162b979394
RpcEnvironment: add environment type enum RpcEnvironmentType
2019-01-27 10:33:42 +01:00
Dietmar Maurer
e82dad9700
also pass rpcenv to async handlers
2019-01-27 10:18:52 +01:00
Dietmar Maurer
d2ab5f19e2
api2/node/services.rs: start services api
2019-01-26 17:54:18 +01:00
Dietmar Maurer
0f253593c6
move rpc environment implementation to separate files
2019-01-26 15:08:02 +01:00
Dietmar Maurer
6049b71f41
api: pass RpcEnvirnment to api handlers
2019-01-26 14:50:37 +01:00
Dietmar Maurer
d96d82736d
use crate log and syslog
2019-01-26 10:56:11 +01:00
Dietmar Maurer
e182ab4a4a
api2/node/syslog.rs: improve error handling
2019-01-26 10:23:52 +01:00
Dietmar Maurer
0441886842
api2/node/syslog.rs: implement dump journal
2019-01-25 18:20:51 +01:00
Dietmar Maurer
4f9a726833
api2/node/syslog.rs: add syslog api schema
2019-01-25 17:17:30 +01:00
Dietmar Maurer
1631c54fca
tools::get_hardware_address - fix FILENAME, use file_get_contents()
2019-01-25 12:23:47 +01:00
Dietmar Maurer
af2fddea4f
api2/node/dns.rs: implement concurrent update protection
2019-01-25 11:38:59 +01:00
Dietmar Maurer
de6b0721fa
pi2/node/dns.rs: return file digest
2019-01-25 11:03:41 +01:00
Dietmar Maurer
2296860027
tools.rs: move digest_to_hex() into tools
2019-01-25 10:58:28 +01:00
Dietmar Maurer
53157ca6cf
tools.rs: implement file_get_contents()
2019-01-25 10:44:40 +01:00
Dietmar Maurer
46b79b9ee5
api2/node/dns.rs: improve dns api
2019-01-25 10:15:32 +01:00
Dietmar Maurer
332dcc2277
backup/chunk_store.rs: fix test case - use absolute path
2019-01-25 10:14:25 +01:00
Dietmar Maurer
8f973f8181
api2/node/dns.rs: implement resolv.conf parser
2019-01-24 14:59:40 +01:00
Dietmar Maurer
784252dbc5
backup/datastore.rs: use single lazy_static block
2019-01-24 14:58:47 +01:00
Dietmar Maurer
e6ffeb918c
api2/node/time.rs: implement set timezone
2019-01-24 12:05:06 +01:00
Dietmar Maurer
cd8b91f78c
api2/node/time.rs: add return schema
2019-01-24 11:07:11 +01:00
Dietmar Maurer
447787ab7c
tools.rs: implement file_read_firstline
2019-01-24 10:43:30 +01:00
Dietmar Maurer
0463602a79
api2/node/time.rs: implement read api
2019-01-23 15:14:14 +01:00
Dietmar Maurer
b2b3485d5f
src/api2/node: start node configuration api
2019-01-23 13:09:17 +01:00
Dietmar Maurer
d15009c0ce
server/rest.rs: fake login cookie
2019-01-23 12:49:10 +01:00
Dietmar Maurer
248cb51862
add missing api2.rs
2019-01-22 13:56:27 +01:00
Dietmar Maurer
7e13b2d67f
api2/subscription.rs: add subscription api class
2019-01-22 12:50:19 +01:00
Dietmar Maurer
576e3bf252
rename api3 back to api2
...
There is no real need to change the path, so using api2 we can reuse
all helpers (like tools from proxmox widget toolkit).
2019-01-22 12:10:38 +01:00
Dietmar Maurer
1adb353d6f
lient/http_client.rs: simplify code
2019-01-22 10:36:04 +01:00
Dietmar Maurer
41c039e143
bin/proxmox-backup-client.rs: implement list command
2019-01-21 18:58:14 +01:00
Dietmar Maurer
1fdb4c6ffa
client/http_client.rs: implement get, share common code
2019-01-21 18:56:48 +01:00
Dietmar Maurer
83dbd80bd8
api3/admin/datastore.rs: implement list backups
2019-01-21 11:22:55 +01:00
Dietmar Maurer
7ca80246ea
backup/datastore.rs: use DateTime<Utc> to store backup time
2019-01-21 10:51:52 +01:00
Dietmar Maurer
be0084b001
white space fix
2019-01-21 10:32:07 +01:00
Dietmar Maurer
e25736b4c3
backup/datastore.rs: implement list_backups
2019-01-20 18:02:27 +01:00
Dietmar Maurer
cce1676a9e
tools.rs: use FnMut for scandir callback
2019-01-20 17:49:11 +01:00
Dietmar Maurer
443f3743bb
tools.rs: new scandir() helper
2019-01-20 17:31:43 +01:00
Dietmar Maurer
68469eebb4
backup/chunk_store.rs: verify base path is absolute
2019-01-20 16:49:22 +01:00
Dietmar Maurer
059ca7c3a1
remove crate tokio-codec (seems to be part of tokio now)
2019-01-20 14:28:06 +01:00
Dietmar Maurer
8eeef8492a
tools/wrapped_reader_stream.rs: allocate Vec<u8> for buffer
2019-01-20 11:01:18 +01:00
Dietmar Maurer
ff8bdf3b64
tools/wrapped_reader_stream.rs: only implement Stream (without AsyncRead)
...
Looks more efficent ...
2019-01-20 10:41:21 +01:00
Dietmar Maurer
8e89d9cafe
tools/wrapped_reader_stream.rs: use tokio_threadpool::blocking
2019-01-20 10:28:51 +01:00
Dietmar Maurer
ce7ba139f1
catar: improve bash completion
2019-01-20 09:55:20 +01:00
Dietmar Maurer
29f34b8e1a
/api3/admin/datastore.rs: add missing subdir to directory index
2019-01-20 09:47:32 +01:00
Dietmar Maurer
0b05fd5830
api3/admin/datastore/catar.rs: implement download
2019-01-20 09:39:32 +01:00
Dietmar Maurer
7f0d67cf79
tools/wrapped_reader_stream.rs: new helper class
...
Will use that to download catar files.
2019-01-20 09:38:28 +01:00
Dietmar Maurer
6a4c091616
api3/admin/datastore/catar.rs: start impl. download
2019-01-19 17:27:02 +01:00
Dietmar Maurer
50cfb695ae
api/router.rs: rename ApiUploadMethod to ApiAsyncMethod
...
We can use this for uploads and downloads ...
2019-01-19 16:42:43 +01:00
Dietmar Maurer
379ea0edb6
Revert "/api/schema.rs: implement Schema::Option"
...
This reverts commit 0a35462c1e
.
I am not sure this add much value, and the old approach needs
less memory. If we really need single optional values, we can still
implement such Option while keeping the hash based approach...
2019-01-19 12:53:32 +01:00
Dietmar Maurer
80f069656d
Revert "api/schema.rs: avoid Option(Option( nesting"
...
This reverts commit ffdac1af01
.
2019-01-19 12:52:51 +01:00
Dietmar Maurer
ca3d861513
remove unused type PropertyMap
2019-01-18 17:54:06 +01:00
Dietmar Maurer
ffdac1af01
api/schema.rs: avoid Option(Option( nesting
2019-01-18 17:49:19 +01:00
Dietmar Maurer
0a35462c1e
/api/schema.rs: implement Schema::Option
2019-01-18 17:40:37 +01:00
Dietmar Maurer
728797d0c1
reduce compiler warnings
2019-01-18 16:50:15 +01:00
Dietmar Maurer
383e857749
move complete_file_name() helper into tools.rs
2019-01-18 13:42:52 +01:00
Dietmar Maurer
73e844dbfe
bin/proxmox-backup-client.rs: improve file name completion
2019-01-18 13:40:05 +01:00
Dietmar Maurer
95cea65b04
backup/datastore.rs: list all index files using walkdir crate
2019-01-18 12:24:58 +01:00
Dietmar Maurer
ff3d3100d4
backup/datastore.rs: try to create useful directory layout
...
store/type/ID/TIME/xxx.yyy
2019-01-18 12:01:37 +01:00
Dietmar Maurer
0d38dcb43c
cleanup nodename()
2019-01-18 10:13:45 +01:00
Dietmar Maurer
74a69302a7
tools.rs: new helper nodename()
2019-01-18 09:58:15 +01:00
Dietmar Maurer
5a778d92b3
api3/admin/datastore.rs: rename parameter "name" to "store"
2019-01-18 08:33:11 +01:00
Dietmar Maurer
244d9b17a8
bin/proxmox-backup-client.rs: implement file name completions - first try ...
2019-01-17 14:24:20 +01:00
Dietmar Maurer
83bdac1e3b
api3/admin/datastore/upload_catar.rs: verify content type ("application/x-proxmox-backup-catar")
2019-01-17 12:43:29 +01:00
Dietmar Maurer
06aeb76a5c
client/catar_backup_stream.rs: remove debug code
2019-01-17 12:16:36 +01:00
Dietmar Maurer
0fe5d605e7
tools.rs: new helper required_string_param()
...
To extract parameters from Value.
2019-01-17 12:14:02 +01:00
Dietmar Maurer
0ee0ad5bf3
api/router.rs: return Result in upload handler
2019-01-17 12:03:38 +01:00
Dietmar Maurer
23bb8780d4
proxmox-backup-client: implement upload to server
2019-01-17 11:38:22 +01:00
Dietmar Maurer
597641fdd5
client/http_client.rs: new helper class
2019-01-17 11:38:03 +01:00
Dietmar Maurer
264f52cf70
api3/admin/datastore.rs: change api patch for upload_catar
2019-01-17 09:16:50 +01:00
Wolfgang Bumiller
37b98249f4
upload: don't return NotReady
...
We need to kick (poll) the stream's next element, so simply
reiterate, the try_read!() will then return NotReady from
the next stream item for us.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 15:17:26 +01:00
Wolfgang Bumiller
1dcd52e5df
cleanup previous commit...
...
This kind of notation is weird...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 14:19:13 +01:00
Wolfgang Bumiller
6a50b77e72
listen on :: instead of 0.0.0.0
...
Since this supports both ipv6 & ipv4 unless v6only is
explicitly enabled.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 14:18:16 +01:00
Dietmar Maurer
cf16af2ab3
server/rest.rs: correctly pass query/url parameters
2019-01-16 13:58:36 +01:00
Dietmar Maurer
2085142ed4
reduce compiler warnings
2019-01-16 10:19:49 +01:00
Dietmar Maurer
e8edbbd49c
client/catar_backup_stream.rs: new helper for catar uploads to server
2019-01-16 10:15:39 +01:00
Dietmar Maurer
150f1bd8f6
backup/datastore.rs: use Arc for chunk_store references
2019-01-15 12:36:16 +01:00
Dietmar Maurer
1629d2ad7b
api3/admin/datastore/upload_catar.rs: implement upload future
2019-01-15 11:38:26 +01:00
Dietmar Maurer
7e21da6e23
api/router.rs: allow different types of api methods
...
Added a prototype for file/backup uploads.
2019-01-14 12:26:04 +01:00
Dietmar Maurer
b41d1aab8c
catar: define all feature flags
2019-01-12 16:49:02 +01:00
Dietmar Maurer
4c0fd4871e
catar/encoder.rs: allow to pass list of devices
...
We can use this to implement the --one-file-system like options. This
is a bit more convenient, because this way we can select to include
multiple devices (which is required by container backups).
2019-01-12 15:43:20 +01:00
Dietmar Maurer
a8a0495612
catar/encoder.rs: disable backup of virtual file systems
2019-01-12 11:56:53 +01:00
Dietmar Maurer
d7bfac8605
catar/encoder.rs: only test fat attribute on MSDOS and FUSE file systems
2019-01-12 10:28:26 +01:00
Dietmar Maurer
13d98013b2
/catar/encoder.rs: do not store data from virtual file systems
2019-01-12 10:20:08 +01:00
Dietmar Maurer
80881f60ef
catar/encoder.rs: add helpers to test file system type
2019-01-12 09:18:21 +01:00
Dietmar Maurer
1c4804cf67
catar/encoder.rs: pass magic fs number to encode_dir
2019-01-12 08:51:44 +01:00
Dietmar Maurer
dce0ebd505
catar/encoder.rs: use O_DIRECTORY to open dirs
2019-01-11 16:53:17 +01:00
Dietmar Maurer
490683ec93
catar: encode/decode sockets and fifos
2019-01-11 13:26:05 +01:00
Dietmar Maurer
20e2043a08
catar/decoder.rs: corrently pass mode to mknodat
2019-01-11 13:12:55 +01:00
Dietmar Maurer
a7e3713122
catar: encode/decode devices
2019-01-11 12:22:00 +01:00
Dietmar Maurer
1bdb3130f7
catar/format_definition.rs: add more feature flags
2019-01-11 10:44:57 +01:00
Dietmar Maurer
5c76c2f331
catar/encoder.rs: store fat file system attributes
2019-01-11 10:18:22 +01:00
Dietmar Maurer
8c1dfa6c72
catar/encoder.rs: cleanups
2019-01-11 10:01:51 +01:00
Dietmar Maurer
4f6892eb74
catar/encoder.rs: encode linux fs attributes, cleanup encoder
...
Try to avoid duplicate stat calls (assume file tree is stable during backup).
2019-01-11 09:20:10 +01:00
Dietmar Maurer
ddbdf80d51
avoid compiler warnings
2019-01-11 08:41:33 +01:00
Dietmar Maurer
373ef4a504
backup/archive_index.rs: correctly check for errors
2019-01-10 11:19:54 +01:00
Dietmar Maurer
23f68e53b4
catar/decoder.rs: restore mode, uid, gid and mtime
2019-01-10 09:02:50 +01:00
Dietmar Maurer
25f60394d2
catar/decoder.rs: avoid slashes in file names
2019-01-10 09:02:50 +01:00
Dietmar Maurer
9b1bb5a277
catar/decoder.rs: impl restore dirs, files, and symlinks
2019-01-10 09:01:36 +01:00
Wolfgang Bumiller
1ed86a0bae
rest: rename utf-8-checked 'bytes' to 'utf8'
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-08 14:22:43 +01:00
Wolfgang Bumiller
9fec7ae593
rest: don't copy the body
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-08 14:21:54 +01:00
Wolfgang Bumiller
d9fbcc4631
tools/timer: fixup doc formatting
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-08 14:11:10 +01:00
Dietmar Maurer
676bf71ad4
catar/decoder.rs: restore_sequential - read payload
2019-01-08 09:26:07 +01:00
Dietmar Maurer
c6a0d3fc6b
catar/decoder.rs: improve restore_sequential
2019-01-07 19:18:59 +01:00
Dietmar Maurer
b6ebfb8dab
catar/decoder.rs: start implementing sequential decoder, cleanups
...
And use [derive(Endian)] for endian conversion.
2019-01-07 19:07:03 +01:00
Dietmar Maurer
02c7d8e522
catar: implement create
2019-01-07 13:25:41 +01:00
Dietmar Maurer
e86c49247e
bin/catar.rs: add list command
2019-01-06 17:42:23 +01:00
Dietmar Maurer
3d8c24ecf8
catar/decoder.rs: start implementation
2019-01-06 17:27:57 +01:00
Dietmar Maurer
4b864ad436
format_definition.rs: remove wrong docs
2019-01-06 17:27:22 +01:00
Dietmar Maurer
4624fe29cc
impl Read for BufferedArchiveReader
2019-01-06 10:04:45 +01:00
Dietmar Maurer
343370504b
impl Seek for BufferedArchiveReader
2019-01-06 09:35:39 +01:00
Dietmar Maurer
318564ac03
buffered_read: return empty buffer on eof
2019-01-06 09:17:28 +01:00
Dietmar Maurer
0a72e26704
define and use BufferedReader trait
2019-01-05 17:28:20 +01:00
Dietmar Maurer
1006acd626
remove dead code
2019-01-05 16:55:21 +01:00
Dietmar Maurer
51b499db74
tools.rs: improve docs
2019-01-05 16:53:28 +01:00
Dietmar Maurer
39c6bd86cc
backup/archive_index.rs: implement BufferedArchiveReader
...
Implement relativly fast random read using binary search.
2019-01-05 14:47:56 +01:00
Dietmar Maurer
060c4811a0
backup/chunk_store.rs: resize buffer by powers of two
2019-01-05 11:34:49 +01:00
Dietmar Maurer
9409255aae
catar/encoder.rs: code cleanup
...
Note: Rust Vec resize uses len(), not capacity()!
2019-01-04 17:23:01 +01:00
Dietmar Maurer
df9973e8f5
backup/chunk_store.rs: avoid allocation in read_chunk
2019-01-04 17:16:56 +01:00
Dietmar Maurer
7394ca3e95
white space cleanups
2019-01-04 12:51:43 +01:00
Dietmar Maurer
96df2fb44b
backup/chunk_store.rs: implement read_chunk()
2019-01-04 12:50:54 +01:00
Dietmar Maurer
691c89a0fb
proxmox-backup-manager: add garbage collection cli
2019-01-04 11:34:32 +01:00
Dietmar Maurer
07b4694a33
cleanup crate names
2019-01-04 10:49:52 +01:00
Dietmar Maurer
4bdba5f4f1
rename backup-client to proxmox-backup-client
2019-01-04 10:41:37 +01:00
Dietmar Maurer
a689f6df0e
rename pbs to proxmox-backup-manager
2019-01-04 10:39:32 +01:00
Dietmar Maurer
bc616633d5
backup/chunk_store.rs: only use one directory level
2019-01-04 10:35:22 +01:00
Dietmar Maurer
030cc2a903
move admin API to /admin/ subfolder
2019-01-04 10:32:27 +01:00
Dietmar Maurer
b663789bdb
backup/archive_index.rs: pass correct size to unmap
2019-01-04 09:28:41 +01:00
Dietmar Maurer
48d0d35688
backup/image_index.rs: use little endian
2019-01-04 08:45:45 +01:00
Wolfgang Bumiller
4968bc3ac0
cli::command: wrap usage errors in a UsageError
...
So we can distinguish them and show usage output
conditionally.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-03 16:08:51 +01:00
Dietmar Maurer
c7fa487250
catar/encoder.rs: make sure we write all data with write_all()
2019-01-03 15:47:32 +01:00
Dietmar Maurer
050f8a6355
chunker.rs: make sure chunk_size_avg is a power of two
2019-01-03 15:10:54 +01:00
Dietmar Maurer
ea4ea34baf
improve chunker speed (avoid modulo)
2019-01-03 14:35:19 +01:00
Dietmar Maurer
3329ae8c2e
add benchmark for chunker
...
We get about 300MB/s - so this is still not fast enough
Note: This is quite slow with debug target, so please compile with --release
2019-01-02 19:18:53 +01:00
Dietmar Maurer
0b8e75ed16
image_index.rs: verify file size
2019-01-02 18:14:02 +01:00
Dietmar Maurer
77703d95aa
implement garbage collection for .aidx files
2019-01-02 14:27:04 +01:00
Dietmar Maurer
a360f6fa2d
image_index.rs: verify header magic/version
2019-01-02 13:13:13 +01:00
Dietmar Maurer
44b3f62b42
fix typo
2019-01-02 12:56:04 +01:00
Dietmar Maurer
594fa52016
disable debug code
2019-01-02 12:55:18 +01:00
Dietmar Maurer
5032b57b44
archive_index.rs: correctly write .aidx file
2019-01-02 12:54:40 +01:00
Dietmar Maurer
580dc84bac
image_index.rs: print stats
2019-01-02 12:53:49 +01:00
Dietmar Maurer
5e7a09be0d
src/backup/archive_index.rs: use close() instead of flush()
...
Also pass a reference to the encoder.
2019-01-02 11:02:56 +01:00
Dietmar Maurer
94a882e900
src/backup/archive_index.rs: implement flush()
2018-12-31 18:01:07 +01:00
Dietmar Maurer
0433db1949
src/backup/archive_index.rs: first try
2018-12-31 17:30:08 +01:00
Dietmar Maurer
cbdd8c54ae
create backup mod in backup.rs, improve docu
2018-12-31 16:08:04 +01:00
Dietmar Maurer
cb4412b18e
move chunker.rs to backup/
2018-12-31 15:46:16 +01:00
Dietmar Maurer
1c287cb19a
rc/catar/chunker.rs: first try
2018-12-31 13:01:06 +01:00
Dietmar Maurer
bf205f94c5
avoid compiler warnings
2018-12-31 10:54:25 +01:00
Dietmar Maurer
1b0dc9f680
src/catar/encoder.rs: correctly sort goodbye items by hash key
2018-12-31 10:11:28 +01:00
Dietmar Maurer
c60d34bdbf
new catar binary
...
currently used for debugging
2018-12-30 18:02:46 +01:00
Dietmar Maurer
e75eac73ca
fix doc syntax
2018-12-30 17:43:53 +01:00
Dietmar Maurer
0866748de6
catar cleanups ...
2018-12-30 17:32:52 +01:00
Dietmar Maurer
6cd28d200e
binary_search_tree.rs: fix docs
2018-12-30 15:34:43 +01:00
Dietmar Maurer
389e562524
src/catar/encoder.rs: auto-resize file_copy_buffer, limit number of dirs
2018-12-30 14:09:59 +01:00
Dietmar Maurer
4fa71e0573
improve catar docs
2018-12-30 13:47:27 +01:00
Dietmar Maurer
48147efd12
add a comment about posible improvements
2018-12-29 19:43:25 +01:00
Dietmar Maurer
0b78833d8e
src/catar/binary_search_tree.rs: add regression tests and fix one bug
2018-12-29 18:32:03 +01:00
Dietmar Maurer
b17d7149d2
src/catar/binary_search_tree.rs: improve docu
2018-12-29 17:38:50 +01:00
Dietmar Maurer
985567fb45
src/catar/encoder.rs: cleanup, factor out write_goodbye_table
2018-12-29 17:26:32 +01:00
Dietmar Maurer
95bd5dfec7
src/catar/encoder.rs: correctly sort goodby items
2018-12-29 17:00:48 +01:00
Dietmar Maurer
46b6fbd6ae
use external crate siphasher
...
Because std:#️⃣ :SipHasher is deprecated.
2018-12-28 19:50:07 +01:00
Dietmar Maurer
f0f3029e26
src/catar/encoder.rs: fix bug in file copy
2018-12-28 15:02:42 +01:00
Dietmar Maurer
d05f93215e
src/catar/encoder.rs: fix symlinks
2018-12-28 15:02:42 +01:00
Wolfgang Bumiller
7ebb173352
getopt: cleanup: add trailing commas
...
It is customary in rust to always use trailing commas. (also
suggested by rustfmt)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Wolfgang Bumiller
2767c5d39b
getopt: cleanup: don't condense 'if' statements too much
...
In a language which enforces curly braces this looks weird
and rustfmt doesn't like it.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Wolfgang Bumiller
7d95c10da0
getopt: whitespace cleanup
...
to make rustfmt happy
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Dietmar Maurer
a0cc09b5f0
src/catar/encoder.rs: fix binary format, write goodby table
...
We still ned to sort the table (BST) ...
2018-12-28 14:27:00 +01:00
Wolfgang Bumiller
03fb895197
getopt: condense nested match to reduce indentation
...
The `match value` statement is the only thing covering the
entire RawArgument::Option case. `rustfmt` suggests this
more condensed way of writing this case.
See the `git diff -w` of this patch:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 9755af2..4db4579 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,8 +72,7 @@ pub fn parse_arguments<T: AsRef<str>>(
| RawArgument::Separator => {
| break;
| }
|- RawArgument::Option { name, value } => {
|- match value {
|+ RawArgument::Option { name, value } => match value {
| None => {
| let mut want_bool = false;
| let mut can_default = false;
|@@ -125,7 +124,6 @@ pub fn parse_arguments<T: AsRef<str>>(
| data.push((name, v));
| }
| }
|- }
| RawArgument::Argument { value } => {
| rest.push(value);
| }
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:16:58 +01:00
Wolfgang Bumiller
4d0ea9978b
getopt: let parse_arguments() take a slice of AsRef<str>
...
We don't need the content to be owned strings, and we don't
need the list to be a Vec, since we only care about being
able to iterate through and copy out portions of the strings
we need, so take an &[T] where T: AsRef<str>.
This avoids .iter().map(to_string).collect() before calling
parse_arguments().
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:05:05 +01:00
Wolfgang Bumiller
2482c095b1
getopt: remove skip logic
...
The 'skip' variable was set to indicate that the "rest of
the args" is to be copied into the 'rest' vec. We can do
this directly and avoid the 'if' case in the loop
altogether.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:59:44 +01:00
Wolfgang Bumiller
99da3a073d
getopt: indentation reduction
...
See the `git diff -w` output:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 6548a01..517cc37 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,7 +72,10 @@ pub fn parse_arguments(
| while pos < args.len() {
| if skip {
| rest.push(args[pos].clone());
|- } else {
|+ pos += 1;
|+ continue;
|+ }
|+
| match parse_argument(&args[pos]) {
| RawArgument::Separator => {
| skip = true;
|@@ -135,7 +138,6 @@ pub fn parse_arguments(
| rest.push(value);
| }
| }
|- }
|
| pos += 1;
| }
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:54:20 +01:00
Wolfgang Bumiller
ad8a98f7e4
parse_arguments: work with utf8 bytes and reduce indentation
...
We are only caring about '=' and '-' which are single-byte
codepoints, so there's no reason not to work on a byte
slice.
Also, some indentation reduction:
Transform
if (a) {
...
return A;
}
return B;
into
if (!a)
return B;
return A;
and
if (a)
if (b)
foo()
into
if (a && b)
return;
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:47:29 +01:00
Dietmar Maurer
248c17af39
src/catar/encoder.rs: write filenames
2018-12-28 11:48:47 +01:00
Dietmar Maurer
2e4ae0e239
src/catar/encoder.rs: write file data
2018-12-28 10:44:12 +01:00
Dietmar Maurer
3192ae968c
src/catar/encoder.rs: write entry and symlink data
2018-12-28 09:55:26 +01:00
Dietmar Maurer
50ea43962e
make map_struct functions public
2018-12-28 08:04:46 +01:00
Dietmar Maurer
d2b03f2397
catar/encoder.rs: improve error handling
2018-12-28 07:45:15 +01:00
Dietmar Maurer
0ff559990c
cleanups, avoid compiler warnings
2018-12-28 07:14:12 +01:00
Dietmar Maurer
45281d4927
catar/encoder: detect symlink and regular files
2018-12-27 14:24:31 +01:00
Dietmar Maurer
fb8365b79f
catar/encoder.rs: first try
2018-12-27 13:15:47 +01:00
Wolfgang Bumiller
fba3437f1e
timer: setup_timeout_handler cannot fail
...
and if it does it panics anyway
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-27 10:36:28 +01:00
Dietmar Maurer
bcd879cfb3
backup-client: allow to specify directories (prepare for catar backup)
2018-12-27 10:11:11 +01:00
Dietmar Maurer
b62b6cadf4
start implementing catar
2018-12-27 09:22:23 +01:00
Dietmar Maurer
dc3de618ed
new helper map_struct and map_struct_mut
2018-12-27 09:20:17 +01:00
Dietmar Maurer
a198d74fc0
cleanup: reduce compiler warnings
2018-12-25 13:29:27 +01:00
Dietmar Maurer
991de6007b
remove outdated comment
2018-12-25 13:24:50 +01:00
Dietmar Maurer
7ee2aa1b94
touch_chunk: use libc::lutimensat
2018-12-25 12:27:25 +01:00
Dietmar Maurer
2c32fdde86
move lookup_datastore() to backup/datastore.rs
2018-12-22 17:37:25 +01:00
Dietmar Maurer
64e53b2835
gather usage statistics during garbage collection
2018-12-22 16:58:16 +01:00
Dietmar Maurer
28b96b56e1
open_file_locked: improve error message
2018-12-22 15:59:55 +01:00
Dietmar Maurer
176e4af964
sweep_used_chunks: print percentage
2018-12-22 15:39:05 +01:00
Dietmar Maurer
7b2b40a893
use openssl for faster hashing
2018-12-22 14:31:59 +01:00
Dietmar Maurer
e95950e40a
chunk_store: reduce number of directories
...
Else, scans everything takes too long ...
2018-12-22 14:04:05 +01:00
Dietmar Maurer
15e9b4ed60
src/api3/datastore.rs: new file
2018-12-21 13:38:41 +01:00
Dietmar Maurer
3c140f60cd
use return value (avoid compiler warning)
2018-12-21 12:44:20 +01:00
Dietmar Maurer
03e4753d8e
fix mutability for chunk store
2018-12-21 12:15:26 +01:00
Dietmar Maurer
2d9d143a8f
backup-client: add optional chunk-size parameter
2018-12-21 11:18:08 +01:00
Dietmar Maurer
f0819fe5a5
backup-client: do not start garbage collection after backup
2018-12-21 09:24:55 +01:00
Dietmar Maurer
d62e6e2264
backup-client: use 1M chunks, make chunk_size configurable
2018-12-21 08:36:57 +01:00
Dietmar Maurer
c34eb16651
backup-clinet: specify target file name
2018-12-20 14:09:31 +01:00
Dietmar Maurer
c2d9b3914a
add test code to access static global state
2018-12-20 14:03:42 +01:00
Dietmar Maurer
fe0e04c69c
use proxmox-backup as package name
2018-12-20 10:45:14 +01:00
Dietmar Maurer
f7dd683b46
fix tests
2018-12-19 13:54:22 +01:00
Dietmar Maurer
277fc5a3ce
try to improve error messages
2018-12-19 13:40:26 +01:00
Dietmar Maurer
eae8aa3aa9
cleanups: avoid compiler warnings
2018-12-19 12:49:23 +01:00
Dietmar Maurer
6c20a13d3c
cleanup - avoid too much indentation
2018-12-19 12:40:26 +01:00
Dietmar Maurer
1c43c56b06
cleanup
2018-12-19 12:30:37 +01:00
Dietmar Maurer
2bf5f6b220
ChunkStore::sweep_used_chunks - use openat to speedup directory walks
2018-12-19 12:20:01 +01:00
Wolfgang Bumiller
1628a4c731
use timers with a signal for file locking
...
* rename lock_file -> open_file_locked,
* add lock_file as a function working on already-opened
files
* change timeout types to std::time::Duration
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-19 11:23:00 +01:00
Wolfgang Bumiller
8cf6e764ad
add tools::timer module
...
This wraps POSIX timers via timer_create(2).
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-19 11:23:00 +01:00
Dietmar Maurer
365bb90f17
move lock_file to tools.rs
2018-12-19 10:02:24 +01:00
Dietmar Maurer
08481a0b60
impl sweep_used_chunks, first try
2018-12-19 09:51:33 +01:00
Dietmar Maurer
6ea3a0b7fc
move sweep_used_chunks to chunk_store
2018-12-18 11:18:55 +01:00
Dietmar Maurer
3d5c11e5d2
start impl garbage collection
2018-12-18 11:06:03 +01:00
Dietmar Maurer
529de6c7a3
start implementing DataStore
2018-12-17 13:00:39 +01:00
Dietmar Maurer
4818c8b6f7
start implementing ImageIndexReader
2018-12-16 14:44:44 +01:00
Dietmar Maurer
a27a3ee4ba
avoid compiler warnings
2018-12-16 13:57:59 +01:00
Dietmar Maurer
af3e7d7551
fix chunk store file paths
2018-12-16 13:52:16 +01:00
Dietmar Maurer
0cd9d4206f
improve error messages
2018-12-16 13:43:19 +01:00
Dietmar Maurer
4fbb72a8b4
image_index.rs: implement atomic write/rename
2018-12-16 13:39:21 +01:00
Dietmar Maurer
798881a68a
rebane ImageIndex to ImageIndexWriter
2018-12-16 11:51:16 +01:00
Dietmar Maurer
5e5b7f1c2d
use u64 to store time
2018-12-16 11:48:03 +01:00
Dietmar Maurer
d13e3745b3
image_index.rs: write header
2018-12-15 17:05:49 +01:00
Dietmar Maurer
606ce64bb0
image_index.rs:write idx of chunks (use mmap)
2018-12-15 14:51:05 +01:00
Dietmar Maurer
f8838fe913
backup-client: add bash completion for datastore names
2018-12-15 11:24:39 +01:00
Dietmar Maurer
5f0c2d5652
remove debug println
2018-12-15 11:16:27 +01:00
Dietmar Maurer
43eeef28d1
move file_chunker to tools
2018-12-15 11:14:41 +01:00
Dietmar Maurer
48e8689505
file_chunker: use Read trait
2018-12-15 11:08:49 +01:00
Dietmar Maurer
fc8b19368a
backup-client: pass chunk_size as parameter to file_chunker
2018-12-15 10:58:48 +01:00
root
a914a77448
backup-client: read file by chunks
2018-12-14 13:39:41 +01:00
root
ff5d37074d
bin/backup-client.rs: nbew tool to play with chunk store
2018-12-14 08:28:56 +01:00
root
15b64d4606
api3/config/datastore.rs: create datastore files
2018-12-13 14:41:14 +01:00
Dietmar Maurer
7f4e639b37
improve completion
2018-12-12 13:00:58 +01:00
Dietmar Maurer
e3d40bb8cd
complete_datastore_name: write more compact code
2018-12-12 12:41:59 +01:00
Dietmar Maurer
30d2e99c77
implement completion functions
2018-12-12 12:19:26 +01:00
Dietmar Maurer
91643d9012
section_config.rs: remove debug output
2018-12-12 12:19:26 +01:00
Dietmar Maurer
d64d80d20a
add comment
2018-12-12 11:23:04 +01:00
Dietmar Maurer
5ba6968922
add comment to make intensions more clear
2018-12-12 11:21:00 +01:00
Dietmar Maurer
ca9caffa23
cleanup parameter types
2018-12-12 10:37:03 +01:00
Dietmar Maurer
38555b5072
implement completion for enums
2018-12-12 08:33:50 +01:00
Dietmar Maurer
2f0258959b
improve completion
2018-12-12 08:18:38 +01:00
Dietmar Maurer
f46403cc65
improve bash completion
2018-12-12 07:51:43 +01:00
Dietmar Maurer
b6e8dd39aa
impl bash completion: first try
2018-12-11 14:21:05 +01:00
Dietmar Maurer
c21de379ed
install bash completion scripts
...
Completion itself is not implemented.
2018-12-11 12:54:10 +01:00
Dietmar Maurer
6460764dbb
cli/command.rs: add new type CliCommandMap
2018-12-11 11:31:36 +01:00
Dietmar Maurer
bf7f103944
implement builder pattern for CliCommand
2018-12-11 11:12:13 +01:00
Dietmar Maurer
baed30b702
cli/command.rs: implement prefix match
2018-12-10 18:13:55 +01:00
Dietmar Maurer
8f62336b0f
impl From for CommandLineInterface
2018-12-10 13:51:10 +01:00
Dietmar Maurer
9f6ab1fc58
rename CmdDef to CommandLineInterface
2018-12-10 13:40:10 +01:00
Dietmar Maurer
b7329c8a1a
src/cli/command.rs: move code into separate file
2018-12-10 13:36:52 +01:00
Dietmar Maurer
211fabd795
try to support nested cli command definitions
2018-12-10 13:28:38 +01:00
Dietmar Maurer
34d3ba52eb
api3/config/datastore.rs: implement delete
2018-12-09 16:52:32 +01:00
Dietmar Maurer
1a7bc3dd5e
file_set_contents: correctly use perm parameter
2018-12-09 16:41:54 +01:00
Dietmar Maurer
f12f8ff1a6
implement tools::file_set_contents
2018-12-09 16:37:48 +01:00
Dietmar Maurer
652c11900d
api3/config/datastore.rs: impl create
2018-12-09 12:51:31 +01:00
Dietmar Maurer
ea0b8b6ec0
bin/pbs-datastore.rs: start implementing cli commands
2018-12-09 11:59:32 +01:00
Dietmar Maurer
5b34c2607d
rename entry to item
2018-12-09 10:25:56 +01:00
Dietmar Maurer
b65eaac62d
api3/config/datastore.rs: return real data
2018-12-09 10:23:19 +01:00
Dietmar Maurer
6349ed60b4
config/datastore.rs - open file for reading
2018-12-09 10:22:39 +01:00
Dietmar Maurer
ce55dbbcc6
simplify code
2018-12-09 09:42:17 +01:00
Dietmar Maurer
461e62fcdf
remove some compiler warnings
2018-12-08 14:55:54 +01:00
Dietmar Maurer
567713b4c3
rename data_store to datastore
2018-12-08 14:51:08 +01:00
Dietmar Maurer
6ce50400c5
cleanup api3 structure
2018-12-08 14:44:55 +01:00
Dietmar Maurer
678d72df6b
add data_store configuration
2018-12-08 13:58:45 +01:00
Dietmar Maurer
391a2e43ff
chunk_store.insert_chunk: return information about existing chunks
2018-12-08 11:25:11 +01:00
Dietmar Maurer
12bb93b3ad
code cleanup
2018-12-08 11:20:02 +01:00
Dietmar Maurer
022fb421b2
lock with timeout
...
clumsy, but I have node idea how to make it better
2018-12-08 10:57:09 +01:00
Dietmar Maurer
2989f6bf20
improve error messages
2018-12-08 08:38:42 +01:00
Dietmar Maurer
b8d4766a4d
code cleanup
2018-12-08 08:21:20 +01:00
Dietmar Maurer
45773720ab
depend on nix, use flock
...
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2018-12-07 18:14:07 +01:00
Dietmar Maurer
c5d82e5f34
chunk_store: use mutex
2018-12-07 16:12:45 +01:00
Dietmar Maurer
128b37feea
impl insert_chunk
2018-12-07 14:44:56 +01:00
Dietmar Maurer
35cf5daa53
start experiments with simple chunk store
2018-12-07 10:51:53 +01:00
Dietmar Maurer
b22751b20c
getopts.rs: fix test
2018-12-07 09:02:24 +01:00
Dietmar Maurer
a4b7c3f2df
more cleanups
2018-12-06 10:41:57 +01:00
Dietmar Maurer
d3369f819d
code cleanup
2018-12-06 10:23:45 +01:00
Dietmar Maurer
c578fcd9e2
simplify formatter code
2018-12-05 18:22:56 +01:00
Dietmar Maurer
4b26195c6a
extract version from environment
2018-12-05 14:40:45 +01:00
Dietmar Maurer
95b492add8
api: add /version
2018-12-05 14:03:15 +01:00
Dietmar Maurer
1571873d7b
add output formatter
2018-12-05 12:43:22 +01:00
Dietmar Maurer
083418619e
move src/api/server.rs -> src/server/rest.rs
2018-12-05 10:16:23 +01:00