Commit Graph

637 Commits

Author SHA1 Message Date
Dietmar Maurer 4b2cdeb9a6 implement relead_timezone flag 2019-02-01 10:04:46 +01:00
Dietmar Maurer c82bc1a1f9 depend on pam-auth, use pam for root login
Please use username "root@pam" for login via GUI.
2019-02-01 09:30:50 +01:00
Dietmar Maurer a154a8e8a4 delay unauthorized request (rate limit) 2019-01-31 14:34:21 +01:00
Dietmar Maurer 5996577ab6 move http error class to router.rs 2019-01-31 13:22:30 +01:00
Dietmar Maurer b9903d6331 server/rest.rs: verify auth cookie 2019-01-31 12:22:00 +01:00
Dietmar Maurer f484eed3c2 Utils.js: fix cookie handling
Use unsecure cookie foör testing.
2019-01-31 10:08:08 +01:00
Dietmar Maurer 9f49fe1d5d avoid compiler warnings 2019-01-30 18:25:37 +01:00
Dietmar Maurer 34f956bc25 api2/access.rs: add ticket api 2019-01-30 15:16:10 +01:00
Wolfgang Bumiller 1bf446a33e fix timer_delete call
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-30 12:04:20 +01:00
Dietmar Maurer cf671670a0 auth_helpers.rs: add timestamp to csrf token 2019-01-29 17:50:03 +01:00
Dietmar Maurer 1e76cbc6a2 auth_helpers.rs: implement assemble_csrf_prevention_token 2019-01-29 17:41:45 +01:00
Dietmar Maurer d01e2420f7 load auth keys on startup 2019-01-29 17:21:58 +01:00
Dietmar Maurer 6c30068ebf auth_helpers.rs: split code into separate file 2019-01-29 16:55:49 +01:00
Dietmar Maurer 39a90ca6c5 bin/proxmox-backup-api.rs: gererate secret for csrf token 2019-01-29 13:15:33 +01:00
Dietmar Maurer 26027a151e tools/ticket.rs: remove debug statements 2019-01-29 13:04:01 +01:00
Dietmar Maurer 8d04280b29 tools/ticket.rs: add basic ticket support 2019-01-29 12:59:07 +01:00
Dietmar Maurer c8f3f9b1af server/rest.rs: simplify proxy code
Only pass neccessary parameters.
2019-01-28 18:22:16 +01:00
Dietmar Maurer a859fa95a9 add protected flag for some api methods 2019-01-28 18:16:47 +01:00
Dietmar Maurer a3da38ddfd server/rest.rs: implement proxy_sync_api_request 2019-01-28 18:06:42 +01:00
Dietmar Maurer f120483353 server/rest.rs: add proxy_sync_api_request() dummy 2019-01-28 17:30:39 +01:00
Dietmar Maurer 4e4df8e2f0 api/router.rs: add new 'protected' flag to ApiMethod 2019-01-28 17:18:42 +01:00
Dietmar Maurer 5bb3398171 server/formatter.rs: fix extjs error format 2019-01-28 13:44:48 +01:00
Dietmar Maurer 02c7a75520 src/bin/proxmox-backup-proxy.rs: implement unpriviledged server
We want to run the public server as user www-data. Requests needing
root priviledges needs to be proxied to the proxmox-backup.service, which
now listens to 127.0.0.1:82.
2019-01-28 13:29:58 +01:00
Dietmar Maurer 48849593ee api2/node/services.rs: implement service commands 2019-01-27 12:40:31 +01:00
Dietmar Maurer d7d23785f0 RpcEnvironment: implement set_user() and get_user() 2019-01-27 10:52:26 +01:00
Dietmar Maurer 162b979394 RpcEnvironment: add environment type enum RpcEnvironmentType 2019-01-27 10:33:42 +01:00
Dietmar Maurer e82dad9700 also pass rpcenv to async handlers 2019-01-27 10:18:52 +01:00
Dietmar Maurer d2ab5f19e2 api2/node/services.rs: start services api 2019-01-26 17:54:18 +01:00
Dietmar Maurer 0f253593c6 move rpc environment implementation to separate files 2019-01-26 15:08:02 +01:00
Dietmar Maurer 6049b71f41 api: pass RpcEnvirnment to api handlers 2019-01-26 14:50:37 +01:00
Dietmar Maurer d96d82736d use crate log and syslog 2019-01-26 10:56:11 +01:00
Dietmar Maurer e182ab4a4a api2/node/syslog.rs: improve error handling 2019-01-26 10:23:52 +01:00
Dietmar Maurer 0441886842 api2/node/syslog.rs: implement dump journal 2019-01-25 18:20:51 +01:00
Dietmar Maurer 4f9a726833 api2/node/syslog.rs: add syslog api schema 2019-01-25 17:17:30 +01:00
Dietmar Maurer 1631c54fca tools::get_hardware_address - fix FILENAME, use file_get_contents() 2019-01-25 12:23:47 +01:00
Dietmar Maurer af2fddea4f api2/node/dns.rs: implement concurrent update protection 2019-01-25 11:38:59 +01:00
Dietmar Maurer de6b0721fa pi2/node/dns.rs: return file digest 2019-01-25 11:03:41 +01:00
Dietmar Maurer 2296860027 tools.rs: move digest_to_hex() into tools 2019-01-25 10:58:28 +01:00
Dietmar Maurer 53157ca6cf tools.rs: implement file_get_contents() 2019-01-25 10:44:40 +01:00
Dietmar Maurer 46b79b9ee5 api2/node/dns.rs: improve dns api 2019-01-25 10:15:32 +01:00
Dietmar Maurer 332dcc2277 backup/chunk_store.rs: fix test case - use absolute path 2019-01-25 10:14:25 +01:00
Dietmar Maurer 8f973f8181 api2/node/dns.rs: implement resolv.conf parser 2019-01-24 14:59:40 +01:00
Dietmar Maurer 784252dbc5 backup/datastore.rs: use single lazy_static block 2019-01-24 14:58:47 +01:00
Dietmar Maurer e6ffeb918c api2/node/time.rs: implement set timezone 2019-01-24 12:05:06 +01:00
Dietmar Maurer cd8b91f78c api2/node/time.rs: add return schema 2019-01-24 11:07:11 +01:00
Dietmar Maurer 447787ab7c tools.rs: implement file_read_firstline 2019-01-24 10:43:30 +01:00
Dietmar Maurer 0463602a79 api2/node/time.rs: implement read api 2019-01-23 15:14:14 +01:00
Dietmar Maurer b2b3485d5f src/api2/node: start node configuration api 2019-01-23 13:09:17 +01:00
Dietmar Maurer d15009c0ce server/rest.rs: fake login cookie 2019-01-23 12:49:10 +01:00
Dietmar Maurer 248cb51862 add missing api2.rs 2019-01-22 13:56:27 +01:00
Dietmar Maurer 7e13b2d67f api2/subscription.rs: add subscription api class 2019-01-22 12:50:19 +01:00
Dietmar Maurer 576e3bf252 rename api3 back to api2
There is no real need to change the path, so using api2 we can reuse
all helpers (like tools from proxmox widget toolkit).
2019-01-22 12:10:38 +01:00
Dietmar Maurer 1adb353d6f lient/http_client.rs: simplify code 2019-01-22 10:36:04 +01:00
Dietmar Maurer 41c039e143 bin/proxmox-backup-client.rs: implement list command 2019-01-21 18:58:14 +01:00
Dietmar Maurer 1fdb4c6ffa client/http_client.rs: implement get, share common code 2019-01-21 18:56:48 +01:00
Dietmar Maurer 83dbd80bd8 api3/admin/datastore.rs: implement list backups 2019-01-21 11:22:55 +01:00
Dietmar Maurer 7ca80246ea backup/datastore.rs: use DateTime<Utc> to store backup time 2019-01-21 10:51:52 +01:00
Dietmar Maurer be0084b001 white space fix 2019-01-21 10:32:07 +01:00
Dietmar Maurer e25736b4c3 backup/datastore.rs: implement list_backups 2019-01-20 18:02:27 +01:00
Dietmar Maurer cce1676a9e tools.rs: use FnMut for scandir callback 2019-01-20 17:49:11 +01:00
Dietmar Maurer 443f3743bb tools.rs: new scandir() helper 2019-01-20 17:31:43 +01:00
Dietmar Maurer 68469eebb4 backup/chunk_store.rs: verify base path is absolute 2019-01-20 16:49:22 +01:00
Dietmar Maurer 059ca7c3a1 remove crate tokio-codec (seems to be part of tokio now) 2019-01-20 14:28:06 +01:00
Dietmar Maurer 8eeef8492a tools/wrapped_reader_stream.rs: allocate Vec<u8> for buffer 2019-01-20 11:01:18 +01:00
Dietmar Maurer ff8bdf3b64 tools/wrapped_reader_stream.rs: only implement Stream (without AsyncRead)
Looks more efficent ...
2019-01-20 10:41:21 +01:00
Dietmar Maurer 8e89d9cafe tools/wrapped_reader_stream.rs: use tokio_threadpool::blocking 2019-01-20 10:28:51 +01:00
Dietmar Maurer ce7ba139f1 catar: improve bash completion 2019-01-20 09:55:20 +01:00
Dietmar Maurer 29f34b8e1a /api3/admin/datastore.rs: add missing subdir to directory index 2019-01-20 09:47:32 +01:00
Dietmar Maurer 0b05fd5830 api3/admin/datastore/catar.rs: implement download 2019-01-20 09:39:32 +01:00
Dietmar Maurer 7f0d67cf79 tools/wrapped_reader_stream.rs: new helper class
Will use that to download catar files.
2019-01-20 09:38:28 +01:00
Dietmar Maurer 6a4c091616 api3/admin/datastore/catar.rs: start impl. download 2019-01-19 17:27:02 +01:00
Dietmar Maurer 50cfb695ae api/router.rs: rename ApiUploadMethod to ApiAsyncMethod
We can use this for uploads and downloads ...
2019-01-19 16:42:43 +01:00
Dietmar Maurer 379ea0edb6 Revert "/api/schema.rs: implement Schema::Option"
This reverts commit 0a35462c1e.

I am not sure this add much value, and the old approach needs
less memory. If we really need single optional values, we can still
implement such Option while keeping the hash based approach...
2019-01-19 12:53:32 +01:00
Dietmar Maurer 80f069656d Revert "api/schema.rs: avoid Option(Option( nesting"
This reverts commit ffdac1af01.
2019-01-19 12:52:51 +01:00
Dietmar Maurer ca3d861513 remove unused type PropertyMap 2019-01-18 17:54:06 +01:00
Dietmar Maurer ffdac1af01 api/schema.rs: avoid Option(Option( nesting 2019-01-18 17:49:19 +01:00
Dietmar Maurer 0a35462c1e /api/schema.rs: implement Schema::Option 2019-01-18 17:40:37 +01:00
Dietmar Maurer 728797d0c1 reduce compiler warnings 2019-01-18 16:50:15 +01:00
Dietmar Maurer 383e857749 move complete_file_name() helper into tools.rs 2019-01-18 13:42:52 +01:00
Dietmar Maurer 73e844dbfe bin/proxmox-backup-client.rs: improve file name completion 2019-01-18 13:40:05 +01:00
Dietmar Maurer 95cea65b04 backup/datastore.rs: list all index files using walkdir crate 2019-01-18 12:24:58 +01:00
Dietmar Maurer ff3d3100d4 backup/datastore.rs: try to create useful directory layout
store/type/ID/TIME/xxx.yyy
2019-01-18 12:01:37 +01:00
Dietmar Maurer 0d38dcb43c cleanup nodename() 2019-01-18 10:13:45 +01:00
Dietmar Maurer 74a69302a7 tools.rs: new helper nodename() 2019-01-18 09:58:15 +01:00
Dietmar Maurer 5a778d92b3 api3/admin/datastore.rs: rename parameter "name" to "store" 2019-01-18 08:33:11 +01:00
Dietmar Maurer 244d9b17a8 bin/proxmox-backup-client.rs: implement file name completions - first try ... 2019-01-17 14:24:20 +01:00
Dietmar Maurer 83bdac1e3b api3/admin/datastore/upload_catar.rs: verify content type ("application/x-proxmox-backup-catar") 2019-01-17 12:43:29 +01:00
Dietmar Maurer 06aeb76a5c client/catar_backup_stream.rs: remove debug code 2019-01-17 12:16:36 +01:00
Dietmar Maurer 0fe5d605e7 tools.rs: new helper required_string_param()
To extract parameters from Value.
2019-01-17 12:14:02 +01:00
Dietmar Maurer 0ee0ad5bf3 api/router.rs: return Result in upload handler 2019-01-17 12:03:38 +01:00
Dietmar Maurer 23bb8780d4 proxmox-backup-client: implement upload to server 2019-01-17 11:38:22 +01:00
Dietmar Maurer 597641fdd5 client/http_client.rs: new helper class 2019-01-17 11:38:03 +01:00
Dietmar Maurer 264f52cf70 api3/admin/datastore.rs: change api patch for upload_catar 2019-01-17 09:16:50 +01:00
Wolfgang Bumiller 37b98249f4 upload: don't return NotReady
We need to kick (poll) the stream's next element, so simply
reiterate, the try_read!() will then return NotReady from
the next stream item for us.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 15:17:26 +01:00
Wolfgang Bumiller 1dcd52e5df cleanup previous commit...
This kind of notation is weird...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 14:19:13 +01:00
Wolfgang Bumiller 6a50b77e72 listen on :: instead of 0.0.0.0
Since this supports both ipv6 & ipv4 unless v6only is
explicitly enabled.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 14:18:16 +01:00
Dietmar Maurer cf16af2ab3 server/rest.rs: correctly pass query/url parameters 2019-01-16 13:58:36 +01:00
Dietmar Maurer 2085142ed4 reduce compiler warnings 2019-01-16 10:19:49 +01:00
Dietmar Maurer e8edbbd49c client/catar_backup_stream.rs: new helper for catar uploads to server 2019-01-16 10:15:39 +01:00
Dietmar Maurer 150f1bd8f6 backup/datastore.rs: use Arc for chunk_store references 2019-01-15 12:36:16 +01:00
Dietmar Maurer 1629d2ad7b api3/admin/datastore/upload_catar.rs: implement upload future 2019-01-15 11:38:26 +01:00
Dietmar Maurer 7e21da6e23 api/router.rs: allow different types of api methods
Added a prototype for file/backup uploads.
2019-01-14 12:26:04 +01:00
Dietmar Maurer b41d1aab8c catar: define all feature flags 2019-01-12 16:49:02 +01:00
Dietmar Maurer 4c0fd4871e catar/encoder.rs: allow to pass list of devices
We can use this to implement the --one-file-system like options. This
is a bit more convenient, because this way we can select to include
multiple devices (which is required by container backups).
2019-01-12 15:43:20 +01:00
Dietmar Maurer a8a0495612 catar/encoder.rs: disable backup of virtual file systems 2019-01-12 11:56:53 +01:00
Dietmar Maurer d7bfac8605 catar/encoder.rs: only test fat attribute on MSDOS and FUSE file systems 2019-01-12 10:28:26 +01:00
Dietmar Maurer 13d98013b2 /catar/encoder.rs: do not store data from virtual file systems 2019-01-12 10:20:08 +01:00
Dietmar Maurer 80881f60ef catar/encoder.rs: add helpers to test file system type 2019-01-12 09:18:21 +01:00
Dietmar Maurer 1c4804cf67 catar/encoder.rs: pass magic fs number to encode_dir 2019-01-12 08:51:44 +01:00
Dietmar Maurer dce0ebd505 catar/encoder.rs: use O_DIRECTORY to open dirs 2019-01-11 16:53:17 +01:00
Dietmar Maurer 490683ec93 catar: encode/decode sockets and fifos 2019-01-11 13:26:05 +01:00
Dietmar Maurer 20e2043a08 catar/decoder.rs: corrently pass mode to mknodat 2019-01-11 13:12:55 +01:00
Dietmar Maurer a7e3713122 catar: encode/decode devices 2019-01-11 12:22:00 +01:00
Dietmar Maurer 1bdb3130f7 catar/format_definition.rs: add more feature flags 2019-01-11 10:44:57 +01:00
Dietmar Maurer 5c76c2f331 catar/encoder.rs: store fat file system attributes 2019-01-11 10:18:22 +01:00
Dietmar Maurer 8c1dfa6c72 catar/encoder.rs: cleanups 2019-01-11 10:01:51 +01:00
Dietmar Maurer 4f6892eb74 catar/encoder.rs: encode linux fs attributes, cleanup encoder
Try to avoid duplicate stat calls (assume file tree is stable during backup).
2019-01-11 09:20:10 +01:00
Dietmar Maurer ddbdf80d51 avoid compiler warnings 2019-01-11 08:41:33 +01:00
Dietmar Maurer 373ef4a504 backup/archive_index.rs: correctly check for errors 2019-01-10 11:19:54 +01:00
Dietmar Maurer 23f68e53b4 catar/decoder.rs: restore mode, uid, gid and mtime 2019-01-10 09:02:50 +01:00
Dietmar Maurer 25f60394d2 catar/decoder.rs: avoid slashes in file names 2019-01-10 09:02:50 +01:00
Dietmar Maurer 9b1bb5a277 catar/decoder.rs: impl restore dirs, files, and symlinks 2019-01-10 09:01:36 +01:00
Wolfgang Bumiller 1ed86a0bae rest: rename utf-8-checked 'bytes' to 'utf8'
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-08 14:22:43 +01:00
Wolfgang Bumiller 9fec7ae593 rest: don't copy the body
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-08 14:21:54 +01:00
Wolfgang Bumiller d9fbcc4631 tools/timer: fixup doc formatting
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-08 14:11:10 +01:00
Dietmar Maurer 676bf71ad4 catar/decoder.rs: restore_sequential - read payload 2019-01-08 09:26:07 +01:00
Dietmar Maurer c6a0d3fc6b catar/decoder.rs: improve restore_sequential 2019-01-07 19:18:59 +01:00
Dietmar Maurer b6ebfb8dab catar/decoder.rs: start implementing sequential decoder, cleanups
And use [derive(Endian)] for endian conversion.
2019-01-07 19:07:03 +01:00
Dietmar Maurer 02c7d8e522 catar: implement create 2019-01-07 13:25:41 +01:00
Dietmar Maurer e86c49247e bin/catar.rs: add list command 2019-01-06 17:42:23 +01:00
Dietmar Maurer 3d8c24ecf8 catar/decoder.rs: start implementation 2019-01-06 17:27:57 +01:00
Dietmar Maurer 4b864ad436 format_definition.rs: remove wrong docs 2019-01-06 17:27:22 +01:00
Dietmar Maurer 4624fe29cc impl Read for BufferedArchiveReader 2019-01-06 10:04:45 +01:00
Dietmar Maurer 343370504b impl Seek for BufferedArchiveReader 2019-01-06 09:35:39 +01:00
Dietmar Maurer 318564ac03 buffered_read: return empty buffer on eof 2019-01-06 09:17:28 +01:00
Dietmar Maurer 0a72e26704 define and use BufferedReader trait 2019-01-05 17:28:20 +01:00
Dietmar Maurer 1006acd626 remove dead code 2019-01-05 16:55:21 +01:00
Dietmar Maurer 51b499db74 tools.rs: improve docs 2019-01-05 16:53:28 +01:00
Dietmar Maurer 39c6bd86cc backup/archive_index.rs: implement BufferedArchiveReader
Implement relativly fast random read using binary search.
2019-01-05 14:47:56 +01:00
Dietmar Maurer 060c4811a0 backup/chunk_store.rs: resize buffer by powers of two 2019-01-05 11:34:49 +01:00
Dietmar Maurer 9409255aae catar/encoder.rs: code cleanup
Note: Rust Vec resize uses len(), not capacity()!
2019-01-04 17:23:01 +01:00
Dietmar Maurer df9973e8f5 backup/chunk_store.rs: avoid allocation in read_chunk 2019-01-04 17:16:56 +01:00
Dietmar Maurer 7394ca3e95 white space cleanups 2019-01-04 12:51:43 +01:00
Dietmar Maurer 96df2fb44b backup/chunk_store.rs: implement read_chunk() 2019-01-04 12:50:54 +01:00
Dietmar Maurer 691c89a0fb proxmox-backup-manager: add garbage collection cli 2019-01-04 11:34:32 +01:00
Dietmar Maurer 07b4694a33 cleanup crate names 2019-01-04 10:49:52 +01:00
Dietmar Maurer 4bdba5f4f1 rename backup-client to proxmox-backup-client 2019-01-04 10:41:37 +01:00
Dietmar Maurer a689f6df0e rename pbs to proxmox-backup-manager 2019-01-04 10:39:32 +01:00
Dietmar Maurer bc616633d5 backup/chunk_store.rs: only use one directory level 2019-01-04 10:35:22 +01:00
Dietmar Maurer 030cc2a903 move admin API to /admin/ subfolder 2019-01-04 10:32:27 +01:00
Dietmar Maurer b663789bdb backup/archive_index.rs: pass correct size to unmap 2019-01-04 09:28:41 +01:00
Dietmar Maurer 48d0d35688 backup/image_index.rs: use little endian 2019-01-04 08:45:45 +01:00
Wolfgang Bumiller 4968bc3ac0 cli::command: wrap usage errors in a UsageError
So we can distinguish them and show usage output
conditionally.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-03 16:08:51 +01:00
Dietmar Maurer c7fa487250 catar/encoder.rs: make sure we write all data with write_all() 2019-01-03 15:47:32 +01:00
Dietmar Maurer 050f8a6355 chunker.rs: make sure chunk_size_avg is a power of two 2019-01-03 15:10:54 +01:00
Dietmar Maurer ea4ea34baf improve chunker speed (avoid modulo) 2019-01-03 14:35:19 +01:00
Dietmar Maurer 3329ae8c2e add benchmark for chunker
We get about 300MB/s - so this is still not fast enough

Note: This is quite slow with debug target, so please compile with --release
2019-01-02 19:18:53 +01:00
Dietmar Maurer 0b8e75ed16 image_index.rs: verify file size 2019-01-02 18:14:02 +01:00
Dietmar Maurer 77703d95aa implement garbage collection for .aidx files 2019-01-02 14:27:04 +01:00
Dietmar Maurer a360f6fa2d image_index.rs: verify header magic/version 2019-01-02 13:13:13 +01:00
Dietmar Maurer 44b3f62b42 fix typo 2019-01-02 12:56:04 +01:00
Dietmar Maurer 594fa52016 disable debug code 2019-01-02 12:55:18 +01:00
Dietmar Maurer 5032b57b44 archive_index.rs: correctly write .aidx file 2019-01-02 12:54:40 +01:00
Dietmar Maurer 580dc84bac image_index.rs: print stats 2019-01-02 12:53:49 +01:00
Dietmar Maurer 5e7a09be0d src/backup/archive_index.rs: use close() instead of flush()
Also pass a reference to the encoder.
2019-01-02 11:02:56 +01:00
Dietmar Maurer 94a882e900 src/backup/archive_index.rs: implement flush() 2018-12-31 18:01:07 +01:00
Dietmar Maurer 0433db1949 src/backup/archive_index.rs: first try 2018-12-31 17:30:08 +01:00
Dietmar Maurer cbdd8c54ae create backup mod in backup.rs, improve docu 2018-12-31 16:08:04 +01:00
Dietmar Maurer cb4412b18e move chunker.rs to backup/ 2018-12-31 15:46:16 +01:00
Dietmar Maurer 1c287cb19a rc/catar/chunker.rs: first try 2018-12-31 13:01:06 +01:00
Dietmar Maurer bf205f94c5 avoid compiler warnings 2018-12-31 10:54:25 +01:00
Dietmar Maurer 1b0dc9f680 src/catar/encoder.rs: correctly sort goodbye items by hash key 2018-12-31 10:11:28 +01:00
Dietmar Maurer c60d34bdbf new catar binary
currently used for debugging
2018-12-30 18:02:46 +01:00
Dietmar Maurer e75eac73ca fix doc syntax 2018-12-30 17:43:53 +01:00
Dietmar Maurer 0866748de6 catar cleanups ... 2018-12-30 17:32:52 +01:00
Dietmar Maurer 6cd28d200e binary_search_tree.rs: fix docs 2018-12-30 15:34:43 +01:00
Dietmar Maurer 389e562524 src/catar/encoder.rs: auto-resize file_copy_buffer, limit number of dirs 2018-12-30 14:09:59 +01:00
Dietmar Maurer 4fa71e0573 improve catar docs 2018-12-30 13:47:27 +01:00
Dietmar Maurer 48147efd12 add a comment about posible improvements 2018-12-29 19:43:25 +01:00
Dietmar Maurer 0b78833d8e src/catar/binary_search_tree.rs: add regression tests and fix one bug 2018-12-29 18:32:03 +01:00
Dietmar Maurer b17d7149d2 src/catar/binary_search_tree.rs: improve docu 2018-12-29 17:38:50 +01:00
Dietmar Maurer 985567fb45 src/catar/encoder.rs: cleanup, factor out write_goodbye_table 2018-12-29 17:26:32 +01:00
Dietmar Maurer 95bd5dfec7 src/catar/encoder.rs: correctly sort goodby items 2018-12-29 17:00:48 +01:00
Dietmar Maurer 46b6fbd6ae use external crate siphasher
Because std:#️⃣:SipHasher is deprecated.
2018-12-28 19:50:07 +01:00
Dietmar Maurer f0f3029e26 src/catar/encoder.rs: fix bug in file copy 2018-12-28 15:02:42 +01:00
Dietmar Maurer d05f93215e src/catar/encoder.rs: fix symlinks 2018-12-28 15:02:42 +01:00
Wolfgang Bumiller 7ebb173352 getopt: cleanup: add trailing commas
It is customary in rust to always use trailing commas. (also
suggested by rustfmt)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Wolfgang Bumiller 2767c5d39b getopt: cleanup: don't condense 'if' statements too much
In a language which enforces curly braces this looks weird
and rustfmt doesn't like it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Wolfgang Bumiller 7d95c10da0 getopt: whitespace cleanup
to make rustfmt happy

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Dietmar Maurer a0cc09b5f0 src/catar/encoder.rs: fix binary format, write goodby table
We still ned to sort the table (BST) ...
2018-12-28 14:27:00 +01:00
Wolfgang Bumiller 03fb895197 getopt: condense nested match to reduce indentation
The `match value` statement is the only thing covering the
entire RawArgument::Option case. `rustfmt` suggests this
more condensed way of writing this case.

See the `git diff -w` of this patch:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 9755af2..4db4579 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,8 +72,7 @@ pub fn parse_arguments<T: AsRef<str>>(
|             RawArgument::Separator => {
|                 break;
|             }
|-            RawArgument::Option { name, value } => {
|-                match value {
|+            RawArgument::Option { name, value } => match value {
|                 None => {
|                     let mut want_bool = false;
|                     let mut can_default = false;
|@@ -125,7 +124,6 @@ pub fn parse_arguments<T: AsRef<str>>(
|                     data.push((name, v));
|                 }
|             }
|-            }
|             RawArgument::Argument { value } => {
|                 rest.push(value);
|             }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:16:58 +01:00
Wolfgang Bumiller 4d0ea9978b getopt: let parse_arguments() take a slice of AsRef<str>
We don't need the content to be owned strings, and we don't
need the list to be a Vec, since we only care about being
able to iterate through and copy out portions of the strings
we need, so take an &[T] where T: AsRef<str>.

This avoids .iter().map(to_string).collect() before calling
parse_arguments().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:05:05 +01:00
Wolfgang Bumiller 2482c095b1 getopt: remove skip logic
The 'skip' variable was set to indicate that the "rest of
the args" is to be copied into the 'rest' vec. We can do
this directly and avoid the 'if' case in the loop
altogether.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:59:44 +01:00
Wolfgang Bumiller 99da3a073d getopt: indentation reduction
See the `git diff -w` output:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 6548a01..517cc37 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,7 +72,10 @@ pub fn parse_arguments(
|     while pos < args.len() {
|         if skip {
|             rest.push(args[pos].clone());
|-        } else {
|+            pos += 1;
|+            continue;
|+        }
|+
|         match parse_argument(&args[pos]) {
|             RawArgument::Separator => {
|                 skip = true;
|@@ -135,7 +138,6 @@ pub fn parse_arguments(
|                 rest.push(value);
|             }
|         }
|-        }
|
|         pos += 1;
|     }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:54:20 +01:00
Wolfgang Bumiller ad8a98f7e4 parse_arguments: work with utf8 bytes and reduce indentation
We are only caring about '=' and '-' which are single-byte
codepoints, so there's no reason not to work on a byte
slice.

Also, some indentation reduction:

Transform
    if (a) {
        ...
        return A;
    }
    return B;
into
    if (!a)
        return B;
    return A;

and
    if (a)
        if (b)
            foo()
into
    if (a && b)
        return;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:47:29 +01:00
Dietmar Maurer 248c17af39 src/catar/encoder.rs: write filenames 2018-12-28 11:48:47 +01:00
Dietmar Maurer 2e4ae0e239 src/catar/encoder.rs: write file data 2018-12-28 10:44:12 +01:00
Dietmar Maurer 3192ae968c src/catar/encoder.rs: write entry and symlink data 2018-12-28 09:55:26 +01:00
Dietmar Maurer 50ea43962e make map_struct functions public 2018-12-28 08:04:46 +01:00
Dietmar Maurer d2b03f2397 catar/encoder.rs: improve error handling 2018-12-28 07:45:15 +01:00
Dietmar Maurer 0ff559990c cleanups, avoid compiler warnings 2018-12-28 07:14:12 +01:00
Dietmar Maurer 45281d4927 catar/encoder: detect symlink and regular files 2018-12-27 14:24:31 +01:00
Dietmar Maurer fb8365b79f catar/encoder.rs: first try 2018-12-27 13:15:47 +01:00
Wolfgang Bumiller fba3437f1e timer: setup_timeout_handler cannot fail
and if it does it panics anyway

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-27 10:36:28 +01:00
Dietmar Maurer bcd879cfb3 backup-client: allow to specify directories (prepare for catar backup) 2018-12-27 10:11:11 +01:00
Dietmar Maurer b62b6cadf4 start implementing catar 2018-12-27 09:22:23 +01:00
Dietmar Maurer dc3de618ed new helper map_struct and map_struct_mut 2018-12-27 09:20:17 +01:00
Dietmar Maurer a198d74fc0 cleanup: reduce compiler warnings 2018-12-25 13:29:27 +01:00
Dietmar Maurer 991de6007b remove outdated comment 2018-12-25 13:24:50 +01:00
Dietmar Maurer 7ee2aa1b94 touch_chunk: use libc::lutimensat 2018-12-25 12:27:25 +01:00
Dietmar Maurer 2c32fdde86 move lookup_datastore() to backup/datastore.rs 2018-12-22 17:37:25 +01:00
Dietmar Maurer 64e53b2835 gather usage statistics during garbage collection 2018-12-22 16:58:16 +01:00
Dietmar Maurer 28b96b56e1 open_file_locked: improve error message 2018-12-22 15:59:55 +01:00
Dietmar Maurer 176e4af964 sweep_used_chunks: print percentage 2018-12-22 15:39:05 +01:00
Dietmar Maurer 7b2b40a893 use openssl for faster hashing 2018-12-22 14:31:59 +01:00
Dietmar Maurer e95950e40a chunk_store: reduce number of directories
Else, scans everything takes too long ...
2018-12-22 14:04:05 +01:00
Dietmar Maurer 15e9b4ed60 src/api3/datastore.rs: new file 2018-12-21 13:38:41 +01:00
Dietmar Maurer 3c140f60cd use return value (avoid compiler warning) 2018-12-21 12:44:20 +01:00
Dietmar Maurer 03e4753d8e fix mutability for chunk store 2018-12-21 12:15:26 +01:00
Dietmar Maurer 2d9d143a8f backup-client: add optional chunk-size parameter 2018-12-21 11:18:08 +01:00
Dietmar Maurer f0819fe5a5 backup-client: do not start garbage collection after backup 2018-12-21 09:24:55 +01:00
Dietmar Maurer d62e6e2264 backup-client: use 1M chunks, make chunk_size configurable 2018-12-21 08:36:57 +01:00
Dietmar Maurer c34eb16651 backup-clinet: specify target file name 2018-12-20 14:09:31 +01:00
Dietmar Maurer c2d9b3914a add test code to access static global state 2018-12-20 14:03:42 +01:00
Dietmar Maurer fe0e04c69c use proxmox-backup as package name 2018-12-20 10:45:14 +01:00
Dietmar Maurer f7dd683b46 fix tests 2018-12-19 13:54:22 +01:00
Dietmar Maurer 277fc5a3ce try to improve error messages 2018-12-19 13:40:26 +01:00
Dietmar Maurer eae8aa3aa9 cleanups: avoid compiler warnings 2018-12-19 12:49:23 +01:00
Dietmar Maurer 6c20a13d3c cleanup - avoid too much indentation 2018-12-19 12:40:26 +01:00
Dietmar Maurer 1c43c56b06 cleanup 2018-12-19 12:30:37 +01:00
Dietmar Maurer 2bf5f6b220 ChunkStore::sweep_used_chunks - use openat to speedup directory walks 2018-12-19 12:20:01 +01:00
Wolfgang Bumiller 1628a4c731 use timers with a signal for file locking
* rename lock_file -> open_file_locked,
* add lock_file as a function working on already-opened
  files
* change timeout types to std::time::Duration

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-19 11:23:00 +01:00
Wolfgang Bumiller 8cf6e764ad add tools::timer module
This wraps POSIX timers via timer_create(2).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-19 11:23:00 +01:00
Dietmar Maurer 365bb90f17 move lock_file to tools.rs 2018-12-19 10:02:24 +01:00
Dietmar Maurer 08481a0b60 impl sweep_used_chunks, first try 2018-12-19 09:51:33 +01:00
Dietmar Maurer 6ea3a0b7fc move sweep_used_chunks to chunk_store 2018-12-18 11:18:55 +01:00
Dietmar Maurer 3d5c11e5d2 start impl garbage collection 2018-12-18 11:06:03 +01:00
Dietmar Maurer 529de6c7a3 start implementing DataStore 2018-12-17 13:00:39 +01:00
Dietmar Maurer 4818c8b6f7 start implementing ImageIndexReader 2018-12-16 14:44:44 +01:00
Dietmar Maurer a27a3ee4ba avoid compiler warnings 2018-12-16 13:57:59 +01:00
Dietmar Maurer af3e7d7551 fix chunk store file paths 2018-12-16 13:52:16 +01:00
Dietmar Maurer 0cd9d4206f improve error messages 2018-12-16 13:43:19 +01:00
Dietmar Maurer 4fbb72a8b4 image_index.rs: implement atomic write/rename 2018-12-16 13:39:21 +01:00
Dietmar Maurer 798881a68a rebane ImageIndex to ImageIndexWriter 2018-12-16 11:51:16 +01:00
Dietmar Maurer 5e5b7f1c2d use u64 to store time 2018-12-16 11:48:03 +01:00
Dietmar Maurer d13e3745b3 image_index.rs: write header 2018-12-15 17:05:49 +01:00
Dietmar Maurer 606ce64bb0 image_index.rs:write idx of chunks (use mmap) 2018-12-15 14:51:05 +01:00
Dietmar Maurer f8838fe913 backup-client: add bash completion for datastore names 2018-12-15 11:24:39 +01:00
Dietmar Maurer 5f0c2d5652 remove debug println 2018-12-15 11:16:27 +01:00
Dietmar Maurer 43eeef28d1 move file_chunker to tools 2018-12-15 11:14:41 +01:00