tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
4,021 Commits 2 Branches 54 Tags 14 MiB
cb4865466e
Commit Graph

3108 Commits

Author SHA1 Message Date
Dietmar Maurer
1c43c56b06 cleanup 2018-12-19 12:30:37 +01:00
Dietmar Maurer
2bf5f6b220 ChunkStore::sweep_used_chunks - use openat to speedup directory walks 2018-12-19 12:20:01 +01:00
Wolfgang Bumiller
1628a4c731 use timers with a signal for file locking 
* rename lock_file -> open_file_locked,
* add lock_file as a function working on already-opened
  files
* change timeout types to std::time::Duration

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-19 11:23:00 +01:00
Wolfgang Bumiller
8cf6e764ad add tools::timer module 
This wraps POSIX timers via timer_create(2).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-19 11:23:00 +01:00
Dietmar Maurer
365bb90f17 move lock_file to tools.rs 2018-12-19 10:02:24 +01:00
Dietmar Maurer
08481a0b60 impl sweep_used_chunks, first try 2018-12-19 09:51:33 +01:00
Dietmar Maurer
6ea3a0b7fc move sweep_used_chunks to chunk_store 2018-12-18 11:18:55 +01:00
Dietmar Maurer
3d5c11e5d2 start impl garbage collection 2018-12-18 11:06:03 +01:00
Dietmar Maurer
529de6c7a3 start implementing DataStore 2018-12-17 13:00:39 +01:00
Dietmar Maurer
4818c8b6f7 start implementing ImageIndexReader 2018-12-16 14:44:44 +01:00
Dietmar Maurer
a27a3ee4ba avoid compiler warnings 2018-12-16 13:57:59 +01:00
Dietmar Maurer
af3e7d7551 fix chunk store file paths 2018-12-16 13:52:16 +01:00
Dietmar Maurer
0cd9d4206f improve error messages 2018-12-16 13:43:19 +01:00
Dietmar Maurer
4fbb72a8b4 image_index.rs: implement atomic write/rename 2018-12-16 13:39:21 +01:00
Dietmar Maurer
798881a68a rebane ImageIndex to ImageIndexWriter 2018-12-16 11:51:16 +01:00
Dietmar Maurer
5e5b7f1c2d use u64 to store time 2018-12-16 11:48:03 +01:00
Dietmar Maurer
d13e3745b3 image_index.rs: write header 2018-12-15 17:05:49 +01:00
Dietmar Maurer
606ce64bb0 image_index.rs:write idx of chunks (use mmap) 2018-12-15 14:51:05 +01:00
Dietmar Maurer
f8838fe913 backup-client: add bash completion for datastore names 2018-12-15 11:24:39 +01:00
Dietmar Maurer
5f0c2d5652 remove debug println 2018-12-15 11:16:27 +01:00
Dietmar Maurer
43eeef28d1 move file_chunker to tools 2018-12-15 11:14:41 +01:00
Dietmar Maurer
48e8689505 file_chunker: use Read trait 2018-12-15 11:08:49 +01:00
Dietmar Maurer
fc8b19368a backup-client: pass chunk_size as parameter to file_chunker 2018-12-15 10:58:48 +01:00
root
a914a77448 backup-client: read file by chunks 2018-12-14 13:39:41 +01:00
root
ff5d37074d bin/backup-client.rs: nbew tool to play with chunk store 2018-12-14 08:28:56 +01:00
root
15b64d4606 api3/config/datastore.rs: create datastore files 2018-12-13 14:41:14 +01:00
Dietmar Maurer
7f4e639b37 improve completion 2018-12-12 13:00:58 +01:00
Dietmar Maurer
e3d40bb8cd complete_datastore_name: write more compact code 2018-12-12 12:41:59 +01:00
Dietmar Maurer
30d2e99c77 implement completion functions 2018-12-12 12:19:26 +01:00
Dietmar Maurer
91643d9012 section_config.rs: remove debug output 2018-12-12 12:19:26 +01:00
Dietmar Maurer
d64d80d20a add comment 2018-12-12 11:23:04 +01:00
Dietmar Maurer
5ba6968922 add comment to make intensions more clear 2018-12-12 11:21:00 +01:00
Dietmar Maurer
ca9caffa23 cleanup parameter types 2018-12-12 10:37:03 +01:00
Dietmar Maurer
38555b5072 implement completion for enums 2018-12-12 08:33:50 +01:00
Dietmar Maurer
2f0258959b improve completion 2018-12-12 08:18:38 +01:00
Dietmar Maurer
f46403cc65 improve bash completion 2018-12-12 07:51:43 +01:00
Dietmar Maurer
b6e8dd39aa impl bash completion: first try 2018-12-11 14:21:05 +01:00
Dietmar Maurer
c21de379ed install bash completion scripts 
Completion itself is not implemented.
 2018-12-11 12:54:10 +01:00
Dietmar Maurer
6460764dbb cli/command.rs: add new type CliCommandMap 2018-12-11 11:31:36 +01:00
Dietmar Maurer
bf7f103944 implement builder pattern for CliCommand 2018-12-11 11:12:13 +01:00
Dietmar Maurer
baed30b702 cli/command.rs: implement prefix match 2018-12-10 18:13:55 +01:00
Dietmar Maurer
8f62336b0f impl From for CommandLineInterface 2018-12-10 13:51:10 +01:00
Dietmar Maurer
9f6ab1fc58 rename CmdDef to CommandLineInterface 2018-12-10 13:40:10 +01:00
Dietmar Maurer
b7329c8a1a src/cli/command.rs: move code into separate file 2018-12-10 13:36:52 +01:00
Dietmar Maurer
211fabd795 try to support nested cli command definitions 2018-12-10 13:28:38 +01:00
Dietmar Maurer
34d3ba52eb api3/config/datastore.rs: implement delete 2018-12-09 16:52:32 +01:00
Dietmar Maurer
1a7bc3dd5e file_set_contents: correctly use perm parameter 2018-12-09 16:41:54 +01:00
Dietmar Maurer
f12f8ff1a6 implement tools::file_set_contents 2018-12-09 16:37:48 +01:00
Dietmar Maurer
652c11900d api3/config/datastore.rs: impl create 2018-12-09 12:51:31 +01:00
Dietmar Maurer
ea0b8b6ec0 bin/pbs-datastore.rs: start implementing cli commands 2018-12-09 11:59:32 +01:00
Dietmar Maurer
5b34c2607d rename entry to item 2018-12-09 10:25:56 +01:00
Dietmar Maurer
b65eaac62d api3/config/datastore.rs: return real data 2018-12-09 10:23:19 +01:00
Dietmar Maurer
6349ed60b4 config/datastore.rs - open file for reading 2018-12-09 10:22:39 +01:00
Dietmar Maurer
ce55dbbcc6 simplify code 2018-12-09 09:42:17 +01:00
Dietmar Maurer
461e62fcdf remove some compiler warnings 2018-12-08 14:55:54 +01:00
Dietmar Maurer
567713b4c3 rename data_store to datastore 2018-12-08 14:51:08 +01:00
Dietmar Maurer
6ce50400c5 cleanup api3 structure 2018-12-08 14:44:55 +01:00
Dietmar Maurer
678d72df6b add data_store configuration 2018-12-08 13:58:45 +01:00
Dietmar Maurer
391a2e43ff chunk_store.insert_chunk: return information about existing chunks 2018-12-08 11:25:11 +01:00
Dietmar Maurer
12bb93b3ad code cleanup 2018-12-08 11:20:02 +01:00
Dietmar Maurer
022fb421b2 lock with timeout 
clumsy, but I have node idea how to make it better
 2018-12-08 10:57:09 +01:00
Dietmar Maurer
2989f6bf20 improve error messages 2018-12-08 08:38:42 +01:00
Dietmar Maurer
b8d4766a4d code cleanup 2018-12-08 08:21:20 +01:00
Dietmar Maurer
45773720ab depend on nix, use flock 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2018-12-07 18:14:07 +01:00
Dietmar Maurer
c5d82e5f34 chunk_store: use mutex 2018-12-07 16:12:45 +01:00
Dietmar Maurer
128b37feea impl insert_chunk 2018-12-07 14:44:56 +01:00
Dietmar Maurer
35cf5daa53 start experiments with simple chunk store 2018-12-07 10:51:53 +01:00
Dietmar Maurer
b22751b20c getopts.rs: fix test 2018-12-07 09:02:24 +01:00
Dietmar Maurer
a4b7c3f2df more cleanups 2018-12-06 10:41:57 +01:00
Dietmar Maurer
d3369f819d code cleanup 2018-12-06 10:23:45 +01:00
Dietmar Maurer
c578fcd9e2 simplify formatter code 2018-12-05 18:22:56 +01:00
Dietmar Maurer
4b26195c6a extract version from environment 2018-12-05 14:40:45 +01:00
Dietmar Maurer
95b492add8 api: add /version 2018-12-05 14:03:15 +01:00
Dietmar Maurer
1571873d7b add output formatter 2018-12-05 12:43:22 +01:00
Dietmar Maurer
083418619e move src/api/server.rs -> src/server/rest.rs 2018-12-05 10:16:23 +01:00
Dietmar Maurer
5c7a1b15c8 start the GUI 2018-12-04 17:53:10 +01:00
Dietmar Maurer
826bb98286 set content type for static file download 2018-12-02 11:00:52 +01:00
Dietmar Maurer
8adbdb0a54 fix file download, listen to 0.0.0.0 2018-12-01 15:21:25 +01:00
Dietmar Maurer
f4c514c13d remove www/pbs-index.html.tt, hardcode into rust for now 2018-12-01 13:37:49 +01:00
Dietmar Maurer
e3871182c5 rename bin to src/bin/api-test-server.rs 2018-11-30 12:48:45 +01:00
Dietmar Maurer
73f29c34d7 improve command line parser 2018-11-30 12:10:26 +01:00
Dietmar Maurer
d42398666c improve error message 2018-11-30 12:07:27 +01:00
Dietmar Maurer
bfb1d69abc fix compiler warnings, add storage/config.rs 2018-11-30 11:15:26 +01:00
Dietmar Maurer
9131286246 remove missleading comment 2018-11-30 08:47:26 +01:00
Dietmar Maurer
a27b905c28 use VecDeque instead of LinkedList 2018-11-29 17:28:28 +01:00
Dietmar Maurer
c6ed6cac12 SectionConfig::write - return raw string 2018-11-29 16:47:40 +01:00
Dietmar Maurer
ae3a512dd9 SectionConfig::write - verify data 2018-11-29 13:18:15 +01:00
Dietmar Maurer
aeb4c8217b schema: implement verify_json 2018-11-29 13:06:43 +01:00
Dietmar Maurer
9c1b42d28e simplify var names 2018-11-29 09:33:27 +01:00
Dietmar Maurer
e189c93b8c section_config: add write - first try 2018-11-28 14:09:35 +01:00
Dietmar Maurer
92fb0784d4 section_config: add new struct SectionConfigData 2018-11-28 08:25:26 +01:00
Dietmar Maurer
e3a2217050 avoid clone 2018-11-27 14:10:16 +01:00
Dietmar Maurer
ee7fc4335a SectionConfig::parse - return Value 2018-11-27 12:54:40 +01:00
Dietmar Maurer
826698d56f SectionConfig: add schema for ID property 2018-11-27 12:34:56 +01:00
Dietmar Maurer
bdb631da88 section_config: improve parser 2018-11-27 11:55:21 +01:00
Dietmar Maurer
1fa897cfed section_config: parse section content 2018-11-27 09:01:36 +01:00
Dietmar Maurer
803b95a01d SectionConfigPlugin: add schema 2018-11-27 08:41:25 +01:00
Dietmar Maurer
b943ed8d2b section_config: further parser improvements 2018-11-26 18:22:07 +01:00
Dietmar Maurer
0d97734e92 use is_empty() to check for empty string 2018-11-26 18:02:29 +01:00
Dietmar Maurer
dd193a61d0 section_config: improve parser 2018-11-26 18:01:24 +01:00
Dietmar Maurer
07d3fa7289 add some inline documentation 2018-11-24 17:52:43 +01:00
Dietmar Maurer
6656ba9d32 StringSchema: add method to check string length 2018-11-24 12:37:05 +01:00
Dietmar Maurer
ab672c124e ArraySchema: add method to check array length 2018-11-24 12:33:44 +01:00
Dietmar Maurer
ed04ce8093 parse_property_string: allow arrays 
So that we can build simple lists like "1,2,3,4"
 2018-11-24 12:21:56 +01:00
Dietmar Maurer
84669dc18d rename test case 2018-11-24 11:21:01 +01:00
Dietmar Maurer
b436389d47 parse_property_string: first try 2018-11-24 11:20:17 +01:00
Dietmar Maurer
a23dd9bb42 add test for VerifyFn 2018-11-24 08:30:23 +01:00
Dietmar Maurer
715f22fb4a remove outdated test code 2018-11-24 08:15:08 +01:00
Dietmar Maurer
757d9b4769 ArraySchema: Add min_length and max_length 2018-11-24 08:12:23 +01:00
Dietmar Maurer
3e1497fb75 remove description from ApiMethod 
We use the description from parameters ObjectSchema instead.
 2018-11-23 17:02:50 +01:00
Dietmar Maurer
768e01091a implement Into Arc<Schema> 2018-11-23 13:18:41 +01:00
Dietmar Maurer
8a41cfdd55 implement ArraySchema 2018-11-23 12:05:55 +01:00
Dietmar Maurer
7edeec7b06 remove parameter macro, implement ObjectSchema builder 
And store "optional" attribute inside properties hash.
 2018-11-23 11:34:15 +01:00
Dietmar Maurer
82df76fff0 start using builder pattern instead of macros 2018-11-23 09:55:38 +01:00
Dietmar Maurer
dc454ef096 implement schema registry 2018-11-22 11:23:49 +01:00
Dietmar Maurer
00c908df85 use Arc pointer to properties 2018-11-20 17:20:50 +01:00
Dietmar Maurer
e471d69946 section_config.rs: add some test code 2018-11-19 08:07:22 +01:00
Dietmar Maurer
222454221c add section_config.rs 2018-11-19 06:47:39 +01:00
Dietmar Maurer
948c74f4ca move future test code into extra file 2018-11-19 06:41:50 +01:00
Dietmar Maurer
0f52af5b23 implement simply test future for storage operations 2018-11-18 15:01:19 +01:00
Dietmar Maurer
be45ccd203 fix getopts when called without any argument 2018-11-18 14:57:18 +01:00
Dietmar Maurer
a19f223dea impl argument parameters 2018-11-18 10:09:13 +01:00
Dietmar Maurer
c78bcf07ad getopts: more tests 2018-11-18 09:34:43 +01:00
Dietmar Maurer
62c40162ee improve getopt parser 2018-11-18 08:55:21 +01:00
Dietmar Maurer
a653882dd9 use ObjectSchema for parameters 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2018-11-18 08:46:26 +01:00
Dietmar Maurer
5d9f0eae6b another testcase for boolean argument parser 2018-11-17 11:29:42 +01:00
Dietmar Maurer
2770fbf3c3 getopt parser fixes and cleanups 2018-11-17 11:28:26 +01:00
Dietmar Maurer
7556cab45e getopts: allow "-option" and "--option" 2018-11-17 10:02:35 +01:00
Dietmar Maurer
0c9ce2bbf0 getopts: improve error handling 2018-11-17 09:57:26 +01:00
Dietmar Maurer
ffff48fcfe cleanup variable names 2018-11-16 16:47:23 +01:00
Dietmar Maurer
845901f41d implement getopt parser - first try 2018-11-16 13:14:11 +01:00
Dietmar Maurer
c548fa250a router: no need to use Fn (fn also works for static closures) 2018-11-16 11:12:00 +01:00
Dietmar Maurer
5206d1d951 fix typo 2018-11-16 09:16:37 +01:00
Dietmar Maurer
e7ea17deb0 handle uri parameters correctly 2018-11-16 09:15:33 +01:00
Dietmar Maurer
484eb9ac19 Router: allow to set match parameter name 2018-11-16 08:29:23 +01:00
Dietmar Maurer
2ab1f7119c Router: implement missing builder methods 2018-11-16 08:24:10 +01:00
Dietmar Maurer
fc937099b3 remove stale macro 2018-11-16 07:35:19 +01:00
Dietmar Maurer
fe476ce884 allow closure handlers 2018-11-15 17:47:59 +01:00
Dietmar Maurer
cbbeff1a6d add missing file 2018-11-15 17:07:55 +01:00
Dietmar Maurer
f17db0abfd cleanup module names 2018-11-15 17:07:10 +01:00
Dietmar Maurer
c0262d7480 rename Jss to Schema 2018-11-15 16:56:28 +01:00
Dietmar Maurer
426a48b04f fix parameter macro for empty object 2018-11-15 16:48:36 +01:00
Dietmar Maurer
8a7005321b improve router 2018-11-15 13:28:15 +01:00
Dietmar Maurer
e63e99d646 rename MethodInfo to Router 2018-11-15 11:46:13 +01:00
Dietmar Maurer
10be1d290a avoid macros 2018-11-15 10:57:52 +01:00
Dietmar Maurer
4dcf43d266 rename ApiError into HttpError 2018-11-15 10:42:01 +01:00
Dietmar Maurer
279ecfdfb4 remove unneeded lifetime annotations 2018-11-15 10:25:59 +01:00
Dietmar Maurer
f0b10921b8 reorder content 2018-11-15 10:18:01 +01:00
Dietmar Maurer
16b48b81c5 move ApiConfig into extra file 2018-11-15 10:14:08 +01:00
Dietmar Maurer
9bc17e8dcd move server code to api_server.rs, implement Service and NewService 2018-11-15 10:05:37 +01:00
Dietmar Maurer
08bd8d476a remove more test code, cleanups 2018-11-14 17:32:17 +01:00
Dietmar Maurer
98d82428b0 remove asnyc test code 2018-11-14 14:39:01 +01:00
Dietmar Maurer
5599e2633c correctly use Arc pointer 2018-11-14 14:28:54 +01:00
Dietmar Maurer
1a53be1443 factor out server code into api_server.rs 2018-11-14 13:22:33 +01:00
Dietmar Maurer
a9696f7beb use Arc for ApiServer 2018-11-14 11:57:01 +01:00
Dietmar Maurer
2c10fd5cb1 simplify code 2018-11-14 11:38:26 +01:00
Dietmar Maurer
324a5bd095 remove service implementation, use service_fn again 2018-11-13 14:47:03 +01:00
Dietmar Maurer
a974251ee6 improve error handling, use a Service 2018-11-13 12:36:56 +01:00
Dietmar Maurer
035cce945e remove static directory cache 2018-11-12 14:11:04 +01:00
Dietmar Maurer
198fab6f2a santtize paths in handle_request 2018-11-12 13:19:53 +01:00
Dietmar Maurer
a0efdca18b code cleanup 2018-11-11 17:19:24 +01:00
Dietmar Maurer
78d0783b00 handle_static_file_download: optimize small files 
Avoid chuncked transfer for small files.
 2018-11-11 17:10:42 +01:00
Dietmar Maurer
579fbe7dc8 implement static file download 
Fully async using futures and stream.
 2018-11-11 15:54:05 +01:00
Dietmar Maurer
cec9f02ebc limit allowed body size 2018-11-11 13:24:14 +01:00
Dietmar Maurer
3cdec2a010 use header::CONTENT_TYPE 2018-11-11 12:55:30 +01:00
Dietmar Maurer
ef1f0e655a add code to create directory name cache 2018-11-10 17:17:41 +01:00
Dietmar Maurer
3c1d7afc42 another way to read files using tokio_codec 2018-11-10 15:12:45 +01:00
Dietmar Maurer
805aec1572 play around with async tasks 2018-11-10 12:06:39 +01:00
Dietmar Maurer
6639c14bd9 use better http status codes 2018-11-10 10:32:25 +01:00
Dietmar Maurer
260c1ee8c6 cleanup futures 2018-11-10 10:00:48 +01:00
Dietmar Maurer
c819ec8dde call api method and return result 2018-11-09 13:48:57 +01:00
Dietmar Maurer
b82472c0d2 use futures - first try 2018-11-09 12:14:18 +01:00
Dietmar Maurer
4beaacb6f4 code cleanup 2018-11-07 13:58:09 +01:00
Dietmar Maurer
a908292008 implement string enum validation 2018-11-07 13:25:47 +01:00
Dietmar Maurer
0a33b12851 try to reduce space usage in ApiStringFormat by using Box 2018-11-07 13:01:14 +01:00
Dietmar Maurer
abbf40fc2e define a separate enum ApiStringFormat 2018-11-07 12:55:33 +01:00
Dietmar Maurer
d46287e083 implement string regex matcher 2018-11-07 12:35:52 +01:00
Dietmar Maurer
144f0f5eda remove unnecessary test 2018-11-07 12:16:16 +01:00
Dietmar Maurer
7b5316ddfa improve error messages 2018-11-07 12:14:52 +01:00
Dietmar Maurer
7c32470aea implement some string tests 2018-11-07 12:11:09 +01:00
Dietmar Maurer
08a13bb456 implement integer schema tests 2018-11-07 11:55:08 +01:00
Dietmar Maurer
e72677bf50 pass ApiMethod to handler 2018-11-07 11:06:37 +01:00
Dietmar Maurer
ab9e6de21c test for required parameters 2018-11-06 14:18:13 +01:00
Dietmar Maurer
b8bcc7630c try to write unit test 2018-11-06 13:58:05 +01:00
Dietmar Maurer
cfa5886c33 implement simple schema parser 2018-11-06 13:10:10 +01:00
Dietmar Maurer
42d569c186 move url related code back to main.rs 2018-11-06 09:35:11 +01:00
Dietmar Maurer
6d77fb4063 move parse_query into json_schema 2018-11-05 15:20:27 +01:00
Dietmar Maurer
0dde2f04d0 avoid static references 2018-11-03 15:10:21 +01:00
Dietmar Maurer
331ab992d7 use const for default object initializers 2018-11-03 10:50:24 +01:00
Dietmar Maurer
504b359724 another way to initialize the api tree 2018-11-03 10:42:48 +01:00
Dietmar Maurer
0e6967d648 move test code 2018-11-03 09:08:01 +01:00
Dietmar Maurer
18671ca530 convert find_method_info function into a method 2018-11-02 10:01:47 +01:00
Dietmar Maurer
8741bc2079 use reference to Jss inside PropertyMap 2018-11-02 09:50:26 +01:00
Dietmar Maurer
016aa6cbab define macro for propertymap 2018-11-02 09:44:18 +01:00
Dietmar Maurer
79fa17d986 use macro to declare static methodinfo items 2018-11-01 18:10:36 +01:00
Dietmar Maurer
bfcba4fd86 new subdirmap macro, cleanups 2018-11-01 15:41:08 +01:00
Dietmar Maurer
763220cefa use edition 2018 (beta) 
This is expected to be stable by the end of the year.
 2018-11-01 14:42:27 +01:00
Dietmar Maurer
1ac3e212c2 avoid warning about unused vars 2018-11-01 14:19:02 +01:00
Dietmar Maurer
f1c0021436 remove hardcoded static lifetime 2018-11-01 14:16:41 +01:00
Dietmar Maurer
89feb6acdf use refs in ApiMethod 2018-11-01 13:57:53 +01:00
Dietmar Maurer
b90dcc34bb cleanup lifetime in find_method_info 2018-11-01 13:54:04 +01:00
Dietmar Maurer
3cd244b9fa static_map: remove starnge type borrow 2018-11-01 13:45:10 +01:00
Dietmar Maurer
d11f14f77d do not use phf 2018-11-01 13:05:45 +01:00
Dietmar Maurer
28e47cea55 start implementing request handler 2018-11-01 11:30:49 +01:00
Dietmar Maurer
886e5ce8f7 split code into several files 2018-10-31 10:42:14 +01:00
Dietmar Maurer
22f0adf26a try to define an static api 2018-10-30 14:06:37 +01:00
Dietmar Maurer
d6a4ba7192 move code into lib.rs 2018-10-30 10:40:05 +01:00
Dietmar Maurer
d8d978ebc4 initial version 2018-10-30 10:04:30 +01:00