Commit Graph

373 Commits

Author SHA1 Message Date
Dietmar Maurer 5e7a09be0d src/backup/archive_index.rs: use close() instead of flush()
Also pass a reference to the encoder.
2019-01-02 11:02:56 +01:00
Dietmar Maurer 94a882e900 src/backup/archive_index.rs: implement flush() 2018-12-31 18:01:07 +01:00
Dietmar Maurer 0433db1949 src/backup/archive_index.rs: first try 2018-12-31 17:30:08 +01:00
Dietmar Maurer cbdd8c54ae create backup mod in backup.rs, improve docu 2018-12-31 16:08:04 +01:00
Dietmar Maurer cb4412b18e move chunker.rs to backup/ 2018-12-31 15:46:16 +01:00
Dietmar Maurer 1c287cb19a rc/catar/chunker.rs: first try 2018-12-31 13:01:06 +01:00
Dietmar Maurer bf205f94c5 avoid compiler warnings 2018-12-31 10:54:25 +01:00
Dietmar Maurer 1b0dc9f680 src/catar/encoder.rs: correctly sort goodbye items by hash key 2018-12-31 10:11:28 +01:00
Dietmar Maurer c60d34bdbf new catar binary
currently used for debugging
2018-12-30 18:02:46 +01:00
Dietmar Maurer e75eac73ca fix doc syntax 2018-12-30 17:43:53 +01:00
Dietmar Maurer 0866748de6 catar cleanups ... 2018-12-30 17:32:52 +01:00
Dietmar Maurer 6cd28d200e binary_search_tree.rs: fix docs 2018-12-30 15:34:43 +01:00
Dietmar Maurer 389e562524 src/catar/encoder.rs: auto-resize file_copy_buffer, limit number of dirs 2018-12-30 14:09:59 +01:00
Dietmar Maurer 4fa71e0573 improve catar docs 2018-12-30 13:47:27 +01:00
Dietmar Maurer 48147efd12 add a comment about posible improvements 2018-12-29 19:43:25 +01:00
Dietmar Maurer 0b78833d8e src/catar/binary_search_tree.rs: add regression tests and fix one bug 2018-12-29 18:32:03 +01:00
Dietmar Maurer b17d7149d2 src/catar/binary_search_tree.rs: improve docu 2018-12-29 17:38:50 +01:00
Dietmar Maurer 985567fb45 src/catar/encoder.rs: cleanup, factor out write_goodbye_table 2018-12-29 17:26:32 +01:00
Dietmar Maurer 95bd5dfec7 src/catar/encoder.rs: correctly sort goodby items 2018-12-29 17:00:48 +01:00
Dietmar Maurer 46b6fbd6ae use external crate siphasher
Because std:#️⃣:SipHasher is deprecated.
2018-12-28 19:50:07 +01:00
Dietmar Maurer f0f3029e26 src/catar/encoder.rs: fix bug in file copy 2018-12-28 15:02:42 +01:00
Dietmar Maurer d05f93215e src/catar/encoder.rs: fix symlinks 2018-12-28 15:02:42 +01:00
Wolfgang Bumiller 7ebb173352 getopt: cleanup: add trailing commas
It is customary in rust to always use trailing commas. (also
suggested by rustfmt)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Wolfgang Bumiller 2767c5d39b getopt: cleanup: don't condense 'if' statements too much
In a language which enforces curly braces this looks weird
and rustfmt doesn't like it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Wolfgang Bumiller 7d95c10da0 getopt: whitespace cleanup
to make rustfmt happy

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:27:38 +01:00
Dietmar Maurer a0cc09b5f0 src/catar/encoder.rs: fix binary format, write goodby table
We still ned to sort the table (BST) ...
2018-12-28 14:27:00 +01:00
Wolfgang Bumiller 03fb895197 getopt: condense nested match to reduce indentation
The `match value` statement is the only thing covering the
entire RawArgument::Option case. `rustfmt` suggests this
more condensed way of writing this case.

See the `git diff -w` of this patch:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 9755af2..4db4579 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,8 +72,7 @@ pub fn parse_arguments<T: AsRef<str>>(
|             RawArgument::Separator => {
|                 break;
|             }
|-            RawArgument::Option { name, value } => {
|-                match value {
|+            RawArgument::Option { name, value } => match value {
|                 None => {
|                     let mut want_bool = false;
|                     let mut can_default = false;
|@@ -125,7 +124,6 @@ pub fn parse_arguments<T: AsRef<str>>(
|                     data.push((name, v));
|                 }
|             }
|-            }
|             RawArgument::Argument { value } => {
|                 rest.push(value);
|             }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:16:58 +01:00
Wolfgang Bumiller 4d0ea9978b getopt: let parse_arguments() take a slice of AsRef<str>
We don't need the content to be owned strings, and we don't
need the list to be a Vec, since we only care about being
able to iterate through and copy out portions of the strings
we need, so take an &[T] where T: AsRef<str>.

This avoids .iter().map(to_string).collect() before calling
parse_arguments().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 14:05:05 +01:00
Wolfgang Bumiller 2482c095b1 getopt: remove skip logic
The 'skip' variable was set to indicate that the "rest of
the args" is to be copied into the 'rest' vec. We can do
this directly and avoid the 'if' case in the loop
altogether.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:59:44 +01:00
Wolfgang Bumiller 99da3a073d getopt: indentation reduction
See the `git diff -w` output:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 6548a01..517cc37 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,7 +72,10 @@ pub fn parse_arguments(
|     while pos < args.len() {
|         if skip {
|             rest.push(args[pos].clone());
|-        } else {
|+            pos += 1;
|+            continue;
|+        }
|+
|         match parse_argument(&args[pos]) {
|             RawArgument::Separator => {
|                 skip = true;
|@@ -135,7 +138,6 @@ pub fn parse_arguments(
|                 rest.push(value);
|             }
|         }
|-        }
|
|         pos += 1;
|     }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:54:20 +01:00
Wolfgang Bumiller ad8a98f7e4 parse_arguments: work with utf8 bytes and reduce indentation
We are only caring about '=' and '-' which are single-byte
codepoints, so there's no reason not to work on a byte
slice.

Also, some indentation reduction:

Transform
    if (a) {
        ...
        return A;
    }
    return B;
into
    if (!a)
        return B;
    return A;

and
    if (a)
        if (b)
            foo()
into
    if (a && b)
        return;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-28 13:47:29 +01:00
Dietmar Maurer 248c17af39 src/catar/encoder.rs: write filenames 2018-12-28 11:48:47 +01:00
Dietmar Maurer 2e4ae0e239 src/catar/encoder.rs: write file data 2018-12-28 10:44:12 +01:00
Dietmar Maurer 3192ae968c src/catar/encoder.rs: write entry and symlink data 2018-12-28 09:55:26 +01:00
Dietmar Maurer 50ea43962e make map_struct functions public 2018-12-28 08:04:46 +01:00
Dietmar Maurer d2b03f2397 catar/encoder.rs: improve error handling 2018-12-28 07:45:15 +01:00
Dietmar Maurer 0ff559990c cleanups, avoid compiler warnings 2018-12-28 07:14:12 +01:00
Dietmar Maurer 45281d4927 catar/encoder: detect symlink and regular files 2018-12-27 14:24:31 +01:00
Dietmar Maurer fb8365b79f catar/encoder.rs: first try 2018-12-27 13:15:47 +01:00
Wolfgang Bumiller fba3437f1e timer: setup_timeout_handler cannot fail
and if it does it panics anyway

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-27 10:36:28 +01:00
Dietmar Maurer bcd879cfb3 backup-client: allow to specify directories (prepare for catar backup) 2018-12-27 10:11:11 +01:00
Dietmar Maurer b62b6cadf4 start implementing catar 2018-12-27 09:22:23 +01:00
Dietmar Maurer dc3de618ed new helper map_struct and map_struct_mut 2018-12-27 09:20:17 +01:00
Dietmar Maurer a198d74fc0 cleanup: reduce compiler warnings 2018-12-25 13:29:27 +01:00
Dietmar Maurer 991de6007b remove outdated comment 2018-12-25 13:24:50 +01:00
Dietmar Maurer 7ee2aa1b94 touch_chunk: use libc::lutimensat 2018-12-25 12:27:25 +01:00
Dietmar Maurer 2c32fdde86 move lookup_datastore() to backup/datastore.rs 2018-12-22 17:37:25 +01:00
Dietmar Maurer 64e53b2835 gather usage statistics during garbage collection 2018-12-22 16:58:16 +01:00
Dietmar Maurer 28b96b56e1 open_file_locked: improve error message 2018-12-22 15:59:55 +01:00
Dietmar Maurer 176e4af964 sweep_used_chunks: print percentage 2018-12-22 15:39:05 +01:00
Dietmar Maurer 7b2b40a893 use openssl for faster hashing 2018-12-22 14:31:59 +01:00
Dietmar Maurer e95950e40a chunk_store: reduce number of directories
Else, scans everything takes too long ...
2018-12-22 14:04:05 +01:00
Dietmar Maurer 15e9b4ed60 src/api3/datastore.rs: new file 2018-12-21 13:38:41 +01:00
Dietmar Maurer 3c140f60cd use return value (avoid compiler warning) 2018-12-21 12:44:20 +01:00
Dietmar Maurer 03e4753d8e fix mutability for chunk store 2018-12-21 12:15:26 +01:00
Dietmar Maurer 2d9d143a8f backup-client: add optional chunk-size parameter 2018-12-21 11:18:08 +01:00
Dietmar Maurer f0819fe5a5 backup-client: do not start garbage collection after backup 2018-12-21 09:24:55 +01:00
Dietmar Maurer d62e6e2264 backup-client: use 1M chunks, make chunk_size configurable 2018-12-21 08:36:57 +01:00
Dietmar Maurer c34eb16651 backup-clinet: specify target file name 2018-12-20 14:09:31 +01:00
Dietmar Maurer c2d9b3914a add test code to access static global state 2018-12-20 14:03:42 +01:00
Dietmar Maurer fe0e04c69c use proxmox-backup as package name 2018-12-20 10:45:14 +01:00
Dietmar Maurer f7dd683b46 fix tests 2018-12-19 13:54:22 +01:00
Dietmar Maurer 277fc5a3ce try to improve error messages 2018-12-19 13:40:26 +01:00
Dietmar Maurer eae8aa3aa9 cleanups: avoid compiler warnings 2018-12-19 12:49:23 +01:00
Dietmar Maurer 6c20a13d3c cleanup - avoid too much indentation 2018-12-19 12:40:26 +01:00
Dietmar Maurer 1c43c56b06 cleanup 2018-12-19 12:30:37 +01:00
Dietmar Maurer 2bf5f6b220 ChunkStore::sweep_used_chunks - use openat to speedup directory walks 2018-12-19 12:20:01 +01:00
Wolfgang Bumiller 1628a4c731 use timers with a signal for file locking
* rename lock_file -> open_file_locked,
* add lock_file as a function working on already-opened
  files
* change timeout types to std::time::Duration

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-19 11:23:00 +01:00
Wolfgang Bumiller 8cf6e764ad add tools::timer module
This wraps POSIX timers via timer_create(2).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2018-12-19 11:23:00 +01:00
Dietmar Maurer 365bb90f17 move lock_file to tools.rs 2018-12-19 10:02:24 +01:00
Dietmar Maurer 08481a0b60 impl sweep_used_chunks, first try 2018-12-19 09:51:33 +01:00
Dietmar Maurer 6ea3a0b7fc move sweep_used_chunks to chunk_store 2018-12-18 11:18:55 +01:00
Dietmar Maurer 3d5c11e5d2 start impl garbage collection 2018-12-18 11:06:03 +01:00
Dietmar Maurer 529de6c7a3 start implementing DataStore 2018-12-17 13:00:39 +01:00
Dietmar Maurer 4818c8b6f7 start implementing ImageIndexReader 2018-12-16 14:44:44 +01:00
Dietmar Maurer a27a3ee4ba avoid compiler warnings 2018-12-16 13:57:59 +01:00
Dietmar Maurer af3e7d7551 fix chunk store file paths 2018-12-16 13:52:16 +01:00
Dietmar Maurer 0cd9d4206f improve error messages 2018-12-16 13:43:19 +01:00
Dietmar Maurer 4fbb72a8b4 image_index.rs: implement atomic write/rename 2018-12-16 13:39:21 +01:00
Dietmar Maurer 798881a68a rebane ImageIndex to ImageIndexWriter 2018-12-16 11:51:16 +01:00
Dietmar Maurer 5e5b7f1c2d use u64 to store time 2018-12-16 11:48:03 +01:00
Dietmar Maurer d13e3745b3 image_index.rs: write header 2018-12-15 17:05:49 +01:00
Dietmar Maurer 606ce64bb0 image_index.rs:write idx of chunks (use mmap) 2018-12-15 14:51:05 +01:00
Dietmar Maurer f8838fe913 backup-client: add bash completion for datastore names 2018-12-15 11:24:39 +01:00
Dietmar Maurer 5f0c2d5652 remove debug println 2018-12-15 11:16:27 +01:00
Dietmar Maurer 43eeef28d1 move file_chunker to tools 2018-12-15 11:14:41 +01:00
Dietmar Maurer 48e8689505 file_chunker: use Read trait 2018-12-15 11:08:49 +01:00
Dietmar Maurer fc8b19368a backup-client: pass chunk_size as parameter to file_chunker 2018-12-15 10:58:48 +01:00
root a914a77448 backup-client: read file by chunks 2018-12-14 13:39:41 +01:00
root ff5d37074d bin/backup-client.rs: nbew tool to play with chunk store 2018-12-14 08:28:56 +01:00
root 15b64d4606 api3/config/datastore.rs: create datastore files 2018-12-13 14:41:14 +01:00
Dietmar Maurer 7f4e639b37 improve completion 2018-12-12 13:00:58 +01:00
Dietmar Maurer e3d40bb8cd complete_datastore_name: write more compact code 2018-12-12 12:41:59 +01:00
Dietmar Maurer 30d2e99c77 implement completion functions 2018-12-12 12:19:26 +01:00
Dietmar Maurer 91643d9012 section_config.rs: remove debug output 2018-12-12 12:19:26 +01:00
Dietmar Maurer d64d80d20a add comment 2018-12-12 11:23:04 +01:00
Dietmar Maurer 5ba6968922 add comment to make intensions more clear 2018-12-12 11:21:00 +01:00
Dietmar Maurer ca9caffa23 cleanup parameter types 2018-12-12 10:37:03 +01:00
Dietmar Maurer 38555b5072 implement completion for enums 2018-12-12 08:33:50 +01:00
Dietmar Maurer 2f0258959b improve completion 2018-12-12 08:18:38 +01:00
Dietmar Maurer f46403cc65 improve bash completion 2018-12-12 07:51:43 +01:00
Dietmar Maurer b6e8dd39aa impl bash completion: first try 2018-12-11 14:21:05 +01:00
Dietmar Maurer c21de379ed install bash completion scripts
Completion itself is not implemented.
2018-12-11 12:54:10 +01:00
Dietmar Maurer 6460764dbb cli/command.rs: add new type CliCommandMap 2018-12-11 11:31:36 +01:00
Dietmar Maurer bf7f103944 implement builder pattern for CliCommand 2018-12-11 11:12:13 +01:00
Dietmar Maurer baed30b702 cli/command.rs: implement prefix match 2018-12-10 18:13:55 +01:00
Dietmar Maurer 8f62336b0f impl From for CommandLineInterface 2018-12-10 13:51:10 +01:00
Dietmar Maurer 9f6ab1fc58 rename CmdDef to CommandLineInterface 2018-12-10 13:40:10 +01:00
Dietmar Maurer b7329c8a1a src/cli/command.rs: move code into separate file 2018-12-10 13:36:52 +01:00
Dietmar Maurer 211fabd795 try to support nested cli command definitions 2018-12-10 13:28:38 +01:00
Dietmar Maurer 34d3ba52eb api3/config/datastore.rs: implement delete 2018-12-09 16:52:32 +01:00
Dietmar Maurer 1a7bc3dd5e file_set_contents: correctly use perm parameter 2018-12-09 16:41:54 +01:00
Dietmar Maurer f12f8ff1a6 implement tools::file_set_contents 2018-12-09 16:37:48 +01:00
Dietmar Maurer 652c11900d api3/config/datastore.rs: impl create 2018-12-09 12:51:31 +01:00
Dietmar Maurer ea0b8b6ec0 bin/pbs-datastore.rs: start implementing cli commands 2018-12-09 11:59:32 +01:00
Dietmar Maurer 5b34c2607d rename entry to item 2018-12-09 10:25:56 +01:00
Dietmar Maurer b65eaac62d api3/config/datastore.rs: return real data 2018-12-09 10:23:19 +01:00
Dietmar Maurer 6349ed60b4 config/datastore.rs - open file for reading 2018-12-09 10:22:39 +01:00
Dietmar Maurer ce55dbbcc6 simplify code 2018-12-09 09:42:17 +01:00
Dietmar Maurer 461e62fcdf remove some compiler warnings 2018-12-08 14:55:54 +01:00
Dietmar Maurer 567713b4c3 rename data_store to datastore 2018-12-08 14:51:08 +01:00
Dietmar Maurer 6ce50400c5 cleanup api3 structure 2018-12-08 14:44:55 +01:00
Dietmar Maurer 678d72df6b add data_store configuration 2018-12-08 13:58:45 +01:00
Dietmar Maurer 391a2e43ff chunk_store.insert_chunk: return information about existing chunks 2018-12-08 11:25:11 +01:00
Dietmar Maurer 12bb93b3ad code cleanup 2018-12-08 11:20:02 +01:00
Dietmar Maurer 022fb421b2 lock with timeout
clumsy, but I have node idea how to make it better
2018-12-08 10:57:09 +01:00
Dietmar Maurer 2989f6bf20 improve error messages 2018-12-08 08:38:42 +01:00
Dietmar Maurer b8d4766a4d code cleanup 2018-12-08 08:21:20 +01:00
Dietmar Maurer 45773720ab depend on nix, use flock
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2018-12-07 18:14:07 +01:00
Dietmar Maurer c5d82e5f34 chunk_store: use mutex 2018-12-07 16:12:45 +01:00
Dietmar Maurer 128b37feea impl insert_chunk 2018-12-07 14:44:56 +01:00
Dietmar Maurer 35cf5daa53 start experiments with simple chunk store 2018-12-07 10:51:53 +01:00
Dietmar Maurer b22751b20c getopts.rs: fix test 2018-12-07 09:02:24 +01:00
Dietmar Maurer a4b7c3f2df more cleanups 2018-12-06 10:41:57 +01:00
Dietmar Maurer d3369f819d code cleanup 2018-12-06 10:23:45 +01:00
Dietmar Maurer c578fcd9e2 simplify formatter code 2018-12-05 18:22:56 +01:00
Dietmar Maurer 4b26195c6a extract version from environment 2018-12-05 14:40:45 +01:00
Dietmar Maurer 95b492add8 api: add /version 2018-12-05 14:03:15 +01:00
Dietmar Maurer 1571873d7b add output formatter 2018-12-05 12:43:22 +01:00
Dietmar Maurer 083418619e move src/api/server.rs -> src/server/rest.rs 2018-12-05 10:16:23 +01:00
Dietmar Maurer 5c7a1b15c8 start the GUI 2018-12-04 17:53:10 +01:00
Dietmar Maurer 826bb98286 set content type for static file download 2018-12-02 11:00:52 +01:00
Dietmar Maurer 8adbdb0a54 fix file download, listen to 0.0.0.0 2018-12-01 15:21:25 +01:00
Dietmar Maurer f4c514c13d remove www/pbs-index.html.tt, hardcode into rust for now 2018-12-01 13:37:49 +01:00
Dietmar Maurer e3871182c5 rename bin to src/bin/api-test-server.rs 2018-11-30 12:48:45 +01:00
Dietmar Maurer 73f29c34d7 improve command line parser 2018-11-30 12:10:26 +01:00
Dietmar Maurer d42398666c improve error message 2018-11-30 12:07:27 +01:00
Dietmar Maurer bfb1d69abc fix compiler warnings, add storage/config.rs 2018-11-30 11:15:26 +01:00
Dietmar Maurer 9131286246 remove missleading comment 2018-11-30 08:47:26 +01:00
Dietmar Maurer a27b905c28 use VecDeque instead of LinkedList 2018-11-29 17:28:28 +01:00