tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
345 Commits 2 Branches 54 Tags 14 MiB
4f6892eb74
Commit Graph

321 Commits

Author SHA1 Message Date
Dietmar Maurer
0433db1949 src/backup/archive_index.rs: first try 2018-12-31 17:30:08 +01:00
Dietmar Maurer
cbdd8c54ae create backup mod in backup.rs, improve docu 2018-12-31 16:08:04 +01:00
Dietmar Maurer
cb4412b18e move chunker.rs to backup/ 2018-12-31 15:46:16 +01:00
Dietmar Maurer
1c287cb19a rc/catar/chunker.rs: first try 2018-12-31 13:01:06 +01:00
Dietmar Maurer
bf205f94c5 avoid compiler warnings 2018-12-31 10:54:25 +01:00
Dietmar Maurer
1b0dc9f680 src/catar/encoder.rs: correctly sort goodbye items by hash key 2018-12-31 10:11:28 +01:00
Dietmar Maurer
c60d34bdbf new catar binary 
currently used for debugging
 2018-12-30 18:02:46 +01:00
Dietmar Maurer
e75eac73ca fix doc syntax 2018-12-30 17:43:53 +01:00
Dietmar Maurer
0866748de6 catar cleanups ... 2018-12-30 17:32:52 +01:00
Dietmar Maurer
6cd28d200e binary_search_tree.rs: fix docs 2018-12-30 15:34:43 +01:00
Dietmar Maurer
389e562524 src/catar/encoder.rs: auto-resize file_copy_buffer, limit number of dirs 2018-12-30 14:09:59 +01:00
Dietmar Maurer
4fa71e0573 improve catar docs 2018-12-30 13:47:27 +01:00
Dietmar Maurer
48147efd12 add a comment about posible improvements 2018-12-29 19:43:25 +01:00
Dietmar Maurer
0b78833d8e src/catar/binary_search_tree.rs: add regression tests and fix one bug 2018-12-29 18:32:03 +01:00
Dietmar Maurer
b17d7149d2 src/catar/binary_search_tree.rs: improve docu 2018-12-29 17:38:50 +01:00
Dietmar Maurer
985567fb45 src/catar/encoder.rs: cleanup, factor out write_goodbye_table 2018-12-29 17:26:32 +01:00
Dietmar Maurer
95bd5dfec7 src/catar/encoder.rs: correctly sort goodby items 2018-12-29 17:00:48 +01:00
Dietmar Maurer
46b6fbd6ae use external crate siphasher 
Because std:#️⃣:SipHasher is deprecated.
 2018-12-28 19:50:07 +01:00
Dietmar Maurer
f0f3029e26 src/catar/encoder.rs: fix bug in file copy 2018-12-28 15:02:42 +01:00
Dietmar Maurer
d05f93215e src/catar/encoder.rs: fix symlinks 2018-12-28 15:02:42 +01:00
Wolfgang Bumiller
7ebb173352 getopt: cleanup: add trailing commas 
It is customary in rust to always use trailing commas. (also
suggested by rustfmt)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:27:38 +01:00
Wolfgang Bumiller
2767c5d39b getopt: cleanup: don't condense 'if' statements too much 
In a language which enforces curly braces this looks weird
and rustfmt doesn't like it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:27:38 +01:00
Wolfgang Bumiller
7d95c10da0 getopt: whitespace cleanup 
to make rustfmt happy

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:27:38 +01:00
Dietmar Maurer
a0cc09b5f0 src/catar/encoder.rs: fix binary format, write goodby table 
We still ned to sort the table (BST) ...
 2018-12-28 14:27:00 +01:00
Wolfgang Bumiller
03fb895197 getopt: condense nested match to reduce indentation 
The `match value` statement is the only thing covering the
entire RawArgument::Option case. `rustfmt` suggests this
more condensed way of writing this case.

See the `git diff -w` of this patch:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 9755af2..4db4579 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,8 +72,7 @@ pub fn parse_arguments<T: AsRef<str>>(
|             RawArgument::Separator => {
|                 break;
|             }
|-            RawArgument::Option { name, value } => {
|-                match value {
|+            RawArgument::Option { name, value } => match value {
|                 None => {
|                     let mut want_bool = false;
|                     let mut can_default = false;
|@@ -125,7 +124,6 @@ pub fn parse_arguments<T: AsRef<str>>(
|                     data.push((name, v));
|                 }
|             }
|-            }
|             RawArgument::Argument { value } => {
|                 rest.push(value);
|             }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:16:58 +01:00
Wolfgang Bumiller
4d0ea9978b getopt: let parse_arguments() take a slice of AsRef<str> 
We don't need the content to be owned strings, and we don't
need the list to be a Vec, since we only care about being
able to iterate through and copy out portions of the strings
we need, so take an &[T] where T: AsRef<str>.

This avoids .iter().map(to_string).collect() before calling
parse_arguments().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:05:05 +01:00
Wolfgang Bumiller
2482c095b1 getopt: remove skip logic 
The 'skip' variable was set to indicate that the "rest of
the args" is to be copied into the 'rest' vec. We can do
this directly and avoid the 'if' case in the loop
altogether.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 13:59:44 +01:00
Wolfgang Bumiller
99da3a073d getopt: indentation reduction 
See the `git diff -w` output:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 6548a01..517cc37 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,7 +72,10 @@ pub fn parse_arguments(
|     while pos < args.len() {
|         if skip {
|             rest.push(args[pos].clone());
|-        } else {
|+            pos += 1;
|+            continue;
|+        }
|+
|         match parse_argument(&args[pos]) {
|             RawArgument::Separator => {
|                 skip = true;
|@@ -135,7 +138,6 @@ pub fn parse_arguments(
|                 rest.push(value);
|             }
|         }
|-        }
|
|         pos += 1;
|     }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 13:54:20 +01:00
Wolfgang Bumiller
ad8a98f7e4 parse_arguments: work with utf8 bytes and reduce indentation 
We are only caring about '=' and '-' which are single-byte
codepoints, so there's no reason not to work on a byte
slice.

Also, some indentation reduction:

Transform
    if (a) {
        ...
        return A;
    }
    return B;
into
    if (!a)
        return B;
    return A;

and
    if (a)
        if (b)
            foo()
into
    if (a && b)
        return;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 13:47:29 +01:00
Dietmar Maurer
248c17af39 src/catar/encoder.rs: write filenames 2018-12-28 11:48:47 +01:00
Dietmar Maurer
2e4ae0e239 src/catar/encoder.rs: write file data 2018-12-28 10:44:12 +01:00
Dietmar Maurer
3192ae968c src/catar/encoder.rs: write entry and symlink data 2018-12-28 09:55:26 +01:00
Dietmar Maurer
50ea43962e make map_struct functions public 2018-12-28 08:04:46 +01:00
Dietmar Maurer
d2b03f2397 catar/encoder.rs: improve error handling 2018-12-28 07:45:15 +01:00
Dietmar Maurer
0ff559990c cleanups, avoid compiler warnings 2018-12-28 07:14:12 +01:00
Dietmar Maurer
45281d4927 catar/encoder: detect symlink and regular files 2018-12-27 14:24:31 +01:00
Dietmar Maurer
fb8365b79f catar/encoder.rs: first try 2018-12-27 13:15:47 +01:00
Wolfgang Bumiller
fba3437f1e timer: setup_timeout_handler cannot fail 
and if it does it panics anyway

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-27 10:36:28 +01:00
Dietmar Maurer
bcd879cfb3 backup-client: allow to specify directories (prepare for catar backup) 2018-12-27 10:11:11 +01:00
Dietmar Maurer
b62b6cadf4 start implementing catar 2018-12-27 09:22:23 +01:00
Dietmar Maurer
dc3de618ed new helper map_struct and map_struct_mut 2018-12-27 09:20:17 +01:00
Dietmar Maurer
a198d74fc0 cleanup: reduce compiler warnings 2018-12-25 13:29:27 +01:00
Dietmar Maurer
991de6007b remove outdated comment 2018-12-25 13:24:50 +01:00
Dietmar Maurer
7ee2aa1b94 touch_chunk: use libc::lutimensat 2018-12-25 12:27:25 +01:00
Dietmar Maurer
2c32fdde86 move lookup_datastore() to backup/datastore.rs 2018-12-22 17:37:25 +01:00
Dietmar Maurer
64e53b2835 gather usage statistics during garbage collection 2018-12-22 16:58:16 +01:00
Dietmar Maurer
28b96b56e1 open_file_locked: improve error message 2018-12-22 15:59:55 +01:00
Dietmar Maurer
176e4af964 sweep_used_chunks: print percentage 2018-12-22 15:39:05 +01:00
Dietmar Maurer
7b2b40a893 use openssl for faster hashing 2018-12-22 14:31:59 +01:00
Dietmar Maurer
e95950e40a chunk_store: reduce number of directories 
Else, scans everything takes too long ...
 2018-12-22 14:04:05 +01:00
Dietmar Maurer
15e9b4ed60 src/api3/datastore.rs: new file 2018-12-21 13:38:41 +01:00
Dietmar Maurer
3c140f60cd use return value (avoid compiler warning) 2018-12-21 12:44:20 +01:00
Dietmar Maurer
03e4753d8e fix mutability for chunk store 2018-12-21 12:15:26 +01:00
Dietmar Maurer
2d9d143a8f backup-client: add optional chunk-size parameter 2018-12-21 11:18:08 +01:00
Dietmar Maurer
f0819fe5a5 backup-client: do not start garbage collection after backup 2018-12-21 09:24:55 +01:00
Dietmar Maurer
d62e6e2264 backup-client: use 1M chunks, make chunk_size configurable 2018-12-21 08:36:57 +01:00
Dietmar Maurer
c34eb16651 backup-clinet: specify target file name 2018-12-20 14:09:31 +01:00
Dietmar Maurer
c2d9b3914a add test code to access static global state 2018-12-20 14:03:42 +01:00
Dietmar Maurer
fe0e04c69c use proxmox-backup as package name 2018-12-20 10:45:14 +01:00
Dietmar Maurer
f7dd683b46 fix tests 2018-12-19 13:54:22 +01:00
Dietmar Maurer
277fc5a3ce try to improve error messages 2018-12-19 13:40:26 +01:00
Dietmar Maurer
eae8aa3aa9 cleanups: avoid compiler warnings 2018-12-19 12:49:23 +01:00
Dietmar Maurer
6c20a13d3c cleanup - avoid too much indentation 2018-12-19 12:40:26 +01:00
Dietmar Maurer
1c43c56b06 cleanup 2018-12-19 12:30:37 +01:00
Dietmar Maurer
2bf5f6b220 ChunkStore::sweep_used_chunks - use openat to speedup directory walks 2018-12-19 12:20:01 +01:00
Wolfgang Bumiller
1628a4c731 use timers with a signal for file locking 
* rename lock_file -> open_file_locked,
* add lock_file as a function working on already-opened
  files
* change timeout types to std::time::Duration

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-19 11:23:00 +01:00
Wolfgang Bumiller
8cf6e764ad add tools::timer module 
This wraps POSIX timers via timer_create(2).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-19 11:23:00 +01:00
Dietmar Maurer
365bb90f17 move lock_file to tools.rs 2018-12-19 10:02:24 +01:00
Dietmar Maurer
08481a0b60 impl sweep_used_chunks, first try 2018-12-19 09:51:33 +01:00
Dietmar Maurer
6ea3a0b7fc move sweep_used_chunks to chunk_store 2018-12-18 11:18:55 +01:00
Dietmar Maurer
3d5c11e5d2 start impl garbage collection 2018-12-18 11:06:03 +01:00
Dietmar Maurer
529de6c7a3 start implementing DataStore 2018-12-17 13:00:39 +01:00
Dietmar Maurer
4818c8b6f7 start implementing ImageIndexReader 2018-12-16 14:44:44 +01:00
Dietmar Maurer
a27a3ee4ba avoid compiler warnings 2018-12-16 13:57:59 +01:00
Dietmar Maurer
af3e7d7551 fix chunk store file paths 2018-12-16 13:52:16 +01:00
Dietmar Maurer
0cd9d4206f improve error messages 2018-12-16 13:43:19 +01:00
Dietmar Maurer
4fbb72a8b4 image_index.rs: implement atomic write/rename 2018-12-16 13:39:21 +01:00
Dietmar Maurer
798881a68a rebane ImageIndex to ImageIndexWriter 2018-12-16 11:51:16 +01:00
Dietmar Maurer
5e5b7f1c2d use u64 to store time 2018-12-16 11:48:03 +01:00
Dietmar Maurer
d13e3745b3 image_index.rs: write header 2018-12-15 17:05:49 +01:00
Dietmar Maurer
606ce64bb0 image_index.rs:write idx of chunks (use mmap) 2018-12-15 14:51:05 +01:00
Dietmar Maurer
f8838fe913 backup-client: add bash completion for datastore names 2018-12-15 11:24:39 +01:00
Dietmar Maurer
5f0c2d5652 remove debug println 2018-12-15 11:16:27 +01:00
Dietmar Maurer
43eeef28d1 move file_chunker to tools 2018-12-15 11:14:41 +01:00
Dietmar Maurer
48e8689505 file_chunker: use Read trait 2018-12-15 11:08:49 +01:00
Dietmar Maurer
fc8b19368a backup-client: pass chunk_size as parameter to file_chunker 2018-12-15 10:58:48 +01:00
root
a914a77448 backup-client: read file by chunks 2018-12-14 13:39:41 +01:00
root
ff5d37074d bin/backup-client.rs: nbew tool to play with chunk store 2018-12-14 08:28:56 +01:00
root
15b64d4606 api3/config/datastore.rs: create datastore files 2018-12-13 14:41:14 +01:00
Dietmar Maurer
7f4e639b37 improve completion 2018-12-12 13:00:58 +01:00
Dietmar Maurer
e3d40bb8cd complete_datastore_name: write more compact code 2018-12-12 12:41:59 +01:00
Dietmar Maurer
30d2e99c77 implement completion functions 2018-12-12 12:19:26 +01:00
Dietmar Maurer
91643d9012 section_config.rs: remove debug output 2018-12-12 12:19:26 +01:00
Dietmar Maurer
d64d80d20a add comment 2018-12-12 11:23:04 +01:00
Dietmar Maurer
5ba6968922 add comment to make intensions more clear 2018-12-12 11:21:00 +01:00
Dietmar Maurer
ca9caffa23 cleanup parameter types 2018-12-12 10:37:03 +01:00
Dietmar Maurer
38555b5072 implement completion for enums 2018-12-12 08:33:50 +01:00
Dietmar Maurer
2f0258959b improve completion 2018-12-12 08:18:38 +01:00
Dietmar Maurer
f46403cc65 improve bash completion 2018-12-12 07:51:43 +01:00
Dietmar Maurer
b6e8dd39aa impl bash completion: first try 2018-12-11 14:21:05 +01:00