tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
401 Commits 2 Branches 54 Tags 14 MiB
059ca7c3a1
Commit Graph

375 Commits

Author SHA1 Message Date
Dietmar Maurer
a7e3713122 catar: encode/decode devices 2019-01-11 12:22:00 +01:00
Dietmar Maurer
1bdb3130f7 catar/format_definition.rs: add more feature flags 2019-01-11 10:44:57 +01:00
Dietmar Maurer
5c76c2f331 catar/encoder.rs: store fat file system attributes 2019-01-11 10:18:22 +01:00
Dietmar Maurer
8c1dfa6c72 catar/encoder.rs: cleanups 2019-01-11 10:01:51 +01:00
Dietmar Maurer
4f6892eb74 catar/encoder.rs: encode linux fs attributes, cleanup encoder 
Try to avoid duplicate stat calls (assume file tree is stable during backup).
 2019-01-11 09:20:10 +01:00
Dietmar Maurer
ddbdf80d51 avoid compiler warnings 2019-01-11 08:41:33 +01:00
Dietmar Maurer
373ef4a504 backup/archive_index.rs: correctly check for errors 2019-01-10 11:19:54 +01:00
Dietmar Maurer
23f68e53b4 catar/decoder.rs: restore mode, uid, gid and mtime 2019-01-10 09:02:50 +01:00
Dietmar Maurer
25f60394d2 catar/decoder.rs: avoid slashes in file names 2019-01-10 09:02:50 +01:00
Dietmar Maurer
9b1bb5a277 catar/decoder.rs: impl restore dirs, files, and symlinks 2019-01-10 09:01:36 +01:00
Wolfgang Bumiller
1ed86a0bae rest: rename utf-8-checked 'bytes' to 'utf8' 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-01-08 14:22:43 +01:00
Wolfgang Bumiller
9fec7ae593 rest: don't copy the body 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-01-08 14:21:54 +01:00
Wolfgang Bumiller
d9fbcc4631 tools/timer: fixup doc formatting 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-01-08 14:11:10 +01:00
Dietmar Maurer
676bf71ad4 catar/decoder.rs: restore_sequential - read payload 2019-01-08 09:26:07 +01:00
Dietmar Maurer
c6a0d3fc6b catar/decoder.rs: improve restore_sequential 2019-01-07 19:18:59 +01:00
Dietmar Maurer
b6ebfb8dab catar/decoder.rs: start implementing sequential decoder, cleanups 
And use [derive(Endian)] for endian conversion.
 2019-01-07 19:07:03 +01:00
Dietmar Maurer
02c7d8e522 catar: implement create 2019-01-07 13:25:41 +01:00
Dietmar Maurer
e86c49247e bin/catar.rs: add list command 2019-01-06 17:42:23 +01:00
Dietmar Maurer
3d8c24ecf8 catar/decoder.rs: start implementation 2019-01-06 17:27:57 +01:00
Dietmar Maurer
4b864ad436 format_definition.rs: remove wrong docs 2019-01-06 17:27:22 +01:00
Dietmar Maurer
4624fe29cc impl Read for BufferedArchiveReader 2019-01-06 10:04:45 +01:00
Dietmar Maurer
343370504b impl Seek for BufferedArchiveReader 2019-01-06 09:35:39 +01:00
Dietmar Maurer
318564ac03 buffered_read: return empty buffer on eof 2019-01-06 09:17:28 +01:00
Dietmar Maurer
0a72e26704 define and use BufferedReader trait 2019-01-05 17:28:20 +01:00
Dietmar Maurer
1006acd626 remove dead code 2019-01-05 16:55:21 +01:00
Dietmar Maurer
51b499db74 tools.rs: improve docs 2019-01-05 16:53:28 +01:00
Dietmar Maurer
39c6bd86cc backup/archive_index.rs: implement BufferedArchiveReader 
Implement relativly fast random read using binary search.
 2019-01-05 14:47:56 +01:00
Dietmar Maurer
060c4811a0 backup/chunk_store.rs: resize buffer by powers of two 2019-01-05 11:34:49 +01:00
Dietmar Maurer
9409255aae catar/encoder.rs: code cleanup 
Note: Rust Vec resize uses len(), not capacity()!
 2019-01-04 17:23:01 +01:00
Dietmar Maurer
df9973e8f5 backup/chunk_store.rs: avoid allocation in read_chunk 2019-01-04 17:16:56 +01:00
Dietmar Maurer
7394ca3e95 white space cleanups 2019-01-04 12:51:43 +01:00
Dietmar Maurer
96df2fb44b backup/chunk_store.rs: implement read_chunk() 2019-01-04 12:50:54 +01:00
Dietmar Maurer
691c89a0fb proxmox-backup-manager: add garbage collection cli 2019-01-04 11:34:32 +01:00
Dietmar Maurer
07b4694a33 cleanup crate names 2019-01-04 10:49:52 +01:00
Dietmar Maurer
4bdba5f4f1 rename backup-client to proxmox-backup-client 2019-01-04 10:41:37 +01:00
Dietmar Maurer
a689f6df0e rename pbs to proxmox-backup-manager 2019-01-04 10:39:32 +01:00
Dietmar Maurer
bc616633d5 backup/chunk_store.rs: only use one directory level 2019-01-04 10:35:22 +01:00
Dietmar Maurer
030cc2a903 move admin API to /admin/ subfolder 2019-01-04 10:32:27 +01:00
Dietmar Maurer
b663789bdb backup/archive_index.rs: pass correct size to unmap 2019-01-04 09:28:41 +01:00
Dietmar Maurer
48d0d35688 backup/image_index.rs: use little endian 2019-01-04 08:45:45 +01:00
Wolfgang Bumiller
4968bc3ac0 cli::command: wrap usage errors in a UsageError 
So we can distinguish them and show usage output
conditionally.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-01-03 16:08:51 +01:00
Dietmar Maurer
c7fa487250 catar/encoder.rs: make sure we write all data with write_all() 2019-01-03 15:47:32 +01:00
Dietmar Maurer
050f8a6355 chunker.rs: make sure chunk_size_avg is a power of two 2019-01-03 15:10:54 +01:00
Dietmar Maurer
ea4ea34baf improve chunker speed (avoid modulo) 2019-01-03 14:35:19 +01:00
Dietmar Maurer
3329ae8c2e add benchmark for chunker 
We get about 300MB/s - so this is still not fast enough

Note: This is quite slow with debug target, so please compile with --release
 2019-01-02 19:18:53 +01:00
Dietmar Maurer
0b8e75ed16 image_index.rs: verify file size 2019-01-02 18:14:02 +01:00
Dietmar Maurer
77703d95aa implement garbage collection for .aidx files 2019-01-02 14:27:04 +01:00
Dietmar Maurer
a360f6fa2d image_index.rs: verify header magic/version 2019-01-02 13:13:13 +01:00
Dietmar Maurer
44b3f62b42 fix typo 2019-01-02 12:56:04 +01:00
Dietmar Maurer
594fa52016 disable debug code 2019-01-02 12:55:18 +01:00
Dietmar Maurer
5032b57b44 archive_index.rs: correctly write .aidx file 2019-01-02 12:54:40 +01:00
Dietmar Maurer
580dc84bac image_index.rs: print stats 2019-01-02 12:53:49 +01:00
Dietmar Maurer
5e7a09be0d src/backup/archive_index.rs: use close() instead of flush() 
Also pass a reference to the encoder.
 2019-01-02 11:02:56 +01:00
Dietmar Maurer
94a882e900 src/backup/archive_index.rs: implement flush() 2018-12-31 18:01:07 +01:00
Dietmar Maurer
0433db1949 src/backup/archive_index.rs: first try 2018-12-31 17:30:08 +01:00
Dietmar Maurer
cbdd8c54ae create backup mod in backup.rs, improve docu 2018-12-31 16:08:04 +01:00
Dietmar Maurer
cb4412b18e move chunker.rs to backup/ 2018-12-31 15:46:16 +01:00
Dietmar Maurer
1c287cb19a rc/catar/chunker.rs: first try 2018-12-31 13:01:06 +01:00
Dietmar Maurer
bf205f94c5 avoid compiler warnings 2018-12-31 10:54:25 +01:00
Dietmar Maurer
1b0dc9f680 src/catar/encoder.rs: correctly sort goodbye items by hash key 2018-12-31 10:11:28 +01:00
Dietmar Maurer
c60d34bdbf new catar binary 
currently used for debugging
 2018-12-30 18:02:46 +01:00
Dietmar Maurer
e75eac73ca fix doc syntax 2018-12-30 17:43:53 +01:00
Dietmar Maurer
0866748de6 catar cleanups ... 2018-12-30 17:32:52 +01:00
Dietmar Maurer
6cd28d200e binary_search_tree.rs: fix docs 2018-12-30 15:34:43 +01:00
Dietmar Maurer
389e562524 src/catar/encoder.rs: auto-resize file_copy_buffer, limit number of dirs 2018-12-30 14:09:59 +01:00
Dietmar Maurer
4fa71e0573 improve catar docs 2018-12-30 13:47:27 +01:00
Dietmar Maurer
48147efd12 add a comment about posible improvements 2018-12-29 19:43:25 +01:00
Dietmar Maurer
0b78833d8e src/catar/binary_search_tree.rs: add regression tests and fix one bug 2018-12-29 18:32:03 +01:00
Dietmar Maurer
b17d7149d2 src/catar/binary_search_tree.rs: improve docu 2018-12-29 17:38:50 +01:00
Dietmar Maurer
985567fb45 src/catar/encoder.rs: cleanup, factor out write_goodbye_table 2018-12-29 17:26:32 +01:00
Dietmar Maurer
95bd5dfec7 src/catar/encoder.rs: correctly sort goodby items 2018-12-29 17:00:48 +01:00
Dietmar Maurer
46b6fbd6ae use external crate siphasher 
Because std:#️⃣:SipHasher is deprecated.
 2018-12-28 19:50:07 +01:00
Dietmar Maurer
f0f3029e26 src/catar/encoder.rs: fix bug in file copy 2018-12-28 15:02:42 +01:00
Dietmar Maurer
d05f93215e src/catar/encoder.rs: fix symlinks 2018-12-28 15:02:42 +01:00
Wolfgang Bumiller
7ebb173352 getopt: cleanup: add trailing commas 
It is customary in rust to always use trailing commas. (also
suggested by rustfmt)

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:27:38 +01:00
Wolfgang Bumiller
2767c5d39b getopt: cleanup: don't condense 'if' statements too much 
In a language which enforces curly braces this looks weird
and rustfmt doesn't like it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:27:38 +01:00
Wolfgang Bumiller
7d95c10da0 getopt: whitespace cleanup 
to make rustfmt happy

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:27:38 +01:00
Dietmar Maurer
a0cc09b5f0 src/catar/encoder.rs: fix binary format, write goodby table 
We still ned to sort the table (BST) ...
 2018-12-28 14:27:00 +01:00
Wolfgang Bumiller
03fb895197 getopt: condense nested match to reduce indentation 
The `match value` statement is the only thing covering the
entire RawArgument::Option case. `rustfmt` suggests this
more condensed way of writing this case.

See the `git diff -w` of this patch:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 9755af2..4db4579 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,8 +72,7 @@ pub fn parse_arguments<T: AsRef<str>>(
|             RawArgument::Separator => {
|                 break;
|             }
|-            RawArgument::Option { name, value } => {
|-                match value {
|+            RawArgument::Option { name, value } => match value {
|                 None => {
|                     let mut want_bool = false;
|                     let mut can_default = false;
|@@ -125,7 +124,6 @@ pub fn parse_arguments<T: AsRef<str>>(
|                     data.push((name, v));
|                 }
|             }
|-            }
|             RawArgument::Argument { value } => {
|                 rest.push(value);
|             }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:16:58 +01:00
Wolfgang Bumiller
4d0ea9978b getopt: let parse_arguments() take a slice of AsRef<str> 
We don't need the content to be owned strings, and we don't
need the list to be a Vec, since we only care about being
able to iterate through and copy out portions of the strings
we need, so take an &[T] where T: AsRef<str>.

This avoids .iter().map(to_string).collect() before calling
parse_arguments().

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 14:05:05 +01:00
Wolfgang Bumiller
2482c095b1 getopt: remove skip logic 
The 'skip' variable was set to indicate that the "rest of
the args" is to be copied into the 'rest' vec. We can do
this directly and avoid the 'if' case in the loop
altogether.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 13:59:44 +01:00
Wolfgang Bumiller
99da3a073d getopt: indentation reduction 
See the `git diff -w` output:
|diff --git a/src/getopts.rs b/src/getopts.rs
|index 6548a01..517cc37 100644
|--- a/src/getopts.rs
|+++ b/src/getopts.rs
|@@ -72,7 +72,10 @@ pub fn parse_arguments(
|     while pos < args.len() {
|         if skip {
|             rest.push(args[pos].clone());
|-        } else {
|+            pos += 1;
|+            continue;
|+        }
|+
|         match parse_argument(&args[pos]) {
|             RawArgument::Separator => {
|                 skip = true;
|@@ -135,7 +138,6 @@ pub fn parse_arguments(
|                 rest.push(value);
|             }
|         }
|-        }
|
|         pos += 1;
|     }

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 13:54:20 +01:00
Wolfgang Bumiller
ad8a98f7e4 parse_arguments: work with utf8 bytes and reduce indentation 
We are only caring about '=' and '-' which are single-byte
codepoints, so there's no reason not to work on a byte
slice.

Also, some indentation reduction:

Transform
    if (a) {
        ...
        return A;
    }
    return B;
into
    if (!a)
        return B;
    return A;

and
    if (a)
        if (b)
            foo()
into
    if (a && b)
        return;

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-28 13:47:29 +01:00
Dietmar Maurer
248c17af39 src/catar/encoder.rs: write filenames 2018-12-28 11:48:47 +01:00
Dietmar Maurer
2e4ae0e239 src/catar/encoder.rs: write file data 2018-12-28 10:44:12 +01:00
Dietmar Maurer
3192ae968c src/catar/encoder.rs: write entry and symlink data 2018-12-28 09:55:26 +01:00
Dietmar Maurer
50ea43962e make map_struct functions public 2018-12-28 08:04:46 +01:00
Dietmar Maurer
d2b03f2397 catar/encoder.rs: improve error handling 2018-12-28 07:45:15 +01:00
Dietmar Maurer
0ff559990c cleanups, avoid compiler warnings 2018-12-28 07:14:12 +01:00
Dietmar Maurer
45281d4927 catar/encoder: detect symlink and regular files 2018-12-27 14:24:31 +01:00
Dietmar Maurer
fb8365b79f catar/encoder.rs: first try 2018-12-27 13:15:47 +01:00
Wolfgang Bumiller
fba3437f1e timer: setup_timeout_handler cannot fail 
and if it does it panics anyway

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2018-12-27 10:36:28 +01:00
Dietmar Maurer
bcd879cfb3 backup-client: allow to specify directories (prepare for catar backup) 2018-12-27 10:11:11 +01:00
Dietmar Maurer
b62b6cadf4 start implementing catar 2018-12-27 09:22:23 +01:00
Dietmar Maurer
dc3de618ed new helper map_struct and map_struct_mut 2018-12-27 09:20:17 +01:00
Dietmar Maurer
a198d74fc0 cleanup: reduce compiler warnings 2018-12-25 13:29:27 +01:00
Dietmar Maurer
991de6007b remove outdated comment 2018-12-25 13:24:50 +01:00
Dietmar Maurer
7ee2aa1b94 touch_chunk: use libc::lutimensat 2018-12-25 12:27:25 +01:00
Dietmar Maurer
2c32fdde86 move lookup_datastore() to backup/datastore.rs 2018-12-22 17:37:25 +01:00
Dietmar Maurer
64e53b2835 gather usage statistics during garbage collection 2018-12-22 16:58:16 +01:00