Commit Graph

258 Commits

Author SHA1 Message Date
Wolfgang Bumiller 340c0bf9e3 pxar: don't clone patterns unnecessarily
The options struct has no Drop handler and is passed by-move
so we can partially move out of it.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2021-01-26 10:24:18 +01:00
Fabian Grünbichler 72064fd0df pxar: extract PxarExtractOptions
same as PxarCreateOptions, but for extraction/restore rather than
create.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:36 +01:00
Fabian Grünbichler 77486a608e pxar: factor out PxarCreateOptions
containing the CLI parameters that are mostly passed-through from the
client to our pxar archive creation wrapper in pxar::create

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:32 +01:00
Fabian Grünbichler e97025ab02 pxar: typedef on_error as ErrorHandler
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-26 09:54:26 +01:00
Fabian Grünbichler 96b7483138 clippy: remove/replace needless explicit lifetimes
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
Fabian Grünbichler a6bd669854 clippy: use matches!
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:23:54 +01:00
Fabian Grünbichler 22a9189ee0 clippy: remove unnecessary closures
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2021-01-20 16:22:59 +01:00
Wolfgang Bumiller dd519bbad1 pxar: stricter file descriptor guards
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-30 11:25:53 +01:00
Fabian Ebner c9097ff801 pxar: avoid including archive root's exclude patterns in .pxarexclude-cli
The patterns from the archive root's .pxarexclude file are already present in
self.patterns when encode_pxarexclude_cli is called. Pass along the number of
CLI patterns and slice accordingly.

Suggested-By: Wolfgang Bumiller <w.bumiller@proxmox.com>
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 13:05:09 +01:00
Fabian Ebner fa4bcbcad0 pxar: only generate .pxarexclude-cli if there were CLI parameters
previously a .pxarexclude entry in the root of the archive caused the file to
be generated as well, because the patterns are read before calling
generate_directory_file_list and within the function it wasn't possible to
distinguish between a pattern coming from the CLI and a pattern coming from
archive/root/.pxarexclude

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:18:08 +01:00
Fabian Ebner 189cdb7427 pxar: include .pxarexclude files in the archive
The documentation states:
.pxarexclude files are treated as regular files and will be included in the
backup archive.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:18:06 +01:00
Fabian Ebner 874bd5454d pxar: fix anchored exclusion at archive root
There is no leading slash in an entry's full_path, causing an anchored
exclude at the root level to fail, e.g. having "/name" as the content of the
file archive/root/.pxarexclude didn't match the file archive/root/name

Fix this by prepending a leading slash before matching.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-12 11:18:04 +01:00
Wolfgang Bumiller 47208b4147 pxar: log when skipping mount points
Clippy complains about the number of paramters we have for
create_archive and it really does need to be made somewhat
less awkward and more usable. For now we just log to stderr
as we previously did. Added todo-comments for this.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-09 12:43:16 +01:00
Wolfgang Bumiller e144810d73 pxar: more concise EOF handling
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-05 10:32:48 +01:00
Dominik Csapak 3c2dd8ad05 pxar/create: handle ErrorKind::Interrupted for file reads
they are not an error and we should retry the read

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-05 10:27:36 +01:00
Dominik Csapak 91e3b38da4 pxar/create: fix endless loop for shrinking files
when a file shrunk during backup, we endlessly looped, reading/copying 0 bytes
we already have code that handles shrunk files, but we forgot to
break from the read loop

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-11-05 10:27:30 +01:00
Dominik Csapak 5499bd3dee fix #2998: encode mtime as i64 instead of u64
saves files mtime as i64 instead of u64 which enables backup of
files with negative mtime

the catalog_decode_i64 is compatible to encoded u64 values (if < 2^63)
but not reverse, so all "old" catalogs can be read with the new
decoder, but catalogs that contain negative mtimes will decode wrongly
on older clients

also remove the arbitrary maximum value of 2^63 - 1 for
encode_u64 (we just use up to 10 bytes now) and correctly
decode them and update the comments accordingly

adds also test for i64 encode/decode and for compatibility between
u64 encode and i64 decode

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
2020-10-29 08:51:10 +01:00
Wolfgang Bumiller 8d841f81ee pxar: anchor pxarexcludes starting with a slash
Given the .pxarexclude file

    foo
    /bar

The following happens:

    exclude: /foo
    exclude: /bar
    exclude: /subdir/foo
    include: /subdir/bar

since the `/bar` line is an absolute path

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-15 12:28:31 +02:00
Wolfgang Bumiller 32a4695c46 pxar: fix relative '!' rules in .pxarexclude
and reduce indentation

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-15 12:18:34 +02:00
Wolfgang Bumiller 8db1468952 more clippy fixups
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-14 13:58:35 +02:00
Wolfgang Bumiller 97bbd1bf9e pxar: pass full path to callback, let verbose flag list files
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-10-14 11:31:42 +02:00
Wolfgang Bumiller 6fd129844d remove DummyCatalogWriter
we're using an `Option` instead now

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-09-24 09:13:54 +02:00
Dietmar Maurer 6a7be83efe avoid chrono dependency, depend on proxmox 0.3.8
- remove chrono dependency

- depend on proxmox 0.3.8

- remove epoch_now, epoch_now_u64 and epoch_now_f64

- remove tm_editor (moved to proxmox crate)

- use new helpers from proxmox 0.3.8
  * epoch_i64 and epoch_f64
  * parse_rfc3339
  * epoch_to_rfc3339_utc
  * strftime_local

- BackupDir changes:
  * store epoch and rfc3339 string instead of DateTime
  * backup_time_to_string now return a Result
  * remove unnecessary TryFrom<(BackupGroup, i64)> for BackupDir

- DynamicIndexHeader: change ctime to i64

- FixedIndexHeader: change ctime to i64
2020-09-15 07:12:57 +02:00
Fabian Grünbichler 7158b304f5 handle invalid mtime when formating entries
otherwise operations like catalog shell panic when viewing pxar archives
containing such entries, e.g. with mtime very far ahead into the future.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-09-11 15:48:43 +02:00
Wolfgang Bumiller 0f9218079a pxar/extract: fixup path stack for errors
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-03 12:20:30 +02:00
Wolfgang Bumiller d9b8e2c795 pxar: better error handling on extract
Errors while applying metadata will not be considered fatal
by default using `pxar extract` unless `--strict` was passed
in which case it'll bail out immediately.

It'll still return an error exit status if something had
failed along the way.

Note that most other errors will still cause it to bail out
(eg. errors creating files, or I/O errors while writing
the contents).

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-08-03 09:40:55 +02:00
Wolfgang Bumiller 30c3c5d66c pxar: create: attempt to use O_NOATIME
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-31 11:46:53 +02:00
Wolfgang Bumiller e51be33807 pxar: create: move common O_ flags to open_file
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-31 11:42:15 +02:00
Wolfgang Bumiller d44185c4a1 fix #2873: if --pattern is used, default to not extracting
The extraction algorithm has a state (bool) indicating
whether we're currently in a positive or negative match
which has always been initialized to true at the beginning,
but when the user provides a `--pattern` argument we need to
start out with a negative match.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-30 09:33:30 +02:00
Wolfgang Bumiller f6c6e09a8a update to pxar 0.3 to support negative timestamps
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-29 08:31:37 +02:00
Wolfgang Bumiller b25deec0be pxar: .pxarexclude: absolute paths and byte based paths
Change the .pxarexclude parser to byte based parsing with
`.split(b'\n')` instead of `.lines()`, to not panic on
non-utf8 paths.

Specially deal with absolute paths by prefixing them with
the current directory.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-15 11:55:48 +02:00
Wolfgang Bumiller 757d0ccc76 warning fixup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 10:37:14 +02:00
Wolfgang Bumiller 032cd1b862 pxar: restore file attributes, improve errors
and use the correct integer types for these operations

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-14 10:25:45 +02:00
Wolfgang Bumiller bf7e2a4648 simpler lost+found pattern
the **/ is not required and currently also mistakenly
doesn't match /lost+found which is probably buggy on the
pathpatterns crate side and needs fixing there

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-09 14:06:42 +02:00
Wolfgang Bumiller 1008a69a13 pxar: less confusing logic
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-07-08 09:58:29 +02:00
Wolfgang Bumiller 7d0754a6d2 pxar: fixup 'vanished-file' logic a bit
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-30 14:41:42 +02:00
Wolfgang Bumiller 5afa0755ea pxar: fix missing newlines in warnings
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-30 14:37:20 +02:00
Wolfgang Bumiller a8e2940ff3 pxar: deal with files changing size during archiving
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-26 11:49:51 +02:00
Wolfgang Bumiller 2564b0834f fix file timestamps in catalog
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:25:31 +02:00
Wolfgang Bumiller 9321bbd1f5 pxar: fix missing subdirectories in catalogs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 11:04:56 +02:00
Wolfgang Bumiller 4264e52220 reuse some extractor code in catalog shell
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:54:54 +02:00
Wolfgang Bumiller 6988b29bdc use O_EXCL when creating files during extraction
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:33:27 +02:00
Wolfgang Bumiller 98c54240e6 pxar: make extractor state more reusable
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-16 10:32:13 +02:00
Wolfgang Bumiller 4482f3fe11 pxar, acl: cleanup acl helper usage
use NixPath for Acl::set_file to avoid memduping the c
string

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:52:18 +02:00
Wolfgang Bumiller cf1bd08131 pxar: fcaps in fuse
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-12 10:37:48 +02:00
Wolfgang Bumiller 3d68536fc2 pxar: support .pxareclude files, error report updates
Report vanished files (instead of erroring out on them),
also only warn about files inaccessible due to permissions
instead of bailing out.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 12:22:18 +02:00
Wolfgang Bumiller 26e78a2efb downgrade some FIXMEs to TODOs
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:09:23 +02:00
Wolfgang Bumiller 5444fa940b turn pxar::flags into bitflags, pxar::Flags
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 11:05:53 +02:00
Wolfgang Bumiller 3014088684 pxar: sort .pxareclude-cli file and fix its mode
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-10 08:49:14 +02:00
Wolfgang Bumiller 239e49f927 pxar: create .pxarexclude-cli file
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-06-09 13:17:59 +02:00