Commit Graph

2218 Commits

Author SHA1 Message Date
Wolfgang Bumiller 5543f7b7b3 bump proxmox crate version requirement
it contained an important api macro bugfix

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-24 13:43:27 +01:00
Christian Ebner 04350b4c08 tools: add access method to LruCache
This will return a mutable reference just like get_mut, but on a cache miss
it will get and insert the missing value via the fetch method provided via the
Cacher trait.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-24 09:32:09 +01:00
Dietmar Maurer 4191018c79 src/api2/admin/datastore.rs - download_file: use BACKUP_ARCHIVE_NAME_SCHEMA for file-name 2020-01-23 13:49:30 +01:00
Dietmar Maurer a92830dc39 src/api2/types.rs: define and use api type GarbageCollectionStatus 2020-01-23 13:40:12 +01:00
Dietmar Maurer dfc58d47ad src/api2/admin/datastore.rs: use api macro for start_garbage_collection 2020-01-23 13:01:21 +01:00
Dietmar Maurer 1dc117bbba src/api2/types.rs: define and use struct StorageStatus 2020-01-23 12:51:30 +01:00
Dietmar Maurer 09b1f7b202 src/api2/types.rs: define struct BackupContent, and use it with list_snapshot_files 2020-01-23 11:17:38 +01:00
Dietmar Maurer 51534c8de9 src/backup/manifest.rs: check if manifest contains files 2020-01-23 11:16:12 +01:00
Dietmar Maurer 68a6a0ee03 src/api2/admin/datastore.rs: use api macro for delete_snapshot 2020-01-23 10:16:45 +01:00
Dietmar Maurer 6abce6c2bb src/backup/datastore.rs: remove_backup_(group/dir) - return Error instead of io::Error 2020-01-23 10:14:46 +01:00
Dietmar Maurer c425bdc9b2 src/api2/pull.rs: delete fanished snapshots 2020-01-23 09:58:57 +01:00
Dietmar Maurer 8a1d68c8b9 src/backup/datastore.rs: improve error messages 2020-01-23 09:58:14 +01:00
Dietmar Maurer 18cc66ee85 src/api2/pull.rs: use new Datastore::last_successful_backup helper
Do not fail if local group does not exists.
2020-01-22 15:06:28 +01:00
Dietmar Maurer 41b373eced src/backup/datastore.rs: new helpers
last_successful_backup: Returns the time of the last successful backup
group_path: Returns the absolute path for a backup_group
snapshot_path: Returns the absolute path for a backup_dir
2020-01-22 15:05:47 +01:00
Thomas Lamprecht 1ca037d81c followup: commit all changes
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-22 14:14:52 +01:00
Thomas Lamprecht 44b65b4ec7 api/datastore: do not allow updating path for now
It's a bit dangerous as it points to all the saved backups, so they
would be seemingly lost after updating the path.
Follow our logic from other products, e.g. in PVE we do not allow to
update the backing path/location of a storage either for similar
reasons.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2020-01-22 13:55:37 +01:00
Christian Ebner 1685c2e341 tools: LRU cache to use for pxar FUSE implementation.
Implements a cache with least recently used cache replacement policy.
Internally the state is tracked by a HashMap (for fast access) and a doubly
linked list (for the access order).

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-22 13:11:42 +01:00
Dietmar Maurer f1d99e3f6a src/bin/proxmox-backup-client.rs: use a std channel to write the catalog 2020-01-22 12:49:08 +01:00
Dietmar Maurer 02141b4d9b src/client/pxar_backup_stream.rs: use a channel instead of a pipe 2020-01-22 11:37:16 +01:00
Dietmar Maurer dcd033a53c src/tools/std_channel_writer.rs: new StdChannelWriter helper class 2020-01-22 11:33:57 +01:00
Wolfgang Bumiller 51f0ab1e8b fixup tests for the new proxmox crate layout
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-22 10:08:40 +01:00
Wolfgang Bumiller 9ea4bce444 bump proxmox crate to 0.1.7
The -sys, -tools and -api crate have now been merged into
the proxmx crate directly. Only macro crates are separate
(but still reexported by the proxmox crate in their
designated locations).

When we need to depend on "parts" of the crate later on
we'll just have to use features.

The reason is mostly that these modules had
inter-dependencies which really make them not independent
enough to be their own crates.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 13:48:37 +01:00
Fabian Grünbichler e881b8a5ef switch to debcargo for generating d/control
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-01-21 11:45:26 +01:00
Dietmar Maurer 3f06d6fbbd use proxmox_backup::tools::runtime::main to run async run_cli_command 2020-01-21 11:34:45 +01:00
Wolfgang Bumiller 077ac6f8b3 runtime: actually park on Pending...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 10:37:16 +01:00
Wolfgang Bumiller 2d51569396 runtime: custom block_on implementation
futures::executor::block_on does not allow nesting, but we
need that...

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-21 10:24:52 +01:00
Wolfgang Bumiller 650e052299 runtime: drop now not required Send and static restrictions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-20 14:09:24 +01:00
Wolfgang Bumiller d973aa827c introduce new runtime tokio helpers
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-20 13:12:40 +01:00
Christian Ebner aac9dbf635 pxar::fuse: remove open, opendir and releasedir callbacks
They are not needed as there is no internal state which needs to be tracked.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-20 12:56:21 +01:00
Christian Ebner a8aff3535d pxar: Include symlink target in DirectoryEntry
This allows to read the target path of a symbolic link in the
Decoder::read_directory_entry() function and stores it in the DirectoryEntry.
By this the Decoder::read_link() function becomes obsolete and is therefore
removed.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-20 12:56:10 +01:00
Dietmar Maurer 138910bcd4 src/client/pxar_backup_stream.rs: use std:🧵:Builder to name the thread
To make debugging easier.
2020-01-18 12:21:22 +01:00
Oguz Bektas 9ffc075b8b todo: remove done suggestion
we now ask for the password twice when creating keys or changing
passphrases.

Signed-off-by: Oguz Bektas <o.bektas@proxmox.com>
2020-01-17 16:13:56 +01:00
Dietmar Maurer 4b4eba0b9e src/api2/pull.rs: implement delete flag for vanished groups 2020-01-17 11:24:55 +01:00
Dietmar Maurer 11d89239c3 src/backup/backup_info.rs: new list_groups helper 2020-01-17 10:42:03 +01:00
Dietmar Maurer b31c8019d7 src/api2/types.rs: define and use GroupListItem 2020-01-17 10:17:18 +01:00
Christian Ebner 2bbbade367 pxar: Refactor fuse and remove unused code.
By ambiguously using the Decoder::read_directory_entry() the code is simplified
and reading of the DirectoryEntry is concentrated into Context::run_in_context().

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-17 09:40:39 +01:00
Christian Ebner 95c9460c4a pxar::fuse: Always use start offset of entries as inode.
Previously it was disciminated based on the entry mode.
For directories, the inode was the offset of the corresponding
goodbye tail mark while for all others it was the offset of the filename.

By simply using the start offset as calculated from the corresponding
goodbye table entry (which yields the archive offset of the filename),
the code is simplified and the more ambiguous read_directory_entry()
function can be used.
The disatvantage of this approach is the need to keep track of the
start and end offsets for each entry, as the end offset is needed in
order to access the goodbye table of directory entries.
The root node still has to be treated special, as it's inode is 1 as per fuse
definition and it has no filename as per the pxar file format definition.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-17 09:40:39 +01:00
Christian Ebner 6744440714 pxar::decoder: refactor Decoder::lookup()
Use Decoder::read_directory_entry() instead of Decoder::attributes() as this
already returns the needed DirectoryEntry.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-17 09:40:39 +01:00
Dietmar Maurer f357390c15 renamed: src/config/remotes.rs -> src/config/remote.rs
And use 'remote' instead of 'remotes' everywhere.
2020-01-16 14:32:06 +01:00
Dietmar Maurer d2dd827877 src/backup/catalog.rs - SenderWriter: use tokio::task::block_in_place
Make sure we do not block the executor.
2020-01-16 14:24:15 +01:00
Dietmar Maurer 94609e2380 src/api2/pull.rs: use reference to remote configuration 2020-01-16 13:55:19 +01:00
Dietmar Maurer 625d19621f src/client/remote_chunk_reader.rs: use tokio::task::block_in_place to avoid blocking
Else we block the current executor.
2020-01-16 13:55:19 +01:00
Fabian Grünbichler dbd3e5da12 update tokio to 0.2.9
Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
2020-01-16 13:23:24 +01:00
Wolfgang Bumiller 6a0dc4a5bc bump proxmox crate version
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-16 11:40:23 +01:00
Dietmar Maurer 51697c283d remove the rust-toolchain file 2020-01-16 10:45:46 +01:00
Dietmar Maurer 933687d959 set toolchain to system 2020-01-16 10:34:45 +01:00
Wolfgang Bumiller 4f0c4523f6 debian: bump dependency versions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-16 10:20:41 +01:00
Wolfgang Bumiller 619495b25e fix use of deprecated items
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-16 08:58:54 +01:00
Wolfgang Bumiller 18c0df4c92 import ordering cleanup
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2020-01-16 08:51:52 +01:00
Christian Ebner ef2d7f2f96 pxar::fuse: remove readdir implementation
By not implementing readdir but only readdirplus, the FUSE_CAP_READDIRPLUS flag
is set while the FUSE_CAP_READDIRPLUS_AUTO flag is not set.
Thereby the kernel will issue only readdirplus calls.
Documentation at:
https://libfuse.github.io/doxygen/fuse-3_88_80_2include_2fuse__common_8h.html#a9b90333ad08d0e1c2ed0134d9305ee87

As the expensive part for accessing and reading the attributes is seeking and
decoding each directory entry, it is usefull to force readdirplus calls.
By this a struct `EntryParam` is returned for each entry, therebye avoiding a
subsequent lookup call.

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2020-01-15 14:44:53 +01:00