tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
2,921 Commits 2 Branches 54 Tags 14 MiB
2aaae9705e
Commit Graph

93 Commits

Author SHA1 Message Date
Dietmar Maurer
39f18b30b6 src/backup/data_blob.rs: new load_from_reader(), which verifies the CRC 
And make verify_crc private for now. We always call load_from_reader() to
verify the CRC.

Also add load_chunk() to datastore.rs (from chunk_store::read_chunk())
 2020-07-28 10:23:16 +02:00
Thomas Lamprecht
c3b090ac8a backup: list images: handle walkdir error, catch "lost+found" 
We support using an ext4 mountpoint directly as datastore and even do
so ourself when creating one through the disk manage code.

Such ext4 ountpoints have a lost+found directory which only root can
traverse into. As the GC list images is done as backup:backup user
walkdir gets an error.

We cannot ignore just all permission errors, as they could lead to
missing some backup indexes and thus possibly sweeping more chunks
than desired. While *normally* that should not happen through our
stack, we had already user report that they do rsyncs to move a
datastore from old to new server and got the permission wrong.

So for now be still very strict, only allow a "lost+found" directory
as immediate child of the datastore base directory, nothing else.

If deemed safe, this can always be made less strict. Possibly by
filtering the known backup-types on the highest level first.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-22 16:01:55 +02:00
Thomas Lamprecht
c47e294ea7 datastore: fix typo 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2020-07-22 15:04:14 +02:00
Wolfgang Bumiller
521a0acb2e DataStore::load_manifest: also return CryptMode 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-07-08 09:19:53 +02:00
Dietmar Maurer
60f9a6ea8f src/backup/datastore.rs: add new helpers to load blobs and verify chunks 2020-06-24 06:58:14 +02:00
Dietmar Maurer
1610c45a86 src/client/pull.rs: also download client.log.blob 2020-05-30 14:51:33 +02:00
Dietmar Maurer
8545480a31 src/bin/proxmox-backup-proxy.rs: add simple task scheduler for garbage collection 2020-05-20 08:59:45 +02:00
Dietmar Maurer
99641a6bbb garbage_collect: call fail_on_abort to abort GV when requested. 2020-05-05 09:06:34 +02:00
Dietmar Maurer
54552dda59 implemnt backup ownership, improve datastore access permissions 2020-04-28 10:22:25 +02:00
Wolfgang Bumiller
f7d4e4b506 switch from failure to anyhow 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-04-17 18:43:30 +02:00
Dietmar Maurer
cf459b1982 gc: log pending removals 2020-04-06 09:50:40 +02:00
Dietmar Maurer
a92830dc39 src/api2/types.rs: define and use api type GarbageCollectionStatus 2020-01-23 13:40:12 +01:00
Dietmar Maurer
6abce6c2bb src/backup/datastore.rs: remove_backup_(group/dir) - return Error instead of io::Error 2020-01-23 10:14:46 +01:00
Dietmar Maurer
8a1d68c8b9 src/backup/datastore.rs: improve error messages 2020-01-23 09:58:14 +01:00
Dietmar Maurer
41b373eced src/backup/datastore.rs: new helpers 
last_successful_backup: Returns the time of the last successful backup
group_path: Returns the absolute path for a backup_group
snapshot_path: Returns the absolute path for a backup_dir
 2020-01-22 15:05:47 +01:00
Dietmar Maurer
4b4eba0b9e src/api2/pull.rs: implement delete flag for vanished groups 2020-01-17 11:24:55 +01:00
Dietmar Maurer
d0187a51a9 src/section_config.rs - convert_to_array: optionally add digest 
datastore::config() -> also return digest
remotes::config() -> also return digest
 2020-01-14 12:57:03 +01:00
Dietmar Maurer
7759eef552 src/backup/datastore.rs: implement cleanup_backup_dir() 2020-01-05 15:15:12 +01:00
Dietmar Maurer
2585a8a4e2 src/backup/chunk_store.rs: implement cond_touch_chunk() 
This will be used by backup sync to test if a chunk already exists.
 2020-01-02 13:26:28 +01:00
Dietmar Maurer
1cf5178ac5 src/backup/datastore.rs: new helper try_shared_chunk_store_lock() 2020-01-02 11:00:33 +01:00
Dietmar Maurer
1e8da0a789 src/backup/manifest.rs: new helper archive_type() 2019-12-31 15:23:41 +01:00
Dietmar Maurer
868c585219 src/backup/datastore.rs: avoid divide by zero 2019-12-19 07:14:23 +01:00
Dietmar Maurer
73e57f244e src/backup/datastore.rs - garbage-collection: fix average chunk size 2019-12-10 11:25:29 +01:00
Dietmar Maurer
4ee8f53d07 remove DataChunk file format - use DataBlob instead 2019-10-06 10:31:06 +02:00
Wolfgang Bumiller
b21851cc5c clippy: collapse identical if branches 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-09-11 13:57:58 +02:00
Wolfgang Bumiller
367f002ee2 use statement cleanup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-08-16 12:28:40 +02:00
Dietmar Maurer
a57360983b src/backup/chunk_store.rs - get_chunk_iterator: return percentage inside iterator item 2019-07-04 09:26:44 +02:00
Dietmar Maurer
a7c72ad9eb src/backup/fixed_index.rs - FixedIndexReader: remove reference to chunk store 2019-07-04 08:13:16 +02:00
Dietmar Maurer
a660978c9a src/backup/datastore.rs: generic index_mark_used_chunks implementation, improve GC stats 2019-07-04 07:57:43 +02:00
Dietmar Maurer
d48a9955a5 src/backup/dynamic_index.rs: introduce ReadChunk trait 2019-06-28 16:35:00 +02:00
Dietmar Maurer
f98ac774ee backup: Add support for client side encryption 
first try ...
 2019-06-13 11:47:23 +02:00
Dietmar Maurer
976595e1a9 src/backup/dynamic_index.rs: split class DynamicIndexWriter 2019-05-29 08:49:57 +02:00
Dietmar Maurer
51a4f63ff7 src/backup/datastore.rs: move last_backup to BackupInfo 2019-05-11 12:07:09 +02:00
Dietmar Maurer
b02a52e3a5 src/backup/datastore.rs: impl last_backup helper 2019-05-11 11:36:46 +02:00
Dietmar Maurer
c09775011a src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups 2019-05-11 10:19:34 +02:00
Dietmar Maurer
f2b99c34f7 src/api2/admin/datastore.rs: implement API to return last GC status 2019-04-11 12:04:25 +02:00
Dietmar Maurer
d4b59ae0b8 src/backup/datastore.rs: pass WorkerTask to GC 
So that we can log to the task log file.
 2019-04-06 17:57:38 +02:00
Dietmar Maurer
92da93b245 abort GC on server shutdown 2019-04-01 12:13:02 +02:00
Dietmar Maurer
11861a482d src/backup/chunk_store.rs: fix GC 
Added option to get oldest_writer timestamp from ProcessLocker.
 2019-03-31 17:21:36 +02:00
Dietmar Maurer
43b1303398 datastore: use new ProcessLocker 
To make sure only one process runs garbage collection while having active writers.
 2019-03-22 09:42:15 +01:00
Dietmar Maurer
515688d1f1 avoid compiler warnings 2019-03-18 10:00:58 +01:00
Dietmar Maurer
58e99e138e src/backup/backup_info.rs: new helper to list files 2019-03-06 11:02:08 +01:00
Wolfgang Bumiller
3b7ade9e78 backup/datastore: expose insert_chunk method 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2019-03-06 10:25:20 +01:00
Dietmar Maurer
b3483782d1 src/backup/backup_info.rs: move code into separate file 
Also changed create_backup_dir() parameters - uses &BackupDir now.
 2019-03-05 07:18:12 +01:00
Dietmar Maurer
2b01a22507 src/backup/datastore.rs: add helper to sort backup lists 2019-03-04 18:20:57 +01:00
Dietmar Maurer
875fb1c01a src/backup/datastore.rs: use unix epoch to create DateTime 
To make sure that we have a timestamp without nanosecond.
 2019-03-04 17:58:22 +01:00
Dietmar Maurer
38f8815925 src/backup/datastore.rs: use new() function to create structs 2019-03-04 17:44:27 +01:00
Dietmar Maurer
9b492eb256 src/backup/datastore.rs: protect BackupDir fields, impl new() 2019-03-04 13:51:36 +01:00
Dietmar Maurer
1e9a94e579 src/backup/datastore.rs: protect BackupGroup fields, impl new() 2019-03-04 13:38:23 +01:00
Dietmar Maurer
f72dfd2d10 src/backup/datastore.rs: make sure timestamp does not contains nanoseconds 
Because we want simple, readable file names.
 2019-03-04 09:37:10 +01:00