tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
4,168 Commits 2 Branches 54 Tags
Commit Graph

3217 Commits

Author SHA1 Message Date
Dominik Csapak
66f4e6a809 server/worker_task: refactor locking of the task list 
also add the functionality of having a 'shared' (read) lock for the list
we will need this later

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-29 08:37:54 +02:00
Dominik Csapak
8074d2b0c3 tools: add logrotate module 
this is a helper to rotate and iterate over log files
there is an iterator for open filehandles as well as
only the filename

also it has the possibilty to rotate them
for compression, zstd is used

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-29 08:33:21 +02:00
Dietmar Maurer
b02d49ab26 proxmox_backup_client key: allow to generate paperkey for master key 2020-09-29 08:29:42 +02:00
Dietmar Maurer
82a0cd2ad4 proxmox_backup_client key: add new paper-key command 2020-09-29 08:29:42 +02:00
Wolfgang Bumiller
ee1a9c3230 parallel_handler: clippy: 'while_let_loop' 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-09-29 08:13:51 +02:00
Wolfgang Bumiller
db24c01106 parallel_handler: explicit Arc::clone 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-09-28 13:40:03 +02:00
Wolfgang Bumiller
ae3cfa8f0d parallel_handler: formatting cleanup, doc comment typo fixup 
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-09-28 13:40:03 +02:00
Dietmar Maurer
b56c111e93 depend on proxmox 0.4.2 2020-09-28 10:50:44 +02:00
Dominik Csapak
bbeb0256f1 server/worker_task: factor out task list rendering 
we will need this later again

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-28 07:31:27 +02:00
Dominik Csapak
005a5b9677 api2/node/tasks: move userfilter to function signature 
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-28 07:18:13 +02:00
Dietmar Maurer
55bee04856 src/tools/parallel_handler.rs: remove unnecessary Sync bound 2020-09-26 16:16:11 +02:00
Dietmar Maurer
42fd40a124 src/bin/proxmox_backup_client/benchmark.rs: avoid compiler warning 2020-09-26 16:13:19 +02:00
Dietmar Maurer
f21508b9e1 src/backup/verify.rs: use ParallelHandler to verify chunks 2020-09-26 11:14:37 +02:00
Dietmar Maurer
ee7a308de4 src/backup/verify.rs: cleanup use clause 2020-09-26 10:23:44 +02:00
Dietmar Maurer
636e674ee7 src/client/pull.rs: simplify code 2020-09-26 10:09:51 +02:00
Dietmar Maurer
b02b374b46 src/tools/parallel_handler.rs: remove static lifetime bound from handler_fn 2020-09-26 09:26:06 +02:00
Dietmar Maurer
1c13afa8f9 src/tools/parallel_handler.rs: join all threads in drop handler 2020-09-26 08:47:56 +02:00
Dietmar Maurer
69b92fab7e src/tools/parallel_handler.rs: remove unnecessary Sync trait bound 2020-09-26 07:38:44 +02:00
Dietmar Maurer
5441708634 src/client/pull.rs: use new ParallelHandler 2020-09-25 12:58:20 +02:00
Dietmar Maurer
3c9b370255 src/tools/parallel_handler.rs: execute closure inside a thread pool 2020-09-25 12:58:20 +02:00
Dominik Csapak
d7a122a026 use jobstate mechanism for verify/garbage_collection schedules 
also changes:
* correct comment about reset (replace 'sync' with 'action')
* check schedule change correctly (only when it is actually changed)

with this changes, we can drop the 'lookup_last_worker' method

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-24 17:06:12 +02:00
Dominik Csapak
6c25588e63 proxy: fix error handling in prune scheduling 
we rely on the jobstate handling to write the error of the worker
into its state file, but we used '?' here in a block which does not
return the error to the block, but to the function/closure instead

so if a prune job failed because of such an '?', we did not write
into the statefile and got a wrong state there

instead use our try_block! macro that wraps the code in a closure

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-24 17:06:09 +02:00
Dietmar Maurer
998db63933 src/client/pull.rs: decode, verify and write in a separate threads 
To maximize throughput.
 2020-09-24 13:12:04 +02:00
Dietmar Maurer
c0fa14d94a src/backup/data_blob.rs: add is_encrypted helper 2020-09-24 13:00:16 +02:00
Wolfgang Bumiller
6fd129844d remove DummyCatalogWriter 
we're using an `Option` instead now

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
 2020-09-24 09:13:54 +02:00
Dietmar Maurer
baae780c99 benchmark: use compressable data to get more realistic result 
And add a benchmatrk to test chunk verify speed (decompress+sha256).
 2020-09-24 08:58:13 +02:00
Dietmar Maurer
09a1da25ed src/backup/data_blob.rs: improve decompress speed 2020-09-24 08:52:35 +02:00
Dietmar Maurer
a83e2ffeab src/api2/reader.rs: use std::fs::read instead of tokio::fs::read 
Because it is about 10%& faster this way.
 2020-09-22 13:27:23 +02:00
Dietmar Maurer
ebbe4958c6 src/client/pull.rs: avoid duplicate downloads using in memory HashSet 2020-09-22 12:34:06 +02:00
Dietmar Maurer
73b2cc4977 src/client/pull.rs: allow up to 20 concurrent download streams 2020-09-22 11:39:31 +02:00
Dietmar Maurer
7ecfde8150 remote_chunk_reader.rs: use Arc for cache_hint to make clone faster 2020-09-22 11:39:31 +02:00
Dietmar Maurer
e0085e6612 src/client/pull.rs: remove temporary manifest 2020-09-21 14:03:01 +02:00
Dietmar Maurer
194da6f867 src/client/pull.rs: open temporary manifest with truncate(true) 
To delete any data if the file already exists.
 2020-09-21 13:53:35 +02:00
Dominik Csapak
5e39918fe1 fix #3017: check array boundaries before using 
else we panic here

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-21 09:22:06 +02:00
Dietmar Maurer
12c65bacf1 src/backup/chunk_store.rs: disable debug output 2020-09-19 15:26:21 +02:00
Dietmar Maurer
ba37f3562d src/backup/datastore.rs - open_with_path: use Path instead of str 2020-09-19 10:01:57 +02:00
Dietmar Maurer
fce4659388 src/backup/datastore.rs: new method open_with_path 
To make testing easier.
 2020-09-19 09:55:21 +02:00
Dominik Csapak
9866de5e3d datastore/prune schedules: use JobState for tracking of schedules 
like the sync jobs, so that if an admin configures a schedule it
really starts the next time that time is reached not immediately

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-19 06:24:37 +02:00
Fabian Grünbichler
be3bd0f90b fix #3015: allow user self-service 
listing, updating or deleting a user is now possible for the user
itself, in addition to higher-privileged users that have appropriate
privileges on '/access/users'.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-18 15:45:11 +02:00
Fabian Grünbichler
3c053adbb5 role api: fix description 
wrongly copy-pasted at some point

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-18 14:55:00 +02:00
Hannes Laimer
c040ec22f7 add verification scheduling to proxmox-backup-proxy 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:14:05 +02:00
Hannes Laimer
2b67de2e3f api2: make verify_schedule deletable 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:12:29 +02:00
Hannes Laimer
477859662a api2: add optional verify-schdule field to create/update datastore endpoint 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:12:16 +02:00
Hannes Laimer
ccd7241e2f add verify_schedule field to DataStoreConfig 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:11:55 +02:00
Hannes Laimer
f37ef25bdd api2: add VERIFY_SCHEDULE_SCHEMA 
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
 2020-09-18 12:11:39 +02:00
Fabian Grünbichler
b93bbab454 fix #3014: allow DataStoreAdmins to list DS config 
filtered by those they are privileged enough to read individually. this
allows such users to configure prune/GC schedules via the GUI (the API
already allowed it previously).

permission-wise, a user with this privilege can already:
- list all stores they have access to (returns just name/comment)
- read the config of each store they have access to individually
(returns full config of that datastore + digest of whole config)

but combines them to
- read configs of all datastores they have access to (returns full
config of those datastores + digest of whole config)

user that have AUDIT on just /datastore without propagate can now no
longer read all configurations (but this could be added it back, it just
seems to make little sense to me).

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
 2020-09-18 12:09:13 +02:00
Dietmar Maurer
1bc1d81a00 move compute_file_csum to src/tools.rs 2020-09-17 10:27:04 +02:00
Dominik Csapak
8f2f3dd710 fix #2942: implement lacp bond mode and bond_xmit_hash_policy 
this was not yet implemented, should be compatible with pve and the gui

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-17 08:36:25 +02:00
Dominik Csapak
85959a99ea api2/network: add bond-primary parameter 
needed for 'active-backup' bond mode

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-17 08:36:14 +02:00
Dominik Csapak
36700a0a87 api2/pull: make pull worker abortable 
by selecting between the pull_future and the abort future

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
 2020-09-17 06:11:33 +02:00