tyler/proxmox-backup
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
5,983 Commits 2 Branches 54 Tags
d0d970f70b9ab3b8427300995cc3802cc6b2d48f
Commit Graph
100 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
Dietmar Maurer 0e994eb938 pbs-api-types: remove proxmox-sys dependency for target wasm 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-12-01 09:49:52 +01:00
Dietmar Maurer 1a211f0d96 pbs-api-types: remove openssl dependency for target wasm 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-12-01 09:28:47 +01:00
Dietmar Maurer f7fde5c81b pbs-api-types: remove libc dependency 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-12-01 09:10:25 +01:00
Dietmar Maurer af5a55509d pbs-api-types: removbe usused nix dependency 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-12-01 09:08:25 +01:00
Dietmar Maurer 0fee4ff2a4 pbs-tape: do not depend on pbs-tools 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 13:17:58 +01:00
Dietmar Maurer 3dcea3ce33 fix typo in comment 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 13:15:35 +01:00
Dietmar Maurer 726b9d4469 use proxmox-sys 0.2.1 and proxmox-io 1.0.1 
And remove unused code from pbs-tools.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 12:30:03 +01:00
Dietmar Maurer 577095e2f7 move pbs-tools/src/percent_encoding.rs to pbs-api-types/src/percent_encoding.rs 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 11:48:52 +01:00
Dietmar Maurer f35e187f16 fix StdChannelWriter usage 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 11:27:20 +01:00
Dietmar Maurer e2b12ce988 StdChannelWriter: avoid using anyhow::Error 
Use a generic implementation to allow different error types.
 2021-11-25 11:14:56 +01:00
Dietmar Maurer 92ef0b56d8 move pbs-tools/src/str.rs to pbs-client/src/pxar/create.rs 
Code is only used there.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 10:43:22 +01:00
Dietmar Maurer 8a8a1850d0 remove trait BufferedRead from pbs-tools/src/io.rs 
We do not need it.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 09:45:47 +01:00
Dietmar Maurer fddb9bcc3e remove pbs-tools/src/sys.rs 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 09:01:29 +01:00
Dietmar Maurer 0df179c2b4 remove pbs-tools/src/cli.rs 
Code is only used once in src/bin/proxmox_backup_debug/inspect.rs, so
move it into that file.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-25 08:33:10 +01:00
Dietmar Maurer 689ed51397 openid_login: improve error message for disabled users. 2021-11-25 07:29:33 +01:00
Dietmar Maurer 3c56335d7b update debian/control 2021-11-25 06:49:26 +01:00
Dietmar Maurer 9eb58647c1 pbs-datastore: use hex::serde feature 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-24 13:06:14 +01:00
Dietmar Maurer 0ff214bedd debian/control: add librust-proxmox-serde 2021-11-24 10:58:01 +01:00
Dietmar Maurer 25877d05ac update to proxmox-sys 0.2 crate 
- imported pbs-api-types/src/common_regex.rs from old proxmox crate
- use hex crate to generate/parse hex digest
- remove all reference to proxmox crate (use proxmox-sys and
  proxmox-serde instead)

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-24 10:32:27 +01:00
Dietmar Maurer c1e6efa8e1 sync job: correctly apply rate limit 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-23 09:42:18 +01:00
Dietmar Maurer e201104d0b docs: update traffic control docs (use HumanBytes) 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-22 09:07:05 +01:00
Dietmar Maurer d63db0863d proxmox-backup-manager traffic: render data human readable 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-22 09:07:05 +01:00
Dietmar Maurer 7a36833103 fix sync job regression test (add RateLimitConfig) 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-22 08:29:43 +01:00
Dietmar Maurer ca6e66aa5a Fingerprint: add new signature method 
commit c42a54795d introcuded a bug by
using fp.to_string(). Replace this with fp.signature() which correctly
returns the full fingerprint instead of the short version.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-22 08:29:43 +01:00
Dietmar Maurer 2d5287fbbc use RateLimitConfig for HttpClient and pull 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-22 07:49:41 +01:00
Dietmar Maurer 6eb756bcab sync-job: add rate limit 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-22 07:49:41 +01:00
Dietmar Maurer 5647219049 pbs-api-types: split out type RateLimitConfig 2021-11-22 07:49:41 +01:00
Dietmar Maurer 193ec30c2b fix proxmox-backup-manager sync-job list 
Property is called 'group-filter' (not 'groups').
 2021-11-21 09:46:46 +01:00
Dietmar Maurer c94723062c pbs-api-types: fix HumanByte::auto_scale 2021-11-21 09:13:02 +01:00
Dietmar MaurerandThomas Lamprecht 118515dbd0 use HumanByte for traffic-control config 
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-11-20 19:35:24 +01:00
Dietmar Maurer 92a8f0bc82 depend on proxmox-async 0.2 2021-11-20 17:14:02 +01:00
Dietmar Maurer bf298a16ef proxmox-rest-server: remove pbs-tools dependency 2021-11-19 18:06:54 +01:00
Dietmar Maurer 9a1b24b6b1 use new proxmox-async crate 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-19 18:03:22 +01:00
Dietmar Maurer 6f6df501a0 fix debian/control: include librust-proxmox-sys 2021-11-19 11:35:28 +01:00
Dietmar Maurer d5790a9f27 use new proxmox-sys crate 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-19 11:06:35 +01:00
Dietmar Maurer 860eaec58f use proxmox::tools::fd::fd_change_cloexec from proxmox 0.15.3 
Depend on proxmox 0.15.3

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-18 13:43:41 +01:00
Dietmar Maurer 26e949d5fe traffic-control api: return current traffic with config 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-18 12:24:33 +01:00
Dietmar Maurer 10beed1199 openid: allow to configure scopes, prompt, ACRs and arbitrary username-claim values 
- no longer set prompt to 'login' (makes auto-login possible)
- new prompt configuration
- allow arbitrary username-claim values

Depend on proxmox-openid 0.9.0.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-18 11:20:55 +01:00
Dietmar Maurer 15cc41b6cb proxmox-systemd: remove crate, use new proxmox-time 1.1.0 instead 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-17 13:07:51 +01:00
Dietmar Maurer c42a54795d move fingerprint helpers from pbs-tools to pbs-api-types 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-17 07:07:40 +01:00
Dietmar Maurer 96ec3801a9 docs: add traffic control section 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-15 13:43:11 +01:00
Dietmar Maurer c4707d0c1d depend on proxmox-shared-memory 0.1.1 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-15 11:35:52 +01:00
Dietmar Maurer 24f9af9e0f add missing file from previous commit 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-14 18:49:29 +01:00
Dietmar Maurer a0172d766b traffic-controls: add API/CLI to show current traffic 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-14 17:21:45 +01:00
Dietmar Maurer 09f999337a update to proxmox-http 0.5.4 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-14 08:27:45 +01:00
Dietmar Maurer e3eb062c09 cached_traffic_control: fix regression tests 
Avoid using shared memory in tests because of permission problems.
 2021-11-14 08:05:40 +01:00
Dietmar Maurer de21d4efdc implement rate limiter in shared memory 
This kind of rate limiter can be used among several processes (as long
as all set the same rate/burst).
 2021-11-13 17:49:38 +01:00
Dietmar Maurer d5f58006d3 cached_traffic_control: use ShareableRateLimit trait object 2021-11-13 17:49:38 +01:00
Dietmar Maurer cb80ffc1de pbs-config: use new SharedMemory helpers from proxmox-shared-memory crate 
depend on proxmox-shared-memory crate.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-13 17:49:38 +01:00
Dietmar Maurer 9e7132c0b3 bump version to 2.0.14-1 2021-11-12 08:12:18 +01:00
Dietmar Maurer bf013be1c4 create /var/lib/proxmox-bnackup at server startup 
This was missing in previous patch...

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-12 08:12:18 +01:00
Dietmar Maurer b935209584 fix directory permission problems 
By carefully setting options on all create_path() calls,
and by creating "/var/lib/proxmox-backup" at api server startup.
 2021-11-12 07:29:18 +01:00
Dietmar Maurer efd4ddc17b debian/control: depend on librust-cidr-dev 2021-11-10 12:23:16 +01:00
Dietmar Maurer e511e0e553 proxmox-backup-proxy: implement traffic control 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer 610150a4b4 implement a traffic control cache for fast rate control limiter lockups 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer 485b2438ac traffic_control: use Memcom to track. config versions 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer bfd12e871f Add traffic control configuration config with API 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer 0c136bfab1 DailyDuration: implement time_match() 2021-11-10 10:15:40 +01:00
Dietmar Maurer 245e2aea23 New DailyDuration type with nom parser 
We will use this to specify timesframes for network rate limits (only
apply limite when inside the time frame).

Note: This is not systemd related, but we can reuse some of the parser
method.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer b9d588ffde implement Servive for RateLimitedStream 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer e4bc3e0e8d proxmox-backup-client: add rate/burst parameter to backup CLI 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:40 +01:00
Dietmar Maurer 2419dc0de9 pbs-client: add option to use the new RateLimiter 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-10 10:15:32 +01:00
Dietmar Maurer 68fd9ca6d6 openid_login: vertify that firstname, lastname and email fits our schema definitions 
If not, we do not copy the values to our user.cfg.
 2021-11-10 06:48:40 +01:00
Dietmar Maurer 4beb7d2dbe correctly lock remote config 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-11-06 17:35:10 +01:00
Dietmar Maurer 7a356a748a bump version to 2.0.13-1 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-21 08:36:41 +02:00
Dietmar Maurer 1c402740a2 update dedian/control 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-21 07:59:45 +02:00
Dietmar Maurer e0a19d3313 use new fsync parameter to replace_file and atomic_open_or_create 
Depend on proxmox 0.15.0 and proxmox-openid 0.8.1

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-21 07:28:32 +02:00
Dietmar Maurer d6473f5359 proxmox-rrd: use fsync instead of syncfs 
syncfs can sync unrelated data, and we do not want that.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-20 11:46:59 +02:00
Dietmar Maurer f5f9ec81d2 proxmox-rrd: fix regression tests 2021-10-19 18:41:03 +02:00
Dietmar Maurer fea950155f proxmox-rrd: improve dev docs 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer ef2944bc24 proxmox-rrd: cleanup - impl FromStr for JournalEntry 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer 934c8724e2 proxmox-rrd: add option to avoid page cache for load/save 
use fadvice(.., POSIX_FADV_DONTNEED) for RRD files. We read those files only once,
and always rewrite them.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer 98eb435d90 proxmox-rrd: use syncfs after writing rrd files 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
 2021-10-19 11:17:09 +02:00
Dietmar Maurer 7f381a6246 proxmox-rrd: use fine grained locking in commit_journal_impl 
Aquire the rrd_map lock for each file (else we block access for a long time)
 2021-10-18 14:55:47 +02:00
Dietmar Maurer c17fbbbc07 proxmox-rrd: log all errors from apply_and_commit_journal_thread (but only once) 2021-10-18 11:57:19 +02:00
Dietmar Maurer d26865c52c proxmox-rrd: cleanup list_old_journals 2021-10-18 10:00:58 +02:00
Dietmar Maurer 2b05008a11 proxmox-rrd: cleanup - use struct instead of tuple 2021-10-16 12:45:03 +02:00
Dietmar Maurer 45700e2ecf proxmox-rrd: move RRDMap into extra file 2021-10-16 12:45:03 +02:00
Dietmar Maurer f84304235b proxmox-rrd: move JournalState into extra file 2021-10-16 12:45:03 +02:00
Dietmar Maurer 0ca41155b2 proxmox-rrd: implement non blocking journal 
Do not block while applying the journal.
 2021-10-16 12:45:03 +02:00
Dietmar Maurer a291ab59ba proxmox-rrd: rename RRDCacheState to JournalState 2021-10-15 09:35:44 +02:00
Dietmar Maurer fce7cd0d36 proxmox-rrd: avoild blocking readers while applying the journal 
By using and extra RwLock<RRDMap> on the rrd data.
 2021-10-15 09:22:07 +02:00
Dietmar Maurer 658357c5a8 proxmox-rrd: log journal apply/flush times, split apply and flush 
We need to apply the journal only once.
 2021-10-15 07:16:41 +02:00
Dietmar Maurer 7484fce24d proxmox-rrd: cleanup - use slot_end_time() 2021-10-14 16:29:00 +02:00
Dietmar Maurer f28a713e2b proxmox-rrd: cleanup - use staturating_add instead of if/else 2021-10-14 16:10:55 +02:00
Dietmar Maurer a9017805b7 proxmox-rrd: improve dev docs 2021-10-14 11:53:54 +02:00
Dietmar Maurer 2e3f94e12f proxmox-rrd: make rrd load callback configurable 2021-10-14 11:41:26 +02:00
Dietmar Maurer d531c7ae61 proxmox-rrd: add more regression tests 2021-10-14 10:55:12 +02:00
Dietmar Maurer 7df1580fa6 proxmox-rrd: add regression tests and two minor fixes 2021-10-14 10:17:07 +02:00
Dietmar Maurer 58f70bccbb proxmox-rrd: pass time and value to update function 2021-10-14 08:12:56 +02:00
Dietmar Maurer fae4f6c509 cleanup: move rrd cache related code into extra file 2021-10-14 07:57:27 +02:00
Dietmar Maurer ddafb28572 proxmox-rrd: add some integration tests (file format tests) 2021-10-13 18:21:23 +02:00
Dietmar Maurer b3f279e2d9 use complete_file_name from proxmox-router 1.1 2021-10-13 14:10:02 +02:00
Dietmar MaurerandThomas Lamprecht bacc99c7f8 proxmox-rrd: add more commands to the rrd cli tool 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar MaurerandThomas Lamprecht 6728d0977b proxmox-rrd: rename last_counter to last_value 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar MaurerandThomas Lamprecht bff7c027c9 proxmox-rrd: protect against negative update time 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar MaurerandThomas Lamprecht 79b3113361 proxmox-rrd: new helpers: slot, slot_start_time & slot_end_time 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar MaurerandThomas Lamprecht 5885767b91 proxmox-rrd: avoid expensive modulo (%) inside loop 
Modulo is very slow, so we try to avoid it inside loops.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar MaurerandThomas Lamprecht ec08247e5c proxmox-rrd: add binary to create/manage rrd files 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00
Dietmar MaurerandThomas Lamprecht 400f081487 proxmox-rrd: split out load_rrd (cleanup) 
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 2021-10-13 13:36:02 +02:00