Commit Graph

255 Commits

Author SHA1 Message Date
Wolfgang Bumiller dd5495d6dc tree-wide: use 'dyn' for all trait objects
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-06-07 13:13:48 +02:00
Dietmar Maurer 4800df7ba6 src/api2/admin/datastore.rs: remove test-upload api 2019-06-07 07:52:11 +02:00
Dietmar Maurer 113898d7e9 src/api2/admin/datastore/upload.rs: remove unused upload api 2019-06-07 07:39:41 +02:00
Dietmar Maurer 986bef16be src/backup.rs; use a macro to define PROXMOX_BACKUP_PROTOCOL_ID_V1
So that we can include it in static doc strings.
2019-06-05 08:41:20 +02:00
Dietmar Maurer c9ec0956cf src/backup.rs: define const PROXMOX_BACKUP_PROTOCOL_ID_V1 2019-06-05 08:12:13 +02:00
Dietmar Maurer 7773ccc11f src/api2.rs: move backup api to /backup 2019-06-05 07:23:21 +02:00
Dietmar Maurer 339ddfcbfa src/cli/command.rs: move doc generator code to src/api_schema/format.rs 2019-06-04 12:32:22 +02:00
Dietmar Maurer 39d6846e1e src/api2/admin/datastore/backup.rs: implement config file upload 2019-06-03 09:45:14 +02:00
Dietmar Maurer 36898ffce6 src/backup/chunk_stream.rs: add optional chunk_size parameter 2019-05-30 13:28:24 +02:00
Dietmar Maurer 96482891ae src/api2/admin/datastore/backup/environment.rs: log upload statistics 2019-05-30 09:21:19 +02:00
Dietmar Maurer 642322b433 src/api2/admin/datastore/backup.rs: use separate api entry points for chunk upload
So that we can provide better statistics (patches follows).
2019-05-30 09:21:19 +02:00
Dietmar Maurer e6389f4e75 src/api2/admin/datastore/backup/environment.rs: do not allow empty backups (no files) 2019-05-29 10:38:57 +02:00
Dietmar Maurer 39e60bd6fa src/api2/admin/datastore/backup.rs: improve debug logging 2019-05-29 10:17:38 +02:00
Dietmar Maurer a42d1f55c3 src/api2/admin/datastore/backup.rs: add debug flag 2019-05-29 09:35:21 +02:00
Dietmar Maurer 976595e1a9 src/backup/dynamic_index.rs: split class DynamicIndexWriter 2019-05-29 08:49:57 +02:00
Dietmar Maurer 60e589a111 src/api2/admin/datastore/backup.rs: log sucessful finish call 2019-05-28 12:55:22 +02:00
Dietmar Maurer 006f3ff407 src/api2/admin/datastore/backup/environment.rs: add more consistency checks 2019-05-28 09:21:31 +02:00
Dietmar Maurer 3dc5b2a203 src/client/http_client.rs: use start of chunk as offset 2019-05-28 09:21:31 +02:00
Dietmar Maurer e3d525feb7 src/api2/admin/datastore/backup.rs: fix typo 2019-05-28 07:13:19 +02:00
Dietmar Maurer a42fa400ee src/api2/admin/datastore/backup.rs: implement fixed sized chunk upload api 2019-05-28 06:18:55 +02:00
Dietmar Maurer a1e7cff3ea src/api2/admin/datastore/backup/upload_chunk.rs: rename upload_dynamic_chunk into upload_chunk 2019-05-27 07:57:43 +02:00
Dietmar Maurer 771953f9eb src/api2/admin/datastore/backup.rs: use smaller window sizes 2019-05-27 07:53:40 +02:00
Dietmar Maurer 05cba08c9c src/client/http_client.rs: append chunks in correct order
We need to wait for response from upload_chunk. After that we can
add the chunk to the index file.
2019-05-26 09:57:34 +02:00
Dietmar Maurer 417cb0731e src/api2/admin/datastore/backup.rs: verify chunk offset
This clearly shows that the current approach does not work - seems we get
chunks out of order.
2019-05-26 08:49:29 +02:00
Wolfgang Bumiller cc84a830c5 RFC: schema: make enums static
I don't see a reason to allow these to be dynamically
modifiable.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-05-23 12:38:51 +02:00
Dietmar Maurer aa1b2e04fe src/client/merge_known_chunks.rs: merge known chunks
To decrease the number of api calls required...
2019-05-23 12:31:09 +02:00
Dietmar Maurer 8bea85b42e src/api2/admin/datastore/backup.rs: verify file size and chunk count on close 2019-05-23 08:50:36 +02:00
Dietmar Maurer 907f5bb0d8 src/api2/admin/datastore/backup/service.rs: improve error logging 2019-05-23 08:05:39 +02:00
Dietmar Maurer a55fcd740f src/api2/admin/datastore/backup.rs: suse POST for finish() api 2019-05-22 13:04:58 +02:00
Dietmar Maurer 5a0b521809 src/api2/admin/datastore/backup.rs: remove test callbacks 2019-05-22 10:55:27 +02:00
Dietmar Maurer d0afeaa831 src/api2/admin/datastore/backup.rs: improve error message 2019-05-22 07:43:54 +02:00
Dietmar Maurer 7f3d2ffab9 src/backup/index.rs: rename ChunkListReader into DigestListEncoder 2019-05-21 12:28:44 +02:00
Dietmar Maurer a95849321c src/api2/admin/datastore/backup.rs: ignore error after finish, register known_chunks 2019-05-21 09:54:11 +02:00
Dietmar Maurer a09c0e38d8 src/api2/admin/datastore/backup/environment.rs: register/lookup known chunks 2019-05-20 18:05:10 +02:00
Dietmar Maurer 82ab72304e src/client/http_client.rs: implement upload_stream 2019-05-20 14:19:24 +02:00
Dietmar Maurer adec8ea263 src/api2/admin/datastore/backup.rs: add speedtest api, improve upload speed
We need to disable tcp Nagle algorythm (set_nodelay), and use larger window size for http2
2019-05-16 10:24:23 +02:00
Dietmar Maurer 372724afea src/api2/admin/datastore/backup.rs: implement finish_backup 2019-05-15 12:58:55 +02:00
Dietmar Maurer bb105f9dae src/api2/admin/datastore/backup.rs: improve logging 2019-05-15 10:09:46 +02:00
Dietmar Maurer a20772525f src/api2/admin/datastore/backup.rs: implement close for dynamic writers 2019-05-15 07:58:05 +02:00
Dietmar Maurer 4e93f8c164 src/api2/types.rs: add schema/format for file names 2019-05-15 07:14:08 +02:00
Dietmar Maurer fbb798f6d2 src/api2/admin/datastore/backup.rs: ignore errors from last_backup 2019-05-13 07:06:56 +02:00
Dietmar Maurer d36113665b src/api2/admin/datastore/backup.rs: implement chunk index download from previous backup 2019-05-11 16:05:50 +02:00
Dietmar Maurer af1d68185c src/api2/admin/datastore/backup.rs: add missing 'archive-name' parameter to schema 2019-05-11 12:25:15 +02:00
Dietmar Maurer 6b95c7df07 src/api2/admin/datastore/backup/environment.rs: do not store path 2019-05-11 12:13:44 +02:00
Dietmar Maurer 51a4f63ff7 src/backup/datastore.rs: move last_backup to BackupInfo 2019-05-11 12:07:09 +02:00
Dietmar Maurer b02a52e3a5 src/backup/datastore.rs: impl last_backup helper 2019-05-11 11:36:46 +02:00
Dietmar Maurer c09775011a src/backup/backup_info.rs: implement list_backup for BackupGroup, cleanups 2019-05-11 10:19:34 +02:00
Dietmar Maurer f9578f3c79 backup-api: allow to create DynamicIndexWriter and add chunks to to 2019-05-10 10:25:40 +02:00
Dietmar Maurer 0506d1ed04 upload_chunk.rs: add comment about possible unnecessary code 2019-05-10 06:15:25 +02:00
Dietmar Maurer 5f42ef009a cleanup: remove unused code 2019-05-09 18:11:39 +02:00
Dietmar Maurer b4b63e520d handle_async_api_request: put rpcenv into a Box
So that we can pass rpcenv into futures.
2019-05-09 18:01:24 +02:00
Dietmar Maurer 2c41096a4b src/api2/admin/datastore/backup/upload_chunk.rs: fix maximal chunk size 2019-05-09 13:29:50 +02:00
Dietmar Maurer 78b5191550 src/server/formatter.rs: further cleanups and renaming ... 2019-05-09 13:28:26 +02:00
Dietmar Maurer 21ee7912fa src/api2/admin/datastore/backup.rs: implement upload chunk 2019-05-09 13:06:09 +02:00
Dietmar Maurer 4ebf0eabb0 src/api2/types.rs: use separate file for schema definitions 2019-05-09 07:44:09 +02:00
Dietmar Maurer bd1507c4fa src/api2/admin/datastore/backup/service.rs: move service code into extra file 2019-05-08 17:36:19 +02:00
Dietmar Maurer 0aadd40b4f src/api2/admin/datastore/backup.rs: add required parameters, cleanups 2019-05-08 13:17:23 +02:00
Dietmar Maurer d95ced64ad src/api2/admin/datastore/backup/environment.rs: use separate file 2019-05-08 12:41:58 +02:00
Dietmar Maurer ca60c371e2 api: rename h2upload to backup 2019-05-08 12:35:15 +02:00
Dietmar Maurer d906b1124f src/api2/admin/datastore/h2upload.rs: code cleanups 2019-05-08 11:36:43 +02:00
Dietmar Maurer 58c8d7d91f rc/api2/admin/datastore/h2upload.rs: implement BackupEnvironment
To pass arbitrary information/state to api methods.
2019-05-08 11:26:54 +02:00
Dietmar Maurer 090ac9f79a src/api2/admin/datastore/h2upload.rs: avoid unwrap 2019-05-08 08:21:18 +02:00
Dietmar Maurer 4e3da4b366 src/api2/admin/datastore/h2upload.rs: do not log client address 2019-05-08 08:15:55 +02:00
Dietmar Maurer e3a44552ee src/api2/admin/datastore/h2upload.rs: code cleanups 2019-05-08 08:12:49 +02:00
Dietmar Maurer a66ab8ae5e src/api2/admin/datastore/h2upload.rs: make worker abortable 2019-05-08 07:14:40 +02:00
Dietmar Maurer 72375ce6c6 src/api2/admin/datastore/h2upload.rs: log to worker task 2019-05-07 14:07:00 +02:00
Dietmar Maurer d9bd06eae8 src/api2/admin/datastore/h2upload.rs: use WorkerTask 2019-05-07 13:42:00 +02:00
Dietmar Maurer 52cf506e48 src/api2/admin/datastore/h2upload.rs: implement async test api call 2019-05-07 12:26:55 +02:00
Dietmar Maurer 92ac375aff src/api2/admin/datastore/h2upload.rs: improve http2 service 2019-05-07 11:24:44 +02:00
Dietmar Maurer 96e95fc179 src/api2/admin/datastore/h2upload.rs: test if version < 2
We can only upgrade http 1.x connections.
2019-05-06 10:29:34 +02:00
Dietmar Maurer cf639a47de rc/client/http_client.rs: add experimental h2 upgrade code 2019-04-29 11:57:58 +02:00
Dietmar Maurer 152764ec15 src/api2/admin/datastore/h2upload.rs: add test code for http2 2019-04-27 08:57:35 +02:00
Dietmar Maurer 6358b68236 cleanup: avoid compiler warnings 2019-04-23 12:17:24 +02:00
Dietmar Maurer 13f1cc17ea src/api_schema/router.rs: implement list_subdirs() helper 2019-04-16 12:07:02 +02:00
Wolfgang Bumiller 062d4916ff api_schema: allow generic api handler functions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-16 11:22:23 +02:00
Dietmar Maurer 1a6c9415ae src/api2/node/services.rs: add proxmox-backup-proxy service 2019-04-15 09:48:02 +02:00
Dietmar Maurer 75bc49bed7 src/server/worker_task.rs: implement abort channel 2019-04-15 09:38:05 +02:00
Dietmar Maurer 912524752e src/api2/admin/datastore/pxar.rs: create worker task for file upload 2019-04-15 08:38:14 +02:00
Dietmar Maurer f2b99c34f7 src/api2/admin/datastore.rs: implement API to return last GC status 2019-04-11 12:04:25 +02:00
Dietmar Maurer a665dea128 src/api2/node/tasks.rs: implement stop_task api 2019-04-10 13:55:05 +02:00
Dietmar Maurer c360bd73d3 src/api2/node/tasks.rs: fix task status api 2019-04-09 15:12:20 +02:00
Dietmar Maurer d8d40dd0c2 src/api2/node.rs: add node parameter 2019-04-09 14:43:30 +02:00
Dietmar Maurer 634132fe10 src/server/upid.rs: moved code into separate file 2019-04-08 18:43:26 +02:00
Dietmar Maurer 5a12c0e2fb src/api2/node/tasks.rs: start upid API 2019-04-07 14:36:57 +02:00
Dietmar Maurer d2a2e02bb7 src/api2/node/tasks.rs: implement userfilter 2019-04-07 13:17:19 +02:00
Dietmar Maurer 3c3bee2e32 src/api2/node/tasks.rs: correctly handle start parameter 2019-04-07 12:41:24 +02:00
Dietmar Maurer 063ca5be77 src/api2/node/tasks.rs: implement list_tasks 2019-04-07 12:18:58 +02:00
Dietmar Maurer d4b59ae0b8 src/backup/datastore.rs: pass WorkerTask to GC
So that we can log to the task log file.
2019-04-06 17:57:38 +02:00
Dietmar Maurer 0f778e0652 src/api2/admin/datastore.rs: use new WorkerTask::new_thread() 2019-04-06 11:27:23 +02:00
Dietmar Maurer 3e6a7deeaa src/api2/admin/datastore.rs: start GC in a separate thread 2019-04-01 12:02:46 +02:00
Dietmar Maurer ad281d1bd9 www/DataStoreConfig.js: add button to start GC 2019-04-01 08:08:34 +02:00
Dietmar Maurer 8968258b66 rename catar into pxar
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
Dietmar Maurer d5c34d98c0 src/catar/decoder.rs: simplify public restore API 2019-03-11 14:31:01 +01:00
Dietmar Maurer 01a1342364 src/api2/admin/datastore.rs: impl list_snapshot_files 2019-03-06 11:02:08 +01:00
Dietmar Maurer dd79265a8a src/api2/admin/datastore/catar.rs: use archive_name directly 2019-03-06 11:02:08 +01:00
Wolfgang Bumiller e2d007f76e api2/admin/datastore: add a backup protocol test api path
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
Dietmar Maurer f8f94534d4 src/api2/access.rs: authenticate_user() - add ticket login 2019-03-05 12:56:21 +01:00
Wolfgang Bumiller 391d310741 backup_info: BackupDir shortcut
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
Dietmar Maurer b3483782d1 src/backup/backup_info.rs: move code into separate file
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
Dietmar Maurer 2b01a22507 src/backup/datastore.rs: add helper to sort backup lists 2019-03-04 18:20:57 +01:00
Dietmar Maurer 875fb1c01a src/backup/datastore.rs: use unix epoch to create DateTime
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
Dietmar Maurer 9b492eb256 src/backup/datastore.rs: protect BackupDir fields, impl new() 2019-03-04 13:51:36 +01:00
Dietmar Maurer 1e9a94e579 src/backup/datastore.rs: protect BackupGroup fields, impl new() 2019-03-04 13:38:23 +01:00
Dietmar Maurer 6f62c9240a src/api2/admin/datastore.rs: imp delete_snapshot 2019-03-03 11:29:00 +01:00
Dietmar Maurer e77a02ed87 catar api: cleanup parameter names 2019-03-03 09:48:32 +01:00
Dietmar Maurer 49dc074007 parameter name cleanups (use '-' instead of '_') 2019-03-02 16:34:10 +01:00
Dietmar Maurer ad20d19877 src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups 2019-03-02 16:28:36 +01:00
Dietmar Maurer 184f17afad src/api2/admin/datastore.rs: impl list_snapshots 2019-03-02 16:20:50 +01:00
Dietmar Maurer 812c6f8716 src/api2/admin/datastore.rs: impl list_backup_groups 2019-03-02 11:29:05 +01:00
Dietmar Maurer e909522f06 src/backup/datastore.rs: use timestamp with zone offset 2019-03-01 17:36:40 +01:00
Dietmar Maurer dc4c09faae src/backup/datastore.rs: split BackupDir 2019-03-01 16:26:35 +01:00
Dietmar Maurer 38b0dfa511 src/backup/datastore.rs: split BackupInfo 2019-03-01 16:26:35 +01:00
Dietmar Maurer 541b908ea4 src/backup/datastore.rs: use DateTime<Utc> instead of unix epoch 2019-02-28 17:03:48 +01:00
Dietmar Maurer 8f57971714 src/api2/admin/datastore.rs: implement prune 2019-02-28 12:51:27 +01:00
Dietmar Maurer 83b7db0209 src/api2/admin/datastore.rs: start prune api
just a dummy for now.
2019-02-27 16:53:17 +01:00
Wolfgang Bumiller 8731e40a7f backup/datastore: create_backup_dir: show if path existed
To enable asserting the creation of a new backup rather than
adding to an existing one.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-27 10:15:36 +01:00
Dietmar Maurer 8c75372b79 src/backup/datastore.rs: list_backups() - include list of index files 2019-02-20 13:37:44 +01:00
Dietmar Maurer 247cdbce72 src/api2/admin/datastore/catar.rs: allow to configure chunk-size 2019-02-19 15:19:12 +01:00
Dietmar Maurer 1c7a88aeff UploadCaTar: currectly write all data using write_all() 2019-02-18 18:31:02 +01:00
Dietmar Maurer ef2f2efbcc improve api_schema module structure 2019-02-17 10:16:33 +01:00
Dietmar Maurer dc9a007b11 rename src/api to src/api_schema 2019-02-17 09:59:20 +01:00
Wolfgang Bumiller 1d77b6cf6b update to pam 0.7 (renamed from pam-auth)
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-15 10:34:40 +01:00
Dietmar Maurer f02e6fc478 api2/admin/datastore/catar.rs: simplify/fix debug message 2019-02-15 10:04:37 +01:00
Dietmar Maurer 541a302224 api2/admin/datastore/catar.rs: use nested file extensions
i.e. archive.catar.didx
2019-02-13 10:37:00 +01:00
Dietmar Maurer e5064ba607 simplify backup lib structure (pub use xxx:*), improve doc 2019-02-12 14:13:31 +01:00
Dietmar Maurer 93d5d77952 rename ArchiveIndex to DynamicIndex
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00
Wolfgang Bumiller ace9e3531a access: use proxmox-backup-auth for pam
allows customization via /etc/pam.d/proxmox-backup-auth

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-07 13:40:22 +01:00
Wolfgang Bumiller 13f8310cac api2: time: use /etc/localtime as fallback
in case there is no /etc/timezone

Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-05 14:14:56 +01:00
Dietmar Maurer 4b2cdeb9a6 implement relead_timezone flag 2019-02-01 10:04:46 +01:00
Dietmar Maurer c82bc1a1f9 depend on pam-auth, use pam for root login
Please use username "root@pam" for login via GUI.
2019-02-01 09:30:50 +01:00
Dietmar Maurer a154a8e8a4 delay unauthorized request (rate limit) 2019-01-31 14:34:21 +01:00
Dietmar Maurer b9903d6331 server/rest.rs: verify auth cookie 2019-01-31 12:22:00 +01:00
Dietmar Maurer 9f49fe1d5d avoid compiler warnings 2019-01-30 18:25:37 +01:00
Dietmar Maurer 34f956bc25 api2/access.rs: add ticket api 2019-01-30 15:16:10 +01:00
Dietmar Maurer a859fa95a9 add protected flag for some api methods 2019-01-28 18:16:47 +01:00
Dietmar Maurer f120483353 server/rest.rs: add proxy_sync_api_request() dummy 2019-01-28 17:30:39 +01:00
Dietmar Maurer 48849593ee api2/node/services.rs: implement service commands 2019-01-27 12:40:31 +01:00
Dietmar Maurer e82dad9700 also pass rpcenv to async handlers 2019-01-27 10:18:52 +01:00
Dietmar Maurer d2ab5f19e2 api2/node/services.rs: start services api 2019-01-26 17:54:18 +01:00
Dietmar Maurer 6049b71f41 api: pass RpcEnvirnment to api handlers 2019-01-26 14:50:37 +01:00
Dietmar Maurer d96d82736d use crate log and syslog 2019-01-26 10:56:11 +01:00
Dietmar Maurer e182ab4a4a api2/node/syslog.rs: improve error handling 2019-01-26 10:23:52 +01:00
Dietmar Maurer 0441886842 api2/node/syslog.rs: implement dump journal 2019-01-25 18:20:51 +01:00
Dietmar Maurer 4f9a726833 api2/node/syslog.rs: add syslog api schema 2019-01-25 17:17:30 +01:00
Dietmar Maurer af2fddea4f api2/node/dns.rs: implement concurrent update protection 2019-01-25 11:38:59 +01:00
Dietmar Maurer de6b0721fa pi2/node/dns.rs: return file digest 2019-01-25 11:03:41 +01:00
Dietmar Maurer 46b79b9ee5 api2/node/dns.rs: improve dns api 2019-01-25 10:15:32 +01:00
Dietmar Maurer 8f973f8181 api2/node/dns.rs: implement resolv.conf parser 2019-01-24 14:59:40 +01:00
Dietmar Maurer e6ffeb918c api2/node/time.rs: implement set timezone 2019-01-24 12:05:06 +01:00
Dietmar Maurer cd8b91f78c api2/node/time.rs: add return schema 2019-01-24 11:07:11 +01:00