78b5191550
src/server/formatter.rs: further cleanups and renaming ...
2019-05-09 13:28:26 +02:00
d55037e471
src/server/formatter.rs: rename format_result to format_data
...
To avoid confusions with Rust Result type.
2019-05-09 13:15:15 +02:00
21ee7912fa
src/api2/admin/datastore/backup.rs: implement upload chunk
2019-05-09 13:06:09 +02:00
4ebf0eabb0
src/api2/types.rs: use separate file for schema definitions
2019-05-09 07:44:09 +02:00
bd1507c4fa
src/api2/admin/datastore/backup/service.rs: move service code into extra file
2019-05-08 17:36:19 +02:00
0aadd40b4f
src/api2/admin/datastore/backup.rs: add required parameters, cleanups
2019-05-08 13:17:23 +02:00
d95ced64ad
src/api2/admin/datastore/backup/environment.rs: use separate file
2019-05-08 12:41:58 +02:00
ca60c371e2
api: rename h2upload to backup
2019-05-08 12:35:15 +02:00
d906b1124f
src/api2/admin/datastore/h2upload.rs: code cleanups
2019-05-08 11:36:43 +02:00
58c8d7d91f
rc/api2/admin/datastore/h2upload.rs: implement BackupEnvironment
...
To pass arbitrary information/state to api methods.
2019-05-08 11:26:54 +02:00
f757b30efc
src/server/rest.rs: use generics to pass RpcEnvironment
2019-05-08 11:09:01 +02:00
2edc341b29
src/tools.rs: Add AsAyn Trait
2019-05-08 11:07:36 +02:00
090ac9f79a
src/api2/admin/datastore/h2upload.rs: avoid unwrap
2019-05-08 08:21:18 +02:00
4e3da4b366
src/api2/admin/datastore/h2upload.rs: do not log client address
2019-05-08 08:15:55 +02:00
e3a44552ee
src/api2/admin/datastore/h2upload.rs: code cleanups
2019-05-08 08:12:49 +02:00
a66ab8ae5e
src/api2/admin/datastore/h2upload.rs: make worker abortable
2019-05-08 07:14:40 +02:00
72375ce6c6
src/api2/admin/datastore/h2upload.rs: log to worker task
2019-05-07 14:07:00 +02:00
d9bd06eae8
src/api2/admin/datastore/h2upload.rs: use WorkerTask
2019-05-07 13:42:00 +02:00
52cf506e48
src/api2/admin/datastore/h2upload.rs: implement async test api call
2019-05-07 12:26:55 +02:00
92ac375aff
src/api2/admin/datastore/h2upload.rs: improve http2 service
2019-05-07 11:24:44 +02:00
78f7e225e4
src/server/rest.rs: make handle_(a)sync_api_request public
2019-05-07 11:23:52 +02:00
75097495ad
RestEnvironment: derive Clone
2019-05-07 11:09:18 +02:00
01bf3b7b5f
move find_method to Router class
2019-05-07 11:08:30 +02:00
3578d99f3e
move normalize_path to tools::normalize_uri_path
2019-05-07 09:44:34 +02:00
96e95fc179
src/api2/admin/datastore/h2upload.rs: test if version < 2
...
We can only upgrade http 1.x connections.
2019-05-06 10:29:34 +02:00
45cdce069e
src/client/http_client.rs: request password at creation time
2019-04-30 11:44:35 +02:00
e45afdff9c
src/server/state.rs: use new BroadcastData helper
2019-04-30 10:21:48 +02:00
824b5ee4ee
src/tools/broadcast_future.rs: cleanup, decompose into two classes
...
In order to make it more usable.
2019-04-30 08:57:20 +02:00
d82ed9b0f5
tools: add AsyncMutex
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-29 14:19:03 +02:00
3a9bea3918
src/bin/h2test.rs: test h2 upgrade
2019-04-29 11:59:18 +02:00
cf639a47de
rc/client/http_client.rs: add experimental h2 upgrade code
2019-04-29 11:57:58 +02:00
5a2df00004
src/client/http_client.rs: cleanup, make login fully async
2019-04-28 10:55:03 +02:00
3dceb9b304
src/tools/broadcast_future.rs: add new constructor new_oneshot()
...
To simplify usage. Also added a test case.
2019-04-27 10:56:49 +02:00
152764ec15
src/api2/admin/datastore/h2upload.rs: add test code for http2
2019-04-27 08:57:35 +02:00
490be29ed6
src/tools/broadcast_future.rs: new helper class
2019-04-26 17:57:57 +02:00
e888155712
proxy: allow .key/.pem files instead of a pkcs12/.pfx file
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-26 10:47:33 +02:00
8b7c0def84
src/tools/procfs.rs: clean up tab indentation and use whitespaces instead
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-25 13:14:52 +02:00
5e5eed5c3b
use double-fork for reload
...
To ensure the new process' parent is pid 1, so systemd won't
complain about supervising a process it does not own.
Fixes the following log spam on reloads:
Apr 25 10:50:54 deb-dev systemd[1]: proxmox-backup.service: Supervising process 1625 which is not our child. We'll most likely not notice when it exits.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:02:12 +00:00
efd1536eb7
add a wrapper around nix::unistd::pipe2
...
Using O_CLOEXEC by default, and returning Fd handles to
ensure they get dropped on bail!() or panic!() if the RawFds
aren't used yet.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:02:12 +00:00
897982e237
tools: add read/write convenience helpers
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:02:12 +00:00
d98c9a7a38
use service Type=notify
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:01:33 +00:00
9c351a36c4
tools/daemon: add sd_notify wrapper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-25 11:01:28 +00:00
d8871f2f18
src/tools/procfs.rs: avoid useless allocations
2019-04-24 09:35:15 +02:00
ae694f44ef
src/tools/procfs.rs: add tests for read_proc_net_route() and read_proc_net_ipv6_route()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-24 06:43:52 +02:00
ec8db837fb
src/tools/procfs.rs: refactor read_proc_net_route() and read_proc_net_ipv6_route()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-24 06:43:32 +02:00
eb46c3d1ec
src/tools/procfs.rs: implement read_proc_net_ipv6_route()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-23 14:31:23 +02:00
6358b68236
cleanup: avoid compiler warnings
2019-04-23 12:17:24 +02:00
e06ffb10d1
src/api_schema/router.rs: fix typo (subdira -> subdir)
2019-04-16 12:20:17 +02:00
13f1cc17ea
src/api_schema/router.rs: implement list_subdirs() helper
2019-04-16 12:07:02 +02:00
062d4916ff
api_schema: allow generic api handler functions
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-16 11:22:23 +02:00
286f0d4099
src/tools/procfs.rs: implement read_proc_net_route()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-15 11:13:16 +02:00
1a6c9415ae
src/api2/node/services.rs: add proxmox-backup-proxy service
2019-04-15 09:48:02 +02:00
75bc49bed7
src/server/worker_task.rs: implement abort channel
2019-04-15 09:38:05 +02:00
912524752e
src/api2/admin/datastore/pxar.rs: create worker task for file upload
2019-04-15 08:38:14 +02:00
1be71fb02b
src/server/worker_task.rs_ do not write ERROR: prefix into task index
2019-04-15 08:37:13 +02:00
418def7a78
src/server/worker_task.rs: move common code into log_result
...
Make new() pub
2019-04-15 07:49:03 +02:00
37b87869b5
src/server/upid.rs: check for illegal characters
2019-04-15 07:47:51 +02:00
d2981e2738
src/tools/procfs.rs: implement read_proc_net_dev()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-13 11:52:59 +02:00
13abfdc57a
src/tools/procfs.rs: implement read_memory_usage()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-13 11:52:01 +02:00
3005ef48b6
src/tools/procfs.rs: Rename variable and struct to be more consistent
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-12 12:13:34 +02:00
cc93bacb72
src/tools/procfs.rs: simplify read_cpuinfo()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-12 12:13:14 +02:00
f2b99c34f7
src/api2/admin/datastore.rs: implement API to return last GC status
2019-04-11 12:04:25 +02:00
59961b8954
src/server/command_socket.rs: check control socket permissions
2019-04-11 10:51:59 +02:00
ba70040dbf
src/server/worker_task.rs: change task list sort order - latest on top
2019-04-11 09:43:33 +02:00
d3f4c08f33
src/server/worker_task.rs: catch panics in worker threads
2019-04-11 07:56:06 +02:00
620dccf1a1
tools/daemon: dup the TcpListener file descriptor
...
Now that we let hyper shutdown gracefully we need an owned
version of the listening socket to prevent it from closing
before running the reload preparations.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-10 15:20:10 +02:00
d96bb7f163
tools: add Fd helper
...
stores a raw file descriptor with a drop handler for
safekeeping in closures
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-04-10 15:20:10 +02:00
1cd33633aa
/src/tools: Cleanup tempfiles in folder
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-10 13:56:22 +02:00
a665dea128
src/api2/node/tasks.rs: implement stop_task api
2019-04-10 13:55:05 +02:00
98a181f055
src/server/worker_task.rs: log message when we call request_abort()
2019-04-10 13:54:28 +02:00
2b6a3e1396
src/tools/procfs.rs: implement read_cpuinfo()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-10 13:11:11 +02:00
321070b4fa
src/server/worker_task.rs: implement abort_worker (via command_socket)
2019-04-10 12:42:24 +02:00
cfb2d3c17c
src/server/command_socket.rs: correctly handle/spawn handle parallel connections
2019-04-10 11:05:00 +02:00
9b002cbc5f
src/server/worker_task.rs: use abstract socket
2019-04-10 09:03:17 +02:00
5f550fd99f
start hyper server using with_graceful_shutdown()
...
Without, hyper keeps some futures running, and the server does not
correctly shutdown.
2019-04-10 08:24:32 +02:00
7f2078de59
src/tools/procfs.rs: avoid compiler warning
2019-04-10 06:40:29 +02:00
1f6fbf1c57
src/tools/procfs.rs: implement read_meminfo()
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-09 15:21:49 +02:00
c360bd73d3
src/api2/node/tasks.rs: fix task status api
2019-04-09 15:12:20 +02:00
0bfd87bc5e
src/server/worker_task.rs: speedup upid_read_status()
2019-04-09 15:11:48 +02:00
d8d40dd0c2
src/api2/node.rs: add node parameter
2019-04-09 14:43:30 +02:00
7ae0bf52dd
src/server/command_socket.rs: code cleanup - fix error message
2019-04-09 12:47:42 +02:00
d607b8861b
src/server/worker_task.rs: implement task control socket
2019-04-09 12:15:06 +02:00
e201753629
src/server/command_socket.rs: implement auto_remove flag
...
Remove the socket file on close.
2019-04-09 11:47:23 +02:00
634132fe10
src/server/upid.rs: moved code into separate file
2019-04-08 18:43:26 +02:00
78a39e0517
src/server/command_socket.rs: simple command socket
2019-04-08 17:59:39 +02:00
e3f41f2165
src/tools/daemon.rs: use new ServerState handler
2019-04-08 14:00:23 +02:00
7a630df709
implement server state/signal handling, depend on tokio-signal
2019-04-08 13:59:07 +02:00
5a12c0e2fb
src/api2/node/tasks.rs: start upid API
2019-04-07 14:36:57 +02:00
b75b968147
src/server/worker_task.rs: implement is_active()
2019-04-07 14:36:25 +02:00
4494d07819
src/server/worker_task.rs: make log_path a method on UPID
2019-04-07 13:51:46 +02:00
d2a2e02bb7
src/api2/node/tasks.rs: implement userfilter
2019-04-07 13:17:19 +02:00
3c3bee2e32
src/api2/node/tasks.rs: correctly handle start parameter
2019-04-07 12:41:24 +02:00
063ca5be77
src/api2/node/tasks.rs: implement list_tasks
2019-04-07 12:18:58 +02:00
d4b59ae0b8
src/backup/datastore.rs: pass WorkerTask to GC
...
So that we can log to the task log file.
2019-04-06 17:57:38 +02:00
35950380a9
src/server/worker_task.rs: carefully handle file permissions
2019-04-06 17:53:12 +02:00
1619a72063
src/tools.rs: implement create_dir_chown
...
Combinded mkdir with chown.
2019-04-06 16:50:23 +02:00
eea8131952
src/tools.rs: implement file_set_contents_full()
2019-04-06 13:53:43 +02:00
0f778e0652
src/api2/admin/datastore.rs: use new WorkerTask::new_thread()
2019-04-06 11:27:23 +02:00
244abab769
src/tools/file_logger.rs: fix test
2019-04-06 11:24:37 +02:00
660c684640
src/server/worker_task.rs: return upid_str
2019-04-06 11:23:53 +02:00
93aebb38bc
src/server/worker_task.rs: implement read_task_list()
2019-04-06 10:17:11 +02:00
4a36a2c999
src/server/worker_task.rs: improve UPID format description
2019-04-06 09:36:50 +02:00
882594c5e9
src/server.rs: improve crate layout
2019-04-06 09:17:25 +02:00
4b01c983f0
src/server/worker_task.rs: implement persistent task list
2019-04-05 12:37:35 +02:00
a2474d328f
src/tools/procfs.rs: implement read_proc_uptime and read_proc_uptime_ticks
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-05 06:22:21 +02:00
08daa81af6
procfs: implement check_process_running and check_process_running_pstart
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-04 13:54:17 +02:00
9a0d0ff774
server/worker_task.rs: implement UPID parser
2019-04-04 13:49:35 +02:00
49cf9f3d0b
src/tools.rs: file_get_json() - add new default parameter
2019-04-04 12:24:18 +02:00
79ad7b7ec2
src/tools/procfs.rs: split regex into several lines
2019-04-04 09:25:19 +02:00
479f6e40ec
src/server/worker_task.rs: implement task registry
2019-04-04 08:58:39 +02:00
52ddc2522d
src/tools/procfs.rs: use libc::pid_t
2019-04-04 08:05:43 +02:00
d3d7ab1273
src/tools/file_logger.rs: new - accept AsRef<Path>
2019-04-03 14:13:33 +02:00
3c2012f97a
src/tools/procfs.rs: start helper tools to read procfs
2019-04-03 13:39:20 +02:00
3489936ebb
src/tools/file_logger.rs: change timestamp format to rfc3339
2019-04-03 08:58:43 +02:00
92da93b245
abort GC on server shutdown
2019-04-01 12:13:02 +02:00
9136f857cc
add global var to indicate server shutdown requests
2019-04-01 12:05:11 +02:00
3e6a7deeaa
src/api2/admin/datastore.rs: start GC in a separate thread
2019-04-01 12:02:46 +02:00
ad281d1bd9
www/DataStoreConfig.js: add button to start GC
2019-04-01 08:08:34 +02:00
49d123ee3a
src/server/rest.rs: use formatter to encode errors
2019-04-01 08:04:12 +02:00
7d4ef127cb
src/server/rest.rs: generate csrf token if we have a valid ticket
...
This is important if the user reloads the browser page.
2019-04-01 07:52:30 +02:00
11861a482d
src/backup/chunk_store.rs: fix GC
...
Added option to get oldest_writer timestamp from ProcessLocker.
2019-03-31 17:21:36 +02:00
d85987aeeb
fix last commit: the filename var was not ment to be removed, sorry
2019-03-31 16:16:14 +02:00
15a77c4c2e
src/backup/chunk_store.rs: avoid create/unlink race
2019-03-31 10:03:01 +02:00
c374f05499
src/backup.rs: improve GC problem description
2019-03-31 09:44:35 +02:00
c8ec450e37
src/backup.rs: start explaining different GC algorithm
2019-03-30 17:21:40 +01:00
8a4757342c
src/backup.rs: describe the garbage collection problem
2019-03-30 16:26:52 +01:00
10eea49d81
src/backup/fixed_index.rs: store reference to file
...
Keep it open and locked while index is in use.
2019-03-27 10:17:03 +01:00
c597a92c81
index file readers: aquire shared lock
2019-03-27 10:11:14 +01:00
0465218953
src/backup.rs: add documentation about ChunkStore locking
2019-03-22 10:14:50 +01:00
43b1303398
datastore: use new ProcessLocker
...
To make sure only one process runs garbage collection while having active writers.
2019-03-22 09:42:15 +01:00
abfc001f25
src/tools/process_locker.rs: new() - use generic path parameter
2019-03-22 09:41:32 +01:00
a650f50301
src/tools/process_locker.rs: implement inter-process reader-writer locks
2019-03-22 08:05:51 +01:00
eb90c9e372
tools: style fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-20 09:57:13 +01:00
e04d1bfadc
Cleanup of if statement with unwraps by a more compact match statement
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-03-20 09:49:18 +01:00
8346f0d59b
src/server/rest.rs: correctly extract content type
2019-03-19 12:50:15 +01:00
a690ecacf0
daemon: simplify daemon creation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 12:12:54 +01:00
e4311382d9
tools: daemon: rename some structs
...
Reloadable resources are now 'Reloadable' instead of
'ReexecContinue'.
The struct handling the reload is a 'Reloader', not a
'ReexecStore'.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 11:09:46 +01:00
5e7bc50af1
add reload support to api daemon
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
a2ca713701
add reload support to proxy
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
af70c1811b
tools: implement ReexecContinue for tokio's TcpListener
...
This is the only thing we currently need to keep alive for
reloads.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
4422ba2c7f
tools: daemon: add a default signalfd helper
...
Proxy and daemon for now just want to handle reload via
`SIGHUP`, so provide a helper creating the signalfd stream
doing that - this is simply a filtered stream which passes
the remaining signals through, so it can be used exactly
like the signalfd stream could before to add more signals.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
dce94d0e09
tools: add daemon helpers
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
ff7049d481
tools: add fd_change_cloexec helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
1c042cdc6c
proxy: listener error handling fixup
...
If the listening socket goes into some error state we'll get
std::io::Errors rather than higher level errors from the
native_tls::TlsAcceptor, those are usually fatal. (Ran into
this after performing a shutdown() on the file descriptor,
after which the future just endlessly loops in accept().)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
60864b1b56
src/pxar.rs: improve docu
2019-03-18 12:27:30 +01:00
515688d1f1
avoid compiler warnings
2019-03-18 10:00:58 +01:00
ba10f2b0d4
src/bin/dynamic-files.rs: test case for file shrink during backup
2019-03-16 14:22:14 +01:00
af572aaa4c
pxar: implement hardlinks
...
So we are no longer compatible with catar ...
2019-03-16 11:02:12 +01:00
812dd935ce
src/pxar/encoder.rs: detect hardlinks
2019-03-15 12:19:51 +01:00
896e803feb
src/pxar/sequential_decoder.rs: dump_entry - fix FIFO/SOCKET handling
2019-03-15 12:15:38 +01:00
7dfa17c7cc
src/pxar/binary_search_tree.rs: fix test
2019-03-15 11:34:31 +01:00
9307279faf
cleanup: remove dead code
2019-03-15 10:18:28 +01:00
46cfe7ab7e
src/pxar/decoder.rs: implement seekable decoder
2019-03-15 09:36:05 +01:00
0e21b87be3
renamed: src/pxar/decoder.rs -> src/pxar/sequential_decoder.rs
2019-03-15 08:36:02 +01:00
3dbfe5b142
simplify pxar module structure
2019-03-15 08:24:32 +01:00
1ef46b81d3
src/bin/pxar.rs: implement extract
2019-03-15 08:03:44 +01:00
02c93163cf
src/pxar/decoder.rs: correctly call callback()
2019-03-15 08:02:04 +01:00
5defa71bf9
rename PxarBackupWriter into PxarDecodeWriter
2019-03-15 07:20:22 +01:00
b005ed12d5
src/client/http_client.rs: avoid compiler warning
2019-03-15 07:16:26 +01:00
9e26abf16b
src/cli/command.rs: fix iterator usage
2019-03-15 07:12:40 +01:00
f98746ba6c
src/pxar/decoder.rs: improve docu
2019-03-15 07:12:11 +01:00
0b2d7fdc81
src/pxar/decoder.rs: avoid compiler warnings
2019-03-14 17:45:47 +01:00
37940aa1f9
src/bin/pxar.rs: implement dump/list
2019-03-14 17:43:11 +01:00
40360fde6e
src/pxar/decoder.rs: implement dump_archive
...
use this to debug archive problems ...
2019-03-14 13:10:27 +01:00
8968258b66
rename catar into pxar
...
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
7c4dd94670
rename catar into pxar
...
Seems impossible to stay compatible with the original catar specification,
so we change the name to avoid confusions.
2019-03-14 10:14:32 +01:00
806500cd09
src/tools.rs: improve file name completion
...
Use current working directory by default.
2019-03-14 08:17:35 +01:00
08dc340a12
src/bin/proxmox-backup-client.rs: complete archive names
2019-03-14 08:09:35 +01:00
6f90a6a764
protocol: cleanup finish_backup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-13 14:16:17 +01:00
b238851832
src/bin/proxmox-backup-client.rs: implement completion for snapshots
2019-03-13 13:31:29 +01:00
4981134708
src/bin/proxmox-backup-client.rs: implement chunk-size completion
2019-03-13 12:26:01 +01:00
98f0b97293
src/bin/proxmox-backup-client.rs: cleanup
2019-03-13 12:17:39 +01:00
024f11bb55
src/bin/proxmox-backup-client.rs: implement completion for backup groups
2019-03-13 11:56:37 +01:00
edd3c8c605
src/client/backup_repo.rs: implement FromStr trait
2019-03-13 10:09:39 +01:00
874acb7039
src/client/backup_repo.rs: implement Display trait
2019-03-13 09:57:36 +01:00
d0a03d40ce
src/bin/proxmox-backup-client.rs: implement repository completions
2019-03-13 09:47:12 +01:00
bff110308b
src/bin/proxmox-backup-client.rs: fix BACKUPSPEC_REGEX - correctly specify non-capture group
...
And improve bash completion.
2019-03-13 07:23:59 +01:00
1a71509aad
src/cli/command.rs: fix completion - never display argument parameters as option
2019-03-12 14:53:41 +01:00
496a67846f
src/cli/command.rs: pass parsed parameters to completion function
2019-03-12 14:39:51 +01:00
b5fa28251c
src/cli/getopts.rs: factor out reusable code
2019-03-12 14:38:47 +01:00
e39efdbd33
src/cli/command.rs: use a HashMap to store fixed param as key/value pairs
2019-03-12 12:54:16 +01:00
f4e4d58327
src/cli/command.rs: improve completions
2019-03-12 12:07:45 +01:00
4ba7b6099c
signalfd cleanup
...
- add comment about the boxed file descriptor (and others)
- reexport SigSet and Signal for convenience
- remove Result wrapper (only used once)
- enforce drop order of the PollEvented2 struct with respect
to the boxed fd
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-12 11:07:09 +01:00
f54c19989c
tools: add async signalfd handler
...
This provides a Stream<Item = siginfo> via nix' signalfd,
by wrapping it in tokio's PollEvented2 struct to allow
polling via tokio's event loop.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-12 08:04:28 +01:00
d5c34d98c0
src/catar/decoder.rs: simplify public restore API
2019-03-11 14:31:01 +01:00
8e39232acc
src/bin/proxmox-backup-client.rs: strip .didx file extensions
2019-03-11 10:54:02 +01:00
74cdb52107
src/bin/proxmox-backup-client.rs: include file extension in backupspec
2019-03-11 10:27:28 +01:00
fc2bf37ead
src/catar/decoder.rs: cleanup style
2019-03-11 08:14:28 +01:00
656b23e107
src/catar/decoder.rs: dir_mkdirat - add create_new flag
2019-03-11 07:27:45 +01:00
e9c9409a99
src/catar/decoder.rs: implement real sequential decoder (no Seek)
...
I copied the old code to src/catar/inspector.rs. Will modify
that later to implement a random access decoder ...
2019-03-08 16:55:54 +01:00
eed6db3923
src/bin/proxmox-backup-client.rs: impl option all-file-systems
2019-03-08 09:36:40 +01:00
06178f13fa
move chunker into proxmox_protocol
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-08 09:29:47 +01:00
e3c30c50a6
src/bin/catar.rs: implement all-file-systems option
2019-03-08 09:28:12 +01:00
65092b1e27
src/catar/encoder.rs: log skipped mount points
2019-03-08 09:25:00 +01:00
6b64d25d64
src/catar/encoder.rs: remove debug output
2019-03-08 08:23:07 +01:00
2689810c92
src/bin/catar.rs: implement verbose flags
2019-03-08 08:20:56 +01:00
79c281fdac
src/catar/encoder.rs: use simply flag "all_file_systems"
2019-03-08 08:14:26 +01:00
25e77d3877
src/cli/command.rs: print_bash_completion - avoid crash
2019-03-07 12:14:26 +01:00
141f062e08
src/backup/chunk_store.rs: use zstd compression insteadf of lz4
...
Provides better compressionm rate, and is still fast.
2019-03-07 11:42:59 +01:00
d2690f74bb
src/backup/chunk_store.rs: read_chunk - clear buffer before calling read_to_end
2019-03-07 10:40:01 +01:00
2ae7d196b9
src/bin/proxmox-backup-client.rs: correctly strip file extension
2019-03-06 11:18:46 +01:00
9f91249351
src/bin/proxmox-backup-client.rs: impl restore
2019-03-06 11:02:08 +01:00
01a1342364
src/api2/admin/datastore.rs: impl list_snapshot_files
2019-03-06 11:02:08 +01:00
58e99e138e
src/backup/backup_info.rs: new helper to list files
2019-03-06 11:02:08 +01:00
dd79265a8a
src/api2/admin/datastore/catar.rs: use archive_name directly
2019-03-06 11:02:08 +01:00
a6b7551311
src/client/http_client.rs: impl download
2019-03-06 11:02:08 +01:00
6716f30bbc
add protocol test client
...
This uses futures for everything which is mostly useful as a
test to see if the protocol crate's non-blocking I/O support
can handle it...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
e2d007f76e
api2/admin/datastore: add a backup protocol test api path
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
e3062f87b1
fixed_index: split add_chunk/add_digest
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
c3bb97e59c
fixed_index: impl Send for FixedIndexWriter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
3b7ade9e78
backup/datastore: expose insert_chunk method
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
91ebea8262
router: add upgrade method
...
This is just an Async GET method, but semantically
different, so it gets its own router method.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
80822b95b1
src/bin/proxmox-backup-client.rs: sort list results
2019-03-06 07:20:51 +01:00
fba30411fb
src/bin/proxmox-backup-client.rs: allow to set backup-id
2019-03-06 07:02:52 +01:00
21ea0158b4
src/client/http_client.rs: remove debug statement
2019-03-06 06:45:51 +01:00
ba3a60b2a4
src/client/http_client.rs: store/load ticket in xdg runtime dir, depend on crate xdg
2019-03-05 12:56:21 +01:00
f8f94534d4
src/api2/access.rs: authenticate_user() - add ticket login
2019-03-05 12:56:21 +01:00
e5662b04e9
src/tools/ticket.rs: define const TICKET_LIFETIME
2019-03-05 12:56:21 +01:00
53cafb5925
src/tools.rs: impl file_get_json()
2019-03-05 12:56:21 +01:00
391d310741
backup_info: BackupDir shortcut
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
93b49ce38a
backup_info: use two independent Into<String> types
...
there's no reason to force them to be the same input type
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 09:13:08 +01:00
3ec3ec3f9a
src/bin/proxmox-backup-client.rs: print backup duration
2019-03-05 08:11:40 +01:00
d57474e055
src/backup/backup_info.rs: improve docs
2019-03-05 07:28:13 +01:00
b3483782d1
src/backup/backup_info.rs: move code into separate file
...
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
2b01a22507
src/backup/datastore.rs: add helper to sort backup lists
2019-03-04 18:20:57 +01:00
875fb1c01a
src/backup/datastore.rs: use unix epoch to create DateTime
...
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
38f8815925
src/backup/datastore.rs: use new() function to create structs
2019-03-04 17:44:27 +01:00
9b492eb256
src/backup/datastore.rs: protect BackupDir fields, impl new()
2019-03-04 13:51:36 +01:00
1e9a94e579
src/backup/datastore.rs: protect BackupGroup fields, impl new()
2019-03-04 13:38:23 +01:00
cdebd467e6
src/bin/proxmox-backup-client.rs: simply log to stdout
2019-03-04 09:38:34 +01:00
f72dfd2d10
src/backup/datastore.rs: make sure timestamp does not contains nanoseconds
...
Because we want simple, readable file names.
2019-03-04 09:37:10 +01:00
219ef0e6ca
src/catar/encoder.rs: add verbose flags
2019-03-04 08:01:09 +01:00
e0b8ce6424
src/catar/encoder.rs: skip RAMFS
...
unless explicitly selected.
2019-03-04 06:48:47 +01:00
597a92039e
src/bin/proxmox-backup-client.rs: rename create into backup
2019-03-03 11:46:53 +01:00
6f62c9240a
src/api2/admin/datastore.rs: imp delete_snapshot
2019-03-03 11:29:00 +01:00
af926291e0
avoid compiler warning
2019-03-03 09:52:06 +01:00
e77a02ed87
catar api: cleanup parameter names
2019-03-03 09:48:32 +01:00
421c9ba1d9
src/bin/proxmox-backup-client.rs: use new json_object_to_query
2019-03-03 09:44:25 +01:00
f5f13ebc5a
src/tools.rs: new helper json_object_to_query
2019-03-03 08:51:37 +01:00
49dc074007
parameter name cleanups (use '-' instead of '_')
2019-03-02 16:34:10 +01:00
ad20d19877
src/api2/admin/datastore.rs: rename get_group_list to list_groups, cleanups
2019-03-02 16:28:36 +01:00
184f17afad
src/api2/admin/datastore.rs: impl list_snapshots
2019-03-02 16:20:50 +01:00
af53186e6a
src/tools.rs: add simply string join
2019-03-02 16:15:45 +01:00
812c6f8716
src/api2/admin/datastore.rs: impl list_backup_groups
2019-03-02 11:29:05 +01:00
e909522f06
src/backup/datastore.rs: use timestamp with zone offset
2019-03-01 17:36:40 +01:00
dc4c09faae
src/backup/datastore.rs: split BackupDir
2019-03-01 16:26:35 +01:00
38b0dfa511
src/backup/datastore.rs: split BackupInfo
2019-03-01 16:26:35 +01:00
dcd15d10bb
backup/index: remove IndexIterator
...
removed the need for it, to avoid copies
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-01 13:29:36 +01:00
17d6979afd
src/bin/proxmox-backup-client.rs: generate a log file
2019-03-01 09:35:41 +01:00
3b151414f8
src/tools/file_logger.rs: class to log into files
2019-03-01 09:34:29 +01:00
5a2892f0ae
src/bin/proxmox-backup-client.rs: use chrono::DateTime
2019-03-01 06:59:32 +01:00
51144821f9
src/bin/proxmox-backup-client.rs: login before starting backup
2019-03-01 06:48:41 +01:00
541b908ea4
src/backup/datastore.rs: use DateTime<Utc> instead of unix epoch
2019-02-28 17:03:48 +01:00
b55cee9227
src/cli/command.rs: generate_usage_str() - fix option separator (newline)
2019-02-28 16:20:00 +01:00
c4c5174000
src/cli/command.rs: get_schema_type_text() - display min/max
2019-02-28 16:11:49 +01:00
df67070b1c
client/backup_repo: allow ip addresses for BackupRepository
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
5de2bced2d
datastore: add generic open_index
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
7bc1d72778
add IndexFile trait
...
We want to be able to iterate through digests of index
files, but without always having to distinguish between
dynamic and fixed types, so add a trait we can use as a
trait object.
Unfortunately the iterator needs to yield copies as
iterators cannot yield values with life times when
represented as trait objects (Box<dyn Iterator<Item = ?>>)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
5be4065baf
make FixedIndexReader Send and update comments
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
fded74d0c7
style: BufferedReader -> BufferedRead
...
Single-method traits usually use the same name as their
method and aren't usually the 'noun' (which is usually an
implementation of them instead).
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-28 15:26:40 +01:00
8f57971714
src/api2/admin/datastore.rs: implement prune
2019-02-28 12:51:27 +01:00
83b7db0209
src/api2/admin/datastore.rs: start prune api
...
just a dummy for now.
2019-02-27 16:53:17 +01:00
0ffbccce5e
rc/server/rest.rs: allow to pass parameters as application/json
2019-02-27 12:37:53 +01:00
f98ea63de2
src/bin/proxmox-backup-client.rs: fix bash completion
2019-02-27 12:37:53 +01:00
8731e40a7f
backup/datastore: create_backup_dir: show if path existed
...
To enable asserting the creation of a new backup rather than
adding to an existing one.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-27 10:15:36 +01:00
25f1650b71
src/bin/proxmox-backup-client.rs: minor code cleanup
2019-02-27 07:21:57 +01:00
9bc3ddb822
src/cli/command.rs: improve generate_usage_str
...
correctly format positional array arguments ({<name>}).
2019-02-26 12:40:51 +01:00
ae0be2dda9
src/bin/proxmox-backup-client.rs: allow to upload more that one directory
2019-02-26 12:27:28 +01:00
a477d6889a
src/client/http_client.rs: login once, store and reuse ticket/token
2019-02-26 11:59:10 +01:00
d02c2ed701
src/cli/getopts.rs: cleanup, avoid compiler warning
2019-02-26 11:55:06 +01:00
f8dfbb45c0
src/tools.rs: new helper required_array_param()
2019-02-26 08:48:17 +01:00
aaba6faace
src/cli/getopts.rs: support positional array arguments
2019-02-26 08:47:34 +01:00
f33fa27307
src/cli/command.rs: improve error message
2019-02-26 08:46:36 +01:00
92a173feb0
src/api_schema/schema.rs: improve ParameterError display function
2019-02-26 07:11:24 +01:00
f661f37427
backup/dynamic_index: split out add_chunk
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:50:45 +01:00
9ac6ec868a
backup/chunk_store: split insert_chunk
...
The protocol handler will receive chunk data plus a hash
pre-calculated by the client. It will verify the hash before
sending it up to the datastore in order to respond to the
client with an error on a mismatch, so there's no need to
recalculate the hash another time.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
3a50ddd0c9
backup/chunk_store: mark ChunkIterator as fused
...
Ranges are FusedIterators and it can be useful to propagate
that property to users.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
eff25ecaf2
backup/chunk_store: make percentage output optional
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
d59397e60e
backup/datastore: expose chunk_store.get_chunk_iterator
...
will be used by the protocol handler to iterate over
existing chunks
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-25 13:44:29 +01:00
8eb1203bda
src/backup/chunk_stat.rs: log write speed
2019-02-25 13:21:00 +01:00
7e3365554e
src/backup/chunk_stat.rs: new struct to track chunk statistics
2019-02-25 12:52:10 +01:00
798f7fa065
src/backup/chunk_store.rs: return info about compressed chunk size
...
So that we can generate better statistics ..
2019-02-25 11:36:05 +01:00
78216a5ab1
src/backup/chunk_store.rs: use lz4 for compression
...
And depend on lz4 lib now.
2019-02-25 10:00:28 +01:00
793b0f4d77
src/cli/command.rs: improve help command completion
2019-02-23 18:07:32 +01:00
6949d91564
src/cli/command.rs: cleanup, avoid array manipulation
2019-02-23 17:50:33 +01:00
9d78d57960
src/cli/command.rs: implement bash completion for help
2019-02-23 17:38:10 +01:00
698d9d4402
src/cli/command.rs: add help command
2019-02-23 15:10:48 +01:00
8b6dd2240d
src/cli/command.rs::allow to generate ReST docu
2019-02-23 11:29:18 +01:00
2f3f2bb77f
src/cli/command.rs: improve usage imformation
2019-02-22 17:40:37 +01:00
f252ca654c
src/backup/chunker.rs: simplify break test
...
To make it a little bit faster ...
2019-02-22 13:02:11 +01:00
6adb40eefc
src/cli/command.rs: start implementing generate_usage_str()
2019-02-22 12:02:59 +01:00
33256db6dd
src/cli/command.rs: simplify cli handler
2019-02-22 11:15:10 +01:00
61834c1cbf
src/backup/chunker.rs: another try to speedup the chunker
2019-02-22 11:15:10 +01:00
532a69eb04
src/cli/getopts.rs: improve docs
2019-02-21 09:36:52 +01:00
4de0e142a0
cleanup cli library structure
2019-02-21 09:08:03 +01:00
56458d9764
http_client: on a tty, read password if no env var is set
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 15:03:34 +01:00
c9b296f117
tools: add tty helper module
...
with read_password() and stdin_isatty() functions
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-20 14:43:07 +01:00
8cc0d6aff3
src/bin/proxmox-backup-client.rs: implement garbage-collection sub-command
2019-02-20 14:10:45 +01:00
81da38c143
src/client/http_client.rs: implement post
...
without parameters for now...
2019-02-20 14:09:55 +01:00
8c75372b79
src/backup/datastore.rs: list_backups() - include list of index files
2019-02-20 13:37:44 +01:00
14691fc1f3
src/bin/test_chunk_size.rs: avoid compilöer warnings, exit after 1GB
2019-02-20 12:05:28 +01:00
1604ec2654
src/bin/test_chunk_size.rs: another chunker test - gather some stats
2019-02-20 11:50:10 +01:00
fa4ebf0fce
src/backup/chunker.rs: simplify code
2019-02-20 11:17:19 +01:00
1b9c959c32
src/backup/chunker.rs: change window size back to 48
...
Using 64 is a bad idea, because it always computes hash 0 for
constant data streams.
2019-02-20 11:01:09 +01:00
247cdbce72
src/api2/admin/datastore/catar.rs: allow to configure chunk-size
2019-02-19 15:19:12 +01:00
c584aa21fd
src/backup/chunker.rs: start() - correctly store hash in self.h
2019-02-19 12:40:12 +01:00
cd1598302a
src/backup/chunker.rs: use 64 byte chunker window size
...
This results in faster code.
2019-02-19 12:27:37 +01:00
9a3e869cf5
src/backup/chunker.rs: simplify buzzhash start code
2019-02-19 12:17:17 +01:00
1c7a88aeff
UploadCaTar: currectly write all data using write_all()
2019-02-18 18:31:02 +01:00
4d2881dd4c
src/backup/chunker.rs: correctly compute window index 'idx'
...
And add a simple test.
2019-02-18 16:59:23 +01:00
a4a5c78c01
src/client/http_client.rs: send csrf token
2019-02-18 16:59:23 +01:00
fc7f03523c
cleanup
...
Error::from is already a function taking 1 parameter,
there's no need to wrap it with `|e| Error::from(e)`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:21:27 +01:00
8fdef1a81c
fix typo: confi{g}dir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-18 13:11:42 +01:00
1cb99c234e
src/server/rest.rs: correctly insert NoLogExtension()
2019-02-18 06:54:12 +01:00
591f570b22
src/client/http_client.rs: trim path
2019-02-18 06:24:28 +01:00
7f1685230a
src/server/rest.rs: get_index() include username and CSRF token
...
When we have an valid ticket. Also delay get_index() if called with
an invalid ticket.
2019-02-17 19:28:32 +01:00
141de8374a
src/server/rest.rs: factor our normalize_path()
2019-02-17 17:31:53 +01:00
8225aa2ff6
src/server/rest.rs: improve logs for unauthorized request
2019-02-17 17:18:44 +01:00
54d15ab5c5
src/api_schema/schema.rs: improve docu
2019-02-17 12:13:06 +01:00
51ebd07974
src/api_schema/router.rs: add docu for RpcEnvironmentType
2019-02-17 11:59:18 +01:00
d21c2710da
api_schema/router.rs: improve doc
2019-02-17 10:30:41 +01:00
ef2f2efbcc
improve api_schema module structure
2019-02-17 10:16:33 +01:00
dc9a007b11
rename src/api to src/api_schema
2019-02-17 09:59:20 +01:00
5ddf8cb10a
cleanup auth code, verify CSRF prevention token
2019-02-16 15:52:55 +01:00
95aee7e032
tools/common_regex.rs: add more docu
2019-02-16 12:36:01 +01:00
99be8844c7
buildcfg.rs: improve doc
2019-02-16 12:31:39 +01:00
a8f268afbb
fix configuration dir permission
...
Carefully set and check permissions ...
2019-02-16 12:19:13 +01:00
aada2a9719
config/datastore.rs: remove unused code
2019-02-16 10:10:45 +01:00
d11594db4c
config/datastore.rs: return empty config if file does not exist
2019-02-16 10:06:08 +01:00
12400210a2
tools::file_get_contents - improve error message
2019-02-16 09:42:00 +01:00
6235a41862
tools::file_read_firstline - improve error message
2019-02-16 09:36:29 +01:00
17ed456c2e
change proxy user from www-data to backup
2019-02-16 09:29:04 +01:00
5d14eb6a76
tools.rs: new helper to get uid/gid for the sepcified system user.
...
Or is there an easier way to get that info?
2019-02-15 15:33:12 +01:00
f0a6112451
backup/datastore.rs: improve error reporting
2019-02-15 14:33:03 +01:00
9b50c16103
section_config.rs: simplify parser by using new try_block macro
2019-02-15 12:13:15 +01:00
fe651dd6e5
tools.rs: fix doc tests
2019-02-15 11:33:51 +01:00
e80503d2a6
tools.rs: document try_block macro
2019-02-15 11:20:45 +01:00
f0dbba8cbe
tools.rs: new try_block macro
2019-02-15 10:59:12 +01:00
1d77b6cf6b
update to pam 0.7 (renamed from pam-auth)
...
It now supports custom conversation methods, so instead of
new() we ask for a default authenticator taking a password
via with_password(). Since the password is now handled by
the now separate conversation handler, `set_credentials()`
is now called on the handler we get via `.get_handler()`.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-15 10:34:40 +01:00
d4736445fe
server/rest.rs: add method to log message
2019-02-15 10:16:12 +01:00
f02e6fc478
api2/admin/datastore/catar.rs: simplify/fix debug message
2019-02-15 10:04:37 +01:00
44c00c0dfd
server/rest.rs: log full error messages
2019-02-15 09:55:12 +01:00
7171b3e079
server/rest.rs: use a protocol extension to avoid double log
...
Instead of modifying the response header itself.
2019-02-14 16:04:24 +01:00
7e03988caf
avoid double logging of proxied requests
2019-02-14 13:28:41 +01:00
af6f80d3db
Revert "backup/datastore: fn open is a private implementation detail"
...
This reverts commit 3dacd9a660
2019-02-14 13:13:49 +01:00
78a1fa6722
src/server/rest.rs: log failed requests
2019-02-14 13:07:34 +01:00
2cc90617dd
tools/fs: mark ReadDir as Send
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
62f2422f6a
backup/chunk_store: verify chunk file names
...
get_chunk_iterator() should skip over files which aren't an
actual chunk
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
3dacd9a660
backup/datastore: fn open is a private implementation detail
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
9739aca41f
backup/chunk_store: open base dir in get_chunk_iterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 12:37:31 +01:00
db4868ef4f
src/client/backup_repo.rs - improve docu
2019-02-14 11:17:08 +01:00
151c6ce27d
src/client.rs: improve library structure and docu
2019-02-14 11:11:39 +01:00
dd1bfe6c12
tools/timer: add module documentation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:13:42 +01:00
c7f481b642
backup: replace ChunkIterator
...
Instead we just build the iterator using our
tools::fs::read_subdir iterator as follows:
Use the index range (0..0x10000) as an iterator, map() it
to yield a subdirectory iterator for each index, then
flatten it.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
121f18efb7
tools: use scan_subdir iterator for scandir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
32286b039c
tools/fs: add file_name_utf8_unchecked shortcut
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
806d7a6a35
tools/fs: change scan_subdir to use filter_file_name_regex
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
53e1e7ca72
tools/fs: add filter_file_name_regex to FileIterOps
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
23fba9d777
tools/fs: add FileIterOps trait and FileTypeFilter
...
FileIterOps will provide operations on file entry iterators.
For now this provides a file type filter.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
1db416150d
tools/fs: let ReadDir iterator yield ReadDirEntry
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
b12505da69
tools/fs: add ReadDirEntry helper
...
This wraps nix::dir::Entry with the parent and additionally
exposes the parent directory file descriptor since we need
it to perform openat().
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
a25f863afd
tools/fs: add scan_subdir helper
...
This filters the results of read_subdir with a regex.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
b4d5787de9
tools: start fs submodule, add read_subdir
...
This creates an iterator relative to a RawFd via
nix::dir::Dir over nix::dir::Entrys.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-14 10:09:59 +01:00
0dffe3f99a
src/client/http_client.rs: try to login
...
use an environment var to store passphrase (PBS_PASSWORD)
2019-02-13 14:31:43 +01:00
33d64b81e0
src/bin/proxmox-backup-client.rs: define and use BackupRepository
2019-02-13 12:30:52 +01:00
4a3f65172c
client: use hyper-tls for now
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
42d6e4fb05
tools: borrow: implement Deref for Tied
...
Eg. if you have an x: Tied<Dir, Iterator> now you can simply
call x.next() as it'll dereference to the Iterator.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-13 11:03:09 +01:00
541a302224
api2/admin/datastore/catar.rs: use nested file extensions
...
i.e. archive.catar.didx
2019-02-13 10:37:00 +01:00
e5064ba607
simplify backup lib structure (pub use xxx:*), improve doc
2019-02-12 14:13:31 +01:00
d78345bcfc
src/backup.rs - improve doc
2019-02-12 13:27:11 +01:00
93d5d77952
rename ArchiveIndex to DynamicIndex
...
also changed the file extension from .aidx to .didx
2019-02-12 12:05:33 +01:00
91a905b6dd
rename ImageIndex to FixedIndex
...
also changed the file extension fron .iidx to .fidx
2019-02-12 11:50:45 +01:00
82bc0ad40c
backup: fixup warnings and readd lost file type check
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:43:43 +01:00
9349d2a1d2
backup: cleanup long lines
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
fdd71f5286
backup: use ChunkIterator for sweep_unused_chunks
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
c9e7f4dc8b
backup: add optional progress callback to ChunkIterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
4070096c6a
backup: add ChunkIterator
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
6ed25cbe3a
tools: add Tied value
...
Tie two dependent values together, such as a nix::Dir with
its nix::dir::Iter<'a> where 'a is tied to the Dir's
lifetime, making it otherwise impossible to return them or
store them in a struct.
Alternatively we could try the 'rental' crate.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
9ac1045c7b
bin: manager: fixup datastore parameter name
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-12 10:39:11 +01:00
8f8d5a420c
avoid compiler warning
2019-02-11 15:12:01 +01:00
aa5a4060ab
bin/proxmox-backup-api.rs: improve error handling
2019-02-11 15:11:42 +01:00
4223d9f800
bin/proxmox-backup-proxy.rs: improve error handling
2019-02-11 14:43:26 +01:00
ace9e3531a
access: use proxmox-backup-auth for pam
...
allows customization via /etc/pam.d/proxmox-backup-auth
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-07 13:40:22 +01:00
13f8310cac
api2: time: use /etc/localtime as fallback
...
in case there is no /etc/timezone
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-05 14:14:56 +01:00
a7cac14ed9
tools: use std::fs::read for file_get_contents
...
Until we add the extra options like we have in pve, just use
the already existing (since 1.26) shortcut.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:35:32 +01:00
e937034c86
remove some rather inconvenient debug output
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:34:38 +01:00
9d02e5c38a
proxy: use <configdir>/server.pfx for tls
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:19:32 +01:00
9f4962d396
introduce buildcfg module and PROXMOX_CONFIGDIR
...
buildcfg.rs should contain convenience variables or macros
for using build-time configured variables
For now we replace hardcoded "/etc/proxmox-backup/<foo>"
with configdir!("<foo>").
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 15:13:08 +01:00
0d176f3681
proxy: use TLS via tokio-tls
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-04 14:56:07 +01:00
e0a5d1ca2f
tests: chunk_store: cleanup .testdir
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
a4f9188300
tests: import lazy_static as needed
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
6285b251e7
don't hardcode /usr/share paths
...
Add a defines.mk file defining the standard default layout
for installation (PREFIX, DATAROOTDIR, LIBDIR etc.).
Add a $(JSDIR) variable and use env!("JSDIR") in place of
the hardcoded /usr/share/javascript/proxmox-backup path.
Have defines.mk include an *optional* local.mak to override
JSDIR to www/ during local development.
Add `make install` targets and remove debian/install.
Note that service files are now in $libdir/proxmox-backup/
as there's usually no point in starting them from the
command line, so /usr/sbin makes no sense.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
8e4df103dd
remove useless command option from daemons
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-02-01 10:41:54 +01:00
6278365fd9
improve ApiMethod docu
2019-02-01 10:38:07 +01:00
4b2cdeb9a6
implement relead_timezone flag
2019-02-01 10:04:46 +01:00
c82bc1a1f9
depend on pam-auth, use pam for root login
...
Please use username "root@pam" for login via GUI.
2019-02-01 09:30:50 +01:00
a154a8e8a4
delay unauthorized request (rate limit)
2019-01-31 14:34:21 +01:00
5996577ab6
move http error class to router.rs
2019-01-31 13:22:30 +01:00
b9903d6331
server/rest.rs: verify auth cookie
2019-01-31 12:22:00 +01:00
f484eed3c2
Utils.js: fix cookie handling
...
Use unsecure cookie foör testing.
2019-01-31 10:08:08 +01:00
9f49fe1d5d
avoid compiler warnings
2019-01-30 18:25:37 +01:00
34f956bc25
api2/access.rs: add ticket api
2019-01-30 15:16:10 +01:00
1bf446a33e
fix timer_delete call
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-30 12:04:20 +01:00
cf671670a0
auth_helpers.rs: add timestamp to csrf token
2019-01-29 17:50:03 +01:00
1e76cbc6a2
auth_helpers.rs: implement assemble_csrf_prevention_token
2019-01-29 17:41:45 +01:00
d01e2420f7
load auth keys on startup
2019-01-29 17:21:58 +01:00
6c30068ebf
auth_helpers.rs: split code into separate file
2019-01-29 16:55:49 +01:00
39a90ca6c5
bin/proxmox-backup-api.rs: gererate secret for csrf token
2019-01-29 13:15:33 +01:00
26027a151e
tools/ticket.rs: remove debug statements
2019-01-29 13:04:01 +01:00
8d04280b29
tools/ticket.rs: add basic ticket support
2019-01-29 12:59:07 +01:00
c8f3f9b1af
server/rest.rs: simplify proxy code
...
Only pass neccessary parameters.
2019-01-28 18:22:16 +01:00
a859fa95a9
add protected flag for some api methods
2019-01-28 18:16:47 +01:00
a3da38ddfd
server/rest.rs: implement proxy_sync_api_request
2019-01-28 18:06:42 +01:00
f120483353
server/rest.rs: add proxy_sync_api_request() dummy
2019-01-28 17:30:39 +01:00
4e4df8e2f0
api/router.rs: add new 'protected' flag to ApiMethod
2019-01-28 17:18:42 +01:00
5bb3398171
server/formatter.rs: fix extjs error format
2019-01-28 13:44:48 +01:00
02c7a75520
src/bin/proxmox-backup-proxy.rs: implement unpriviledged server
...
We want to run the public server as user www-data. Requests needing
root priviledges needs to be proxied to the proxmox-backup.service, which
now listens to 127.0.0.1:82.
2019-01-28 13:29:58 +01:00
48849593ee
api2/node/services.rs: implement service commands
2019-01-27 12:40:31 +01:00
d7d23785f0
RpcEnvironment: implement set_user() and get_user()
2019-01-27 10:52:26 +01:00
162b979394
RpcEnvironment: add environment type enum RpcEnvironmentType
2019-01-27 10:33:42 +01:00
e82dad9700
also pass rpcenv to async handlers
2019-01-27 10:18:52 +01:00
d2ab5f19e2
api2/node/services.rs: start services api
2019-01-26 17:54:18 +01:00
0f253593c6
move rpc environment implementation to separate files
2019-01-26 15:08:02 +01:00
6049b71f41
api: pass RpcEnvirnment to api handlers
2019-01-26 14:50:37 +01:00
d96d82736d
use crate log and syslog
2019-01-26 10:56:11 +01:00
e182ab4a4a
api2/node/syslog.rs: improve error handling
2019-01-26 10:23:52 +01:00
0441886842
api2/node/syslog.rs: implement dump journal
2019-01-25 18:20:51 +01:00
4f9a726833
api2/node/syslog.rs: add syslog api schema
2019-01-25 17:17:30 +01:00
1631c54fca
tools::get_hardware_address - fix FILENAME, use file_get_contents()
2019-01-25 12:23:47 +01:00
af2fddea4f
api2/node/dns.rs: implement concurrent update protection
2019-01-25 11:38:59 +01:00
de6b0721fa
pi2/node/dns.rs: return file digest
2019-01-25 11:03:41 +01:00
2296860027
tools.rs: move digest_to_hex() into tools
2019-01-25 10:58:28 +01:00
53157ca6cf
tools.rs: implement file_get_contents()
2019-01-25 10:44:40 +01:00
46b79b9ee5
api2/node/dns.rs: improve dns api
2019-01-25 10:15:32 +01:00
332dcc2277
backup/chunk_store.rs: fix test case - use absolute path
2019-01-25 10:14:25 +01:00
8f973f8181
api2/node/dns.rs: implement resolv.conf parser
2019-01-24 14:59:40 +01:00
784252dbc5
backup/datastore.rs: use single lazy_static block
2019-01-24 14:58:47 +01:00
e6ffeb918c
api2/node/time.rs: implement set timezone
2019-01-24 12:05:06 +01:00
cd8b91f78c
api2/node/time.rs: add return schema
2019-01-24 11:07:11 +01:00
447787ab7c
tools.rs: implement file_read_firstline
2019-01-24 10:43:30 +01:00
0463602a79
api2/node/time.rs: implement read api
2019-01-23 15:14:14 +01:00
b2b3485d5f
src/api2/node: start node configuration api
2019-01-23 13:09:17 +01:00
d15009c0ce
server/rest.rs: fake login cookie
2019-01-23 12:49:10 +01:00
248cb51862
add missing api2.rs
2019-01-22 13:56:27 +01:00
7e13b2d67f
api2/subscription.rs: add subscription api class
2019-01-22 12:50:19 +01:00
576e3bf252
rename api3 back to api2
...
There is no real need to change the path, so using api2 we can reuse
all helpers (like tools from proxmox widget toolkit).
2019-01-22 12:10:38 +01:00
1adb353d6f
lient/http_client.rs: simplify code
2019-01-22 10:36:04 +01:00
41c039e143
bin/proxmox-backup-client.rs: implement list command
2019-01-21 18:58:14 +01:00
1fdb4c6ffa
client/http_client.rs: implement get, share common code
2019-01-21 18:56:48 +01:00
83dbd80bd8
api3/admin/datastore.rs: implement list backups
2019-01-21 11:22:55 +01:00
7ca80246ea
backup/datastore.rs: use DateTime<Utc> to store backup time
2019-01-21 10:51:52 +01:00
be0084b001
white space fix
2019-01-21 10:32:07 +01:00
e25736b4c3
backup/datastore.rs: implement list_backups
2019-01-20 18:02:27 +01:00
cce1676a9e
tools.rs: use FnMut for scandir callback
2019-01-20 17:49:11 +01:00
443f3743bb
tools.rs: new scandir() helper
2019-01-20 17:31:43 +01:00
68469eebb4
backup/chunk_store.rs: verify base path is absolute
2019-01-20 16:49:22 +01:00
059ca7c3a1
remove crate tokio-codec (seems to be part of tokio now)
2019-01-20 14:28:06 +01:00
8eeef8492a
tools/wrapped_reader_stream.rs: allocate Vec<u8> for buffer
2019-01-20 11:01:18 +01:00
ff8bdf3b64
tools/wrapped_reader_stream.rs: only implement Stream (without AsyncRead)
...
Looks more efficent ...
2019-01-20 10:41:21 +01:00
8e89d9cafe
tools/wrapped_reader_stream.rs: use tokio_threadpool::blocking
2019-01-20 10:28:51 +01:00
ce7ba139f1
catar: improve bash completion
2019-01-20 09:55:20 +01:00
29f34b8e1a
/api3/admin/datastore.rs: add missing subdir to directory index
2019-01-20 09:47:32 +01:00
0b05fd5830
api3/admin/datastore/catar.rs: implement download
2019-01-20 09:39:32 +01:00
7f0d67cf79
tools/wrapped_reader_stream.rs: new helper class
...
Will use that to download catar files.
2019-01-20 09:38:28 +01:00
6a4c091616
api3/admin/datastore/catar.rs: start impl. download
2019-01-19 17:27:02 +01:00
50cfb695ae
api/router.rs: rename ApiUploadMethod to ApiAsyncMethod
...
We can use this for uploads and downloads ...
2019-01-19 16:42:43 +01:00
379ea0edb6
Revert "/api/schema.rs: implement Schema::Option"
...
This reverts commit 0a35462c1e
2019-01-19 12:53:32 +01:00
80f069656d
Revert "api/schema.rs: avoid Option(Option( nesting"
...
This reverts commit ffdac1af01
2019-01-19 12:52:51 +01:00
ca3d861513
remove unused type PropertyMap
2019-01-18 17:54:06 +01:00
ffdac1af01
api/schema.rs: avoid Option(Option( nesting
2019-01-18 17:49:19 +01:00
0a35462c1e
/api/schema.rs: implement Schema::Option
2019-01-18 17:40:37 +01:00
728797d0c1
reduce compiler warnings
2019-01-18 16:50:15 +01:00
383e857749
move complete_file_name() helper into tools.rs
2019-01-18 13:42:52 +01:00
73e844dbfe
bin/proxmox-backup-client.rs: improve file name completion
2019-01-18 13:40:05 +01:00
95cea65b04
backup/datastore.rs: list all index files using walkdir crate
2019-01-18 12:24:58 +01:00
ff3d3100d4
backup/datastore.rs: try to create useful directory layout
...
store/type/ID/TIME/xxx.yyy
2019-01-18 12:01:37 +01:00
0d38dcb43c
cleanup nodename()
2019-01-18 10:13:45 +01:00
74a69302a7
tools.rs: new helper nodename()
2019-01-18 09:58:15 +01:00
5a778d92b3
api3/admin/datastore.rs: rename parameter "name" to "store"
2019-01-18 08:33:11 +01:00
244d9b17a8
bin/proxmox-backup-client.rs: implement file name completions - first try ...
2019-01-17 14:24:20 +01:00
83bdac1e3b
api3/admin/datastore/upload_catar.rs: verify content type ("application/x-proxmox-backup-catar")
2019-01-17 12:43:29 +01:00
06aeb76a5c
client/catar_backup_stream.rs: remove debug code
2019-01-17 12:16:36 +01:00
0fe5d605e7
tools.rs: new helper required_string_param()
...
To extract parameters from Value.
2019-01-17 12:14:02 +01:00
0ee0ad5bf3
api/router.rs: return Result in upload handler
2019-01-17 12:03:38 +01:00
23bb8780d4
proxmox-backup-client: implement upload to server
2019-01-17 11:38:22 +01:00
597641fdd5
client/http_client.rs: new helper class
2019-01-17 11:38:03 +01:00
264f52cf70
api3/admin/datastore.rs: change api patch for upload_catar
2019-01-17 09:16:50 +01:00
37b98249f4
upload: don't return NotReady
...
We need to kick (poll) the stream's next element, so simply
reiterate, the try_read!() will then return NotReady from
the next stream item for us.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 15:17:26 +01:00
1dcd52e5df
cleanup previous commit...
...
This kind of notation is weird...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 14:19:13 +01:00
6a50b77e72
listen on :: instead of 0.0.0.0
...
Since this supports both ipv6 & ipv4 unless v6only is
explicitly enabled.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-01-16 14:18:16 +01:00
cf16af2ab3
server/rest.rs: correctly pass query/url parameters
2019-01-16 13:58:36 +01:00
2085142ed4
reduce compiler warnings
2019-01-16 10:19:49 +01:00
e8edbbd49c
client/catar_backup_stream.rs: new helper for catar uploads to server
2019-01-16 10:15:39 +01:00
150f1bd8f6
backup/datastore.rs: use Arc for chunk_store references
2019-01-15 12:36:16 +01:00
1629d2ad7b
api3/admin/datastore/upload_catar.rs: implement upload future
2019-01-15 11:38:26 +01:00
7e21da6e23
api/router.rs: allow different types of api methods
...
Added a prototype for file/backup uploads.
2019-01-14 12:26:04 +01:00
b41d1aab8c
catar: define all feature flags
2019-01-12 16:49:02 +01:00
4c0fd4871e
catar/encoder.rs: allow to pass list of devices
...
We can use this to implement the --one-file-system like options. This
is a bit more convenient, because this way we can select to include
multiple devices (which is required by container backups).
2019-01-12 15:43:20 +01:00
a8a0495612
catar/encoder.rs: disable backup of virtual file systems
2019-01-12 11:56:53 +01:00
d7bfac8605
catar/encoder.rs: only test fat attribute on MSDOS and FUSE file systems
2019-01-12 10:28:26 +01:00
Dietmar Maurer
Wolfgang Bumiller
Christian Ebner