Dietmar Maurer
c360bd73d3
src/api2/node/tasks.rs: fix task status api
2019-04-09 15:12:20 +02:00
Dietmar Maurer
0bfd87bc5e
src/server/worker_task.rs: speedup upid_read_status()
2019-04-09 15:11:48 +02:00
Dietmar Maurer
d8d40dd0c2
src/api2/node.rs: add node parameter
2019-04-09 14:43:30 +02:00
Dietmar Maurer
7ae0bf52dd
src/server/command_socket.rs: code cleanup - fix error message
2019-04-09 12:47:42 +02:00
Dietmar Maurer
d607b8861b
src/server/worker_task.rs: implement task control socket
2019-04-09 12:15:06 +02:00
Dietmar Maurer
e201753629
src/server/command_socket.rs: implement auto_remove flag
...
Remove the socket file on close.
2019-04-09 11:47:23 +02:00
Dietmar Maurer
634132fe10
src/server/upid.rs: moved code into separate file
2019-04-08 18:43:26 +02:00
Dietmar Maurer
78a39e0517
src/server/command_socket.rs: simple command socket
2019-04-08 17:59:39 +02:00
Dietmar Maurer
e3f41f2165
src/tools/daemon.rs: use new ServerState handler
2019-04-08 14:00:23 +02:00
Dietmar Maurer
7a630df709
implement server state/signal handling, depend on tokio-signal
2019-04-08 13:59:07 +02:00
Dietmar Maurer
5a12c0e2fb
src/api2/node/tasks.rs: start upid API
2019-04-07 14:36:57 +02:00
Dietmar Maurer
b75b968147
src/server/worker_task.rs: implement is_active()
2019-04-07 14:36:25 +02:00
Dietmar Maurer
4494d07819
src/server/worker_task.rs: make log_path a method on UPID
2019-04-07 13:51:46 +02:00
Dietmar Maurer
d2a2e02bb7
src/api2/node/tasks.rs: implement userfilter
2019-04-07 13:17:19 +02:00
Dietmar Maurer
3c3bee2e32
src/api2/node/tasks.rs: correctly handle start parameter
2019-04-07 12:41:24 +02:00
Dietmar Maurer
063ca5be77
src/api2/node/tasks.rs: implement list_tasks
2019-04-07 12:18:58 +02:00
Dietmar Maurer
d4b59ae0b8
src/backup/datastore.rs: pass WorkerTask to GC
...
So that we can log to the task log file.
2019-04-06 17:57:38 +02:00
Dietmar Maurer
35950380a9
src/server/worker_task.rs: carefully handle file permissions
2019-04-06 17:53:12 +02:00
Dietmar Maurer
1619a72063
src/tools.rs: implement create_dir_chown
...
Combinded mkdir with chown.
2019-04-06 16:50:23 +02:00
Dietmar Maurer
eea8131952
src/tools.rs: implement file_set_contents_full()
2019-04-06 13:53:43 +02:00
Dietmar Maurer
0f778e0652
src/api2/admin/datastore.rs: use new WorkerTask::new_thread()
2019-04-06 11:27:23 +02:00
Dietmar Maurer
244abab769
src/tools/file_logger.rs: fix test
2019-04-06 11:24:37 +02:00
Dietmar Maurer
660c684640
src/server/worker_task.rs: return upid_str
2019-04-06 11:23:53 +02:00
Dietmar Maurer
93aebb38bc
src/server/worker_task.rs: implement read_task_list()
2019-04-06 10:17:11 +02:00
Dietmar Maurer
4a36a2c999
src/server/worker_task.rs: improve UPID format description
2019-04-06 09:36:50 +02:00
Dietmar Maurer
882594c5e9
src/server.rs: improve crate layout
2019-04-06 09:17:25 +02:00
Dietmar Maurer
4b01c983f0
src/server/worker_task.rs: implement persistent task list
2019-04-05 12:37:35 +02:00
Christian Ebner
a2474d328f
src/tools/procfs.rs: implement read_proc_uptime and read_proc_uptime_ticks
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-05 06:22:21 +02:00
Christian Ebner
08daa81af6
procfs: implement check_process_running and check_process_running_pstart
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-04-04 13:54:17 +02:00
Dietmar Maurer
9a0d0ff774
server/worker_task.rs: implement UPID parser
2019-04-04 13:49:35 +02:00
Dietmar Maurer
49cf9f3d0b
src/tools.rs: file_get_json() - add new default parameter
2019-04-04 12:24:18 +02:00
Dietmar Maurer
79ad7b7ec2
src/tools/procfs.rs: split regex into several lines
2019-04-04 09:25:19 +02:00
Dietmar Maurer
479f6e40ec
src/server/worker_task.rs: implement task registry
2019-04-04 08:58:39 +02:00
Dietmar Maurer
52ddc2522d
src/tools/procfs.rs: use libc::pid_t
2019-04-04 08:05:43 +02:00
Dietmar Maurer
d3d7ab1273
src/tools/file_logger.rs: new - accept AsRef<Path>
2019-04-03 14:13:33 +02:00
Dietmar Maurer
3c2012f97a
src/tools/procfs.rs: start helper tools to read procfs
2019-04-03 13:39:20 +02:00
Dietmar Maurer
3489936ebb
src/tools/file_logger.rs: change timestamp format to rfc3339
2019-04-03 08:58:43 +02:00
Dietmar Maurer
92da93b245
abort GC on server shutdown
2019-04-01 12:13:02 +02:00
Dietmar Maurer
9136f857cc
add global var to indicate server shutdown requests
2019-04-01 12:05:11 +02:00
Dietmar Maurer
3e6a7deeaa
src/api2/admin/datastore.rs: start GC in a separate thread
2019-04-01 12:02:46 +02:00
Dietmar Maurer
ad281d1bd9
www/DataStoreConfig.js: add button to start GC
2019-04-01 08:08:34 +02:00
Dietmar Maurer
49d123ee3a
src/server/rest.rs: use formatter to encode errors
2019-04-01 08:04:12 +02:00
Dietmar Maurer
7d4ef127cb
src/server/rest.rs: generate csrf token if we have a valid ticket
...
This is important if the user reloads the browser page.
2019-04-01 07:52:30 +02:00
Dietmar Maurer
11861a482d
src/backup/chunk_store.rs: fix GC
...
Added option to get oldest_writer timestamp from ProcessLocker.
2019-03-31 17:21:36 +02:00
Dietmar Maurer
d85987aeeb
fix last commit: the filename var was not ment to be removed, sorry
2019-03-31 16:16:14 +02:00
Dietmar Maurer
15a77c4c2e
src/backup/chunk_store.rs: avoid create/unlink race
2019-03-31 10:03:01 +02:00
Dietmar Maurer
c374f05499
src/backup.rs: improve GC problem description
2019-03-31 09:44:35 +02:00
Dietmar Maurer
c8ec450e37
src/backup.rs: start explaining different GC algorithm
2019-03-30 17:21:40 +01:00
Dietmar Maurer
8a4757342c
src/backup.rs: describe the garbage collection problem
2019-03-30 16:26:52 +01:00
Dietmar Maurer
10eea49d81
src/backup/fixed_index.rs: store reference to file
...
Keep it open and locked while index is in use.
2019-03-27 10:17:03 +01:00
Dietmar Maurer
c597a92c81
index file readers: aquire shared lock
2019-03-27 10:11:14 +01:00
Dietmar Maurer
0465218953
src/backup.rs: add documentation about ChunkStore locking
2019-03-22 10:14:50 +01:00
Dietmar Maurer
43b1303398
datastore: use new ProcessLocker
...
To make sure only one process runs garbage collection while having active writers.
2019-03-22 09:42:15 +01:00
Dietmar Maurer
abfc001f25
src/tools/process_locker.rs: new() - use generic path parameter
2019-03-22 09:41:32 +01:00
Dietmar Maurer
a650f50301
src/tools/process_locker.rs: implement inter-process reader-writer locks
2019-03-22 08:05:51 +01:00
Wolfgang Bumiller
eb90c9e372
tools: style fixup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-20 09:57:13 +01:00
Christian Ebner
e04d1bfadc
Cleanup of if statement with unwraps by a more compact match statement
...
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
2019-03-20 09:49:18 +01:00
Dietmar Maurer
8346f0d59b
src/server/rest.rs: correctly extract content type
2019-03-19 12:50:15 +01:00
Wolfgang Bumiller
a690ecacf0
daemon: simplify daemon creation
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 12:12:54 +01:00
Wolfgang Bumiller
e4311382d9
tools: daemon: rename some structs
...
Reloadable resources are now 'Reloadable' instead of
'ReexecContinue'.
The struct handling the reload is a 'Reloader', not a
'ReexecStore'.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-19 11:09:46 +01:00
Wolfgang Bumiller
5e7bc50af1
add reload support to api daemon
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
a2ca713701
add reload support to proxy
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
af70c1811b
tools: implement ReexecContinue for tokio's TcpListener
...
This is the only thing we currently need to keep alive for
reloads.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
4422ba2c7f
tools: daemon: add a default signalfd helper
...
Proxy and daemon for now just want to handle reload via
`SIGHUP`, so provide a helper creating the signalfd stream
doing that - this is simply a filtered stream which passes
the remaining signals through, so it can be used exactly
like the signalfd stream could before to add more signals.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
dce94d0e09
tools: add daemon helpers
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
ff7049d481
tools: add fd_change_cloexec helper
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Wolfgang Bumiller
1c042cdc6c
proxy: listener error handling fixup
...
If the listening socket goes into some error state we'll get
std::io::Errors rather than higher level errors from the
native_tls::TlsAcceptor, those are usually fatal. (Ran into
this after performing a shutdown() on the file descriptor,
after which the future just endlessly loops in accept().)
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-18 12:29:18 +01:00
Dietmar Maurer
60864b1b56
src/pxar.rs: improve docu
2019-03-18 12:27:30 +01:00
Dietmar Maurer
515688d1f1
avoid compiler warnings
2019-03-18 10:00:58 +01:00
Dietmar Maurer
ba10f2b0d4
src/bin/dynamic-files.rs: test case for file shrink during backup
2019-03-16 14:22:14 +01:00
Dietmar Maurer
af572aaa4c
pxar: implement hardlinks
...
So we are no longer compatible with catar ...
2019-03-16 11:02:12 +01:00
Dietmar Maurer
812dd935ce
src/pxar/encoder.rs: detect hardlinks
2019-03-15 12:19:51 +01:00
Dietmar Maurer
896e803feb
src/pxar/sequential_decoder.rs: dump_entry - fix FIFO/SOCKET handling
2019-03-15 12:15:38 +01:00
Dietmar Maurer
7dfa17c7cc
src/pxar/binary_search_tree.rs: fix test
2019-03-15 11:34:31 +01:00
Dietmar Maurer
9307279faf
cleanup: remove dead code
2019-03-15 10:18:28 +01:00
Dietmar Maurer
46cfe7ab7e
src/pxar/decoder.rs: implement seekable decoder
2019-03-15 09:36:05 +01:00
Dietmar Maurer
0e21b87be3
renamed: src/pxar/decoder.rs -> src/pxar/sequential_decoder.rs
2019-03-15 08:36:02 +01:00
Dietmar Maurer
3dbfe5b142
simplify pxar module structure
2019-03-15 08:24:32 +01:00
Dietmar Maurer
1ef46b81d3
src/bin/pxar.rs: implement extract
2019-03-15 08:03:44 +01:00
Dietmar Maurer
02c93163cf
src/pxar/decoder.rs: correctly call callback()
2019-03-15 08:02:04 +01:00
Dietmar Maurer
5defa71bf9
rename PxarBackupWriter into PxarDecodeWriter
2019-03-15 07:20:22 +01:00
Dietmar Maurer
b005ed12d5
src/client/http_client.rs: avoid compiler warning
2019-03-15 07:16:26 +01:00
Dietmar Maurer
9e26abf16b
src/cli/command.rs: fix iterator usage
2019-03-15 07:12:40 +01:00
Dietmar Maurer
f98746ba6c
src/pxar/decoder.rs: improve docu
2019-03-15 07:12:11 +01:00
Dietmar Maurer
0b2d7fdc81
src/pxar/decoder.rs: avoid compiler warnings
2019-03-14 17:45:47 +01:00
Dietmar Maurer
37940aa1f9
src/bin/pxar.rs: implement dump/list
2019-03-14 17:43:11 +01:00
Dietmar Maurer
40360fde6e
src/pxar/decoder.rs: implement dump_archive
...
use this to debug archive problems ...
2019-03-14 13:10:27 +01:00
Dietmar Maurer
8968258b66
rename catar into pxar
...
To avoid confusion with the casync implementation.
2019-03-14 10:54:09 +01:00
Dietmar Maurer
7c4dd94670
rename catar into pxar
...
Seems impossible to stay compatible with the original catar specification,
so we change the name to avoid confusions.
2019-03-14 10:14:32 +01:00
Dietmar Maurer
806500cd09
src/tools.rs: improve file name completion
...
Use current working directory by default.
2019-03-14 08:17:35 +01:00
Dietmar Maurer
08dc340a12
src/bin/proxmox-backup-client.rs: complete archive names
2019-03-14 08:09:35 +01:00
Wolfgang Bumiller
6f90a6a764
protocol: cleanup finish_backup
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-13 14:16:17 +01:00
Dietmar Maurer
b238851832
src/bin/proxmox-backup-client.rs: implement completion for snapshots
2019-03-13 13:31:29 +01:00
Dietmar Maurer
4981134708
src/bin/proxmox-backup-client.rs: implement chunk-size completion
2019-03-13 12:26:01 +01:00
Dietmar Maurer
98f0b97293
src/bin/proxmox-backup-client.rs: cleanup
2019-03-13 12:17:39 +01:00
Dietmar Maurer
024f11bb55
src/bin/proxmox-backup-client.rs: implement completion for backup groups
2019-03-13 11:56:37 +01:00
Dietmar Maurer
edd3c8c605
src/client/backup_repo.rs: implement FromStr trait
2019-03-13 10:09:39 +01:00
Dietmar Maurer
874acb7039
src/client/backup_repo.rs: implement Display trait
2019-03-13 09:57:36 +01:00
Dietmar Maurer
d0a03d40ce
src/bin/proxmox-backup-client.rs: implement repository completions
2019-03-13 09:47:12 +01:00
Dietmar Maurer
bff110308b
src/bin/proxmox-backup-client.rs: fix BACKUPSPEC_REGEX - correctly specify non-capture group
...
And improve bash completion.
2019-03-13 07:23:59 +01:00
Dietmar Maurer
1a71509aad
src/cli/command.rs: fix completion - never display argument parameters as option
2019-03-12 14:53:41 +01:00
Dietmar Maurer
496a67846f
src/cli/command.rs: pass parsed parameters to completion function
2019-03-12 14:39:51 +01:00
Dietmar Maurer
b5fa28251c
src/cli/getopts.rs: factor out reusable code
2019-03-12 14:38:47 +01:00
Dietmar Maurer
e39efdbd33
src/cli/command.rs: use a HashMap to store fixed param as key/value pairs
2019-03-12 12:54:16 +01:00
Dietmar Maurer
f4e4d58327
src/cli/command.rs: improve completions
2019-03-12 12:07:45 +01:00
Wolfgang Bumiller
4ba7b6099c
signalfd cleanup
...
- add comment about the boxed file descriptor (and others)
- reexport SigSet and Signal for convenience
- remove Result wrapper (only used once)
- enforce drop order of the PollEvented2 struct with respect
to the boxed fd
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-12 11:07:09 +01:00
Wolfgang Bumiller
f54c19989c
tools: add async signalfd handler
...
This provides a Stream<Item = siginfo> via nix' signalfd,
by wrapping it in tokio's PollEvented2 struct to allow
polling via tokio's event loop.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-12 08:04:28 +01:00
Dietmar Maurer
d5c34d98c0
src/catar/decoder.rs: simplify public restore API
2019-03-11 14:31:01 +01:00
Dietmar Maurer
8e39232acc
src/bin/proxmox-backup-client.rs: strip .didx file extensions
2019-03-11 10:54:02 +01:00
Dietmar Maurer
74cdb52107
src/bin/proxmox-backup-client.rs: include file extension in backupspec
2019-03-11 10:27:28 +01:00
Dietmar Maurer
fc2bf37ead
src/catar/decoder.rs: cleanup style
2019-03-11 08:14:28 +01:00
Dietmar Maurer
656b23e107
src/catar/decoder.rs: dir_mkdirat - add create_new flag
2019-03-11 07:27:45 +01:00
Dietmar Maurer
e9c9409a99
src/catar/decoder.rs: implement real sequential decoder (no Seek)
...
I copied the old code to src/catar/inspector.rs. Will modify
that later to implement a random access decoder ...
2019-03-08 16:55:54 +01:00
Dietmar Maurer
eed6db3923
src/bin/proxmox-backup-client.rs: impl option all-file-systems
2019-03-08 09:36:40 +01:00
Wolfgang Bumiller
06178f13fa
move chunker into proxmox_protocol
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-08 09:29:47 +01:00
Dietmar Maurer
e3c30c50a6
src/bin/catar.rs: implement all-file-systems option
2019-03-08 09:28:12 +01:00
Dietmar Maurer
65092b1e27
src/catar/encoder.rs: log skipped mount points
2019-03-08 09:25:00 +01:00
Dietmar Maurer
6b64d25d64
src/catar/encoder.rs: remove debug output
2019-03-08 08:23:07 +01:00
Dietmar Maurer
2689810c92
src/bin/catar.rs: implement verbose flags
2019-03-08 08:20:56 +01:00
Dietmar Maurer
79c281fdac
src/catar/encoder.rs: use simply flag "all_file_systems"
2019-03-08 08:14:26 +01:00
Dietmar Maurer
25e77d3877
src/cli/command.rs: print_bash_completion - avoid crash
2019-03-07 12:14:26 +01:00
Dietmar Maurer
141f062e08
src/backup/chunk_store.rs: use zstd compression insteadf of lz4
...
Provides better compressionm rate, and is still fast.
2019-03-07 11:42:59 +01:00
Dietmar Maurer
d2690f74bb
src/backup/chunk_store.rs: read_chunk - clear buffer before calling read_to_end
2019-03-07 10:40:01 +01:00
Dietmar Maurer
2ae7d196b9
src/bin/proxmox-backup-client.rs: correctly strip file extension
2019-03-06 11:18:46 +01:00
Dietmar Maurer
9f91249351
src/bin/proxmox-backup-client.rs: impl restore
2019-03-06 11:02:08 +01:00
Dietmar Maurer
01a1342364
src/api2/admin/datastore.rs: impl list_snapshot_files
2019-03-06 11:02:08 +01:00
Dietmar Maurer
58e99e138e
src/backup/backup_info.rs: new helper to list files
2019-03-06 11:02:08 +01:00
Dietmar Maurer
dd79265a8a
src/api2/admin/datastore/catar.rs: use archive_name directly
2019-03-06 11:02:08 +01:00
Dietmar Maurer
a6b7551311
src/client/http_client.rs: impl download
2019-03-06 11:02:08 +01:00
Wolfgang Bumiller
6716f30bbc
add protocol test client
...
This uses futures for everything which is mostly useful as a
test to see if the protocol crate's non-blocking I/O support
can handle it...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
Wolfgang Bumiller
e2d007f76e
api2/admin/datastore: add a backup protocol test api path
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:26:12 +01:00
Wolfgang Bumiller
e3062f87b1
fixed_index: split add_chunk/add_digest
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Wolfgang Bumiller
c3bb97e59c
fixed_index: impl Send for FixedIndexWriter
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Wolfgang Bumiller
3b7ade9e78
backup/datastore: expose insert_chunk method
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Wolfgang Bumiller
91ebea8262
router: add upgrade method
...
This is just an Async GET method, but semantically
different, so it gets its own router method.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-06 10:25:20 +01:00
Dietmar Maurer
80822b95b1
src/bin/proxmox-backup-client.rs: sort list results
2019-03-06 07:20:51 +01:00
Dietmar Maurer
fba30411fb
src/bin/proxmox-backup-client.rs: allow to set backup-id
2019-03-06 07:02:52 +01:00
Dietmar Maurer
21ea0158b4
src/client/http_client.rs: remove debug statement
2019-03-06 06:45:51 +01:00
Dietmar Maurer
ba3a60b2a4
src/client/http_client.rs: store/load ticket in xdg runtime dir, depend on crate xdg
2019-03-05 12:56:21 +01:00
Dietmar Maurer
f8f94534d4
src/api2/access.rs: authenticate_user() - add ticket login
2019-03-05 12:56:21 +01:00
Dietmar Maurer
e5662b04e9
src/tools/ticket.rs: define const TICKET_LIFETIME
2019-03-05 12:56:21 +01:00
Dietmar Maurer
53cafb5925
src/tools.rs: impl file_get_json()
2019-03-05 12:56:21 +01:00
Wolfgang Bumiller
391d310741
backup_info: BackupDir shortcut
...
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 10:15:06 +01:00
Wolfgang Bumiller
93b49ce38a
backup_info: use two independent Into<String> types
...
there's no reason to force them to be the same input type
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
2019-03-05 09:13:08 +01:00
Dietmar Maurer
3ec3ec3f9a
src/bin/proxmox-backup-client.rs: print backup duration
2019-03-05 08:11:40 +01:00
Dietmar Maurer
d57474e055
src/backup/backup_info.rs: improve docs
2019-03-05 07:28:13 +01:00
Dietmar Maurer
b3483782d1
src/backup/backup_info.rs: move code into separate file
...
Also changed create_backup_dir() parameters - uses &BackupDir now.
2019-03-05 07:18:12 +01:00
Dietmar Maurer
2b01a22507
src/backup/datastore.rs: add helper to sort backup lists
2019-03-04 18:20:57 +01:00
Dietmar Maurer
875fb1c01a
src/backup/datastore.rs: use unix epoch to create DateTime
...
To make sure that we have a timestamp without nanosecond.
2019-03-04 17:58:22 +01:00
Dietmar Maurer
38f8815925
src/backup/datastore.rs: use new() function to create structs
2019-03-04 17:44:27 +01:00